[Bug]: Unable to change MilvusVectorStore uri parameter to a local Milvus path

Bug Description
MilvusVectorStore takes in uri parameter which when given as a local Milvus path (which works on MilvusLite), does not change its path and sticks to ./milvus_llamaindex.db as a default.
Unable to change the uri has implication on other parts of the code when the vector store needs to be fetched again in a different session after all the docs have been inserted.
Version
v0.12.1
Steps to Reproduce
Consider the following code from Milvus demo example, with the only change in LLM and embedding model used to be from HF instead of OpenAI.:
%pip install llama-index-vector-stores-milvus
%pip install llama-index
%pip install pymilvus>=2.4.2
%pip install llama-index-llms-huggingface-api
!pip install "huggingface_hub[inference]"
%pip install llama-index-embeddings-huggingface

! mkdir -p 'data/'
! wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/docs/examples/data/paul_graham/paul_graham_essay.txt' -O 'data/paul_graham_essay.txt'
! wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/docs/examples/data/10k/uber_2021.pdf' -O 'data/uber_2021.pdf'

from llama_index.embeddings.huggingface import HuggingFaceEmbedding
from llama_index.llms.huggingface_api import HuggingFaceInferenceAPI
from llama_index.core import Settings
from google.colab import userdata

# set embed model for index and quering
embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-small-en-v1.5")
Settings.embed_model = embed_model

llm = HuggingFaceInferenceAPI(
model_name="meta-llama/Meta-Llama-3-8B-Instruct",
token = userdata.get('HF_TOKEN')
)
Settings.llm = llm

from llama_index.core import SimpleDirectoryReader

# load documents
documents = SimpleDirectoryReader(
    input_files=["./data/paul_graham_essay.txt"]
).load_data()

print("Document ID:", documents[0].doc_id)

# Create an index over the documents
from llama_index.core import VectorStoreIndex, StorageContext
from llama_index.vector_stores.milvus import MilvusVectorStore

vector_store = MilvusVectorStore(
    uri="./milvus_something.db", dim=384, overwrite=True
)
print(vector_store.uri)
#prints ./milvus_llamaindex.db instead of ./milvus_something.db

storage_context = StorageContext.from_defaults(vector_store=vector_store)
index = VectorStoreIndex.from_documents(
    documents, storage_context=storage_context
)

query_engine = index.as_query_engine()
res = query_engine.query("What did the author learn?")
print(res)

Relevant Logs/Tracbacks
--2024-11-23 11:49:00--  https://raw.githubusercontent.com/run-llama/llama_index/main/docs/docs/examples/data/paul_graham/paul_graham_essay.txt
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|<IP_ADDRESS>|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 75042 (73K) [text/plain]
Saving to: ‘data/paul_graham_essay.txt’

data/paul_graham_es 100%[===================>]  73.28K  --.-KB/s    in 0.002s

2024-11-23 11:49:00 (41.1 MB/s) - ‘data/paul_graham_essay.txt’ saved [75042/75042]

--2024-11-23 11:49:00--  https://raw.githubusercontent.com/run-llama/llama_index/main/docs/docs/examples/data/10k/uber_2021.pdf
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|<IP_ADDRESS>|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1880483 (1.8M) [application/octet-stream]
Saving to: ‘data/uber_2021.pdf’

data/uber_2021.pdf  100%[===================>]   1.79M  --.-KB/s    in 0.01s

2024-11-23 11:49:01 (188 MB/s) - ‘data/uber_2021.pdf’ saved [1880483/1880483]

DEBUG:pymilvus.milvus_client.milvus_client:Created new connection using: 3387021e2381447792762a5d40b37cea
DEBUG:pymilvus.milvus_client.milvus_client:Successfully created collection: llamacollection
Document ID: 83b63d52-3e07-40e0-94ed-d9527ef70414
DEBUG:pymilvus.milvus_client.milvus_client:Successfully created an index on collection: llamacollection
./milvus_llamaindex.db
1. AI, as practiced at the time, was a hoax. 2. There was an unbridgeable gap between what AI programs could do and actually understanding natural language. 3. Lisp was interesting for its own sake and not just for its association with AI. 4. The author could salvage Lisp from the wreckage of his plans. 5. The author could write essays and publish them online, which was a new medium. 6. Working on things that weren't prestigious could be a sign of something real to be discovered and the right kind of motives. 7. The author could learn from his experiences and write about them in essays. 8. The author could use the web to publish his essays and reach a wider audience. 9. The author could work on different projects, such as spam filters, painting, and cooking. 10. The author could use his experiences to teach others and share his knowledge. 11. The author could use his writing to express himself and share his thoughts with others. 12. The author could use his experiences to learn and grow as a person. 13. The author could use his writing to inspire others and share his passion with them. 14. The author could use his experiences to create

Looking at the latest source code, it's directly passing the uri to the milvus client
https://github.com/run-llama/llama_index/blob/b452e003c2fc0684e650022a720bde193162e37b/llama-index-integrations/vector_stores/llama-index-vector-stores-milvus/llama_index/vector_stores/milvus/base.py#L273

Is there a motivation behind using self.client at places and self._milvusclient in other?
Although it should not matter, but just curious:
@property
def client(self) -> Any:
    """Get client."""
    return self._milvusclient

https://github.com/run-llama/llama_index/blob/b452e003c2fc0684e650022a720bde193162e37b/llama-index-integrations/vector_stores/llama-index-vector-stores-milvus/llama_index/vector_stores/milvus/base.py#L273
https://github.com/run-llama/llama_index/blob/b452e003c2fc0684e650022a720bde193162e37b/llama-index-integrations/vector_stores/llama-index-vector-stores-milvus/llama_index/vector_stores/milvus/base.py#L283
I would also like to add that the bug that I encounter with a wrong uri being returned, does not mean that the ./milvus_something.db was not created. It was. But rather the vector_store was pointing to a wong (default in this case) uri instead.

If it creates the proper db, it's a pretty mild bug I guess?
Could be related to setting the field on the class directly as a string vs. A pydantic field object. Will take a look

Note that collection_name, even though initialized the same way as uri in the __init__, can be overrided by the user input:
https://github.com/run-llama/llama_index/blob/b452e003c2fc0684e650022a720bde193162e37b/llama-index-integrations/vector_stores/llama-index-vector-stores-milvus/llama_index/vector_stores/milvus/base.py#L188C5-L188C20
output of the test code:
stores_text=True is_embedding_query=True stores_node=True uri='./milvus_llamaindex.db' token='' collection_name='some_collection' dim=384 embedding_field='embedding' doc_id_field='doc_id' similarity_metric='IP' consistency_level='Session' overwrite=True text_key=None output_fields=[] index_config={} search_config={} collection_properties=None batch_size=100 enable_sparse=False sparse_embedding_field='sparse_embedding' sparse_embedding_function=None hybrid_ranker='RRFRanker' hybrid_ranker_params={} index_management=<IndexManagement.CREATE_IF_NOT_EXISTS: 'create_if_not_exists'> scalar_field_names=None scalar_field_types=None

If anyone stumbles upon this issue because they are trying to load a milvus vector store with preinserted docs, make sure to instantiate milvus client with overwrite=False during storage context loading otherwise your vector store will be wiped off even before llamaindex loads it. Also use storage context for this task and persist it while saving and loading. The above script would have component like this :
# saving storage context with our vector store
storage_context = StorageContext.from_defaults(vector_store=vector_store)
index = VectorStoreIndex.from_documents(
    documents, storage_context=storage_context
)
storage_context.persist(persist_dir="./storage")

# loading storage context that contains our vector store
storage_context = StorageContext.from_defaults(
    docstore=SimpleDocumentStore.from_persist_dir(persist_dir="./storage"),
    vector_store=vector_store,
    index_store=SimpleIndexStore.from_persist_dir(persist_dir="./storage"),
)
indices = load_indices_from_storage(storage_context)

query_engine = indices[0].as_query_engine()
res = query_engine.query("What did the author learn?")
print(res)

Ofcourse saving and loading storage context during different sessions would work in this case. Also, I did not find any other way other than using storage context.
Whether milvus is actually storing the data can be checked with a small script like this:
from pymilvus import MilvusClient

# 1. Create a milvus client
client = MilvusClient(
    uri="./milvus_something.db",
)

# 3. List collections
print(client.list_collections() )
print(client.get_collection_stats(collection_name="some_collection"))

# fetches first record only
res = client.query(
    collection_name="some_collection",
    filter="",
    offset=1,
    limit=1
)

print(res)


feat(application integration): Add service account sample for creating service account

…g auth configs
Description
Fixes #
Note: If you are not associated with Google, open an issue for discussion before submitting a pull request.
Checklist
Readiness

[* ] Yes, merge this PR after it is approved
[ ] No, don't merge this PR after it is approved

Style

[* ] My sample follows the rules described for Terraform in the Effective Samples style guide
[ ] My sample follows the other requirements and best practices in the Contributing
guide

Testing

[* ] I have performed tests described in the Contributing guide:

[ ] Tests pass: terraform apply
[ ] Lint pass: terraform fmt check

Intended location

[ ] Yes, this sample will be (or already is) included on cloud.google.com
Location(s):

[ ] No, this sample won't be included on cloud.google.com
Reason:

API enablement

[ ] If the sample needs an API enabled to pass testing, I have added the service to the Test setup file

Review

[ ] If this sample adds a new directory, I have added codeowners to the CODEOWNERS file

/gcbrun

/gcbrun

cl/657845149


Cannot Install and Import Xpdf

from xpdf_python import to_text
An exception has occurred, use %tb to see the full traceback.
SystemExit: Did not detect correctly installed xpdf. Please follow install instructions at: https://github.com/ecatkins/xpdf_python.
UserWarning: To exit: use 'exit', 'quit', or Ctrl-D. [interactiveshell.py:2971]

The problem is that you didn't install xpdf and the automatic installer script is outdated, you can find updated version in my fork of this repo, or you can simply install xpdf following the instruction in the official website.

Does work from terminal. But does not work out of a python script or when installed as a pip package


[ATIP] Fix android swipe step bug

-Fix android swipe step bug

test locally ok. @Honry please help to review it.

LGTM


Create a 2D map

Using pre-defined mapping method, create a 2D map of one room.

You can navigate through environment using teleoperation package which is already installed


Support "LIMIT ?" in UPDATE and DELETE for MySQL

PostgreSQL doesn't support UPDATE-LIMIT
issue #2131

Fixes #2131


add pixi export

#191

This change is

This creates a zip with the spritesheet and a json metadatafile in the pixijs format.
But to download only the json metadatafile and separate it from the png spritesheet download may be better.

Thanks @PSeitz this looks great!
I was about to suggest moving this to the "other" tab, but I guess it can be useful to set the number of columns for the export here, so let's leave it in the "PNG" tab.
I think we should open another issue to rework a bit the UI of this tab. Maybe group the image download / data-uri options (or even remove the data-uri button completely ... this one got released by mistake, I wanted to have it only for development initially).
I am not really using Pixi personally, so if @hassanshaikley wants to also give some feedback that'd be great!

Code change looks good. Will merge soon.

This is awesome! : )

Thanks for the feedback @hassanshaikley
Merging and opening follow up issues to track my review comments.
Thanks for the great work @PSeitz !


Added additional check to scrollbar drawing.

This should resolve #1234.

@aodhneine looks good to me! :+1: Thank you so much for your contribution!


Reconvert linters to use tools

fixes: https://linear.app/trunk/issue/TRUNK-6640/have-linter-definitions-use-tools

I think the testing failures are arising from the (accidental?) taplo KGV bump


Makefile: Download versioned protofiles

Currently the sdk/generated.proto and uast/generated.proto files are included in the repository. To correctly support SDK versioning, it should download them from gopkg.in redirect URL.

Duplicated of #36


Invalid diagnostics generated by compiler

Description:
Observed $subject by a Ballerina Hackathon attendee. Consider the below example code snippet
import ballerina/io;

type EmployeeFillUpSummary record {|int id;|};
xmlns "blaaa" as s;
public function abc(string inputFile) {
      do {
        //xml xmlPaylod = check io:fileReadXml(inputFile);
        io:ReadableByteChannel rbc = check io:openReadableFile(inputFile);
        io:ReadableCharacterChannel rch = new (rbc, "UTF8");
        xml payload = check rch.readXml();
        io:println("#1->", payload);
        map<EmployeeFillUpSummary> empMap={};
        EmployeeFillUpSummary[] emA = [];
        var _ = check from var x in payload/<s:FuelEvent>
            do {
                xml gallons = x/<s:gallons>;
                decimal gal  = check decimal:fromString(gallons.data().trim());
                xml odometer = x/<s:odometerReading>;
                int odm = check int:fromString(odometer.data().trim());
                xml gasPrice = x/<s:gasPrice>;
                decimal gp = check decimal:fromString(gasPrice.data().trim());
                string empid = check x.employeeId;
                int employeeId = check int:fromString(empid);
                io:println("2 gallons ", gallons.data(), " odometer ", odometer.data(), " gasPrice ", gasPrice.data(), " employeeId ", employeeId);
                empMap
            };
    };
}

Compiler produces an unknown type error  for empMap (which is invalid). It seems like root cause would the parser recovery logic.

@lochana-chathura @rdulmina Any comments on this?

Minimal version
type EmployeeFillUpSummary record {|int id;|};
public function abc(string inputFile) {
    map<EmployeeFillUpSummary> empMap={};
    empMap
}

On a separate note, I think it is better if we can rearrange the diagnostic in a way, syntax errors are at the top, and then the semantics errors. Because one has to fix the syntax errors first and then only the semantics. And also if we have syntax errors, semantic errors are based on them.

Given the code is incomplete, I think this is okay. Recovers as local-var-decl,
public function abc(string inputFile) {
    map<EmployeeFillUpSummary> empMap = {};
    empMap MISSING[Identifier] MISSING[;] // empMap will be treated as a type-ref
}

At the parser level, we don't have the semantic info and we don't know empMap is a type-reference or a variable reference. Immediately after the user enters the = token next to the empMap, the ambiguity is resolved so the diagnostic will go away.
We can make the recovery empMap MISSING[=] MISSING[;]. But, it doesn't make sense to me. For example,
type T int;

public function abc(string inputFile) {
    // option-1
    // T will be treated as a type-ref. So no error which is correct.
    T MISSING[Identifier] MISSING[;]

    // option-2
    // T will be treated as variable-ref.
    // So the compile will give "cannot assign a value to a type definition"
    // Although the user was expecting write something like T x = 10;
    T MISSING[=] MISSING[;]
}

If we keep "type name starts with a capital letter" convention aside. Below is a perfectly valid code.
type empMap map<any>;

public function abc(string inputFile) {
    empMap _ = {};  // Here empMap is a type ref.
}

Hmm. Lochana, sorry I won't fully agree with this.
We should guide users accurate manner when the code is incomplete. That's the situation user need proper guidance and help. Hence we shouldn't ignore this case just because the code is incomplete (and provide incorrect diagnostics)
Just to be clear, are we saying that our diagnostics are valid only when the code is complete, and incomplete code snippets could generate invalid diagnostics?
I do get the reason from the parser perspective, but again providing unknown type diagnostic is incorrect. Even if we write some random string that also gives unknown type, this should be something like unresolvable symbol or cannot resolve symbol.

Agree with your point as well.
We can consider making recovery of foo, foo MISSING[;]. Then we will not have this unknown type error.
Instead, the user will be given  "Invalid expression statement". That is the java behavior as well.

Assigned to @SasinduDilshara


Feature: Support flexible priority in solve task

Checklist

[x] I added a descriptive title
[x] I searched open requests and couldn't find a duplicate

What is the idea?
Following https://github.com/conda/rattler/pull/598, it would be very useful if rattler/resolvo also supported flexible priority: With flexible channel priority, the solver may reach into lower priority channels to fulfill dependencies, rather than raising an unsatisfiable error
Why is this needed?
The main use case I see for this is user requesting for a package A==2.0.0 and getting an error because this newer version of the package is not present in the first priority channel.
What should happen?
Frankly, I'm not sure I have enough knowledge of the internals to be very opinionated here. But I'd like to see A==2.0.0 be satisfied if possible, given the users acknowledgement that they're searching for a specific version
Additional Context
No response

Thats for the feature request. I am not entirely sure how to implement this yet.
@wolfv would sorting the packages by channel priority work?


ARROW-5838: [C++] Delegate OPENSSL_ROOT_DIR to bundled gRPC

Codecov Report

Merging #4795 into master will increase coverage by 1.64%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #4795      +/-   ##
==========================================
+ Coverage   87.43%   89.07%   +1.64%
==========================================
  Files         996      719     -277
  Lines      139677   100035   -39642
  Branches     1418        0    -1418
==========================================
- Hits       122124    89109   -33015
+ Misses      17191    10926    -6265
+ Partials      362        0     -362

Impacted Files
Coverage Δ

go/arrow/ipc/writer.go

js/src/util/fn.ts

go/arrow/memory/memory_avx2_amd64.go

rust/datafusion/src/execution/filter.rs

rust/arrow/src/csv/writer.rs

rust/datafusion/src/bin/main.rs

go/arrow/ipc/file_reader.go

js/src/builder/index.ts

js/src/enum.ts

go/arrow/array/builder.go

... and 267 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a9a82ec...1aee6d4. Read the comment docs.

Seems there are something wrong with master branch commits.

We've rebased master branch for 0.14.0 release:  https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide#ReleaseManagementGuide-Rebasingthemasterbranchonlocalreleasebranch
All of our pull requests needs to rebase on master.

I'll test it locally.


Microsoft Office Bar

Good day All,
Does anyone know how to remove the Microsoft Office App bar at the top of the screen?

I'm having a similar issue. It happened to me after the session expired and I refreshed. I tried uninstalling, then reinstalling and it's still an issue.

I have the same issue, feels weird. Comparing to the screenshots of the app in the presentation, which look really good i get a really ugly gui.


API: category id type

The ids of feeds & entries is i64 while categories are inconsistent. Sometimes its a string and other times i64. I feel like categories should also have an i64 id property.
I guess this exception was done to be able to represent the "all" category? If that's the only case, maybe /category/get should just return the top level categories & top level feeds? Or maybe reserve the id 0 or -1 for "all"?

You're right it's inconsistent and it's indeed because there are the 'all' and 'starred' categories.
Actually I feel like moving to a string id across the board instead of a long number would be the way to go, as it would prevent the need for special numeric values.
Probably for 4.x though since it would be a breaking change.


PHP's DateTime type and SQL Server's Date Type mismatch

Hello,
I'm not sure if this is expected behavior or a bug, but this is an issue that I'm having so I thought I would report it to find out.
I'm running SQLSRV 5.9.0 with PHP 8.0 and MSODBCSQL <IP_ADDRESS>-1 on Ubuntu 20.04, accessing SQL Server 15.0.2080.9 Development Edition running on a Windows 10 Pro machine (my dev machine, but we are also having this issue on another SQL Server 12.0.6164.21 Web Edition).
We have a table that stores data in the Date format. I'm trying to run a SELECT query with a WHERE clause that looks for rows with a specific Date, but I'm parameterizing the query with a DateTime object for that date. I would expect for that SELECT query to select the rows with that date, but it instead returns nothing (I expected that it would return since I can run an INSERT parameterized with the DateTime and it works fine. It also throws no errors.)
Here's code to reproduce:
<?php
$db = sqlsrv_connect('databaseConnectionString', array(
    "UID" => "databaseAdmin",
    "PWD" => "databaseAdminPassword",
    "CharacterSet" => "UTF-8"
));

$now = new DateTime();

sqlsrv_begin_transaction($db);
sqlsrv_query($db, "CREATE DATABASE testDateBug");
sqlsrv_query($db, "USE testDateBug");
sqlsrv_query($db, "CREATE TABLE testTable (
    testID int IDENTITY PRIMARY KEY,
    testDate date NOT NULL
    )");
sqlsrv_query($db, "INSERT INTO testTable (testDate) VALUES (?)", array($now));
$result = sqlsrv_query($db, "SELECT testID FROM testTable WHERE testDate = ?", array($now->format("Y-m-d")));
$hasRows = sqlsrv_has_rows($result);
echo "Has Rows? " . ($hasRows?"TRUE":"FALSE") . PHP_EOL; //Returns TRUE
$result = sqlsrv_query($db, "SELECT testID FROM testTable WHERE testDate = ?", array($now));
$hasRows = sqlsrv_has_rows($result);
echo "Has Rows? " . ($hasRows?"TRUE":"FALSE") . PHP_EOL; //Returns FALSE...
sqlsrv_rollback($db);
sqlsrv_close($db);

Hi @timnash , this is by design. The column is of type Date, and your first query by specifying the format $now->format("Y-m-d") is correct. In this case, your input is a string, and there is no conversion behind the scene, so it's straightforward.
However, with your second query, since the parameter is passed in as a PHP DateTime object, the sqlsrv driver first converts it to a datetimeoffset and then to a string such that no details will be missed.
With no timezone specified, UTC is the default, so the second SQL statement is essentially equivalent to the one shown below, which explains why no row is matched:
SELECT testID FROM testTable WHERE testDate = "2021-07-05 16:58:40.543746 +00:00"
The reason why insertion works is that the input object is converted to Date type during insertion.
That being said, with the sqlsrv driver, there is a way to indicate the data type of your input parameter:
array($value [, $direction [, $phpType [, $sqlType]]])
Hence, you need to change your php script like this:
$param = array(array($now, null, null, SQLSRV_SQLTYPE_DATE));
$result = sqlsrv_query($db, "SELECT testID FROM testTable WHERE testDate = ?", $param);
$hasRows = sqlsrv_has_rows($result);
echo "Has Rows? " . ($hasRows?"TRUE":"FALSE") . PHP_EOL;

Please give it a try. Hope this clarifies things.

Thank you, that's a very comprehensive answer.


fix(config): change GED2150 extension to json

Renamed GED2150 extension from sjon to json
Fixed a type in the description

Ah damn, I should look more closely...
Thanks!

I should look more closely...

So should I, that was my doing 🙈


Fix style issue with iTunes App Store on homepage

References:

https://en.wikipedia.org/wiki/ITunes
https://www.apple.com/itunes/

Looks like that build timed out.

Thanks for this!
After #1163 try.html is generated, so could you please make these changes in lib/all-badge-examples.js instead, and then run npm run build?
try.html will no longer be tracked after #1194.

Closing this in favour of https://github.com/badges/shields/pull/1211
(I couldn't work out how to update this PR)


[Feature] Using a open-source LLM instead of Open AI

STEPS:

(optional) if you haven't tried it, download PYGMALION-AI Metharme 1.3B , a 2.9GB high-context AUTOTASKING ENGINE, add the metharme template to the bottom of your prompt <|system|><|user|>Input<|model|>

create prelaunchers

launch textgen, confirm api address

launch gpt-engineer

you're done, have fun

prelauncher script i use for textgen (modify args and gpu settings to meet your individual needs and hardware)
create a fake/dummy/spoofed key, doesn't matter what it is, as long as it matches what you put in gpt-engineer
launch(textgen).sh
#!/bin/bash
export HOST_PORT=7861
export CUDA_VISIBLE_DEVICES=0
export TORCH_USE_CUDA_DSA=1
export TORCH_CUDA_ARCH_LIST=8.6+PTX
export PYTORCH_CUDA_ALLOC_CONF=garbage_collection_threshold:0.25,max_split_size_mb:128M
export OPENAI_API_KEY="dummy" # export ANY string in this field, including the default textgen key "dummy" it works!

python3 ./server.py --model TheBloke_orca_mini_7B-GPTQ --loader exllama --listen --listen-port 7861 --api --gpu-split 8_20 --trust-remote-code --extensions openai

prelauncher i use for gpt-engineer
launch(gpte).sh
#!/bin/bash
export OPENAI_API_KEY="dummy" # make sure it matches what you export/set in textgen
export OPENAI_API_BASE=http://<IP_ADDRESS>:5001/v1 #depending on settings, and ports in use when you launched textgen, this may change slightly if 5001 is in use - confirm base with textgen launch-post, prior to launching gpt-engineer

python3 gpt_engineer/main.py projects/MasterCodeBase/ --temperature 1 --steps tdd+

One thing that could be useful is to use the Langchain framework. But remind that actual open sources models that can run on a standard local machine, like tiiuae/falcon-7b are far from beeing enough powerful to be used for this project.
I think that the OpenAI API will remain, in the short term, the only valid option for applications of this type.

Yes, Falcon-7b does a poor job at coding. Although, there are other OpenLLM's trained specifically for coding that are possible to run on a RTX 3090+ or users can get a Runpod.io VPS with 48GB-160GB GPU VRAM and run the 30b/40b models.
These models were trained specifically for coding and appear to do pretty well, adding support for these would be pretty great:

WizardCoder-15B-GPTQ (4-bit) - https://huggingface.co/TheBloke/WizardCoder-15B-1.0-GPTQ
FalCoder 7b (Fine-tuned with CodeAlpaca 20k Instructions Dataset): https://huggingface.co/mrm8488/falcoder-7b

STEPS:

(optional) if you haven't tried it, download PYGMALION-AI Metharme 1.3B , a 2.9GB high-context AUTOTASKING ENGINE, add the metharme template to the bottom of your prompt <|system|><|user|>Input<|model|>
create prelaunchers
launch textgen, confirm api address
launch gpt-engineer
you're done, have fun

prelauncher script i use for textgen (modify args and gpu settings to meet your individual needs and hardware)
create a fake/dummy/spoofed key, doesn't matter what it is, as long as it matches what you put in gpt-engineer
launch(textgen).sh
#!/bin/bash
export HOST_PORT=7861
export CUDA_VISIBLE_DEVICES=0
export TORCH_USE_CUDA_DSA=1
export TORCH_CUDA_ARCH_LIST=8.6+PTX
export PYTORCH_CUDA_ALLOC_CONF=garbage_collection_threshold:0.25,max_split_size_mb:128M
export OPENAI_API_KEY="dummy" # export ANY string in this field, including the default textgen key "dummy" it works!

python3 ./server.py --model TheBloke_orca_mini_7B-GPTQ --loader exllama --listen --listen-port 7861 --api --gpu-split 8_20 --trust-remote-code --extensions openai

prelauncher i use for gpt-engineer
launch(gpte).sh
#!/bin/bash
export OPENAI_API_KEY="dummy" # make sure it matches what you export/set in textgen
export OPENAI_API_BASE=http://<IP_ADDRESS>:5001/v1 #depending on settings, and ports in use when you launched textgen, this may change slightly if 5001 is in use - confirm base with textgen launch-post, prior to launching gpt-engineer

python3 gpt_engineer/main.py projects/MasterCodeBase/ --temperature 1 --steps tdd+

is this confirm working by anyone?

This has been addressed in the PRs #639 #644


[BEAM-952] Use ServiceLoader to register IOChannelFactories.

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

[ ] Make sure the PR title is formatted like:
[BEAM-<Jira issue #>] Description of pull request
[ ] Make sure tests pass via mvn clean verify. (Even better, enable
Travis-CI on your fork and ensure the whole test matrix passes).
[ ] Replace <Jira issue #> in the title with the actual Jira issue
number, if there is one.
[ ] If this contribution is large, please file an Apache
Individual Contributor License Agreement.

R: @lukecwik

PTAL
Updated previous PRs to drop factory factory.
other comments are addressed in fixup.
For documentation, are you asking about how to define IOChannelFactory for other file systems or How to register standard factories in runners?
I want to defer adding new documentations, since we are going to redesign the IOChannelFactory.

PTAL
I kept registerStandardIOFactories backward compatible. We can remove it once worker code is updated.

PTAL
marked registerIOFactoriesAllowOverride as deprecated.

rebased for Jenkins license issue

PTAL

PTAL

LGTM


Persistent storage for certificates in S3

Welcome!

[X] Yes, I've searched similar issues on GitHub and didn't find any.
[X] Yes, I've searched similar issues on the Traefik community forum and didn't find any.

What did you expect to see?
As a Traefik user and using the very useful feature to have it provision SSL Certificates, to avoid having to request new certificates every time a container is re-created (faulty underlying HW, scaling etc), persisting in a different layer, i.e. AWS S3, with a logic of "check first if exists, generate second", would allow for a highly distributed deployment of traefik without risking rate limiting.
As of today one has to persist the certificates storage location and implement that logic externally to traefik.
Thank you for your consideration

Hello @JohnPreston,
Thanks for your interest in Traefik!
As the ACME HA feature was dropped, and as there is no good proposition/design/implementation of the multiple certificate store proposal (#4756), this S3 persistent storage feature proposal cannot be accepted for now.


Max workers to buy

What if when you buy workers it automatically has the max number of workers in the prompt! Then you don't have to write it in every time.

To do that, you just have to do this:
var money = 5;
prompt("How much money do you have?", money);

@GoRedbirds please answer this.

Great! I'll start working on that! @GoRedbirds I might have to wait a little bit because certain events might get in the way so it should be done by 12:00 pm tomorrow.

Done!


Update docs and example for dfuse 0.3.0 when providing WebSocket clients

Issue
Since version 0.3.0, the dfuse client creates a GraphqlStreamClient (see https://github.com/dfuse-io/client-js/blob/v0.3.0/src/client/client.ts#L258). This behavior did not exist before and developers that have chosen to not pollute the global scope will find themselves with the runtime error:

You did not provide a webSocket option and we were not able find a WebSocket object in the global scope to create use.

This is because the StreamClient will have its own WebSocket factory option provided via streamClientOptions.socketOptions.webSocketFactory but the GraphqlStreamClient will have none.
How to Reproduce
To reproduce this issue, simply follow the Example instructions and run the nodejs-fetch-and-websocket-options.ts example from the master branch:
yarn run:example examples/advanced/nodejs-fetch-and-websocket-options.ts

The docs fix
This PR updates the above-mentioned example and the README to reflect the new changes introduced with version 0.3.0: simply provide the WebSocket factory to graphqlStreamClientOptions.socketOptions.webSocketFactory in addition to the already mentioned streamClientOptions.
Alternatives?
An option would be to restructure the DfuseClientOptions interface to accept a WebSocketFactory property so that any client that requires a WebSocket can use the provided one (or, when undefined, fallback to inferring from the global scope) as opposed of providing it manually for every client that you may or may not even use.

Thanks for the PR.
You are right, that causes an error for such cases. I agree for the sharing, there is little reason not to share, and in fact, the refactor was planned but we decided to post-pone it.
We might decide to even drop WebSocketFactory concept and simply ask user to pass a compatible WebSocket object directly that we would simply call.
I'll merge that and look into improving the situation.


[Doc Update] Steps for how to Update the client secret SECRET_BOT_PASSWORD if it is expired for existing published application

Type of issue
Typo
Feedback
Section can be updated: https://learn.microsoft.com/en-us/microsoftteams/platform/toolkit/debug-overview#customize-debug-settings
Steps:

Create new Secret in app registration & keep it for later use
Update .env file for SECRET_BOT_PASSWORD and provision/deploy the application
If you app is already deployed, make sure your app service configuration in azure resource is also updated with newly created app/bot secret.

Page URL
https://learn.microsoft.com/en-us/microsoftteams/platform/toolkit/debug-overview
Content source URL
https://github.com/MicrosoftDocs/msteams-docs/blob/main/msteams-platform/toolkit/debug-overview.md
Author
@surbhigupta
Document Id
ade7805f-ba79-1de6-1a0d-7cc3e221c5d8

Hi @surbhigupta12 , could you please help here to update the document as suggested?


Using any of the available lattices at any given time, instead of a specifically named one

This helps avoid test failures anytime the available lattice(s) changes.

@josh146 I might need your help on this one. It seems that there are a few tests that are failing because I can't seem to do the following:
>>> import pennylane as qml
>>> dev_numpy = qml.device('forest.numpy_wavefunction', wires=2)

which gives me an error message saying pennylane._device.DeviceError: Device does not exist. Make sure the required plugin is installed.
I could trace it back to https://github.com/XanaduAI/pennylane/blob/master/pennylane/init.py#L216, but don't exactly know how to resolve this issue.

Hi @msohaibalam, I'm assuming this a test suite you are running locally, as I can't find the same error message in the Semaphore log? In order to have installed devices become available to PennyLane, the setup.py install command needs to be run on the plugin at some point - the best way is to install the plugin using pip in development mode:
pip install -e .

Usually I get the above error if I forget to install the plugin (for example, by cloning the repository and then directly running pytest)

I thought I had already ran pip install -e . but apparently not! Thanks for the input, my tests (indeed, this was for a local test suite) are now passing. I'll have to dig into the error msgs in the Semaphore log a bit more to really get to the bottom of the failing build though.

@karalekas Semaphore is failing tests that pass locally for me. Most (all?) of them seem to be
E       RuntimeError: Error encountered while parsing the quil program at line 1 and column 5
E       Received an '    ' but was expecting one of [ ' ', NEWLINE ]

Any ideas why this is happening?

I think I've narrowed down the cause of the error. When I run the tests locally, I'm running them against pyquil version 2.7.2. However, I noticed that Semaphore is installing pyquil 2.6 due to caching; running the tests locally on 2.6 causes the same test errors.
I've pushed an update to the setup.py and requirements.txt to specify pyquil>=2.7, and see if that fixes the Semaphore build.

Seems to have worked!

@josh146 Awesome! Merging this in now.


Create Character “emmerson”

Automatically generated by Netlify CMS

👷 Deploy Preview for dynamic-cheesecake-5fbd8b processing.

Name
Link

🔨 Latest commit
cde709254d1e2faf0525e2795b024dcee4071192

🔍 Latest deploy log
https://app.netlify.com/sites/dynamic-cheesecake-5fbd8b/deploys/62eb2fe5e6359c0008bfeac8


Math magicians: events

Hello,
In this PR, I have worked on the calculator functionality feature to ensure that All buttons are working properly.
Here are are what I have achieved:

[ ] Create a new directory called logic inside your project
[ ] Copy the files calculate.js and operate.js into your logic/ directory
[ ] Install big.js using NPM
[ ] Import calculate.js in your Calculator.js component
[ ] Implement click event handlers using the math logic from calculate.js and operate.js in my React component

Kindly check and let me know what you think.
Best regards.
Happy Coding🎉

Hello @zainsadaqat
Thank you🙏🙏  for approving my PR.


Add custom composite action to set up CI environments

It may be possible to set up CI environments (e.g. JDK) in a shared custom action with all workflows.

It seems to work.
Prepare all required actions
Getting action download info
Download action repository 'actions/setup-java@v[2](https://github.com/apache/lucene/runs/5136360849?check_suite_focus=true#step:3:2)' (SHA:f0bb91606209742fe[3](https://github.com/apache/lucene/runs/5136360849?check_suite_focus=true#step:3:3)ea[4](https://github.com/apache/lucene/runs/5136360849?check_suite_focus=true#step:3:4)0199be2f3ef19[5](https://github.com/apache/lucene/runs/5136360849?check_suite_focus=true#step:3:5)ecabf)
Run ./.github/actions/setup-action
  with:
    java-version: 17
Run actions/setup-java@v2
  with:
    distribution: temurin
    java-version: 17
    java-package: jdk
    architecture: x[6](https://github.com/apache/lucene/runs/5136360849?check_suite_focus=true#step:3:6)4
    check-latest: false
    server-id: github
    server-username: GITHUB_ACTOR
    server-password: GITHUB_TOKEN
    overwrite-settings: true
    job-status: success
Resolved Java 1[7](https://github.com/apache/lucene/runs/5136360849?check_suite_focus=true#step:3:7).0.1+[12](https://github.com/apache/lucene/runs/5136360849?check_suite_focus=true#step:3:12) from tool-cache
Setting Java [17](https://github.com/apache/lucene/runs/5136360849?check_suite_focus=true#step:3:17).0.1+12 as the default

Java configuration:
  Distribution: temurin
  Version: 17.0.1+12
  Path: /opt/hostedtoolcache/Java_Temurin-Hotspot_jdk/17.0.1-12/x64

I'm closing this. I think we'd need more complex or fully scratched custom actions not to duplicate the JDK set-up across workflows.


fix: disalbe bounce option for macos, closes #557

I fixed https://github.com/tauri-apps/wry/issues/557.
What kind of change does this PR introduce?

[ ] Bugfix
[x] Feature
[ ] Docs
[ ] New Binding issue #___
[ ] Code style update
[ ] Refactor
[ ] Build-related changes
[ ] Other, please describe:

Does this PR introduce a breaking change?

[ ] Yes, and the changes were approved in issue #___
[x] No

Checklist

[x] When resolving issues, they are referenced in the PR's title (e.g fix: remove a typo, closes #___, #___)
[ ] A change file is added if any packages will require a version bump due to this PR per the instructions in the readme.
[ ] I have added a convincing reason for adding this feature, if necessary

Other information
For compatibility, bounce is On by default.

Does bouncing mean rubber band scrolling?

I thought there was a CSS way to do this too...
body {
  height: 100%;
  padding: 0;
  overflow: auto;
  margin: 0;
  -webkit-overflow-scrolling: touch;
}

or this (if you manage scrolling internally in the layout):
body {
  position: fixed;
}

Does bouncing mean rubber band scrolling?

Yes, If webview's content size is over the window size, rubber band scrolling is good. But, the app using wry always enable rubber band scrolling. I thought it would be better if wry had an enable/disable option.

I thought there was a CSS way to do this too...
body {
height: 100%;
padding: 0;
overflow: auto;
margin: 0;
-webkit-overflow-scrolling: touch;
}

It wasn't working on my project.

WkWebview on macOS doesn't have scroll view somehow.
https://stackoverflow.com/questions/39929552/disable-bounce-scrolling-for-wkwebview-in-macos
Did it solve the issue on your side?

Sorry. This PR's solution works in specific situations. I'll close this PR.

I get horizontal rubber band scrolling, and it's always an annoyance with no benefit. The CSS above doesn't fix it, but the PR does. -webkit-overflow-scrolling isn't supported for me on macOS 10.15.
Is there any reason to actually keep bounce on by default?


THOUSANDS cmdlet in 2016 but links, sysnopsis and help are missing.

Cmdlets not finished, links and sysnopsis not inside the cmdlets object.
Most of Cmdlets with nothing in the man even after a update-help, could Mr Nadella finish the product  we paid please ? :-/
Steps to reproduce
$modules = gmo BitsTransfer -l | select -ExpandProperty name #BitsTransfer as exemple

$log0 = "$home\Desktop\desk.txt"
if (Test-Path $log0) {rm $log0 -Force}

foreach ($module in ($modules | sort))
{
Write-Host "====== $module ======" -ForegroundColor Red

"====== $module ======" | out-file -FilePath $log0 -Append
"**[[powershell:$module]]**" | out-file -FilePath $log0 -Append

    foreach ( $cmdlet in (gcm -Module $module))
    {
    $Synopsis = (get-help $cmdlet.name).Synopsis
    if ($Synopsis -match '\>\]') {$Synopsis = '   '}
    $cmdlet | select name,HelpUri | %{ '|[['+$cmdlet.HelpUri+'|'+$cmdlet.name+']]|'+$Synopsis+' |' }  | out-file -FilePath $log0 -Append
    }
}

Expected behavior
like the ... AWS module for exemple ... 2088 cmdlets like that ... perfectly documented
====== AWSPowerShell ======
**[[powershell:AWSPowerShell]]**
|[[http://docs.aws.amazon.com/powershell/latest/reference/index.html?page=Add-AASScalableTarget.html&tocid=Add-AASScalableTarget|Add-AASScalableTarget]]|Invokes the RegisterScalableTarget operation against Application Auto Scaling. |
|[[http://docs.aws.amazon.com/powershell/latest/reference/index.html?page=Add-ACMCertificateTag.html&tocid=Add-ACMCertificateTag|Add-ACMCertificateTag]]|Invokes the AddTagsToCertificate operation against AWS Certificate Manager. |
|[[http://docs.aws.amazon.com/powershell/latest/reference/index.html?page=Add-ASAAttachmentsToSet.html&tocid=Add-ASAAttachmentsToSet|Add-ASAAttachmentsToSet]]|Invokes the AddAttachmentsToSet operation against AWS Support API. |

Actual behavior
====== BitsTransfer ======
**[[powershell:BitsTransfer]]**
|[[|Add-BitsFile]]|    |
|[[|Complete-BitsTransfer]]|    |
|[[|Get-BitsTransfer]]|    |
|[[|Remove-BitsTransfer]]|    |
|[[|Resume-BitsTransfer]]|    |
|[[|Set-BitsTransfer]]|    |
|[[|Start-BitsTransfer]]|    |
|[[|Suspend-BitsTransfer]]|    |

> $PSVersionTable
PSVersion                      5.1.14393.206
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.14393.206
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           <IP_ADDRESS>

Hi!
  Help content certainly lags behind with the new state of things, but this might be as simple as pulling down the help content.
Cheers!

After running Update-Help, if you're still seeing missing content, please file an issue on PowerShell-Docs.

@RamblingCookieMonster
@joeyaiello
i did update-help before to post this issue.
The result is on server 2016 (AWS), 8 days ago (not on an odlder release of windows like 2012r2).
Microsoft Windows Server 2016 Base - ami-133bc27c
Microsoft Windows 2016 Datacenter edition. [English]

The same request today after an other update-help is better, showing that Microsoft working is working on it, an excellent feature like powershell need that.
====== BitsTransfer ======
**[[powershell:BitsTransfer]]**
|[[|Add-BitsFile]]|Adds one or more files to an existing BITS transfer job. |
|[[|Complete-BitsTransfer]]|Completes a BITS transfer job. |
|[[|Get-BitsTransfer]]|Gets the associated BitsJob object for an existing BITS transfer job. |
|[[|Remove-BitsTransfer]]|Cancels a BITS transfer job. |
|[[|Resume-BitsTransfer]]|Resumes a BITS transfer job. |
|[[|Set-BitsTransfer]]|Modifies the properties of an existing BITS transfer job. |
|[[|Start-BitsTransfer]]|Creates a BITS transfer job. |
|[[|Suspend-BitsTransfer]]|Suspends a BITS transfer job. |

@joeyaiello - this is also linked to the about_* topics not being installed which I've tested and confirmed that this also occurs on Server 2016 as well even after Update-help has been run

I can confirm that if you run Update-Help now on Server 2016 it will install all the about_* topics as expected.


New workflows width inconsistent

If you create a new workflow the first time you switch to it the widths can be wrong

For me, this seems to be fixed here: https://github.com/chrisgoringe/cg-controller/issues/199
When I notice is this: https://github.com/chrisgoringe/cg-controller/issues/201#issuecomment-2435211388


docs: Change broken coc url

close #749
Check List (Check all the applicable boxes)

[x] My code follows the code style of this project.
[] My change requires changes to the documentation.
[x] I have updated the documentation accordingly.
[x] All new and existing tests passed.
[x] This PR does not contain plagiarized content.
[x] The title of my pull request is a short description of the requested changes.

You can also join our Discord community.
Feel free to check out other cool repositories of the Thecyberworld.
Join the Thecyberworld GitHub Organisation by raising an issue (you will be sent an invitation).

@Tamir198 could you change in all the templates as well?
When we create a new template there also we have this redirection to coc and it doesn't work there as well.

Ye sure i will add it into the pr

@ArkadiK94  Changed, if I missed some templates tell me and I will add the changes as well?


Feat(menu): Add Menu component to Carbon web components

Related Ticket(s)
Closes #11543
Description
Add menu component to Carbon web components}
Changelog
New

Added Menu component files and implemented functionality and styling

Closing as CWC is no longer part of this repo


Hello, Messenger

Create the baseline project for Bujobot. Should:

[x] Greet user
[x] Save user in database
[x] Echo message back to user

Facebook is temporarily not allowing new applications on the Messenger platform, not sure for how long. Switching to Twilio to implement conversations.


Default Implicits from cats.effect.IOApp cause BlazeServer to crash when a JVM error occurs

When the default ContextShift and Timer from cats.effect.IOApp is passed to BlazeServerBuilder, if there's a java.lang.Error raised during the processing of one of the HTTP Requests, the error bubbles up to the main thread running the server and kills the app.
This behaviour doesn't seem to happen if the user defines their own ContextShift and Timer
Example
Suppose there's a HTTP Service that is throwing a StackOverflowError for some input data
trait HelloWorld {
  def hello(n: HelloWorld.Name): IO[HelloWorld.Greeting]
}

def impl: HelloWorld = new HelloWorld {
    def hello(n: HelloWorld.Name): IO[HelloWorld.Greeting] = {
      simulateError
    }

    def simulateError: IO[HelloWorld.Greeting] =
      IO.fromEither {
        Either.catchNonFatal {
          throw new StackOverflowError("BOOM BANG!!")
        }
      }
  }

If the user makes a request to the endpoint bound to this implementation:
curl http://localhost:8080/hello/john

Then the Error bubbles up to the main thread and kills the app. However, if we override the default implicits in the app, then this curl request just times out and the app is still running which would be the desired behaviour.
  import cats.effect.{ContextShift, Timer}
  override implicit val contextShift: ContextShift[IO] = IO.contextShift(ExecutionContext.global)
  override implicit val timer: Timer[IO] = IO.timer(ExecutionContext.global)

I have the sample application available here which simulates this error.

On further assessment, this is happening due to how the Pool is defined in cats-effect 2. It is designed to call System.exit(1) if there is a NonFatal error. Is this a sensible default for http4s apps?
https://github.com/typelevel/cats-effect/blob/series/2.x/core/jvm/src/main/scala/cats/effect/internals/PoolUtils.scala#L47

Fatal errors, in general, can leave the JVM in an undefined state. StackOverflowError is probably a bad example because it is far and away the most benign, but many of the others (such as OOM) should not be caught except under very specific circumstances.
Story time: I once had a bug that only occurred in production, and only after prod had been live for a few days. What we were seeing ultimately narrowed down to floating point arithmetic producing the wrong result. It was almost as simple as 2 + 2 and seeing an answer of 5. No objects, nothing fancy, just primitives that were somehow wrong. And again, this only happened after prod had been live for a while.
After an absolutely heroic minimization effort, one of my coworkers finally noticed that, GBs further up in the logs, a StackOverflowError had been printed, seemingly benignly. It didn't really make any sense, particularly since the stack trace didn't actually look like a loop, but it was the first real clue. Digging into it further, we realized the SOE was actually a consequence of something much more subtle: Future at the time was catching all Throwable, not just NonFatal, and it would attempt to wrap those Throwables just like any other. The real problem was that the application was running out of memory, and thus an OutOfMemoryError was being raised within Future, which promptly caught it and tried to allocate a failed Future to carry it, but that allocation failed, which tripped a second OutOfMemoryError (still within Future), etc etc.
When all the dust settled, the OOMs had been completely eaten, and the error printed was a StackOverflowError for reasons I can't quite remember, but by that point, the JVM was still running and the only external consequence was that an HTTP request had timed out, but internally it was in deep trouble. Digging into the JVM spec on this type of situation, what was happening was ultimately that the JVM itself (I think specifically the GC) was in a completely undefined state, with its internal data structures no longer obeying invariants. This in turn was enough to generate the seemingly-impossible and unrelated consequences later on down the line in that same instance, such as floating point arithmetic behaving impossibly.
Moral of the story: be very, very, very careful around catching Throwable. It's fine for some "fatal" errors, like InterruptedException and StackOverflowError (though with the former, you should be sure to handle it appropriately), but in general it's not something you should be doing.
CE3 improves on this situation from CE2 btw. The runtime will still exit on fatal error, but it will propagate the exception back to the unsafeRun call site without allocating or relying on safe points (generally within IOApp), which then prints the exception, removes the application finalizers, and calls sys.exit(-1). So it still takes down the app, but at least the errors are in the right place and will deterministically print.
Ultimately, fatal errors are a bit like segfaults: you simply cannot suppress them in general. My best advice, if you're honestly expecting SOEs from some effect in normal operation, would be to stick a manual try/catch around the root of the SOE before it gets wrapped in IO and suppress the exception manually.

Thanks a lot for sharing that experience Daniel! Very interesting. I definitely agree with you that we shouldn't catch such errors. I'll share what we saw in production which also relates to Fatal errors which we suspect was a StackOverFlow.
So for our application, we have a graceful shutdown mechanism built in that gets called as IO.guarantee { } when we yield the Exit code in the run method of our IOApp. What that guarantee method does is that it waits for all IOs to complete and before shutting down. We use a Ref to keep track of the ongoing IOs that process a HTTP request by incrementing the Ref before executing the IO and subsequently decrementing the Ref once the IO is complete.
What we saw in Production was that there was some Fatal error thrown which we sadly didn't see in the logs since we hadn't configured System error correctly. Since CE 2 catches Fatal errors and calls System.exit(1), it ended up calling that guarantee block (maybe because internally its registered as a shutdown hook?). Since the Ref never got decremented for that IO which had the Fatal error, the guarantee block ended up looping infinitely.
So we investigated if using .guaranteeCase would be a better alternative. In the case of a Fatal error, it matches to the case ExitCase.Canceled instead of case ExitCase.Error(e). And if the app is terminated by sending the SIGTERM signal, it again calls case ExitCase.Canceled instead of case ExitCase.Completed which in effect would mean that case ExitCase.Completed would never be called.
In the end, we decided to override the ExecutionContext of CE 2 and made a 1 line change. Instead of calling System.exit(1) which was triggering the .guarantee block in our codebase, we opted to replace it with Runtime.getRuntime.halt(1) which just terminates the JVM instead of trying to call the shutdown hooks.
Is there a reason as to why CE 2 uses System.exit(1) instead of Runtime.getRuntime.halt(1) when it encounters Fatal errors?

This discussion would be a better fit for the cats-effect issue tracker, since it's not http4s-specific. Curious to hear @djspiewak's take on exit vs halt though!

Continued in https://github.com/typelevel/cats-effect/issues/2586.


If there is no change, don't exit with an error code

Currently when the tool runs, if there is no version change detected, it exits with error code 1. This causes our CI process to record these builds as failed, but in reality they were just one of the commit codes that don't cause version changes (docs, style, etc).
I'm not sure if there is a use-case for this being considered an error.

The use case is: if the version didn't change, abort the deployment. Usually in CI you can do something like this:
semantic-release || true
This will return a zero exit code

The problem with that is it overrides any errors that might happen with the semantic release process itself. Do different reasons exit with different values that can be tested for?

TL;DR; I only run semver if certain files have changed.
I've come around on the exiting with an error code with no change to the version number. It has come in really handy when building docker images in Ci.  I use this to build docker images in Jenkins, and having an error code when there is no change is extremely helpful.
I had to be more deliberate about when I run semver though, and I use changeset to do that
        stage('get version num') {
            when {
                branch 'master'
                changeset 'images/**'
            }
            steps {
                sh 'semantic-release -vf -noci -slug arcgis/image-odjenkins -changelog CHANGELOG.md'
                script {
                    env.VERSION_NUMBER = readFile('.version').trim()
                }
            }
            post {
                success {
                    archiveArtifacts 'CHANGELOG.md'
                }
            }
}

Additionally, I noticed a .ghr file is not being created when there is no change.


Update Python versions page

Contributions

[X] I have read the contribution docs, and understand what's expected of me.

Link to the page on docs.getdbt.com requiring updates
https://docs.getdbt.com/faqs/Core/install-python-compatibility
What part(s) of the page would you like to see updated?
Compatibility matrix:

dbt-core →  Python ↓
1.0
1.1
1.2
1.3
1.4
1.5
1.6
1.7
1.8

3.8
✓
✓
✓
✓
✓
✓
✓
✓
✓

3.9
✓
✓
✓
✓
✓
✓
✓
✓
✓

3.10
✓
✓
✓
✓
✓
✓
✓
✓
✓

3.11
✖
✖
✖
✖
✓
✓
✓
✓
✓

3.12
✖
✖
✖
✖
✖
✖
✖
✓
✓

Additional information
No response

I'm happy to create the PR for this, would that be easier?
cc @runleonarun @matthewshaver

This table is so helpful! If you can open a PR would be great. I can get to it tomorrow!


Properties column predicate

This is an attempt to have some control over the properties being stored in the Properties column. I've added a filter to the PropertiesColumnOptions class that makes the sink only store the properties allowed by the filter.

Thanks for the PR!
Just to get a better understanding of this, what's the scenario driving it for you? Thanks!

Thanks for having a look. We use Serilog to collect heavily enriched log information with several targets. We  would like to be able to only store a subset of those properties in a SQL table, with the same LogEvent being dispatched to other targets 'in full'.
Creating a DataColumn per property would result in a sparse data table, whereas I would ideally like to avoid creating a separate LogEvent for this target to keep the business logic layer ignorant of the logging targets.
As this is just one of my initial ideas, any feedback is welcome.

Thanks!

Thank you!


KeyError: None when converting from Caffe to Tensorflow: maybe related to PReLU?

Platform (like ubuntu 16.04/win10): Ubuntu 18.04
Python version: 3.6.5
Source framework with version (like Tensorflow 1.4.1 with GPU): Caffe 1.0.0 CPU
Destination framework with version (like CNTK 2.3 with GPU): TensorFlow 1.8.0 CPU
Pre-trained model path (webpath or webdisk path): https://github.com/danaitri/Face-detection-cnn . Any of the two .prototxt file and the .caffemodel
Running scripts: mmconvert -sf caffe -in F8_deploy.prototxt -iw G3XT_B.caffemodel -df tensorflow -om ./
Error:
Traceback (most recent call last):
File "/home/robin/Documents/virtualenvfolder/bin/mmconvert", line 11, in
sys.exit(_main())
File "/home/robin/Documents/virtualenvfolder/lib/python3.6/site-packages/mmdnn/conversion/_script/convert.py", line 95, in _main
ret = convertToIR._convert(ir_args)
File "/home/robin/Documents/virtualenvfolder/lib/python3.6/site-packages/mmdnn/conversion/_script/convertToIR.py", line 9, in _convert
transformer = CaffeTransformer(args.network, args.weights, "tensorflow", args.inputShape, phase = args.caffePhase)
File "/home/robin/Documents/virtualenvfolder/lib/python3.6/site-packages/mmdnn/conversion/caffe/transformer.py", line 316, in init
graph = GraphBuilder(def_path, self.input_shape, self.is_train_proto, phase).build()
File "/home/robin/Documents/virtualenvfolder/lib/python3.6/site-packages/mmdnn/conversion/caffe/graph.py", line 447, in build
graph.compute_output_shapes(self.model)
File "/home/robin/Documents/virtualenvfolder/lib/python3.6/site-packages/mmdnn/conversion/caffe/graph.py", line 266, in compute_output_shapes
node.output_shape = TensorShape(*NodeKind.compute_output_shape(node))
File "/home/robin/Documents/virtualenvfolder/lib/python3.6/site-packages/mmdnn/conversion/caffe/graph.py", line 126, in compute_output_shape
return LAYER_DESCRIPTORSnode.kind
KeyError: None
I saw on related errors that this might be due to the presence of PReLU in the .prototxt. Is PReLU supported from Caffe to TensorFlow or not?
Thank you very much!

The PReLU is sooning add into MMdnn. I am creating a pull request, hoping to solve your problem.

The converted code of tensorflow
def KitModel(weight_file = None):
    global __weights_dict
    __weights_dict = load_weights(weight_file)

    data            = tf.placeholder(tf.float32, shape = (None, 160, 120, 3), name = 'data')
    conv1           = convolution(data, group=1, strides=[1, 1], padding='VALID', name='conv1')
    prelu1          = prelu(conv1, name='prelu1')
    conv2_pad       = tf.pad(prelu1, paddings = [[0L, 0L], [0L, 1L], [0L, 1L], [0L, 0L]])
    conv2           = convolution(conv2_pad, group=1, strides=[2, 2], padding='VALID', name='conv2')
    prelu2          = prelu(conv2, name='prelu2')
    conv3           = convolution(prelu2, group=1, strides=[1, 1], padding='VALID', name='conv3')
    prelu3          = prelu(conv3, name='prelu3')
    conv4n          = convolution(prelu3, group=1, strides=[1, 1], padding='VALID', name='conv4n')
    prelu4n         = prelu(conv4n, name='prelu4n')
    conv5n          = convolution(prelu4n, group=1, strides=[1, 1], padding='VALID', name='conv5n')
    prelu5n         = prelu(conv5n, name='prelu5n')
    conv6n          = convolution(prelu5n, group=1, strides=[1, 1], padding='VALID', name='conv6n')
    prelu6n         = prelu(conv6n, name='prelu6n')
    conv7           = convolution(prelu6n, group=1, strides=[1, 1], padding='VALID', name='conv7')
    prob            = tf.nn.softmax(conv7, name = 'prob')
    fpool_pad       = tf.pad(prob, paddings = [[0L, 0L], [1L, 1L], [1L, 1L], [0L, 0L]])
    fpool           = tf.nn.avg_pool(fpool_pad, [1, 3, 3, 1], [1, 1, 1, 1], padding='VALID', name='fpool')
    return data, fpool

Thanks ! Will try it out once it has been merged!

Hi @RobinDtx merged. Please check it. Thanks.

@kitstar @JiahaoYao @RobinDtx
This post is redundant of what is posted at https://github.com/Microsoft/MMdnn/issues/57
Adding this here since the ticket is still open.
I am having the same error which could maybe for different reason.
Here is the prototext file I am trying to convert.
https://github.com/hszhao/PSPNet/blob/master/evaluation/prototxt/pspnet101_cityscapes_713.prototxt
Here is the command I used to convert Caffemodel to IR
mmtoir -f caffe -n path-to-prototxt -w path-to-caffemodel -o caffe_resnet_IR
Here is my stack trace.
Traceback (most recent call last):
File "/data/bdhanani/Experiments_04/Notebooks/cf_to_tf/mmdnn/bin/mmconvert", line 11, in
sys.exit(_main())
File "/data/bdhanani/Experiments_04/Notebooks/cf_to_tf/mmdnn/lib/python2.7/site-packages/mmdnn/conversion/_script/convert.py", line 95, in _main
ret = convertToIR._convert(ir_args)
File "/data/bdhanani/Experiments_04/Notebooks/cf_to_tf/mmdnn/lib/python2.7/site-packages/mmdnn/conversion/_script/convertToIR.py", line 9, in _convert
transformer = CaffeTransformer(args.network, args.weights, "tensorflow", args.inputShape, phase = args.caffePhase)
File "/data/bdhanani/Experiments_04/Notebooks/cf_to_tf/mmdnn/lib/python2.7/site-packages/mmdnn/conversion/caffe/transformer.py", line 318, in init
graph = GraphBuilder(def_path, self.input_shape, self.is_train_proto, phase).build()
File "/data/bdhanani/Experiments_04/Notebooks/cf_to_tf/mmdnn/lib/python2.7/site-packages/mmdnn/conversion/caffe/graph.py", line 450, in build
graph.compute_output_shapes(self.model)
File "/data/bdhanani/Experiments_04/Notebooks/cf_to_tf/mmdnn/lib/python2.7/site-packages/mmdnn/conversion/caffe/graph.py", line 269, in compute_output_shapes
node.output_shape = TensorShape(*NodeKind.compute_output_shape(node))
File "/data/bdhanani/Experiments_04/Notebooks/cf_to_tf/mmdnn/lib/python2.7/site-packages/mmdnn/conversion/caffe/graph.py", line 129, in compute_output_shape
return LAYER_DESCRIPTORSnode.kind
KeyError: None
Regarding the error, my guess is on these two things
The prototxt file has a layer name called "BN" which is nothing but a "BatchNorm" layer. If I parse the file with the layer name "BatchNorm", I get this error:
check failed: param_size <= num_param_blobs (4 vs. 3) Too many params specified for layer conv1_1_3x3_s2/bn
To tackle this error, I changed the layer definition and reduced batch norm parameter from 4 to 3 which afterwards lead me to this error:
Check failed: target_blobs.size() == source_layer.blobs_size() (3 vs. 4) Incompatible number of blobs for layer conv1_1_3x3_s2/bn
Which makes sense, since my weights has 4 blobs for the batch norm layer: scale, offset, mean and variance.
The prototxt file has a layer name called "Interp" which is custom layer implemented in the Caffe.
I am not sure which of these things from above causing this issue.
I have a small suggestions.
Can the error be more informative than just "Keyerror":None ? Like, which layer is problematic or the root cause of the error.
Please let me know if anyone had the same issue or my approach is wrong.
Thanks,
Bee


Remap Alt(Right) to Backspace issue

Microsoft PowerToys version
0.62.1
Running as admin

[ ] Yes

Area(s) with issue?
Keyboard Manager
Steps to reproduce
Remap the Alt (Right) key to the Backspace key.
✔ Expected Behavior
Pressing the Alt (Right) key once would be equivalent to pressing the Backspace key once.
❌ Actual Behavior
Instead of deleting one character at a time, pressing the Alt (Right) key deletes one word at a time. This happens everywhere, e.g. a browser or Word.
Other Software
No response

I suspect that would be an AltGr key, or at least it behaves like so. AltGr = (L)Ctrl+(R)Alt. Left Alt should be good.

That is correct, it's AltGr.


customize keys

Have you considered adding support for overriding keymap bindings?

Hello, thank you for your interest in my little plugin. As far as I know this functionality was enabled with #3.
For example you can simply add the following to your neovim configuration
vim.keymap.set('n', '<leader>l', ":lua require('sqlrun').execute_current_line()<CR>", {noremap = true})
vim.keymap.set('v', '<leader>q', ":lua require('sqlrun').execute_selection()<CR>", {noremap = true})

and replace <leader>l and <leader>l with your favorite key bindings.
I hope that helps.


Add splash screen

Add splash screen, use test logo and beautiful animation. Show your creativity.
Post your doubt here:Discord server

Please assign this task to me.

I will assign. Do follow the project file pattern.

@Wizpna  Follow the instructions. If you have any doubts Post on discord server

I have made a pull request regarding this, please review and merge

@Wizpna I have added comment on pull request.


ResolverContract instantiation failure when using defaultAbi

Hi,
ResolverContract is currently receiving a resolverAbi  param that must be string.
Web3.eth.Contract expects an Object as constructor parameter, also, since the ABI generaly is into json format (truffle and other frameworks), probably it would be better to just expect a json object.
By the way, if you dont specify the resolverAbi to the resolver, it will use defaultAbi. The default one is a json file and it fails when trying to convert it again to an object (cannot parse an object).
class ResolverContract {

/**
     * Constructor.
     * @param {string} nodeProvider A web3 provider to use to communicate with the blockchain. Defaults to the public RSK node.
	 * @param {string} resolverAddress The address of the RNS Resolver. Defaults to the public RNS Resolver.
	 * @param {string} resolverAbi The abi of the RNS Resolver. Defaults to the public RNS Resolver's abi.
	 * @returns {ResolverContract}
     */

	constructor(nodeProvider = defaultNode, resolverAddress = defaultResolverAddress, resolverAbi = defaultAbi){
		this.web3 = new Web3(nodeProvider);
		this.address = resolverAddress;
		this.abi = resolverAbi;
                // the JSON.parse fails when using defaultAbi since it is a json object already
		this.resolverContract = new this.web3.eth.Contract(JSON.parse(this.abi), this.address);
	}
}

Hello,
Please consider to push a hotfix or minor release to npm, to include this contribution in the SDK Library.
Another topic is it posible that the defaultResolverAddress is out of date? After a reset of the public testnet?
const defaultResolverAddress = "0x59313581e7d735793962e16b8d0b10636bf53ae7"

According to https://docs.rns.rifos.org/RNS-Testnet/ the testnet resolver contract is 0x480fee7506db34dde84cfec85057a6bd96b38e1b
If that's the case I could do a small PR to correct this.
Thank you very much!

Hi @sebastinez! PRs are always welcomed! You are right, the testnet address is wrong.


What is the fastest way to resolve a stream url for YT Music?

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

[X] I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

[X] I'm asking a question and not reporting a bug or requesting a feature
[X] I've looked through the README
[X] I've verified that I have updated yt-dlp to nightly or master (update instructions)
[X] I've searched known issues and the bugtracker for similar questions including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue

Please make sure the question is worded well enough to be understood
Hello people 👋  , Music Assistant dev here.
We use your amazing product over at Music Assistant to integrate playback for YT Music. Now, Google has been making our life harder and harder to properly support YT Music by first dropping OAuth support. We implementing cookie authentication as a workaround, but now Google has also broken the ios client in combination with providing cookies.
We were previously using the following config that resulted in stream urls being resolved super fast (< 2 sec):
ydl_opts = {
      "cookiefile": StringIO(self._netscape_cookie),
      # This enforces a player client and skips unnecessary scraping to increase speed
      "extractor_args": {
          "youtube": {"skip": ["translated_subs", "dash"], "player_client": ["ios"]}
      },
  }

Now, since this is broken, we are using the default player client:
ydl_opts = {
    "cookiefile": StringIO(self._netscape_cookie),
    # This enforces a player client and skips unnecessary scraping to increase speed
    "extractor_args": {
        "youtube": {
            "skip": ["translated_subs", "dash"],
            "player_client": ["web_music"],
            "player_skip": ["webpage"],
        }
    },
}

Unfortunately, this can sometimes take around 10 secs to resolve a stream url. As you can imagine, this does not provide a nice user experience when a user clicks a song in the front end. The reason we use authentication is twofold:

We want to grab the HQ 141 streams to ensure the highest quality playback
We want to verify that a user has YT Premium. As far as I know, only subscribed users can obtain a 141 stream.

Question: Is there any way to speed up resolving YT Music stream urls after the new restrictions that have been put on us?
Again, thanks a lot for the amazing work here and I hope we can find a way to keep the music alive 💪 .
Cheers,
Marvin
Provide verbose output that clearly demonstrates the problem

[ ] Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
[ ] If using API, add 'verbose': True to YoutubeDL params instead
[ ] Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output
No response

There isn't any faster way than what you are currently using AFAIK.

Thanks for your response! Too bad, I guess it's a restriction the users will have to live with.