bug: Failed to use Flan-T5 model

Describe the bug
When trying to use Flan-T5 model I keep getting the following error:
ValueError: You have to specify either decoder_input_ids or decoder_inputs_embeds

I follow the instructions from your README
To reproduce
In one terminal
virtualenv venv -p python3.9
source venv/bin/activate
pip install "openllm[flan-t5]"
openllm start google/flan-t5-large

In another terminal
source venv/bin/activate
export OPENLLM_ENDPOINT=http://localhost:3000
openllm query 'What are large language models?'

Logs
vLLM is not available. Note that PyTorch backend is not as performant as vLLM and you should always consider using vLLM for production.
To disable these warnings, set 'OPENLLM_DISABLE_WARNING=True'
🚀Tip: run 'openllm build google/flan-t5-large --serialization safetensors' to create a BentoLLM for 'google/flan-t5-large'
2023-11-14T15:54:24+0100 [INFO] [cli] Prometheus metrics for HTTP BentoServer from "_service:svc" can be accessed at http://localhost:3000/metrics.
2023-11-14T15:54:25+0100 [INFO] [cli] Starting production HTTP BentoServer from "_service:svc" listening on http://<IP_ADDRESS>:3000 (Press CTRL+C to quit)
2023-11-14T15:54:53+0100 [INFO] [api_server:llm-flan-t5-service:14] <IP_ADDRESS>:41378 (scheme=http,method=POST,path=/v1/metadata,type=application/json,length=2) (status=200,type=application/json,length=890) 1.641ms (trace=d7ec3a0e3bbe24d191ab86aa1ac6ba33,span=d98a00bc5bba1b98,sampled=1,service.name=llm-flan-t5-service)
2023-11-14T15:54:54+0100 [ERROR] [runner:llm-flan-t5-runner:1] Exception in ASGI application
Traceback (most recent call last):
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/uvicorn/protocols/http/h11_impl.py", line 408, in run_asgi
result = await app(  # type: ignore[func-returns-value]
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/uvicorn/middleware/proxy_headers.py", line 84, in __call__
return await self.app(scope, receive, send)
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/starlette/applications.py", line 116, in __call__
await self.middleware_stack(scope, receive, send)
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/starlette/middleware/errors.py", line 186, in __call__
raise exc
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/starlette/middleware/errors.py", line 164, in __call__
await self.app(scope, receive, _send)
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/bentoml/_internal/server/http/traffic.py", line 26, in __call__
await self.app(scope, receive, send)
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/opentelemetry/instrumentation/asgi/__init__.py", line 596, in __call__
await self.app(scope, otel_receive, otel_send)
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/bentoml/_internal/server/http/instruments.py", line 252, in __call__
await self.app(scope, receive, wrapped_send)
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/bentoml/_internal/server/http/access.py", line 126, in __call__
await self.app(scope, receive, wrapped_send)
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/starlette/_exception_handler.py", line 55, in wrapped_app
raise exc
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/starlette/_exception_handler.py", line 44, in wrapped_app
await app(scope, receive, sender)
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/starlette/routing.py", line 746, in __call__
await route.handle(scope, receive, send)
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/starlette/routing.py", line 288, in handle
await self.app(scope, receive, send)
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/starlette/routing.py", line 75, in app
await wrap_app_handling_exceptions(app, request)(scope, receive, send)
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/starlette/_exception_handler.py", line 55, in wrapped_app
raise exc
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/starlette/_exception_handler.py", line 44, in wrapped_app
await app(scope, receive, sender)
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/starlette/routing.py", line 73, in app
await response(scope, receive, send)
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/starlette/responses.py", line 259, in __call__
await wrap(partial(self.listen_for_disconnect, receive))
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 597, in __aexit__
raise exceptions[0]
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/starlette/responses.py", line 255, in wrap
await func()
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/starlette/responses.py", line 244, in stream_response
async for chunk in self.body_iterator:
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/bentoml/_internal/server/runner_app.py", line 356, in streamer
async for p in payload:
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/bentoml/_internal/server/runner_app.py", line 214, in inner
async for data in ret:
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/openllm/_runners.py", line 118, in generate_iterator
async for generation_output in self.forward(prompt_token_ids, request_id, stop=stop, **attrs):
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/openllm/_runners.py", line 148, in forward
out = self.model(torch.as_tensor([prompt_token_ids], device=self.model.device), use_cache=True)
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/transformers/models/t5/modeling_t5.py", line 1746, in forward
decoder_outputs = self.decoder(
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/transformers/models/t5/modeling_t5.py", line 1011, in forward
raise ValueError(f"You have to specify either {err_msg_prefix}input_ids or {err_msg_prefix}inputs_embeds")
ValueError: You have to specify either decoder_input_ids or decoder_inputs_embeds
2023-11-14T15:54:54+0100 [ERROR] [api_server:llm-flan-t5-service:14] Exception in ASGI application
Traceback (most recent call last):
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/uvicorn/protocols/http/h11_impl.py", line 408, in run_asgi
result = await app(  # type: ignore[func-returns-value]
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/uvicorn/middleware/proxy_headers.py", line 84, in __call__
return await self.app(scope, receive, send)
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/starlette/applications.py", line 116, in __call__
await self.middleware_stack(scope, receive, send)
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/starlette/middleware/errors.py", line 186, in __call__
raise exc
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/starlette/middleware/errors.py", line 164, in __call__
await self.app(scope, receive, _send)
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/bentoml/_internal/server/http/traffic.py", line 26, in __call__
await self.app(scope, receive, send)
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/bentoml/_internal/server/http/instruments.py", line 135, in __call__
await self.app(scope, receive, wrapped_send)
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/opentelemetry/instrumentation/asgi/__init__.py", line 596, in __call__
await self.app(scope, otel_receive, otel_send)
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/bentoml/_internal/server/http/access.py", line 126, in __call__
await self.app(scope, receive, wrapped_send)
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/starlette/_exception_handler.py", line 55, in wrapped_app
raise exc
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/starlette/_exception_handler.py", line 44, in wrapped_app
await app(scope, receive, sender)
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/starlette/routing.py", line 746, in __call__
await route.handle(scope, receive, send)
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/starlette/routing.py", line 288, in handle
await self.app(scope, receive, send)
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/starlette/routing.py", line 75, in app
await wrap_app_handling_exceptions(app, request)(scope, receive, send)
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/starlette/_exception_handler.py", line 55, in wrapped_app
raise exc
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/starlette/_exception_handler.py", line 44, in wrapped_app
await app(scope, receive, sender)
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/starlette/routing.py", line 73, in app
await response(scope, receive, send)
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/starlette/responses.py", line 259, in __call__
await wrap(partial(self.listen_for_disconnect, receive))
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 597, in __aexit__
raise exceptions[0]
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/starlette/responses.py", line 255, in wrap
await func()
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/starlette/responses.py", line 244, in stream_response
async for chunk in self.body_iterator:
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/openllm/_service.py", line 46, in generate_stream_v1
async for it in llm.generate_iterator(**llm_model_class(**input_dict).model_dump()):
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/openllm/_llm.py", line 554, in generate_iterator
async for out in self.runner.generate_iterator.async_stream(
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/bentoml/_internal/runner/runner_handle/remote.py", line 330, in async_stream_method
async for b, end_of_http_chunk in resp.content.iter_chunks():
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/aiohttp/streams.py", line 51, in __anext__
rv = await self._stream.readchunk()
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/aiohttp/streams.py", line 433, in readchunk
await self._wait("readchunk")
File "/home/m.gawinecki/tmp/venv/lib/python3.9/site-packages/aiohttp/streams.py", line 304, in _wait
await waiter
aiohttp.client_exceptions.ClientPayloadError: Response payload is not completed

Environment
Environment variable
BENTOML_DEBUG=''
BENTOML_QUIET=''
BENTOML_BUNDLE_LOCAL_BUILD=''
BENTOML_DO_NOT_TRACK=''
BENTOML_CONFIG=''
BENTOML_CONFIG_OPTIONS=''
BENTOML_PORT=''
BENTOML_HOST=''
BENTOML_API_WORKERS=''

System information
bentoml: 1.1.9
python: 3.9.16
platform: Linux-5.4.0-166-generic-x86_64-with-glibc2.31
uid_gid:<PHONE_NUMBER>:1354815571
pip_packages

accelerate==0.24.1
aiohttp==3.8.6
aiosignal==1.3.1
annotated-types==0.6.0
anyio==3.7.1
appdirs==1.4.4
asgiref==3.7.2
async-timeout==4.0.3
attrs==23.1.0
bentoml==1.1.9
bitsandbytes==0.41.2.post2
black==23.11.0
build==0.10.0
cattrs==23.1.2
certifi==2023.7.22
charset-normalizer==3.3.2
circus==0.18.0
click==8.1.7
click-option-group==0.5.6
cloudpickle==3.0.0
coloredlogs==15.0.1
contextlib2==21.6.0
cuda-python==12.3.0
dataclasses-json==0.6.2
datasets==2.14.6
deepmerge==1.1.0
Deprecated==1.2.14
dill==0.3.7
distlib==0.3.7
distro==1.8.0
exceptiongroup==1.1.3
fairscale==0.4.13
fastcore==1.5.29
filelock==3.13.1
frozenlist==1.4.0
fs==2.4.16
fsspec==2023.10.0
ghapi==1.0.4
greenlet==3.0.1
h11==0.14.0
httpcore==1.0.2
httpx==0.25.1
huggingface-hub==0.19.1
humanfriendly==10.0
idna==3.4
importlib-metadata==6.8.0
inflection==0.5.1
Jinja2==3.1.2
jsonpatch==1.33
jsonpointer==2.4
langchain==0.0.335
langsmith==0.0.64
markdown-it-py==3.0.0
MarkupSafe==2.1.3
marshmallow==3.20.1
mdurl==0.1.2
mpmath==1.3.0
multidict==6.0.4
multiprocess==0.70.15
mypy-extensions==1.0.0
networkx==3.2.1
numpy==1.26.2
nvidia-cublas-cu12==<IP_ADDRESS>
nvidia-cuda-cupti-cu12==12.1.105
nvidia-cuda-nvrtc-cu12==12.1.105
nvidia-cuda-runtime-cu12==12.1.105
nvidia-cudnn-cu12==<IP_ADDRESS>
nvidia-cufft-cu12==<IP_ADDRESS>
nvidia-curand-cu12==<IP_ADDRESS>
nvidia-cusolver-cu12==<IP_ADDRESS>
nvidia-cusparse-cu12==<IP_ADDRESS>
nvidia-ml-py==11.525.150
nvidia-nccl-cu12==2.18.1
nvidia-nvjitlink-cu12==12.3.52
nvidia-nvtx-cu12==12.1.105
openllm==0.4.6
openllm-client==0.4.6
openllm-core==0.4.6
opentelemetry-api==1.20.0
opentelemetry-instrumentation==0.41b0
opentelemetry-instrumentation-aiohttp-client==0.41b0
opentelemetry-instrumentation-asgi==0.41b0
opentelemetry-sdk==1.20.0
opentelemetry-semantic-conventions==0.41b0
opentelemetry-util-http==0.41b0
optimum==1.14.0
orjson==3.9.10
packaging==23.2
pandas==2.1.3
pathspec==0.11.2
pip-requirements-parser==32.0.1
pip-tools==7.3.0
platformdirs==3.11.0
prometheus-client==0.18.0
psutil==5.9.6
pyarrow==14.0.1
pydantic==2.5.0
pydantic-core==2.14.1
Pygments==2.16.1
pyparsing==3.1.1
pyproject-hooks==1.0.0
python-dateutil==2.8.2
python-json-logger==2.0.7
python-multipart==0.0.6
pytz==2023.3.post1
PyYAML==6.0.1
pyzmq==25.1.1
regex==2023.10.3
requests==2.31.0
rich==13.6.0
safetensors==0.4.0
schema==0.7.5
scipy==1.11.3
sentencepiece==0.1.99
simple-di==0.1.5
six==1.16.0
sniffio==1.3.0
SQLAlchemy==2.0.23
starlette==0.32.0.post1
sympy==1.12
tenacity==8.2.3
tokenizers==0.14.1
tomli==2.0.1
torch==2.1.0
tornado==6.3.3
tqdm==4.66.1
transformers==4.35.0
triton==2.1.0
typing-extensions==4.8.0
typing-inspect==0.9.0
tzdata==2023.3
urllib3==2.1.0
uvicorn==0.24.0.post1
virtualenv==20.24.6
watchfiles==0.21.0
wrapt==1.16.0
xxhash==3.4.1
yarl==1.9.2
zipp==3.17.0

Copy-and-paste the text below in your GitHub issue and FILL OUT the two last points.

transformers version: 4.35.0
Platform: Linux-5.4.0-166-generic-x86_64-with-glibc2.31
Python version: 3.9.16
Huggingface_hub version: 0.19.1
Safetensors version: 0.4.0
Accelerate version: 0.24.1
Accelerate config:    not found
PyTorch version (GPU?): 2.1.0+cu121 (True)
Tensorflow version (GPU?): not installed (NA)
Flax version (CPU?/GPU?/TPU?): not installed (NA)
Jax version: not installed
JaxLib version: not installed
Using GPU in script?: Don't know. Just following the steps from README
Using distributed or parallel set-up in script?: Don't know. Just following the steps from README

System information (Optional)
No response

I have now updated the main branch to bring back support for encoder-decoder models.
Will include this in 0.4.15


AMBARI-23867 : Remove dependency on org.gridgain:ignite-shmem 1.0.0 in Ambari Metrics Collector

What changes were proposed in this pull request?
Remove dependency that cause CVE.
How was this patch tested?
mvn clean install. Manually tested by removing the JAR on a deployed AMS instance and restart Metrics collector.

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/Ambari-Github-PullRequest-Builder/2314/
Test PASSed.


[RFC] Verification key permissions

This is an excellent write-up @nholland94!

!ci-build-me

!ci-build-me

!approved-for-mainnet

!ci-build-me


Solve issue #850

Support sub keys for global.jwt entry
Added 2 examples in yaml files, oidc-trust and uaa

@Tallicia @peterhaochen47 any concerns ? If no I will merge this


connect timeout clarification

The "connect timeout" setting mentions that, if it's set to 0, the value from sp_configure will be used.  But there is no "connect timeout" setting in sp_configure.
Based on practical testing (which I mentioned on DBA Stack Exchange here), I think this really refers to the "remote login timeout" setting.

@markingmyname
IMPORTANT: When the changes are ready for publication, add a #sign-off comment to signal that the PR is ready for the review team to merge.
#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team

Hey Josh, this looks good. We'll get this merged once a PM confirms.

Thanks, Randolph!

Sorry for the delay. Feedback from the product group was to add some more language around it, so I'll pull this into an internal PR and you'll see an update in the next day or so. When the internal PR is merged, this PR will automatically close, and you'll still get credit.


First (rough) stab at oidc4vc comparison

help very much needed.

I was listening to this SSI Orbit podcast and I took a note on the following:

It was developed as a specialized protocol for credential issuance and credential presentation use cases, and with a side use case for pseudonymous user authentication.
It can also work with DIDComm to bootstrap other protocol workflows.

screenshot from Karim presentation at Aires WG call:

LGTM


[TODO]: Rewrite using ES6 and build with babel and npm

This need to be done deadly before add new feature 👊


I am trying to convert an audio file of about 28 minutes in .wav format into text using the Speech-to-text API. I am getting the following error:

Azure documentation issue guidance
Thanks for opening an issue in the Azure technical documentation repository.
We use GitHub issues as the primary channel for customer and community feedback about the Azure documentation.
Creating an issue
We prefer that you create documentation feedback issues using the Feedback link on the published article - the feedback control on the doc page creates an issue that contains all the article details so you can focus on the feedback part.
You can also create a feedback issue here in the repo. If you do this, please make sure your issue lists:

[ ] The relevant Azure service or technology.
[ ] A link to the published documentation article that you have feedback about.
[ ] Clear, specific feedback that the author can act on.

Pull requests and article contributions
If you know the change that is needed in an article, we encourage you to submit the changes directly using a pull request. If the change is large, or if you want to contribute an entire article, follow these guidelines:

[ ] Don't surprise us with a big pull request or a pull request with a new article! Submit an issue that describes the details of the proposed large change or new article.
[ ] Include the service or technology area.

We'll route the issue to the appropriate content team for review and discussion.
Tech support and product feedback
If you would like to contact Microsoft about other things, such as product feedback or tech support, please review these guidelines:

If you need technical support using Azure, the paid and free support options are described here: https://azure.microsoft.com/support/options/.

Each article in the Azure technical documentation contains a product feedback button - it's best to submit product feedback directly from a relevant article. Otherwise, you can submit product feedback for most Azure products in the following product feedback forum: https://feedback.azure.com/forums/34192--general-feedback.

@SharvariN-Lab58 Thanks for your question. Please provide us with the link to the documentation you are referring to so we can better assist.

@Karishma-Tiwari-MSFT after a couple of reattempts the same code could transcribe the full audio. Though the code worked, I am not understanding the reason as to why it did not work in the first go.

@SharvariN-Lab58 Can you please share the url of the speech-to-text azure document you have been referring to, it will help me direct this question to the right team.

@SharvariN-Lab58 It looks like your issue is not related to the existing azure documentation.
The scope for these doc issues is to help identify gaps and issues with the documentation.
Since your issue is about troubleshooting the error you are seeing, I would recommend opening a post on  MSDN or Stack Overflow communities. Our engineers will help you troubleshoot your issue there. :)

We will now close this issue. If there are further questions regarding this matter, please tag me in a comment. I will reopen it and we will gladly continue the discussion.


added 3th part libraries notes on development-guidelines.md

notes about 3th part libraries patch/fix

@Undermaken any update on this PR?

Thought It was complete with these changes https://github.com/teamdigitale/io-handbook/commit/ca22a2c47c2348c51c5bc55bdfc112474cc89688 Did I miss something?


WMCore-10-2016-issues

Bug fixes high priority:
#6791 Aborted workflows acquiring WQE and creating/submitting jobs (S)
#6710 force complete not completing (S)
#6187 ACDC for TaskChain wfs creating more jobs than expected (A)
#5809 WQ trying to abort workflows in aborted-archived (S)
#7012 CMSSW Release used for merges within TaskChain (A)
#7115 Files are not injected to PhEDEx but DBS is updated. (S)
#5373 make acquistion era, process string, processing version mandatory in reqmgr
High Priority:
#5643 Name for output root file for multiRun DQM harvesting
#4851 show when a merge job skips a file (S) (needs testing in the new agent)
#5631 redo abort and force complete (S)
#6789 Change abort workflow to complete ALL subscriptions
#6787 Stepchain: sort out duplicate outputModules (A)
#6881 ReqMgr2: filter out some keys before persisting them in the reqmgr2 couch doc
#7054 when acdc is created move the status to assignment-approved
#6917 Request assignment to > 1 team should fail
#6802 Don't delete the Couch data until all fwjr is uploaded to WMArchive
#6634 Create the REST api get all the teams available currently
#6612 Remove WMBSService from WMAgent component
Bug fixes medium priority:
#7118 Combine threshold for Processing and Production job type (S)
#6907 memoryRequirement should not be set for utilitarian job types
#6906 Upload new values to Multicore, Memory and Performance params to workload doc
#6881 ReqMgr2: filter out some keys before persisting them in the reqmgr2 couch doc
#6952 Create outputmodule validate function based on its length (A)
#6845 getChunkInfo doesn't assumes all the files in that chunk has the same location (A)
#7133 Make a list of accepted parameters for creation and another for assignment
#7119 Changing pendingSlotsSitePercent takes no effect in ASW (A)
#4966 Adding tasks info into RC fails silently (A)

@ticoann I'll create the initial list of issues soon, possible before the release planning meeting

Ticket for November issues created in #7366 , though we still have to populate it :)

New ticket created for December's issues #7451


ublox_dgnss: 0.2.2-1 in 'galactic/distribution.yaml' [bloom]

Increasing version of package(s) in repository ublox_dgnss to 0.2.2-1:

upstream repository: https://github.com/aussierobots/ublox_dgnss
release repository: https://github.com/aussierobots/ublox_dgnss-release.git
distro file: galactic/distribution.yaml
bloom version: 0.10.7
previous version for package: 0.2.1-1

I'm getting a build farm issue https://build.ros2.org/job/Gbin_ufv8_uFv8__ublox_dgnss_node__ubuntu_focal_arm64__binary/6/
This package is part of the release and is not already installed. It seems to be a catch 22. Can someone please advise, how to rectify this?
16:07:54 CMake Error at CMakeLists.txt:25 (find_package):
16:07:54   By not providing "Findublox_ubx_msgs.cmake" in CMAKE_MODULE_PATH this
16:07:54   project has asked CMake to find a package configuration file provided by
16:07:54   "ublox_ubx_msgs", but CMake did not find one.
16:07:54
16:07:54   Could not find a package configuration file provided by "ublox_ubx_msgs"
16:07:54   with any of the following names:
16:07:54
16:07:54     ublox_ubx_msgsConfig.cmake
16:07:54     ublox_ubx_msgs-config.cmake
16:07:54
16:07:54   Add the installation prefix of "ublox_ubx_msgs" to CMAKE_PREFIX_PATH or set
16:07:54   "ublox_ubx_msgs_DIR" to a directory containing one of the above files.  If
16:07:54   "ublox_ubx_msgs" provides a separate development package or SDK, be sure it
16:07:54   has been installed.
16:07:54


[Data] modify int16 to parse sql int(16) format to int16

Background
ref #845

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

4 of 4 (100.0%)  changed or added relevant lines in 3 files are covered.
2 unchanged lines in 1 file lost coverage.
Overall coverage increased (+0.004%) to 98.307%

Files with Coverage Reduction
New Missed Lines
%

core/src/executor/aggregate/mod.rs
2
89.74%

Totals

Change from base Build<PHONE_NUMBER>:
0.004%

Covered Lines:
36111

Relevant Lines:
36733

💛  - Coveralls


configs when training/eval  DAIR-V2X dataset?

@wuzirui  Could you release the configs when training/eval  DAIR-V2X dataset, and how many frames did you use for training, thank you in advance.

Sorry, we cannot provide this information due to business issue.


Column mean_exposure doesn't exist."

Hey, I i came across this issue in robyn and it's not going away:

Run all trials and iterations. Use ?robyn_run to check parameter definition
OutputModels <- robyn_run(

InputCollect = InputCollect, # feed in all model specification
cores = NULL, # NULL defaults to (max available - 1)
iterations = 10000, # 2000 recommended for the dummy dataset with no calibration
trials = 8, # 5 recommended for the dummy dataset
ts_validation = TRUE, # 3-way-split time series for NRMSE validation.
add_penalty_factor = FALSE # Experimental feature. Use with caution.
)
Input data has 42 weeks in total: 2023-08-14 to 2024-05-27
Initial model is built on rolling window of 42 week: 2023-08-14 to 2024-05-27
Time-series validation with train_size range of 50%-80% of the data...
Using geometric adstocking with 14 hyperparameters (14 to iterate + 0 fixed) on 15 cores

Starting 8 trials with 10000 iterations each using TwoPointsDE nevergrad algorithm...
Running trial 1 of 8
|                                                                                |   0%Timing stopped at: 0.73 0.11 0.86
Error in { : task 1 failed - "Can't select columns that don't exist.
✖ Column mean_exposure doesn't exist."

I'd really appreciate if anyone can help me out with this one.

dt_input

A tibble: 65 × 6
DATE                  Call RESPONSIVE_SEARCH VIDEO_TRUEVIEW_IN_STREAM    FB Conversion

1 2023-09-18 00:00:00  0                  818.                     75.0  501.        220
2 2023-09-25 00:00:00  0                  805.                     74.2  450.        198
3 2023-10-02 00:00:00  0                  781.                     70.2  500.        211
4 2023-10-09 00:00:00  0                  802.                     55.9  509.        194
5 2023-10-16 00:00:00  0.241              801.                     69.6  483.        240
6 2023-10-23 00:00:00  2.87               797.                    102.   462.        206
7 2023-10-30 00:00:00 18.9                713.                     64.4  460.        224
8 2023-11-06 00:00:00  5.99               740.                     82.3  510.        238
9 2023-11-13 00:00:00  2.86               818.                     72.0  519.        193
10 2023-11-20 00:00:00  0                  718.                     81.3  481.        187
ℹ 55 more rows
ℹ Use print(n = ...) to see more rows

All sign control are now automatically provided: "positive" for media & organic
variables and "default" for all others. User can still customise signs if necessary.
Documentation is available, access it anytime by running: ?robyn_inputs
InputCollect <- robyn_inputs(

dt_input = dt_input,
dt_holidays = dt_prophet_holidays,
date_var = "DATE", # date format must be "2020-01-01"
dep_var = "Conversion", # there should be only one dependent variable
dep_var_type = "conversion", # "revenue" (ROI) or "conversion" (CPA)
prophet_vars = c("trend", "season", "holiday"), # "trend","season", "weekday" & "holiday"
prophet_country = "US", # input country code. Check: dt_prophet_holidays
paid_media_spends = c( "Call",	"RESPONSIVE_SEARCH",	"VIDEO_TRUEVIEW_IN_STREAM",	"FB"
), # mandatory input

paid_media_vars = c( "Call",	"RESPONSIVE_SEARCH",	"VIDEO_TRUEVIEW_IN_STREAM",	"FB"), # mandatory.

paid_media_vars must have same order as paid_media_spends. Use media exposure metrics like

impressions, GRP etc. If not applicable, use spend instead.

#context_vars = c("events"),
#factor_vars = c("events"), # force variables in context_vars or organic_vars to be categorical
#window_start = "2021-05-03",
#window_end = "2023-12-25",

adstock = "geometric" # geometric, weibull_cdf or weibull_pdf.
)
Warning message:
In check_datadim(dt_input, all_ind_vars, rel = 10) :
There are 7 independent variables & 65 data points. We recommend row:column ratio of 10 to 1

print(InputCollect)
Total Observations: 65 (weeks)
Input Table Columns (6):
Date: DATE
Dependent: Conversion [conversion]
Paid Media: Call, RESPONSIVE_SEARCH, VIDEO_TRUEVIEW_IN_STREAM, FB
Paid Media Spend: Call, RESPONSIVE_SEARCH, VIDEO_TRUEVIEW_IN_STREAM, FB
Context:
Organic:
Prophet (Auto-generated): trend, season, holiday on US
Unused variables: None

Date Range: 2023-09-18:2024-12-09
Model Window: 2023-09-18:2024-12-09 (65 weeks)
With Calibration: FALSE
Custom parameters: None
Adstock: geometric
Hyper-parameters: Not set yet

Default media variable for modelling has changed from paid_media_vars to paid_media_spends.
Also, calibration_input are required to be spend names.
hyperparameter names are based on paid_media_spends names too. See right hyperparameter names:
hyper_names(adstock = InputCollect$adstock, all_media = InputCollect$all_media)
[1] "Call_alphas"                     "Call_gammas"                     "Call_thetas"
[4] "FB_alphas"                       "FB_gammas"                       "FB_thetas"
[7] "RESPONSIVE_SEARCH_alphas"        "RESPONSIVE_SEARCH_gammas"        "RESPONSIVE_SEARCH_thetas"
[10] "VIDEO_TRUEVIEW_IN_STREAM_alphas" "VIDEO_TRUEVIEW_IN_STREAM_gammas" "VIDEO_TRUEVIEW_IN_STREAM_thetas"
1. IMPORTANT: set plot = TRUE to create example plots for adstock & saturation
hyperparameters and their influence in curve transformation.
plot_adstock(plot = FALSE)
plot_saturation(plot = FALSE)
4. Set individual hyperparameter bounds. They either contain two values e.g. c(0, 0.5),
or only one value, in which case you'd "fix" that hyperparameter.
Run hyper_limits() to check maximum upper and lower bounds by range
hyper_limits()
thetas alphas gammas shapes scales
1    >=0     >0     >0    >=0    >=0
2     <1    <10    <=1    <20    <=1
Example hyperparameters ranges for Geometric adstock
hyperparameters <- list(

FB_alphas = c(0.5, 3),
FB_gammas = c(0.3, 1),
FB_thetas = c(0, 0.3),
Call_alphas = c(0.5, 3),
Call_gammas = c(0.3, 1),
Call_thetas = c(0, 0.3),
RESPONSIVE_SEARCH_alphas = c(0.5, 3),
RESPONSIVE_SEARCH_gammas = c(0.3, 1),
RESPONSIVE_SEARCH_thetas = c(0, 0.3),
VIDEO_TRUEVIEW_IN_STREAM_alphas = c(0.5, 3),
VIDEO_TRUEVIEW_IN_STREAM_gammas = c(0.3, 1),
VIDEO_TRUEVIEW_IN_STREAM_thetas = c(0, 0.3)
)

InputCollect <- robyn_inputs(InputCollect = InputCollect, hyperparameters = hyperparameters)

Running feature engineering...
Warning message:
In check_hyperparameters(InputCollect$hyperparameters, InputCollect$adstock,  :
Automatically added missing hyperparameter range: 'train_size' = c(0.5, 0.8)
print(InputCollect)
Total Observations: 65 (weeks)
Input Table Columns (6):
Date: DATE
Dependent: Conversion [conversion]
Paid Media: Call, RESPONSIVE_SEARCH, VIDEO_TRUEVIEW_IN_STREAM, FB
Paid Media Spend: Call, RESPONSIVE_SEARCH, VIDEO_TRUEVIEW_IN_STREAM, FB
Context:
Organic:
Prophet (Auto-generated): trend, season, holiday on US
Unused variables: None

Date Range: 2023-09-18:2024-12-09
Model Window: 2023-09-18:2024-12-09 (65 weeks)
With Calibration: FALSE
Custom parameters: None
Adstock: geometric
Hyper-parameters ranges:
FB_alphas: [0.5, 3]
FB_gammas: [0.3, 1]
FB_thetas: [0, 0.3]
Call_alphas: [0.5, 3]
Call_gammas: [0.3, 1]
Call_thetas: [0, 0.3]
RESPONSIVE_SEARCH_alphas: [0.5, 3]
RESPONSIVE_SEARCH_gammas: [0.3, 1]
RESPONSIVE_SEARCH_thetas: [0, 0.3]
VIDEO_TRUEVIEW_IN_STREAM_alphas: [0.5, 3]
VIDEO_TRUEVIEW_IN_STREAM_gammas: [0.3, 1]
VIDEO_TRUEVIEW_IN_STREAM_thetas: [0, 0.3]
train_size: [0.5, 0.8]

Check spend exposure fit if available
if (length(InputCollect$exposure_vars) > 0) {

lapply(InputCollect$modNLS$plots, plot)
}

Run all trials and iterations. Use ?robyn_run to check parameter definition
OutputModels <- robyn_run(

InputCollect = InputCollect, # feed in all model specification
cores = NULL, # NULL defaults to (max available - 1)
iterations = 5000, # 2000 recommended for the dummy dataset with no calibration
trials = 5, # 5 recommended for the dummy dataset
ts_validation = TRUE, # 3-way-split time series for NRMSE validation.
add_penalty_factor = FALSE # Experimental feature. Use with caution.
)
Input data has 65 weeks in total: 2023-09-18 to 2024-12-09
Initial model is built on rolling window of 65 week: 2023-09-18 to 2024-12-09
Time-series validation with train_size range of 50%-80% of the data...
Using geometric adstocking with 14 hyperparameters (14 to iterate + 0 fixed) on 15 cores

Starting 5 trials with 5000 iterations each using TwoPointsDE nevergrad algorithm...
Running trial 1 of 5
|                                                                                |   0%Timing stopped at: 0.88 0.05 0.94
Error in { : task 1 failed - "Can't select columns that don't exist.
✖ Column mean_exposure doesn't exist."

The codes were running fine for me before but started giving me problem today. Please help me with this issue @gufengzhou

I found the error! you need to have an organic variable column as well in the input column now for the model to run. This was not the case before, kindly look into it.

Thanks! Will look into it. @laresbernardo would you have time for this one?

@mahaja1 I can't reproduce your error. I can run the demo without organic using the latest version.

I am still getting it. What options do i have? When i ran the model using traffic column, it is assigning large contribution value to traffic. I want to run it without any organic variable.

I really need help around this. Please look into this.

which version are you using? you can do so by running packageVersion("Robyn"). Your copy paste message above is quite messy. I might need your dataset as well as your robyn_inputs() configs to debug

I wasn't able to reproduce the error either. Potentially changing a hard select(...)' to select(any_of(...))could fix it if that's the case @gufengzhou. There are only 4mean_exposure`'s in the code.


IPv6-specific "Host is unreachable" error that exits the matter runtime

Environment
Chip: ESP32-C3-MINI-1
Hardware: ESP32-C3-DevKitM-1
Platform: esp-idf (Rust std)
Problem
I likely have something misconfigured on my network causing IPv6 broadcasts to yield a surprising "Host is unreachable" error, however the more important issue is that the way the master future is structured in my example (and onoff_light) causes the entire Matter runtime to effectively shutdown and not automatically restart.
An abridged version of the log shows the issue:
I (7607) rs_matter::transport::core: Comissioning started
I (7617) rs_matter::transport::core: Creating queue for 1 exchanges
I (7617) rs_matter::transport::core: Creating 8 handlers
I (7627) rs_matter::transport::core: Handlers size: 9992
I (7637) rs_matter::transport::core: Transport: waiting for incoming packets
I (7647) rs_matter::transport::udp::async_io: Listening on [::]:5353
I (7647) rs_matter::transport::udp::async_io: Joined IPV6 multicast ff02::fb/2
I (7657) rs_matter::transport::udp::async_io: Joined IP multicast <IP_ADDRESS>/<IP_ADDRESS>
I (7667) rs_matter::mdns::builtin: Broadcasting mDNS entry to <IP_ADDRESS>:5353
I (7687) rs_matter::mdns::builtin: Broadcasting mDNS entry to ff02::fb:5353
W (7697) rs_matter::transport::udp::async_io: Error on the network: Os { code: 118, kind: HostUnreachable, message: "Host is unreachable" }
Error: Error::Network

The last line in particular appears to be coming from the master future in the onoff light example: https://github.com/project-chip/rs-matter/blob/main/examples/onoff_light/src/main.rs#L165
This is "fixed" by just disabling IPv6 for me, but I do think this highlights some bigger issues with the error handling robustness inside the runtime.  In particular I'd expect that the IPv4 and IPv6 behaviour be separated into separate futures that can error out independently and that one reaching a terminal state wouldn't negatively impact the other.  Further I think some measure of error handling policy is appropriate (Host is unreachable seems like it should probably be retryable for example).  I could take a crack at a patch but I do worry based on the current state of the code that it might be a bit intrusive.  Any guidance from the maintainers would be greatly appreciated before getting started!
Thanks again for this awesome project, it's renewed my interest big time in IoT :)

The mDNS responder is doing broadcasting. In other words, it is not sending the UDP packet to a specific host, but rather, to the broadcast address ff02::fb which should always be available.
Moreover, not broadcasting via ipv6 is less than ideal to put it mildly. The way Matter is implemented in the field (Google Home and I suspect others) is that it requires IPv6 connectivity - link-local IPv6 addresses suffice, but those are necessary. Moreover, the mDNS responder also needs ipv6 support. I was not able - without it - to get Google Home provisioning to complete.
So where I'm going is that if ipv6 (including for broadcasting) does not work for you, a hard failure for now is probably OK. It is another story why it fails, as per above. Can you pinpoint the exact code line where it fails?

So where I'm going is that if ipv6 (including for broadcasting) does not work for you, a hard failure for now is probably OK. It is another story why it fails, as per above. Can you pinpoint the exact code line where it fails?

Ack'd, I'll dig a little deeper why this isn't working.  My network definitely should be supporting ipv6 as it's a Google WiFi mesh with no custom configuration which makes me think the fault lies somewhere in the rs-matter code somehow but we'll see...

Might be... see, with linked-local ipv6 there is no need of any explicit "ipv6 support" per se. As in, there is no "dhcp" and you don't need a gateway as well.
What esp idf version are you using with the example? It should be 4.4.x which I know works, unless you've explicitly changed it...

Confirmed that 4.4.x fixed this specific issue.  I'll try to debug deeper why 5.x would be broken in this way.

Digging deeper on why this doesn't work in release/v5.0 (and I presume v5.1 but that fails to compile with esp-idf-svc), the story here seems really hairy.  I think espressif might've broke something when attempting to backport fixes to v4.4.  After many hours of debugging, I am fairly confident the offending code is:
espressif/esp-idf/components/lwip @ release/v4.4:
https://github.com/espressif/esp-idf/tree/release/v4.4/components/lwip
https://github.com/espressif/esp-lwip/blob/4f24c9baf9101634b7c690802f424b197b3bb685/src/core/ipv6/ip6.c#L175-L185
espressif/esp-idf/components/lwip @ release/v5.0:
https://github.com/espressif/esp-idf/tree/release/v5.0/components/lwip
https://github.com/espressif/esp-lwip/blob/8dad8d3ee66840deee4acfc1601de4e396c594be/src/core/ipv6/ip6.c#L175-L177
No idea why these things are different or what actual diff introduced this inconsistency.  The v4.4 branch of esp-lwip has only one commit and it seems unrelated like maybe somebody squashed a big merge into one commit (possibly on accident?).  Even weirder is that I can't find any evidence of upstream or esp-lwip having code like this.  There's also support in the v4.4 branch for IPV6_MULTICAST_IF (which probably would also fix the issue matter-rs is seeing), but that support isn't in upstream or v5.0/v5.1, or really anywhere I can see...

My hypothesis:
Rust STD's join_multicast_v6 is (partially) broken on the ESP IDF (join_multicast_v4 was totally broken and I hat to fix it back in time) - in that it likely does not set correctly the proper ipv6 network interface. And therefore it hits the  "fallback paths" in ESP IDF 4.4 and 5.0 which try to derive a network interface (and then use the default one on 4.4 and fail on 5.0).
One test we can try to do is "manually" re-implement join_multicast_v6 here, as I did for join_multicast_v4. If it works, next step is to upstream in libc correct signatures for setsockopt, associated constants, and maybe the ipv6_mreq structure.

Nope, you were right, I think it's not setting the zone flag in the ip6_addr struct which is causing the route to fail.  I'll prep a patch soon to fix it.

After some digging, I have good news.  I believe the issue is that in lwip you have to call ip6_addr_set_zone on an ip6_addr_t (which has an extra u8 zone field at the end) that is then used to route packets.  This is achieved using the scope_id field in SocketAddrV6.  I believe this should be required/important on all platforms, it's just very likely that Linux has a less fragile heuristic to figure this out for you.
See the discussion on the scope_id field here: https://datatracker.ietf.org/doc/html/rfc2553#section-3.3.
So, good news addressing my unknowns above:

We can just pass scope_id into the SocketAddr we use for send_to, confirmed this works.
Nothing needed upstream, IPv6 scopes are implemented properly for newer versions of lwip (found in esp-idf 5.x)

I'll prep a PR to fix this


Support for converting style attributes to HTML attributes

<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 128 128">
  <path d="M2,2 v6 h6 v-6 h-6 z" style="stroke:#FF0000;stroke-width:0.5;fill:none;"/>
</svg>

To
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 128 128">
  <path d="M2,2 v6 h6 v-6 h-6 z" stroke="#FF0000" stroke-width="0.5" fill="none"/>
</svg>

This is useful for font tools, e.g. https://github.com/googlefonts/picosvg/pull/60

Hi! Thank you for reporting! :)
Interesting feature - implementation-wise it should be relatively straightforward:

Config flag
If enabled, then insert collected styles as attributes

Though the underlying library does not provide a way to attach an XML parser / serializer, therefore I am concerned about the use case you provided would not be supported out of the box. However, implementing such a thing is feasible even without kuchiki-rs as there is a standalone XML parser available for Rust.

Any plans to implement this feature this year?

I'd like to have it in css-inline and this year sounds realistic to me :) Non-blocking resolving is at the top of my to-do list, but after that, I'd be able to work on this.
@caseyjhol, as you commented on #139 - are you still interested in moving this forward? I'd be glad to assist in the implementation or provide a more detailed plan.

Forgot about the XML part :(
Maybe a better way to solve the task described in the original message would be:

A way to configure the underlying parser, currently, we only have HTML via html5ever
Add an XML parser that will parse input into the same Document struct
Bind different serializers for each parser, the current one is for HTML only.
Add a different method on CSSInliner which will look for "style" attributes, parse them, and insert individual styles as separate attributes. Some inline_styles_as_attributes or something better

I think that it also will be much faster & simpler as there is no need for selectors at all, just check the nodes for the style attributes.

So, for HTML it would be a different thing that should be integrated into the current inlining process


Use IPAM consistently

If we use both IPs in ansibleVars and IPAM there can be issues with IPAM reserved IPs different from what's used in ansibleVars with duplicates. We've IPAM so that we can avoid duplicates.
There should be single interface to change those. For pre-provisoned nodes we shoud use fixedIP to ensure that we reserve the ctlplane IP we use for the node.
When using IPAM, IPs in ansibleVars would be ignored.

recheck

recheck

recheck


What still needs to happen for clipboard to be supported?

I can see it correctly saves each pixel in sprite_sheet.txt and shows the copied/pasted message yet it does not work, so if you slap [gfx][/gfx] on a string with the data from current selected sprite we can copy and read it back minus [gfx][/gfx] for paste?
Then we need to be able to access to stat(4)/clipboard to read it in a game/tool.
Is that it?
Also I know Pico-8 has intentional limitations about reading/writing clipboard which is quite annoying for my usecase here: https://github.com/lesleyrs/pico-paint
This is to prevent a cartridge from accessing potentially-sensitive data on the host system's clipboard without the user's permission. - from https://pico-8.fandom.com/wiki/Clipboard
I couldn't find an answer why this would be bad on pico-8 as it doesn't have any built-in network capabilities so the worst that could happen is a game displaying your clipboard on screen, which could be annoying if somebody was livestreaming it and happens to copy his passwords. And even then they wouldn't know their username?
IMO that's not worth being a big inconvenience though, I would much rather have a notification on launch that shows if the application could try to access clipboard. Just like the devkit mode notification in pico-8 informing people.
Thx for coming to my TED talk.

Hey there, thank you for the help. This doesn't work like you expect because I didn't use the system clipboard when implementing Runty8's clipboard 😅:
"Copying" right now means writing to an in-memory vec of pixels, and similarly "pasting" is reading from it.
We should probably change this to be like Pico8, using the real clipboard and the [gfx]...[/gfx] formatted strings, I just didn't know that's how it worked at the time.


[ie/RTP] Fix RTP extractor

IMPORTANT: PRs without the template will be CLOSED
Description of your pull request and other information

This PR fully fixes the RTP extractor. It's based on contributions of others (mostly https://github.com/ytdl-org/youtube-dl/pull/28205) and my own work.
Fixes #4661 and #10439. Unlike #10393, this really fixes the extractor.
Template

Before submitting a pull request make sure you have:

[X] At least skimmed through contributing guidelines including yt-dlp coding conventions
[X] Searched the bugtracker for similar pull requests

In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under Unlicense. Check all of the following options that apply:

[X] I am the original author of this code and I am willing to release it under Unlicense
[X] I am not the original author of this code but it is in public domain or released under Unlicense (provide reliable evidence)

What is the purpose of your pull request?

[X] Fix or improvement to an extractor (Make sure to add/update tests)
[ ] New extractor (Piracy websites will not be accepted)
[ ] Core bug fix/improvement
[ ] New feature (It is strongly recommended to open an issue first)

This works and passes all tests, so is there anything missing that should be accounted for?
Would be world to have it included in the next release.

I hate to be this guy but can this PR get some love from a maintainer? It was created 5 weeks ago, plenty of other PRs where created/accepted/closed meanwhile (including NFSW ones) and 2 new official releases came out.
As you may also check on the previous PR #9339, most of the fixes included here have been implemented since March already.

PRs don't get reviewed in order. There are currently around 100 open extractor PRs requiring a review. Each potentially taking hours of work to be brought in to a mergable state.
When a PR gets reviewed depends on several things like the complexity of the PR, complexity of the site, how much work is still needed for that PR, priority of the issue, personal interests, etc.
For this PR specifically, the site is incredibly annoying to work with on its own (irrespective of any PR). I decided to rework the extractor myself in #11638
Please check if that PR is working as intended.


Glob() should not return leaf nodes

This issue was originally created at: 2008-08-27 16:50:20.
This issue was reported by: gregnoel.
gregnoel said at 2008-08-27 16:50:20

Consider this SConscript:

    File('foo')
    print([str(s) for s in Glob('foo')]

Glob() reports the existence of foo despite the fact it does not exist and cannot be built.  The equivalent SConscript using FindFile():

    File('foo')
    print FindFile('foo', '.')

returns None.
I think Glob() should follow the same convention and only return derived files (i.e., files with an attached Builder).

gregnoel said at 2008-09-02 14:32:36

Bug party triage.

gregnoel said at 2008-12-26 13:30:37

Adjust triage of issues.

garyo said at 2012-09-01 10:04:01

de-assigning all tickets assigned to Greg Noel (no longer working on SCons)

There are two questions here for me:

Should Glob return anything that doesn't exist in the file system.  Note that at least one other build system does not: https://docs.bazel.build/versions/master/be/functions.html#glob which includes this comment: "looks only for source files (not generated files nor other targets"
Should Glob return files that cannot be made by SCons, as this issue asks.

Yes
Yes

Any files SCons knows how to build whether they're currently exist or not, as well as files which aren't built by SCons but exist on the filesystem.

I'm going to close this issue.
Changing the existing behavior wouldn't be an improvement.
This issue is from 2008, Glob() was introduced to SCons in 2007.


Add dutch translation

Thank you for translation!
Comments is along the commit.

Thank you one more time!


Custom Authorization

Coz of duende licences and bugs with docker containers, the app needs custom auth.

Its in working progress. When I finish It will be uploaded.


ros2_control: 0.1.1-1 in 'foxy/distribution.yaml' [bloom]

Increasing version of package(s) in repository ros2_control to 0.1.1-1:

upstream repository: https://github.com/ros-controls/ros2_control.git
release repository: https://github.com/bmagyar/ros2_control-release.git
distro file: foxy/distribution.yaml
bloom version: 0.10.0
previous version for package: 0.1.0-1

controller_interface

No changes

controller_manager

No changes

controller_manager_msgs

No changes

hardware_interface

No changes

ros2_control
* Remove transmission_interface from release, add ros2cli to ros_control (#280 <https://github.com/ros-controls/ros2_control/issues/280>)
  * Remove transmission_interface from release, add ros2cli to ros_control
  metapackage
  * patch
* Contributors: Bence Magyar

ros2controlcli

No changes

test_robot_hardware

No changes

@jacobperron happy to go ahead with this as-is

Superseded by https://github.com/ros/rosdistro/pull/27873

Superseded by https://github.com/ros/rosdistro/pull/27873


chai-dom not registered in custom karma webpack setup

I'm trying migrate some of my existing tests over to the @open-wc/testing library, mainly to use the snapshots and chai-dom.
My setup uses webpack with karma to run the tests. This seems to cause chai-dom to not be registered. It looks like register-chai-plugins.js imports chai-dom and does nothing with it. Chai dom is a UMD, so when run this way it exports ChaiDom instead of implicitly calling chai.use with it.
Is there a way to force chai-dom to be registered in this case?

I think we can make it work for both webpack and browser es modules by doing this instead:
// non es module plugins
import 'chai-dom';
import 'sinon-chai';

// es module plugins
import { chaiDomDiff } from '@open-wc/semantic-dom-diff';
import { chaiA11yAxe } from 'chai-a11y-axe';

let chai = window.chai;
if (!chai && require) {
  chai = require('chai');
}

// @ts-ignore
chai.use(chaiDomDiff);
// @ts-ignore
chai.use(chaiA11yAxe);

Could you test if this works on your setup? I don't have a webpack karma setup to test this out.

That didn't work for me, but this does work:
// non es module plugins
import chaiDom from 'chai-dom';
import 'sinon-chai';

// es module plugins
import { chaiDomDiff } from '@open-wc/semantic-dom-diff';
import { chaiA11yAxe } from 'chai-a11y-axe';

// @ts-ignore
chai.use(chaiDomDiff);
// @ts-ignore
chai.use(chaiA11yAxe);

chai.use(chaiDom);

What happens if you run import chaiDom from 'chai-dom'; with your setup? If that returns null or an empty object, then you should be able to check that and only call chai.use(chaiDom) if it is an actual chai plugin.
sinon-chai would have the same problem, since it is a UMD as well.

Unfortunately the browser complains that there is no export from that modules, so we can't do that.

It looks like the author of chai-dom is considering moving towards an es module approach: https://github.com/nathanboktae/chai-dom/issues/38#issuecomment-549221553

as long as chai itself is not an es module it's probably tough 🙈

I was able to find a solution for this in https://github.com/open-wc/open-wc/pull/1142


[BugFix][CherryPick] fix insert overwrite bug with duplicate target partitions for 3.0 (#27545)

Fixes #27521
What type of PR is this:

[x] BugFix
[ ] Feature
[ ] Enhancement
[ ] Refactor
[ ] UT
[ ] Doc
[ ] Tool

Does this PR entail a change in behavior?

[x] Yes, this PR will result in a change in behavior.
[ ] No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

[ ] Interface/UI changes: syntax, type conversion, expression evaluation, display information
[ ] Parameter changes: default values, similar parameters but with different default values
[ ] Policy changes: use new policy to replace old one, functionality automatically enabled
[ ] Feature removed
[ ] Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

[ ] I have added test cases for my bug fix or my new feature
[ ] This pr needs user documentation (for new or modified features or behaviors)

[ ] I have added documentation for my new feature or new function

Bugfix cherry-pick branch check:

[ ] I have checked the version labels which the pr will be auto-backported to the target branch

[ ] 3.1
[ ] 3.0
[ ] 2.5
[ ] 2.4

[BE Incremental Coverage Report]
:heart_eyes: pass : 0 / 0 (0%)

[FE Incremental Coverage Report]
:heart_eyes: pass : 7 / 7 (100.00%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/sql/analyzer/InsertAnalyzer.java
7
7
100.00%
[]

[BE Incremental Coverage Report]
:heart_eyes: pass : 0 / 0 (0%)


Crash when using Delve debugger

I have c crash while debugging with Delve (VS Code).
I updated the go-sqllite yesterday, and after that i've got a crash on both my PC (2 systems with windows - crash). Linux is OK. I have same GOPATH folder for them all.
Debug log is:
Exception 0xc0000005 0x8 0xa8c148 0xa8c148
PC=0xa8c148
signal arrived during external code execution
Error is from here (bold selection):
//package sql:
// Close closes the database, releasing any open resources.
//
// It is rare to Close a DB, as the DB handle is meant to be
// long-lived and shared between many goroutines.
func (db *DB) Close() error {
db.mu.Lock()
if db.closed { // Make DB.Close idempotent
db.mu.Unlock()
return nil
}
close(db.openerCh)
var err error
fns := make([]func() error, 0, len(db.freeConn))
for _, dc := range db.freeConn {
fns = append(fns, dc.closeDBLocked())
}
db.freeConn = nil
db.closed = true
for _, req := range db.connRequests {
close(req)
}
db.mu.Unlock()
for _, fn := range fns {
err1 := fn()
if err1 != nil {
err = err1
}
}
return err
}

Got exactly the same problem, if you found a fix please tell us 😉

It probably is.   Go on non-Linux (eg OSX, Windows) seems to be er... missing some important debug information which delve needs.  Apparently this is being fixed for Go 1.9, but doesn't really help us much now. 😦
As a thought, JetBrains has some delve binaries available with short term fixes applied which might help:
    https://bintray.com/jetbrains/golang/delve
No guarantees though. 😁

@Eun Damn.  I was really looking forward to 1.9 fixing much of the debugging issues. 😦
With the Jetbrains provided Delve binaries, when I unzip any of the recent ones they definitely include a Windows version.  eg:
    https://bintray.com/jetbrains/golang/delve/0.11.359#files/com/jetbrains/delve/0.11.359
In that .zip file, it has three subdirectories with delve executables:

linux
mac
windows

Is it different for you?

@justinclift You are right, I somehow used the old 0.6.65 version... 😮

@Eun Do you have a simple reproducer for the crash?  The Jetbrains people were asking (via email) the other day when I mentioned it to them.
I haven't gotten around to making a simple crash causer yet.  If you have a simple code listing handy with reliably causes the crash, we can point the JetBrains people at it for investigation.

@justinclift
I just noticed go 1.9 is not even out there, so still hope for go 1.9 to fix it.
However here is an example:
a.go:
package main

import (
	"database/sql"

	"fmt"

	_ "github.com/mattn/go-sqlite3"
)

func main() {
	db, err := sql.Open("sqlite3", "test.db")
	if err != nil {
		panic(err)
	}
	_, err = db.Exec(`
		CREATE TABLE IF NOT EXISTS
		Config (
			'Section' TEXT NOT NULL,
			'Key' TEXT NOT NULL,
			'Value' TEXT NOT NULL
		)`)
	if err != nil {
		panic(err)
	}
	fmt.Println("Everything worked")
}

launch.json:
    {
        "name": "TEST",
        "type": "go",
        "request": "launch",
        "mode": "debug",
        "remotePath": "",
        "port": 2345,
        "host": "<IP_ADDRESS>",
        "program": "${workspaceRoot}/a.go",
        "env": {
            "GOPATH": "${workspaceRoot}"
        },
        "args": [],
        "showLog": true,
        "cwd": "${workspaceRoot}"
    }

Operating system is Windows 10 x64 (1607)
Go is go version go1.8 windows/amd64
Hopefully this helps somehow...

Ahhh, that is a good sample. :smile:
For me, trying it out on OSX 10.11.6, running Go 1.8, it errors out with the following when trying to debug:
# command-line-arguments
github.com/mattn/go-sqlite3(__DATA/__data): unexpected reloc for dynamic symbol close
github.com/mattn/go-sqlite3(__DATA/__data): unexpected reloc for dynamic symbol access
github.com/mattn/go-sqlite3(__DATA/__data): unexpected reloc for dynamic symbol getcwd
github.com/mattn/go-sqlite3(__DATA/__data): unexpected reloc for dynamic symbol stat$INODE64
github.com/mattn/go-sqlite3(__DATA/__data): unexpected reloc for dynamic symbol fstat$INODE64
github.com/mattn/go-sqlite3(__DATA/__data): unexpected reloc for dynamic symbol ftruncate
github.com/mattn/go-sqlite3(__DATA/__data): unexpected reloc for dynamic symbol fcntl
github.com/mattn/go-sqlite3(__DATA/__data): unexpected reloc for dynamic symbol read
github.com/mattn/go-sqlite3(__DATA/__data): unexpected reloc for dynamic symbol pread
github.com/mattn/go-sqlite3(__DATA/__data): unexpected reloc for dynamic symbol write
github.com/mattn/go-sqlite3(__DATA/__data): unexpected reloc for dynamic symbol pwrite
github.com/mattn/go-sqlite3(__DATA/__data): unexpected reloc for dynamic symbol fchmod
github.com/mattn/go-sqlite3(__DATA/__data): unexpected reloc for dynamic symbol unlink
github.com/mattn/go-sqlite3(__DATA/__data): unexpected reloc for dynamic symbol mkdir
github.com/mattn/go-sqlite3(__DATA/__data): unexpected reloc for dynamic symbol rmdir
github.com/mattn/go-sqlite3(__DATA/__data): unexpected reloc for dynamic symbol fchown
github.com/mattn/go-sqlite3(__DATA/__data): unexpected reloc for dynamic symbol geteuid
github.com/mattn/go-sqlite3(__DATA/__data): unexpected reloc for dynamic symbol mmap
github.com/mattn/go-sqlite3(__DATA/__data): unexpected reloc for dynamic symbol munmap
github.com/mattn/go-sqlite3(__DATA/__data): unexpected reloc for dynamic symbol readlink
github.com/mattn/go-sqlite3(__DATA/__data): unexpected reloc for dynamic symbol lstat$INODE64
/usr/local/Cellar/go/1.8/libexec/pkg/tool/darwin_amd64/link: too many errors

When running it normally, the output (as expected) is:
Everything worked

There's a bug report with that kind of error on the Gogland issue tracker already, so I'll add your example code in there and see if they're up for resolving it.  It might turn out it needs to be reported on the delve issue tracker as well (unsure).

@Eun I've added your reproducer to the Gogland issue about the problem.  The error I'm seeing appears to be a problem with mattn's driver + OSX + Delve.
So, that's blocking me from even getting it compiled in Debug mode on OSX. 😦

@Eun Out of curiosity, does that reproducer still crash for you when changing it to gwenn/gosqlite driver?  eg from:
_ "github.com/mattn/go-sqlite3"

to:
_ "github.com/gwenn/gosqlite"

For me, it runs and debugs fine.

If this is still active can you compile on OSX with the tag libssqlite3.

Close due no activity.


Nation vaults don't bold [Nation Vault] on creation

Using latest snapshot.
This is mostly a cosmetic thing, but Nation Vaults don't bold like other vaults once they're created.
Not a high priority as all other functions seem to work fine, but thought I would point it out.
Thanks!

Hello @yntel , is that happening only on Nation, or on Town vaults too?

Just Nation vaults. Town vaults bold just fine!

Hello, if it's possible, can you verify that the problem got fixed?
https://github.com/nikosgram/Gringotts/releases/tag/2.11.2-SNAPSHOT-ref98c00
Thank you :)

I can confirm this is fixed in that build you linked. Thanks!


Care about namespace when evaluating pods in the duplicate strategy

If two deployments in different namespaces have identical template hashes, the replicaset name is the same -- with the same name the Remove Duplicates strategy will erroneously evict pods.

Can one of the admins verify this patch?

I signed it

Is there anything else that needs to be done for this?

squash your commits into 1.

If two deployments in different namespaces have identical template hashes, the replicaset name is the same -- with the same name the Remove Duplicates strategy will erroneously evict pods.

could you give/show me an example where you noticed this?


Trigger CI on PRs and trigger tskit.dev rebuild on push

Closes #31 and #21

Hooray! Merging.

Hooray! Merging.


Add documentation in the collector to run the calculator for requests and provide options

Overview
Add the documentation to calculate the rate limit and how it effects the resolution of the collector. Also document that you can increase it with an org wide pat.

Closing this out as done for GitHub. Will create a new piece of work for the GitLab API's.


🛂(frontend) match email if no existing user matches the sub

Purpose
Some OIDC identity providers may provide a random value in the "sub" field instead of an identifying ID. In this case, it may be a good idea to fallback to matching the user on its email field.
Proposal

Add a setting OIDC_FALLBACK_TO_EMAIL_FOR_IDENTIFICATION which defaults to True
When logging-in, if the sub does not match any existing user, try matching on the email field before creating a new user for the unknown sub.

If the sub is random, how random is it? Is there not a risk of getting another user trying to use it?
Is there a way to know from the user info, the sub is random (so we can just ignore it and use the email instead)?


Restructure commands

Codecov Report
Attention: Patch coverage is 50.00000% with 13 lines in your changes missing coverage. Please review.

Project coverage is 55.56%. Comparing base (0e72bb7) to head (cb3d03d).

Files
Patch %
Lines

cmd/root.go
0.00%
10 Missing :warning:

cmd/version/version.go
62.50%
2 Missing and 1 partial :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #77      +/-   ##
==========================================
+ Coverage   51.48%   55.56%   +4.07%
==========================================
  Files          31       32       +1
  Lines        2282     1760     -522
==========================================
- Hits         1175      978     -197
+ Misses       1055      729     -326
- Partials       52       53       +1

Flag
Coverage Δ

Linux
55.56% <50.00%> (+3.94%)
:arrow_up:

Windows
?

macOS
55.56% <50.00%> (+3.94%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


ASUS Z10PA-D8. Doesn't boot only after ReBarState.exe with 32

System

Motherboard: ASUS Z10PA-D8

BIOS Version: Z10PA-D8-ASUS-3801 (https://disk.yandex.ru/d/46p8AXX9CawTVw)

Modded BIOS: (https://disk.yandex.ru/d/uA7qCh1AiP-fxw)

GPU: Gainward RTX 3070 Phantom

[ Y] CSM is turned off

[Y ] 4G decoding is enabled

[Y ] UEFIPatch is applied (see README for more information)

I update VGA driver for GPU
I get original firmware
I turn on 4G and turn of CSM
I add ReBarDxe.ffs in firmware
I insert patch by UEFIPatch_0.28.0_win32
I install firmawe and reboot
I reconfigure bios and reboot
I run ReBarState.exe, set 32 and tried to reboot.
PC doesn't start. If I install back original bios - it boot, if I install patched bios - it boot. But after this app - it doesn't boot.

BIOS POST Code: 87
Pls, help me. I will give you all info you want.
P.S. I have programmator - so I can flash bios)

Try starting with smaller sizes such as 9 (512MB) and increase till it stops working

Try starting with smaller sizes such as 9 (512MB) and increase till it stops working. Also check device manager resources to see if it's being allocated a 64-bit address or not

Any value doesn't work

I think - I made smth wrong with bois rom. But - don't know what wrong I made...

@SlimRG take a screenshot of device manager resources showing the GPU with just 4G decode on

@SlimRG did you see this ?

@SlimRG did you see this ?

yes
But, I'm not at home - will be at home at 09.01.2023

Sorry for waiting
Today is the first day, I'm not drinking)

System
* Motherboard: ASUS Z10PA-D8

* BIOS Version: Z10PA-D8-ASUS-3801 (https://disk.yandex.ru/d/46p8AXX9CawTVw)

* Modded BIOS: (https://disk.yandex.ru/d/uA7qCh1AiP-fxw)

* GPU: Gainward RTX 3070 Phantom

* [ Y] CSM is turned off

* [Y ] 4G decoding is enabled

* [Y ] UEFIPatch is applied (see README for more information)

1. I update VGA driver for GPU

2. I get original firmware

3. I turn on 4G and turn of CSM

4. I add [ReBarDxe.ffs](https://github.com/xCuri0/ReBarUEFI/releases/download/0.2/ReBarDxe.ffs) in firmware

5. I insert patch by UEFIPatch_0.28.0_win32

6. I install firmawe and reboot

7. I reconfigure bios and reboot

8. I run [ReBarState.exe](https://github.com/xCuri0/ReBarUEFI/releases/download/0.2/ReBarState.exe), set 32 and tried to reboot.

9. PC doesn't start. If I install back original bios - it boot, if I install patched bios - it boot. But after this app - it doesn't boot.

BIOS POST Code: 87
Pls, help me. I will give you all info you want.
P.S. I have programmator - so I can flash bios)

Hello, did you install rebar patch? I have same motherboard (but z10pe-d8 WS)


Wrong key inputs on keyboard layouts other than qwerty

Hello,
I use a keyboard layout other than qwerty, and the numbers row doesn't get picked up correctly.
Also, it doesn't pick up capital letters either.
I'm on MacOs, can probably test on my linux machine as well.
Here's what happens with the numbers row:
If I hit where the <S-1> is on a qwerty, I see the qwerty's <S-1> output - even though the OS is using a different layout.
If I hit <S-h>, it just hits <h>.
Is there a way around this issue?
update: on my linux, the problem is the same: the shift keys doesn't get picked up correctly. I'd say that's where the problem is coming from.

Found It!
Dunno if reaper's the culprit upon importing, all the lowercase keys had a duplicate keybinding that included the shift+[key].

Found It! Dunno if reaper's the culprit upon importing, all the lowercase keys had a duplicate keybinding that included the shift+[key].

Hi, @AntoineBalaine. May I ask how exactly you went about adjusting every keybind? Did you do it manually?
Cheers

I'm on MacOs, can probably test on my linux machine as well.

hmm, i have noticed this on my m1 macbook as well. however, this wasn't an issue before for me on intel macs.

Found It! Dunno if reaper's the culprit upon importing, all the lowercase keys had a duplicate keybinding that included the shift+[key].

aaah oh wait you solved hmm i have to check this out later myself

also, great to see some ppl in this repo

@deetuned Yes, I updated every key bind manually - both in TCP actions list, and in midi editor's list.

does it reload keys again with the faulty duplicate shift keys?
@molleweide  yes, it does.
I'm working on a fork - reaper-keys is sooo great, I've been loving every minute with it.

it fore sure, is a recent problem because everything has worked flawlessly before
dunno… Latest reaper's fault, or my system's fault, maybe…?

yeah im starting to think that it might have something to do with the latest reaper versions for sure.

I'm working on a fork - reaper-keys is sooo great, I've been loving every minute with it.

awesome! what kind of music do you use rk for?

I want to do some electronic stuff, but I'm still in the midst of getting comfortable with it.

anything from classical composing to hardstyle lol. im down for chatting about rk anytime!

Very good, let's do it! My email's in my branche's git log. Let's set a time and chat!
Cheers

I submitted a forum post for this. @AntoineBalaine please tell me if Im missing something in the post that should be included.
https://forum.cockos.com/showthread.php?p=2688754#post2688754

I also posted the forum question into Chat GPT without any real results other than it thinking that there could have been some change to how reaper treats assigning command IDs to upper case key combinations.

Dunno… I saw your posts, thanks for looking into the topic.
I have an updated keymap on my mapCleanUp branch , maybe we can see if it's worth merging it and call it a day - and I mean merging only the keymap changes. Would you be able to give it a try and see if it's working first, though?
This might be a linux-only issue, or because I don't use a qwerty layout, or whatever. I don't know if it's worth running in circles about: most users who come across the problem are likely to pop up on the forum or over here…
I've also noticed that reaper got some bugs in the current version - time selections going out-of-sync with the grid, amongst others. At first, I spent some time trying to figure whether it was an rk-issue (since it breaks BigItem movements in the TCP), until I saw people mentioning similar problems on Reaper's pre-release forum. From what it looks like, Cockos is working on a release with bug-fixes for early July.

Yeah ill try the kmap and get back. Ig if it works to export that keymap and import it with correct shift keys then this only had to be done once. Hmmm.

We should find the version manager and figure out which versions are best for rk

This might be a linux-only issue, or because I don't use a qwerty layout, or whatever. I don't know if it's worth running in circles about: most users who come across the problem are likely to pop up on the forum or over here…

No its both linux and macos. Same issue on intel and m1.
And the keyboard layout is not the issue either. Both i and gwatcha are using super wierd custom keyboards and layouts and it used to work fine.
Qmk/kmonad,etc..

Oh wait. I just realized i have to try one other thing as well. Ill come back to here later

now some ppl in the reaper forum have started replying so hopefully we'll get to the bottom with this.
I haven't yet tried your keymap.
Have you tried exporting your updated keymap and then importing it again to see if it works on your own machine?

Yes. It works just fine when I swap it for the defaults and then re-import - on both macOS and Linux.

Allright, so I tried importing your keymap reaper-keys-unix from the branch you linked and the issue still persists for me.
When re-importing, did you do that on macos or linux?

Here are some docs on how keymaps work
https://mespotin.uber.space/Ultraschall/Reaper-Filetype-Descriptions.html#Reaper-kb.ini
Im getting help from Mespotine in the devs forum.
If we resolve the issue I'll try to rewrite the generator script.

@AntoineBalaine can you please try this keymap on your Linux machine and report back:
https://mespotin.uber.space/stuf/reaper-keys-unix-test.ReaperKeyMap

I just figured out where the problem seems to be. You can read the progress on the issue under the forum thread where I tried to document my process.

https://forum.cockos.com/showthread.php?p=2689605#post2689605


Linux compilation error - librocksdb.so.5.10

Hello, I'm compiling on Ubuntu 16.04 64-bit.
I follow the install instructions and at the command:
./scripts/build/daedalus-bridge.sh
I get the following error:
cardano-sl-db-1.0.3: build (lib)
Preprocessing library cardano-sl-db-1.0.3...
[ 5 of 13] Pos.DB.Pure
<command line>: can't load .so/.DLL for: /home/slex29/programs/cardano-sl/.stack-work/install/x86_64-linux/lts-9.1/8.0.2/lib/x86_64-linux-ghc-8.0.2/libHSrocksdb-haskell-1.0.0-ASh627zSIahH8QxTbLrgnb-ghc8.0.2.so (librocksdb.so.5.10: cannot open shared object file: No such file or directory)

I have compiled rocksdb with make shared_lib. When I use the command ls /usr/local/lib/librocksdb* -s  I get the following:
373588 /usr/local/lib/librocksdb.a
0 /usr/local/lib/librocksdb.so.5
109548 /usr/local/lib/librocksdb.so.5.10.0
0 /usr/local/lib/librocksdb.so
0 /usr/local/lib/librocksdb.so.5.10

do you need the user local version or can you not use the standard package?  I am using librocksdb5.8_5.8.7-1_amd64.deb from debian sid on debian, maybe you can find a deb for ubuntu?


Assumes AffectedSOPClass is returned in C-ECHO-RSP

Hi,
pynetdicom assumes that AffectedSOPClass is returned in a C-ECHO-RSP. However, it seems that at least mirth does not return AffectedSOPClass in its response, making pynetdicom unable to properly ping mirth, throwing the following trace:
File "pinger.py", line 38, in
st = assoc.VerificationSOPClass.SCU(1)
File "/usr/local/lib/python2.7/site-packages/netdicom/SOPclass.py", line 60, in SCU
ans, id = self.DIMSE.Receive(Wait=True)
File "/usr/local/lib/python2.7/site-packages/netdicom/DIMSEprovider.py", line 75, in Receive
return tmp.ToParams(), tmp.ID
File "/usr/local/lib/python2.7/site-packages/netdicom/DIMSEmessages.py", line 248, in ToParams
tmp.AffectedSOPClassUID = self.CommandSet[(0x0000,0x0002)]
File "/usr/local/lib/python2.7/site-packages/pydicom-0.9.9-py2.7.egg/dicom/dataset.py", line 277, in getitem
data_elem = dict.getitem(self, tag)
KeyError: (0000, 0002)
the quick fix is to change AffectedSOPClassUID to optional, in DIMSEmessages.py line 247:
    if self.CommandSet.has_key((0x0000,0x0002)):
        tmp.AffectedSOPClassUID = self.CommandSet[(0x0000,0x0002)]

I will file a bug with Mirth in parallel, but I believe pynetdicom should be more lenient.

Looks like this is affecting more SOP classes and more keys like AffectedSOPInstanceUID. This had better be a more generic thing rather than just those classes.


User Experience Improvement and Persistent Data Storage

Working on getting the user experience fine-tuned so that when the user changes a selection and closes the app and reopens it they will default to the previous selection. This should save folks time and make them more likely to want to share data and give them some peace of mind that their metadata is backed up when using the app in offline mode.

[x] Create workflow for saving user inputs on all buttons and switches.
[x] Backup that data in the app.
[x] Backup the data on dropbox.
[ ] Return most recent user selection when they open up the app.
[x] Reorganize folders in dropbox to have user id as the folder name and all data for the user within them.

#13 now has support for the last part.


Limitations of MIOpen Deconvolution

What are possible combinations of datatypes and format tags that MIOpen Deconvolution supports?
Do limitations of Convolution apply for Deconvolution as well? Please let us know if there are any references.
How to test the combination of Deconvolution using the MIOpen driver?

Do limitations of Convolution apply for Deconvolution as well?

Yes.

How to test the combination of Deconvolution using the MIOpen driver?

Use -m trans and engage --trans_output_pad_d/w/h as needed. Run ./bin/MIOpenDriver conv --help for more info.

I have tried the below tensor inputs for both Convolution and Deconvolution (ref attachment).
Src:
miopenStatus_t miopenSetTensorDescriptor(miopenTensorDescriptor_t, miopenDataType_t, int, int *, int *){
MIOpen(HIP):    tensorDesc =
MIOpen(HIP):    dataType = 1
MIOpen(HIP):    nbDims = 4
MIOpen(HIP):    dim.values = { 256 96 1 55 }
MIOpen(HIP):    stride.values = { 5280 55 55 1 }
MIOpen(HIP): }
Weight:
miopenStatus_t miopenSetTensorDescriptor(miopenTensorDescriptor_t, miopenDataType_t, int, int *, int *){
MIOpen(HIP):    tensorDesc =
MIOpen(HIP):    dataType = 1
MIOpen(HIP):    nbDims = 4
MIOpen(HIP):    dim.values = { 3 96 1 11 }
MIOpen(HIP):    stride.values = { 1056 11 11 1 }
MIOpen(HIP): }
Dst:
miopenStatus_t miopenSetTensorDescriptor(miopenTensorDescriptor_t, miopenDataType_t, int, int *, int *){
MIOpen(HIP):    tensorDesc =
MIOpen(HIP):    dataType = 1
MIOpen(HIP):    nbDims = 4
MIOpen(HIP):    dim.values = { 256 3 1 227 }
MIOpen(HIP):    stride.values = { 681 227 227 1 }
MIOpen(HIP): }
While Convolution is running fine and Deconvolution exited with "Invalid filter channel number" MIOpen error.
Please let us know if I'm missing something in understanding the limitations of Deconvolution.
Thanks in advance!
Deconv_test.log
Conv_test.log

https://medium.com/apache-mxnet/transposed-convolutions-explained-with-ms-excel-52d13030c7e8
I think you need to use use CKYX layout instead of KCYX for weights (i.e. {96 3 1 11})

Thanks for the suggestion.

@AryanKarumuri Please close the issue if it is resolved.


Don't import tuf.log in init.py so importing tuf.conf doesn't start logging

:+1: The log file is no longer created if tuf.conf is imported:

(tuf2.7)$ ls tuf.log
ls: cannot access tuf.log: No such file or directory
(tuf2.7)$ python
Python 2.7.6 (default, Mar 22 2014, 22:59:56)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import tuf.conf
>>>
(tuf2.7)$ ls tuf.conf
ls: cannot access tuf.conf: No such file or directory

I think test_init.py will now have to directly import tuf.log though, otherwise a warning is raised about a missing handler.
(tuf2.7)$ python test_init.py
No handlers could be found for logger "tuf.test_keys"
..
----------------------------------------------------------------------
Ran 2 tests in 0.000s

OK

Note that we incorrectly named the logger for this module.  It should be tuf.test_init instead of tuf.test_keys

Shall I do that on this PR or will you do it?

I've created a pull request to fix the issues in my last reply.  I'll go ahead and merge your PR.  Please let me know if #308 is not working as intended.


Registry operator: Avoid restarts when token file changes

Client-go sets the token file location when using in-cluster config:
https://github.com/openshift/cluster-image-registry-operator/blob/39ea65a1ea93043e8b3ec156fde719760a807e1f/vendor/k8s.io/client-go/rest/config.go#L540
This in turn results in the token-file being periodically live-reloaded,
without needing to restart the controller:
https://github.com/openshift/cluster-image-registry-operator/blob/39ea65a1ea93043e8b3ec156fde719760a807e1f/vendor/k8s.io/client-go/rest/config.go#L75-L77
/assign @csrwng

What this PR does / why we need it:
Which issue(s) this PR fixes (optional, use fixes #<issue_number>(, fixes #<issue_number>, ...) format, where issue_number might be a GitHub issue, or a Jira story:
Fixes #
Checklist

[ ] Subject and description added to both, commit and PR.
[ ] Relevant issues have been referenced.
[ ] This change includes docs.
[ ] This change includes unit tests.

/lgtm
/retest-required


Fasta cannot parse native top-level functions

import 'dart-ext:x';
int m(a) native 'string';

library;
import self as self;

static method #main() → dynamic {
  throw "file:///test.dart:2:10: Error: Expected a function body or '=>'.\nint m(a) native 'string';\n         ^";
}

We need to decide whether we keep ability to parse native top-level functions in Analyzer and Fasta, or we remove it everywhere.

Ah, that's a good guess.
Yes, we use NoneTarget in analyzer's KernelContext.
This CL will switch it to a target that enables VM's native.


Update to latest rom

See individual commits for more info.
After this PR rom-csv works with ROM master.

I guess this can be closed?

Oh, yes it is, thanks :)
Closing


Add @azure/cosmos package to the set of artifacts we can ship.

This PR allows the js - cosmos pipeline to deploy @azure/cosmos package.

/azp run js - core


Social share buttons only work for whatsapp on mobile

Currently, facebook and twitter share buttons don't work on mobile. Not sure why - work fine on desktop and in dev tools mobile emulation, sigh. This should be fixed and also added into a test.

Done, turned out this was just local to a pop up blocker setting on my phone 🤦🏻


Add gene type

Context
See the conversation in cell-science-census.
Reference
This is the biotype in ENSEMBL which is the gene_type in the GENCODE GTF.

From Sara Simmonds

Direct integration of gene type metadata in CELLxGENE
Value
Data richness and biological context - including gene type metadata (e.g., protein coding, long non-coding RNAs) gives more biological context for genes (e.g., mitochondria, gene regulation) and can aid in quality control, data processing and analysis.
Support reproducibility and provide consistency - including gene type metadata supports consistent and reproducible data workflows for users and across users. Some users might not know where to get gene type metadata from on their own or users might use different sources creating potential inconsistency.

See https://github.com/chanzuckerberg/single-cell-curation/issues/960#issuecomment-2297274065
Same run should pick up these changes after an implementation is written in gene_processing.py


Passing arrays as clauses

While trying to pass an array as a single argument to a where clause, I had to wrap the array in another array, so it would be unwrapped and used correctly.
var array = ["a" "b"];
squel.select().from("table")
.where("field = ANY(?)", [array]); // array wrapped in array

Does squel support using arrays as arguments in any other ways?

Yeah this is due to Squel needing to be able to handle IN queries, e.g.
squel.select().from('scores').where('a IN ?', [1,2,3])

// SELECT * FROM scores WHERE a IN (1, 2, 3)

Yeah, that example in the docs (at the end of the parameters section) is what led me to the "array-wrapped-in-an-array" solution.
Could you add an example for this to the docs please?
I imagine a new method squel.array(array) (like squel.str(str)) would be overkill?
PS. thanks for the squel lib

Sure, will do. Thanks.

And for those like us stuck using v4 of Squel, I had to create a custom wrapper so I could continue passing arrays as a single where clause:
var array = ["a" "b"];
squel.select().from("table")
.where("field = ANY(?)", new ArrayWrapper(array)); // custom value type

// squel config (prerequisite for above code)
squel.registerValueHandler(ArrayWrapper, unwrapArray);
function ArrayWrapper(arr) {
    this.arr = arr;
}
function unwrapArray(wrapper) {
    return wrapper.arr;
}

& here's the ArrayWraper JSDoc for anyone interested:
/**
 * Wraps an array to be unwrapped & used as is in a SQL query via Squel. e.g.:
 *
 * squel.select().from('table').where("field = ANY(?)", sqlUtils.wrap([1, 2, 3])
 * // SELECT * FROM table WHERE field = ANY(ARRAY[1, 2, 3])
 *
 * This is necessary to circumvent Squel repeating arguments in arrays for IN clauses. e.g.:
 *
 * squel.select().from('table').where('field IN ?', [1, 2, 3])
 * // SELECT * FROM table WHERE field IN (1, 2, 3)
 *
 * @param {array} arr an array
 * @constructor
 */

thanks @DJDaveMark


Allow .. on !Tar command

!Tar command now checks if the url starts by a dot and consider it's on the current folder instead of trying to download the tarball, but when trying to use a tarball in a parent directory I get the next error:
tar: can't open '/work/../nodeos-barebones/out/latest': No such file or directory
ERROR:vagga::builder: Error building container "barebones": step Tar(TarInfo { url: "../nodeos-barebones/out/latest", sha256: None, path: "/", subdir: "" }) failed: Tar exited with status: exited with code 1
ERROR:vagga::wrapper: Error executing _build: Builder exited with code 1
Command <Command "/proc/self/exe" ("vagga_wrapper") "_build" "initramfs"; environ: {"VAGGA_USER_HOME"="/home/piranna","RUST_LOG"="warn","TERM"="xterm",}; uid_map=[UidMap { inside_uid: 0, outside_uid: 1000, count: 1 }, UidMap { inside_uid: 100000, outside_uid: 100000, count: 65536 }]; gid_map=[GidMap { inside_gid: 0, outside_gid: 1000, count: 1 }, GidMap { inside_gid: 100000, outside_gid: 100000, count: 65536 }]> exited with code 124

Seems build process is itself confined and can't access to files outside current folder, isn't it?

Seems build process is itself confined and can't access to files outside current folder, isn't it?

Sure. This is important part of vagga design. May be fixed by #89

Seems build process is itself confined and can't access to files outside current folder, isn't it?
Sure. This is important part of vagga design. May be fixed by #89

Make sense. The solution I'm currently doing is to copy the tarballs of previous layers to the current one so they are available in the current folder, maybe I could do a symlink too. I'm trying to think a solution that could vagga implement but don't find anything... :-/

You can do a hardlink. But you can't do a symlink. Symlinks works like paths with files. And containerization (i.e. chroots) is specifically prevents to read things symlinked from outside of the container.
You can also use bind mount sudo mount --bind ../nodeos-barebones/out ./out

Since I would need to re-create the hardlinks each time since I'm removing the files from their original places, I decided to copy them to their place explicitly.

Probably nothing to do. May be better with #89


Use of UniquePersonList

I am unsure what UniquePersonList does. We need to refactor it to include Patients if needed. If UniquePersonList is sufficient, delete UniqueDoctorList.

UniquePersonList is sufficient, delete UniqueDoctorList.

UniquePersonList is a generic class that can be used with any child class of Person. Useful for future developers that want to implement more Person types and have it work with UniquePersonList


List of all Traits with attributes and commands

I'm looking for the full list of traits with attributes.
Obviously, I can get the list of trait from the developer's page:  developers.google.com/assistant/smarthome/traits
but I need to click on individual traits to get the list of associated attributes and commands?
I'm looking for a place where I can get all of them in one shot.
ideally I would like them in a JSON format.
Does anyone have this?

What is your intention with this list?

I have some powershell scripts that scan the Google documentation and generate some enum classes for my implementation. It'll get you DeviceTypes and DeivceTraits that each supports... but not commands. Unfortunately the Google doc site doesn't 100% follow a single standard, so I was never able to get it to grab the command list in a reliable way.
https://github.com/i8beef/HomeAutio.Mqtt.GoogleHome/blob/master/src/generateDeviceClasses.ps1
https://github.com/i8beef/HomeAutio.Mqtt.GoogleHome/blob/master/src/generateTraitClasses.ps1
Note @Fleker I would love to see a structured document published that we could more easily generate this sort of stuff from, ideally one with Google Devices and the Traits they support, and one of Traits and the Attributes, Commands, and State they support... But that's just a selfish desire for something that would make the lives easier for those of us supporting FULL Google Home support across all device types (it got too hard to manually keep up to date with your full list of device types / trait changes, so I had to switch over to generating as much as I could from documentation). Bonus points for specifying "required" / "required if X" information in that doc somehow so we can generate validations....

I really like that idea as it's currently impossible to get notified about changes. I always need to click through all traits to see if some new stuff is there, like new attribute, new error option, etc.
Please provide the documentation in some sort of machine readable format (e.g. json). Thx!

We recently started working on this repository: https://github.com/actions-on-google/smart-home-schema which could provide the basis of what was requested here.
Let's continue the discussion there (since this repo is more about the Node.js backend for the sample).


Add population data missing for locality in Province 2

Our data set for population is missing data for one locality in Province 2. The data are available in our "Federal Data" in https://github.com/Code4Nepal/data/.
Add the data to the population.sql file.  Please put the data in a logical place -- so that all data for the same district are in rows next to each other. That will make it easier to review in the future.

When you add them, you will add one row for female population and another row for male population. See the current file for examples.
You will need to update the totals for the district for that locality to include these additional data.
You will need to update the totals for Province 2 to include these additional data.
You will need to update the totals for Nepal to include these additional data.

Here are the localities whose data are currently missing.

geo_level
geo_code
name
parent_name
parent_level
parent_code

local
17001
Aaurahi
Dhanusa
district
19

"Aaurahi" in Dhanusa district is "Aurahi Gaunpalika" in the federal population data: https://github.com/Code4Nepal/data/blob/master/Federal Data/753 Local Unit Population and HouseHold/PRADESH 2/Dhanusa/Aurahi Gaunpalika.csv

Completed in #42


Bug: Sidebar + Multiline Input = incorrect handleAutoResize height calculation

React: v15.4.0
React Toolbox: v1.3.4
Description
The handleAutoResize handler for a multiline Input is calculating a larger than expected height value when mounted in a Sidebar whose width is currently transitioning as a result of setting its pinned prop from false to true.
This is because the textarea value is wrapping, due to the width of the Sidebar parent at the time, resulting in the handleAutoResize height calculation being considerably larger than the actual height the element will be when the Sidebar transition is complete.
The following code demonstrates how to reproduce the bug:
import React from 'react';
import { Layout, Sidebar, Button, Input } from 'react-toolbox';

export default class LayoutTest extends React.Component {
    state = {
        sidebarPinned: false,
        value: 'When the sidebar width is transitioned overtime, the child input\'s handleAutoSize function will incorrectly calculate the height due to this long text being wrapped.'
    };

    handleChange = (value) => {
        this.setState({ value });
    };

    toggleSidebar = () => {
        this.setState({ sidebarPinned: !this.state.sidebarPinned });
    };

    render() {
        const buttonLabel = this.state.sidebarPinned ? 'Hide sidebar' : 'Show sidebar';

        return (
            <Layout>
                <Button label={buttonLabel} onClick={this.toggleSidebar} />
                <Sidebar pinned={ this.state.sidebarPinned } width={ 50 }>
                    <div>
                        <Input
                            type="text"
                            value={this.state.value}
                            onChange={this.handleChange}
                            multiline />
                    </div>
                </Sidebar>
            </Layout>
        );
    }
}

Overriding the Sidebar CSS transition-duration property to be 0s results in the calculation of handleAutoResize being correct.
Delaying the call to handleAutoResize in the Input componentDidMount by >=350ms (0.35s being the amount transition-duration is set to for the Sidebar) also results in the calculation being correct.
Both of these solutions feel a bit hacky to me though. Perhaps the Sidebar should be transitioned by a different property instead of width?

Thank you for such a detailed issue! I've tried to reproduce the issue in the latest beta but the components for layout have been rewritten. I think the issue can be solved by debouncing the handleAutoResize but since I couldn't reproduce it it wasn't possible.
Maybe it's just working in the latest so I'm closing. Please, try to reproduce it in the latest version and if it still happens feel free to reopen so I can try to debug it again!


Remove enabling of logToFile for every display manager

Motivation for this change
I don't like to log simultaneously to a file and the journal and I can't understand why services.xserver.displayManager.job.logToFile is set to true for these three display managers.
It is inconsistent, that this option is only set for a subset of all display managers. To disable logToFile I have to use lib.mkForce which is ugly because I am resetting the value to the default.
Things done
I have removed all occurences of services.xserver.displayManager.job.logToFile = true; in every display manager config.

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[x] Fits CONTRIBUTING.md.

Seems reasonable to me, though it might not hurt to mention this change in the 19.03 notes just in case?

Do you think the change in the release notes is okay? I'm not sure about my skills in the english language.. :D

Looking good. Given the mention in the manual I think this is pretty simple to merge into master. Just need to find someone to merge it now...

The first introduction of this was in https://github.com/nixos/nixpkgs/commit/2e088aa2779a91b787a44fd9e3f44a9ef2f2edd9#diff-12d91a9ce5bfb436f1bfb642126ea347R93 (later logsXsession has been renamed to logToFile). I can't find any reason this was done, so I think it's fine to merge this.


User Metadata Download Option

While reviewing the tool, I noticed that the user metadata was not being saved within the account download directory. Since you are already retrieved the Neos User object, I added additional functionality to save the user metadata as well as an opt-in to save save data (exactly the same method as the other types).
User metadata includes standard account information, badges (tags), profile details, Patreon transactions, and other account information. I believe tags include the neosdb id for "custom badges," but I did not check and did not include such functionality to grab these custom badge images.

👍


Optimized modulo operator for Fp61BitPrime

Zero-knowledge proofs use multiplication on this field extensively. For every bit of multiplication output, we perform ~10 multiplications in Fp61BitPrime. That's around $86 \times 10^9$ modulo reduction operations per 35 million MPC multiplications.
My preliminary testing shows ~30% improvement in local runs, will do some Draft runs to confirm

Draft shows 25% improvement: https://draft-mpc.vercel.app/query/view/astir-fang2024-11-04T2255
8 hours before (from #1391)
6 hours with this change


Create another test that starts both a block producer and a relay node

Starting from this test create another one that starts also a relay node.
For now I would say to do this on the same NixOS host. My idea is adding a boolean option to the existing block producer module that when enabled (probably by default it should be true) instead of just creating a systemd service for the block producer it also creates a relay node. Then it makes the relay node able to access the block producer node but blocks it from the external connections.

little trouble is that both nodes boil down to services.cardano-node so need to sth a bit cleverer to run both nodes on one machine

just quickly what i can now think of is:
I wouldn't care to run both nodes on single host, because 1) that's not a real usecase as the whole point of having two nodes was to split block producer from network access completely 2) to just implement it for tests is a waste of time because by the design of cardano-node module you'd probably have to use nixos-containers at which point you're doing something similar to running two machines in tests, so lets just run 2 machines.


KeyPairGeneratorSpec deprecated

https://developer.android.com/reference/android/security/KeyPairGeneratorSpec

ADO Tracking (corpnet reqd):

https://identitydivision.visualstudio.com/Engineering/_workitems/edit/1226180

No code changes necessary. This code path is only used by PoP and the implementation is backwards/forwards compat down to API 18:
https://github.com/AzureAD/microsoft-authentication-library-common-for-android/blob/dev/common/src/main/java/com/microsoft/identity/common/internal/platform/DevicePopManager.java#L1263-L1266

No code changes necessary. This code path is only used by PoP and the implementation is backwards/forwards compat down to API 18:
https://github.com/AzureAD/microsoft-authentication-library-common-for-android/blob/dev/common/src/main/java/com/microsoft/identity/common/internal/platform/DevicePopManager.java#L1263-L1266