VMExit on External Interrupts

Hey,
I am trying to trap external interrupts from the hypervisor-end and my module is as follows:
    vcpu(vcpuid::type id) :
        eapis::intel_x64::vcpu{id}
    {
        ...
        eapis()->add_external_interrupt_handler(
            external_interrupt_handler::handler_delegate_t::create<vcpu,
            &vcpu::external_interrupt_handler>(this)
        );
    }

    bool
    external_interrupt_handler(gsl::not_null<vmcs_t *> vmcs, external_interrupt_handler::info_t &info)
    {
      bfignored(vmcs);
      bfignored(info);
      eapis()->external_interrupt()->disable_exiting();
      return false;
    }

But my system (Linux) hangs each time I do so. Can someone guide me as to what the issue could be?
Also, I am using an old release RC 2.0.2. Please let me know if the release version might have
something to do with this.
Thanks!

The following provides an example of how to do this:
https://github.com/Bareflank/hypervisor/blob/master/bfvmm/integration/arch/intel_x64/vmexit/external_interrupt/trap_all_interrupts.cpp
I think the reason your system is likely hanging is that you are returning
false, which tells the base APIs that you are not handling the external
interrupt, but the base has no way to do this on its own.

Rian

On Mon, Mar 11, 2019 at 7:18 PM Adil Ahmad<EMAIL_ADDRESS>wrote:

Hey,
I am trying to trap external interrupts from the hypervisor-end and my
module is as follows:
vcpu(vcpuid::type id) :
    eapis::intel_x64::vcpu{id}
{
    ...
    eapis()->add_external_interrupt_handler(
        external_interrupt_handler::handler_delegate_t::create<vcpu,
        &vcpu::external_interrupt_handler>(this)
    );
}

bool
external_interrupt_handler(gsl::not_null<vmcs_t *> vmcs, external_interrupt_handler::info_t &info)
{
  bfignored(vmcs);
  bfignored(info);
  eapis()->external_interrupt()->disable_exiting();
  return false;
}

But my system hangs each time I do so. Can someone guide me as to what the
issue could be?
Also, I am using an old release RC 2.0.2. Please let me know if the
release version might have
something to do with this.
Thanks!
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/Bareflank/extended_apis/issues/117, or mute the
thread
https://github.com/notifications/unsubscribe-auth/AFqD43vwnXO4I1oR6108L7qLRZ--Bn75ks5vVwBRgaJpZM4bp3hH
.

 If you don't, the interrupt will be dropped, which will likely cause issues. You can still disable exiting after that, just make sure you actually handle the interrupt you trapped on.

Oh, thanks a bunch for the pointer. The issue was that I wasn't queuing the interrupt and since i am using an old release, I also seem to have to trap on interrupt-windows in order to dump my internal queue of interrupts onto the guest.
Thanks a bunch for the great work!


SCB-909 Add alpha TCC event scanner for timeout and clean up completed globaltx (WIP)

Coverage increased (+0.4%) to 92.124% when pulling 7235686c39fc5eaecb3bb7fb6eaddee6249fb36f on cherrylzhao:master into 6a68423367bcd1acb970bfa25b36f94ca2d58980 on apache:master.


Possible enhancements

Hey!
I've been trying Idaes for a day and found some things that can be improved. The first one occurs when you hover your mouse in the sidebar and it highlights blank lines;

Changing the list format of the ToDos to a text editing one would be better. Instead of filling a form, you could just write and hit enter. Like simple text boxes generated any time you hit enter or something similar. This could improve the workflow or the speed while adding tasks.
The sidebar should show the folder instead of the full path, because if you use any cloud solution, it shows a non-attractive visualization like: C:\Users\%USER%\OneDrive\Documents\%FOLDER%\Idaes
You could move the data shown in the bottom bar to the right, so it doesn't look too long and you can take a quicklook without too much characters. Also, I think there's a typo in the "Messanges" showed in the bottom bar

I will change everything you said after I finished the kanban and calendar view ( Calendar view is finished, kanban is in progress ), except the text flow thing, I would need to recreate everything from todolist and kanban view and its not easy to create. I will add that feature in 1.0.0 stable release.

I now added everything you said, except the todo workflow


refactor: Test Dashboard Layout

🍰 Pullrequest
dito
Todo
find a way to test the transaction updates

Closes. The problem is solved in the Vue Apollo Client PR #701


Tracking updates for CUDA 11.3

CUDA 11.3 was released a while ago. Opening this issue to track roll-out, using the pattern from #47
11.3

[ ]  cudatoolkit
[ ]  nvcc
[ ]  Docker image
[ ]  Windows installer
[ ]  Migrator

Also ref to the docs for adding a new version.
CC @jaimergp

I am working on this now! 💪

Ok, that's it for now, sans the migrator.
We need to wait on cudatoolkit, which will enable docker-images, which will enable nvcc.
The Windows part should be self-sufficient.

We should get https://github.com/conda-forge/cudatoolkit-feedstock/issues/48 done before starting the migrator. Now that we have big packages like pytorch which needs manual compilations, it's a waste of developer time to build them for 11.3 when only packages that depend on nvrtc need to be rebuilt.

Fair enough! I'll comment on the issue then.

Might as well make this a double-issue like #47 was: https://docs.nvidia.com/cuda/cuda-toolkit-release-notes/index.html

I'll add the relevant metadata but we still need to decide whether we split or not (#48).

NVIDIA Docker images were uploaded end of last week. Jaime updated the Docker PR to include 11.4. Those builds seem to work except for the fact that cudatoolkit packages are not available yet


Add dependabot configuration

What does this pull request do?
Adds dependabot config for Gradle and Docker.
What is the intent behind these changes?
We want to make it easier to be aware of out of date dependencies.

I think #12 covers this visibility concern around dependencies.


test label-gun

Alongside a description of your problem/question/feature suggestion, please also include a debug log ID.
You can generate a debug log ID by:

reproducing the problem
selecting the problematic reference(s)
right-clicking them
choose BBT debug log from the popup menu.

If you are sure your question does not relate to items in your library at all, just select any one item in your library.
Please remove this boilerplate before submitting your issue. Thank you.

comment

comment

comment

comment

comment


Add option to JarInfer to deal with signed jars.

[Currently a WIP]
This makes two changes to the JarInfer CLI tool:

By default, the tool will error out when it encounters a signed jar (as per the jarsigner tool), since modifying a signed jar's bytecode and having it still verify is impossible. Users can then exclude such jars from the list of jars sent the tool runs on.
We add an optional flag --strip-jar-signatures for users who explicitly want to run JarInfer on such jars, removing the signature in the process.

This change includes a test, but it might be fragile outside of JDK8 due to the fact that the jarsigner tool's programatic API is not stable. This instability does not affect the JarInfer tool, just the test.
I also might need to clean up some stuff here before it's ready for prime time :)

Committing jar-infer/jar-infer-lib/src/test/resources/testKeyStore.jks is also not my favorite solution, since .jks files will (rightfully) be flagged everywhere as a security risk, and since there is always the risk someone somewhere tries to actually sign stuff with that keystore (All certificate data associated with it is utter nonsense to try and mitigate that possibility)


[Feature] Add streaming ability in chat page using default SDK

Purpose

Currently, on the Chat page, when we ask a question, the system gathers all information (e.g., answer, thought, supportingContent, etc.) and responds to the client through the API.
We are adding streaming capability. Whenever the LLM responds with an answer in a streaming format.

Does this introduce a breaking change?

[x] Yes
[ ] No

Pull Request Type
What kind of change does this Pull Request introduce?

[ ] Bugfix
[x] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Documentation content changes
[ ] Other... Please describe:

How to Test

Get the code

git clone [repo-address]
cd [repo-name]
git checkout [branch-name]
npm install

Test the code

azd up

What to Check
Verify that the following are valid

...

Other Information

hi @IEvangelist @LittleLittleCloud Can you take a look at this PR for adding the streaming feature?
I still have two issues where I’d like your suggestions as well:

A minor UX issue: it displays an incomplete JSON. I’m considering changing the query format to the LLM to expect a different format instead of JSON (because JSON also have some issue with deserialization as well)

I checked the network tab, and the messages are streaming back as expected. However, the UI keeps showing I'm generating an answer... Please wait. until it receives the full response from the server. Only then does it start streaming the response to the frontend.

I'm not sure if this behavior is expected. Could you suggest or comment on which part I should change? Thank you!


[Bugfix] Fix crashing for multimodal when image passed with height == 1

vLLM is failing when the image with Wx1 dimensions are passed. This is because of transformers library throwing exceptions for this cases.
Here is the log that vLLM was crashing:
INFO 10-07 06:31:28 engine.py:288] Added request chat-fadb404935fa4be686a7bb42081775c4.
The channel dimension is ambiguous. Got image shape (1, 499, 3). Assuming channels are the first dimension.
ERROR 10-07 06:31:28 image.py:59] Failed to process image ([<PIL.Image.Image image mode=RGB size=499x1 at 0x7FA654695DF0>])
ERROR 10-07 06:31:28 engine.py:157] ValueError('mean must have 1 elements if it is an iterable, got 3')
ERROR 10-07 06:31:28 engine.py:157] Traceback (most recent call last):
ERROR 10-07 06:31:28 engine.py:157]   File "/usr/local/lib/python3.12/dist-packages/vllm/engine/multiprocessing/engine.py", line 155, in start
ERROR 10-07 06:31:28 engine.py:157]     self.run_engine_loop()
ERROR 10-07 06:31:28 engine.py:157]   File "/usr/local/lib/python3.12/dist-packages/vllm/engine/multiprocessing/engine.py", line 218, in run_engine_loop
ERROR 10-07 06:31:28 engine.py:157]     request_outputs = self.engine_step()
ERROR 10-07 06:31:28 engine.py:157]                       ^^^^^^^^^^^^^^^^^^
ERROR 10-07 06:31:28 engine.py:157]   File "/usr/local/lib/python3.12/dist-packages/vllm/engine/multiprocessing/engine.py", line 236, in engine_step
ERROR 10-07 06:31:28 engine.py:157]     raise e
ERROR 10-07 06:31:28 engine.py:157]   File "/usr/local/lib/python3.12/dist-packages/vllm/engine/multiprocessing/engine.py", line 227, in engine_step
ERROR 10-07 06:31:28 engine.py:157]     return self.engine.step()
ERROR 10-07 06:31:28 engine.py:157]            ^^^^^^^^^^^^^^^^^^
ERROR 10-07 06:31:28 engine.py:157]   File "/usr/local/lib/python3.12/dist-packages/vllm/engine/llm_engine.py", line 1264, in step
ERROR 10-07 06:31:28 engine.py:157]     outputs = self.model_executor.execute_model(
ERROR 10-07 06:31:28 engine.py:157]               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 10-07 06:31:28 engine.py:157]   File "/usr/local/lib/python3.12/dist-packages/vllm/executor/gpu_executor.py", line 130, in execute_model
ERROR 10-07 06:31:28 engine.py:157]     output = self.driver_worker.execute_model(execute_model_req)
ERROR 10-07 06:31:28 engine.py:157]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 10-07 06:31:28 engine.py:157]   File "/usr/local/lib/python3.12/dist-packages/vllm/worker/worker_base.py", line 303, in execute_model
ERROR 10-07 06:31:28 engine.py:157]     inputs = self.prepare_input(execute_model_req)
ERROR 10-07 06:31:28 engine.py:157]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 10-07 06:31:28 engine.py:157]   File "/usr/local/lib/python3.12/dist-packages/vllm/worker/worker_base.py", line 291, in prepare_input
ERROR 10-07 06:31:28 engine.py:157]     return self._get_driver_input_and_broadcast(execute_model_req)
ERROR 10-07 06:31:28 engine.py:157]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 10-07 06:31:28 engine.py:157]   File "/usr/local/lib/python3.12/dist-packages/vllm/worker/worker_base.py", line 253, in _get_driver_input_and_broadcast
ERROR 10-07 06:31:28 engine.py:157]     self.model_runner.prepare_model_input(
ERROR 10-07 06:31:28 engine.py:157]   File "/usr/local/lib/python3.12/dist-packages/vllm/worker/enc_dec_model_runner.py", line 251, in prepare_model_input
ERROR 10-07 06:31:28 engine.py:157]     model_input = self._prepare_model_input_tensors(
ERROR 10-07 06:31:28 engine.py:157]                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 10-07 06:31:28 engine.py:157]   File "/usr/local/lib/python3.12/dist-packages/vllm/worker/model_runner.py", line 1141, in _prepare_model_input_tensors
ERROR 10-07 06:31:28 engine.py:157]     builder.add_seq_group(seq_group_metadata)
ERROR 10-07 06:31:28 engine.py:157]   File "/usr/local/lib/python3.12/dist-packages/vllm/worker/model_runner.py", line 728, in add_seq_group
ERROR 10-07 06:31:28 engine.py:157]     per_seq_group_fn(inter_data, seq_group_metadata)
ERROR 10-07 06:31:28 engine.py:157]   File "/usr/local/lib/python3.12/dist-packages/vllm/worker/model_runner.py", line 660, in _compute_multi_modal_input
ERROR 10-07 06:31:28 engine.py:157]     mm_kwargs = self.multi_modal_input_mapper(mm_data)
ERROR 10-07 06:31:28 engine.py:157]                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 10-07 06:31:28 engine.py:157]   File "/usr/local/lib/python3.12/dist-packages/vllm/multimodal/registry.py", line 126, in map_input
ERROR 10-07 06:31:28 engine.py:157]     input_dict = plugin.map_input(model_config, data_value)
ERROR 10-07 06:31:28 engine.py:157]                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 10-07 06:31:28 engine.py:157]   File "/usr/local/lib/python3.12/dist-packages/vllm/multimodal/base.py", line 279, in map_input
ERROR 10-07 06:31:28 engine.py:157]     return mapper(InputContext(model_config), data, **mm_processor_kwargs)
ERROR 10-07 06:31:28 engine.py:157]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 10-07 06:31:28 engine.py:157]   File "/usr/local/lib/python3.12/dist-packages/vllm/multimodal/image.py", line 56, in _default_input_mapper
ERROR 10-07 06:31:28 engine.py:157]     .preprocess(data, return_tensors="pt") \
ERROR 10-07 06:31:28 engine.py:157]      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 10-07 06:31:28 engine.py:157]   File "/usr/local/lib/python3.12/dist-packages/transformers/models/mllama/image_processing_mllama.py", line 711, in preprocess
ERROR 10-07 06:31:28 engine.py:157]     image = self.normalize(
ERROR 10-07 06:31:28 engine.py:157]             ^^^^^^^^^^^^^^^
ERROR 10-07 06:31:28 engine.py:157]   File "/usr/local/lib/python3.12/dist-packages/transformers/image_processing_utils.py", line 111, in normalize
ERROR 10-07 06:31:28 engine.py:157]     return normalize(
ERROR 10-07 06:31:28 engine.py:157]            ^^^^^^^^^^
ERROR 10-07 06:31:28 engine.py:157]   File "/usr/local/lib/python3.12/dist-packages/transformers/image_transforms.py", line 392, in normalize
ERROR 10-07 06:31:28 engine.py:157]     raise ValueError(f"mean must have {num_channels} elements if it is an iterable, got {len(mean)}")
ERROR 10-07 06:31:28 engine.py:157] ValueError: mean must have 1 elements if it is an iterable, got 3
CRITICAL 10-07 06:31:28 launcher.py:99] MQLLMEngine is already dead, terminating server process

 PR Checklist (Click to Expand)
Thank you for your contribution to vLLM! Before submitting the pull request, please ensure the PR meets the following criteria. This helps vLLM maintain the code quality and improve the efficiency of the review process.
PR Title and Classification
Only specific types of PRs will be reviewed. The PR title is prefixed appropriately to indicate the type of change. Please use one of the following:

[Bugfix] for bug fixes.
[CI/Build] for build or continuous integration improvements.
[Doc] for documentation fixes and improvements.
[Model] for adding a new model or improving an existing model. Model name should appear in the title.
[Frontend] For changes on the vLLM frontend (e.g., OpenAI API server, LLM class, etc.)
[Kernel] for changes affecting CUDA kernels or other compute kernels.
[Core] for changes in the core vLLM logic (e.g., LLMEngine, AsyncLLMEngine, Scheduler, etc.)
[Hardware][Vendor] for hardware-specific changes. Vendor name should appear in the prefix (e.g., [Hardware][AMD]).
[Misc] for PRs that do not fit the above categories. Please use this sparingly.

Note: If the PR spans more than one category, please include all relevant prefixes.
Code Quality
The PR need to meet the following code quality standards:

We adhere to Google Python style guide and Google C++ style guide.
Pass all linter checks. Please use format.sh to format your code.
The code need to be well-documented to ensure future contributors can easily understand the code.
Include sufficient tests to ensure the project to stay correct and robust. This includes both unit tests and integration tests.
Please add documentation to docs/source/ if the PR modifies the user-facing behaviors of vLLM. It helps vLLM user understand and utilize the new features or changes.

Adding or changing kernels
Each custom kernel needs a schema and one or more implementations to be registered with PyTorch.

Make sure custom ops are registered following PyTorch guidelines: Custom C++ and CUDA Operators and The Custom Operators Manual
Custom operations that return Tensors require meta-functions. Meta-functions should be implemented and registered in python so that dynamic dims can be handled automatically. See above documents for a description of meta-functions.
Use torch.libary.opcheck() to test the function registration and meta-function for any registered ops.  See tests/kernels for examples.
When changing the C++ signature of an existing op, the schema must be updated to reflect the changes.
If a new custom type is needed, see the following document: Custom Class Support in PT2.

Notes for Large Changes
Please keep the changes as concise as possible. For major architectural changes (>500 LOC excluding kernel/data/config/test), we would expect a GitHub issue (RFC) discussing the technical design and justification. Otherwise, we will tag it with rfc-required and might not go through the PR.
What to Expect for the Reviews
The goal of the vLLM team is to be a transparent reviewing machine. We would like to make the review process transparent and efficient and make sure no contributor feel confused or frustrated. However, the vLLM team is small, so we need to prioritize some PRs over others. Here is what you can expect from the review process:

 After the PR is submitted, the PR will be assigned to a reviewer. Every reviewer will pick up the PRs based on their expertise and availability.
 After the PR is assigned, the reviewer will provide status update every 2-3 days. If the PR is not reviewed within 7 days, please feel free to ping the reviewer or the vLLM team.
 After the review, the reviewer will put an  action-required label on the PR if there are changes required. The contributor should address the comments and ping the reviewer to re-review the PR.
 Please respond to all comments within a reasonable time frame. If a comment isn't clear or you disagree with a suggestion, feel free to ask for clarification or discuss the suggestion.

Thank You
 Finally, thank you for taking the time to read these guidelines and for your interest in contributing to vLLM. Your contributions make vLLM a great tool for everyone!

Hmm, wouldn't it be better to fix this issue on the HF side? This seems like a bandaid solution.

Hmm, wouldn't it be better to fix this issue on the HF side? This seems like a bandaid solution.

of course, the best way is to fix it from HF side, but for people who can't wait the fix from HF side, they can use this commit I guess.
I've created an issue in HF: https://github.com/huggingface/transformers/issues/34029
Let's track this and maybe upvote/comment in the issue for better prioritization in hf


Fix linter path

Since the closure compiler moved from ant to maven the path to the linter has to be changed too.
This may be considered related to 8b3e2795d3b2d40d67eacc2bfa02c30718eda04c.

Huh, any clue why build on master is still passing even without this change?

Oh... it is broken. It just hasn't run in a while. Thanks.


ability to use standby db to get relations count

Yes, all configurations should be externalized as a matter of course. Having to recompile to change a configuration is really not great.

Looking fwd to this in prod as ability to do read only queries to standby for prod and use it for other cases too.
For externalizing the configuration pls see if we can use this and  do in similar fashion
https://github.com/oneops/oneops/blob/master/oo-commons/src/main/java/com/oneops/config/OneOpsConfig.java#L84


Introduce customer portfolio with Cloud Trade as first customer

Signed-off-by: Tom Kerkhove<EMAIL_ADDRESS>

Word users is probably a bit more fitting instead of customers. Let me look around what other open projects refer to users as

Yeah Prometheus says “some of our users include.”  Like that phrasing or something like it.

Sure, my pleasure!


windows7 build fail

When I use "github.com/andlabs/ui", compile error. However, I do not use "github.com/andlabs/ui", compiled through.
C:\app\go\goroot\pkg\tool\windows_386\link.exe: running g++ failed: exit status 1
C:/app/mingw-w64/mingw-w64/i686-7.2.0-posix-dwarf-rt_v5-rev1/mingw32/bin/../lib/gcc/i686-w64-mingw32/7.2.0/../../../../i686-w64-mingw32/bin/ld.exe: cannot find -loci
collect2.exe: error: ld returned 1 exit status

You need to fix oci8.pc

Thanks, how can I fix it ？

Please modify below to match as your environment.
Libs: -L${libdir} -lclntsh
Cflags: -I${includedir}

Thanks, it does not work, I can not find the reason, so I plan to use "github.com/lxn/walk/".

What? walk is GUI library not database driver.

My point is: when I use gui "github.com/andlabs/ui", I reference go-oci8 compilation errors, but I use other gui libraries (eg "github.com/lxn/walk","github.com/ zserge / webview "), quoted go-oci8 compiler passed. This question is so weird

github.com/jzaikovs/ora
This is written by pure go. It works well at present.

It is not pure Go, it is cgo. It is using https://github.com/rainycape/dl which loads the Oracle instant client which is written in C

It using package syscall.
syscall.NewLazyDLL("oci.dll")

https://github.com/rainycape/dl/blob/1b01514224a1a60a6bcbb0b6b9d3a00ec14ae17f/dl.go#L6

@mattn Can you lock this thread?


How to compile with SIMD?

Im having trouble compiling with SIMD. When trying to compile, I receive the following:
RUSTFLAGS="-C target-feature=+simd128" cargo wasi build --release --package app
<compilation output>
Finished release [optimized] target(s) in 0.09s
  Processing /home/rimi/Documents/example/target/wasm32-wasi/release/app.rustc.wasm
error: failed to process wasm at `/home/rimi/Documents/example/target/wasm32-wasi/release/app.rustc.wasm`
Caused by:
    failed to parse code section
Caused by:
    found type v128
Caused by:
    expected type i32
Caused by:
    The input WebAssembly is invalid

I suspect that there is a need to pass SIMD specific flags to some of the tooling, such as wasm-opt.
However, the same occurs even without the --release flag which should disable running the optimization step, so maybe the error originates from wasm bindgen? Its hard to tell what tool in the toolchain the error is originating from. Same is true for succeeding steps - there is no report on what tool actually succeeded in running
I am trying to target wasmtime and other standalone runtimes.
What kind of flags / steps do I need to take in order to compile with SIMD or is it not supported by cargo-wasi?
I also noted that if I have already compiled without SIMD successfully and then compile with SIMD unsuccessfully and try again, the compilation will simply skip the compilation and other steps altogether and report a success.
The compilation succeeds (as seen in the above error output), sure, but the optimization steps and such that are normally performed afterwards that cause an error are skipped. The files from a previous working non SIMD build are left in the target folder and could be used by a user if they are not careful.
For example:
cargo wasi build --release --package app # Success
RUSTFLAGS="-C target-feature=+simd128" cargo wasi build --release --package app # Error
RUSTFLAGS="-C target-feature=+simd128" cargo wasi build --release --package app # Success, even though shouldnt

Thanks for the report! You're doing all the right things, but the problem is that the dependencies used in the last-published binaries hadn't been updated for the simd proposal as-is today. I just pushed a new version of this tool, can you try that out and see if it fixes the issue for you?

Alas CI is failing, I'll have to fix that before a new version is published.

Ok it should be published now, if you can test the latest version.

to update, I tried the following
cargo wasi self update-check
    Checking for the latest release
error: failed to get successful response from https://crates.io/api/v1/crates/cargo-wasi: 403 Forbidden

I checked with browser if the URL works and it seems to work fine and returns a JSON.
I then continued to try with the normal installation command that worked fine cargo install cargo-wasi.
Afterward, the update command seems to work fine
cargo wasi self update-check
    Checking for the latest release
cargo-wasi v0.1.23 is up-to-date

And as for the SIMD compilation, it seems to work as expected now.

Ok great! I'm gonna close this assuming that error was spurious, but if it keeps cropping up I can try to investigate.


Installation using helm charts doesn't work

Describe the bug
Following the guide, https://github.com/nginxinc/nginx-kubernetes-gateway/blob/main/deploy/helm-chart/README.md I encountered an error.
To Reproduce
$ kind create cluster
Creating cluster "kind" ...
 ✓ Ensuring node image (kindest/node:v1.27.3) 🖼
 ✓ Preparing nodes 📦
 ✓ Writing configuration 📜
 ✓ Starting control-plane 🕹
 ✓ Installing CNI 🔌
 ✓ Installing StorageClass 💾
Set kubectl context to "kind-kind"
You can now use your cluster with:

kubectl cluster-info --context kind-kind

$ kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v0.8.0/standard-install.yaml
customresourcedefinition.apiextensions.k8s.io/gatewayclasses.gateway.networking.k8s.io created
customresourcedefinition.apiextensions.k8s.io/gateways.gateway.networking.k8s.io created
customresourcedefinition.apiextensions.k8s.io/httproutes.gateway.networking.k8s.io created
customresourcedefinition.apiextensions.k8s.io/referencegrants.gateway.networking.k8s.io created
namespace/gateway-system created
validatingwebhookconfiguration.admissionregistration.k8s.io/gateway-api-admission created
service/gateway-api-admission-server created
deployment.apps/gateway-api-admission-server created
serviceaccount/gateway-api-admission created
clusterrole.rbac.authorization.k8s.io/gateway-api-admission created
clusterrolebinding.rbac.authorization.k8s.io/gateway-api-admission created
role.rbac.authorization.k8s.io/gateway-api-admission created
rolebinding.rbac.authorization.k8s.io/gateway-api-admission created
job.batch/gateway-api-admission created
job.batch/gateway-api-admission-patch created
$ helm install my-release oci://ghcr.io/nginxinc/charts/nginx-kubernetes-gateway  --create-namespace -n nginx-gateway
Pulled: ghcr.io/nginxinc/charts/nginx-kubernetes-gateway:0.1.0
Digest: sha256:7a2473543a56073cd72060b8a5bd472ee9149c5e727ac086f558993facb8d2e9
Error: INSTALLATION FAILED: Internal error occurred: failed calling webhook "validate.gateway.networking.k8s.io": failed to call webhook: Post "https://gateway-api-admission-server.gateway-system.svc:443/validate?timeout=10s": dial tcp <IP_ADDRESS>:443: connect: connection refused

Expected behavior
Installation is successful.

Hi @mikolajkniejski , after applying the gateway-api resources, you may have to wait for those Pods to be up and running  (in the gateway-system namespace) before installing nginx kubernetes gateway.

Hi @mikolajkniejski , the webhook is not needed in K8s 1.25 or later. Closing this feel free to reopen if the issue persists.


enable llvm-build for windows

See also https://github.com/wkpark/triton/actions/runs/7043295043 (downloadable artifacts found at the bottom of the page)

[x] supprot windows-latest + python 3.11
[x] Azure upload if available
[x] Upload artifacts for all other builds

LGTM. FYI, I don't think we want to support windows officially but it wouldn't hurt to provide windows LLVM dependencies for those who do.


Threads: binance.exceptions.BinanceAPIException: APIError(code=-1003): Way too many requests; IP banned

I got class with 3 api methods that i call into it:
Common code
"""
client = Client(api_key, api_secret) # CALL ONCE
while True:
   sleep(random.randint(5,10)
   ticker = client.get_ticker(symbol='{}{}'.format(markets[0],markets[1]))
   sleep(1)
   orderbook = client.get_order_book(symbol='{}{}'.format(markets[0],markets[1]),limit=1000)
   sleep(1)
   history = client.get_historical_trades(symbol='{}{}'.format(markets[0],markets[1]),limit=500)
"""

When i use threads(threading.Thread), untill it's less than 3 - all OK. But when it's 3 and more i got
binance.exceptions.BinanceAPIException: APIError(code=-1003): Way too many requests; IP banned until TimeStamp
From documentation:
At the current time Binance rate limits are:
1200 requests per minute
10 orders per second (what mean orders in my case?)
100,000 orders per 24hrs
Is it actual information?
Cause, as i thought, i made 9 requests total (max - 3 methods in 3 threads) at one second(if all random-time sets at one value), and i still got that error. Or it counts not like that? In what moment i get over 1200?
Also i read here https://github.com/binance-exchange/binance-official-api-docs/blob/master/rest-api.md#limits
and here - https://github.com/binance-exchange/binance-official-api-docs/blob/master/errors.md#-1003-too_many_requests in official documentation.
Still don't understand where i get over limits  for my 3 calls (get_ticker, get_order_book, get_historical_trades)? And what max. possible limits for that methods?

Hi @leexxg, the rate limits are based on information from the get_exchange_info() call, you can see the current rate limits there.
Note that some calls have higher weight than others, especially if you're fetching for many symbols at once. Check the official docs for more info https://github.com/binance-exchange/binance-official-api-docs
Otherwise I'd enable debugging on requests and see what is actually sent.
Then hit up the Binance API Telegram group for official support for your issue. https://t.me/binance_api_english


Better way of handling markers in JS

As I mentioned in the other PR I did, I think this would better suited as we won't have tons of unnecessary variables from doing it with the templating system, instead we do it with javascript.

Instead of 25 of those variables + addListeners, we have this:

Merging your latest PR #24 introduced conflicts with this PR. Can you please adjust this PR? it will be very welcome to improve the JS code!
Thanks

@rochacbruno I think I've fixed the conflict

@RubenRocha thanks a lot for improvements!
BTW: there is still a lot of ideas to get from #17


Set workspace as floating

Hello Nikita this is an excellent tiling manager, thanks for the hard work.
I was wondering if it's possible to assign a workplace as floating for all windows? For instance setting workplace 0 so that none of the windows snap there.
All best,
Casey.

I thought the same. To map this request to existing concepts it could be that the configuration has a new callback like this
[on-window-change-workspace]
if.workspace_to = "x"
run = "layout float horizontal"


Allow label edits without restarting server

This PR:

changes the LABELS global to a @cached function, so that label
edits can be viewed during development without restarting the server

removes several cached bits stored in the importjson that can be
computed from labels

restores a little bit of previously-deleted code required for that to
work

Codecov Report

Merging #909 (280f858) into main (71a27fd) will increase coverage by 0.02%.
The diff coverage is 86.00%.

:exclamation: Current head 280f858 differs from pull request most recent head 96255db. Consider uploading reports for the commit 96255db to get more accurate results

@@            Coverage Diff             @@
##             main     #909      +/-   ##
==========================================
+ Coverage   79.14%   79.16%   +0.02%
==========================================
  Files         132      132
  Lines        4473     4507      +34
  Branches      568      576       +8
==========================================
+ Hits         3540     3568      +28
- Misses        823      825       +2
- Partials      110      114       +4

Impacted Files
Coverage Δ

src/CreeDictionary/API/search/presentation.py
81.81% <75.00%> (-2.33%)
:arrow_down:

src/CreeDictionary/CreeDictionary/relabelling.py
95.61% <90.47%> (-1.33%)
:arrow_down:

src/CreeDictionary/API/apps.py
50.00% <100.00%> (ø)

src/CreeDictionary/API/search/types.py
92.56% <100.00%> (ø)

...tionary/CreeDictionary/templatetags/relabelling.py
90.90% <100.00%> (+0.58%)
:arrow_up:

src/CreeDictionary/API/search/espt_crk.py
100.00% <0.00%> (ø)

src/CreeDictionary/API/search/test_espt_crk.py
100.00% <0.00%> (ø)

src/CreeDictionary/CreeDictionary/views.py
81.65% <0.00%> (+0.91%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 71a27fd...96255db. Read the comment docs.


perl version generates warnings/errors on single_greather_than.fasta

$ ./biotool.pl ../test_data/single_greather_than.fasta
FILENAME	TOTAL	NUMSEQ	MIN	AVG	MAX
Use of uninitialized value $top in pattern match (m//) at /Library/Perl/5.18/Bio/SeqIO/fasta.pm line 147,  chunk 1.
Use of uninitialized value $L in numeric lt (<) at ./biotool.pl line 67,  line 1.
Use of uninitialized value in addition (+) at ./biotool.pl line 70,  line 1.
Use of uninitialized value $L in numeric lt (<) at ./biotool.pl line 71,  line 1.
Use of uninitialized value $L in numeric gt (>) at ./biotool.pl line 72,  line 1.
Use of uninitialized value in join or string at ./biotool.pl line 86.
../test_data/single_greather_than.fasta	1	0		0	0

This is difficult to fix due to issues in the upstream Perl library.


fix(experimental elaborator): Only add_generics once for each item

Description
Problem*
Resolves an issue introduced by https://github.com/noir-lang/noir/pull/5082 where add_generics would be called multiple times on a given item. Once when declaring the FuncMeta beforehand, and again when defining it afterward. This is an issue because add_generics will give a fresh set of generics each time. This meant the generics in a function's signature weren't the same type variables as the ones used internally, which led to type errors.
Summary*
Additional Context
Down to 400 errors in the stdlib down from 1050
Documentation*
Check one:

[x] No documentation needed.
[ ] Documentation included in this PR.
[ ] [For Experimental Features] Documentation to be submitted in a separate PR.

PR Checklist*

[x] I have tested the changes locally.
[x] I have formatted the changes with Prettier and/or cargo fmt on default settings.

Wrong merge target


Where is optimize_restarts defined?

It is strange that I can call model.optimize_restarts but I cannot see optimize_restarts function in https://github.com/SheffieldML/GPy/blob/devel/GPy/core/gp.py . Where is optimize_restarts defined?

https://github.com/sods/paramz/blob/master/paramz/model.py#L118


⚠ Fetchers has degraded performance

In df45f22, Fetchers ($STATUS_URL) experienced degraded performance:

HTTP code: 200
Response time: 396 ms

Resolved: Fetchers performance has improved in 56685c9.


UNDP-607: fixed modal component converted it static to dynamic

[UNDP-607]: fixed modal component converted it to static to dynamic
In This pr i have done some code refactor for CTA button and also make the modal box dynamic. Here is the latest html markup to use this feature
example code;

Deploying with    Cloudflare Pages
Latest commit: c5fa061
Status:	 ✅  Deploy successful!
Preview URL: https://de3daad1.design-system.pages.dev
View logs

Note: Modal feature in Dummy env is not work here because i push .mdx file later but on actual environment it will work.


ARTEMIS-2738 implement per-acceptor security domains

How does this map into existing simple, roles.properties and user.properties is there a new security_domains.properties or something?

I think the documents need a bit of work, its not very clear to me how to use this, how it works etc and how it maps to JAAS?

How does this map into existing simple, roles.properties and user.properties is there a new security_domains.properties or something?

I'm not entirely sure what you're asking. Everything maps like it always has. There's no new properties files or anything like that.

I think the documents need a bit of work, its not very clear to me how to use this, how it works etc and how it maps to JAAS?

The document is pretty simple because the configuration and function is simple. Just specify the domain you want to use on your your acceptor using the securityDomain URL parameter. Then any client using that acceptor will be secured with that domain. It maps to JAAS just like the standard stuff. The specified securityDomain overrides the broker's configuration which includes the certificate domain.

What tests did you add for AMQP? I wasn't sure I understood correctly,  but it feels like the CoreSession one you added could been Parameterized to use AMQP / Core.

@jbertram there is one real failures on your PR:
org.apache.activemq.artemis.tests.integration.server.SecureConfigurationTest.testSecureDurableSubscriber[2: protocol=OPENWIRE]

@jbertram as a matter of fact you can probably extend this test for your use?

@clebertsuconic, I added some more tests & removed the debug stuff. I couldn't get org.apache.activemq.artemis.tests.integration.server.SecureConfigurationTest#testSecureDurableSubscriber to fail even though I ran it on repeat for awhile.

Ok, I am about to start a testsuite on my box anyways.. I will bring your commit with mine and if everything is ok I will merge it.

LGTM.. merging


Bundle tiller, write ownerreferences, and write status back to CRs (squashed)

(Note: this is the same PR as the previous PR, but with some git trickery to get DCO to pass)
This is a PR that bundles up what we've been working on in the coreos/lostromos fork:
The two major features are highly coupled, without the CR Status changes, lostromos will lose information about the releases it's managing if it is restarted.
OwnerReferences
The helm controller now writes out OwnerReferences on the objects it creates. Owner references are how kubernetes handles GC policy, and are the "source of truth" (versus labels) over what resources are truly owned by another.
To achieve this part, we needed to make a fairly major change to lostromos: Tiller is now vendored. This is so that we can intercept the templated files after helm templating but before they're written into the cluster. This also means we removed the configuration for talking to an external tiller.
It's not feasible to add ownerreferences to incoming templates (template tags can be anywhere in a yaml file, there's no way to know it's safe to add the ownerreferences in a particular part). This same problem prevented us from adding ownerreferences to the tmplctrl - incoming templates are not parseable as separate yaml files, and adding the bookkeeping to keep track of where we can add ownerreferences (such as which files contain top-level kubernetes objects and which are simply included elsewhere, or looking for and parsing multiple kuberentes objects from a single file) would turn the template controller into helm anyway.
You can see where we hook into tiller in helmctrl/engine.go. We add a new engine (tiller-speak for renderer) that wraps the standard renderer and adds ownerreferences pointing back to the lostromos CR.
One annoying side effect of vendoring tiller: we now vendor kubernetes as well, because tiller depends on the internalclientset. This unfortunate bloat can be removed when tiller is updated to depend only upon client-go.
CR Status
The Helm controller now publishes a status block to the custom resource describing the state of the installation process.
The status block contains the following fields:

phase Current phase of the installation
reason Reason for the phase
release Information about the last successful Helm release

Example status:
status:
  lastTransitionTime: '2018-02-12T21:33:15Z'
  lastUpdateTime: '2018-02-13T17:54:55Z'
  phase: Applied
  reason: ApplySuccessful
  release:
    chart:
      metadata:
        apiVersion: v1
        description: A Helm chart for testing stateless app operator
        name: helloworld
        version: 0.2.0
      templates:
        - data: >-
            e3svKiB2aW06IHNldCBmaWxldHlwZT1tdXN0YWNoZTogKi99fQp7ey8qCkV4cGFuZCB0aGUgbmFtZSBvZiB0aGUgY2hhcnQuCiovfX0Ke3stIGRlZmluZSAibmFtZSIgLX19Cnt7LSBkZWZhdWx0IC5DaGFydC5OYW1lIC5WYWx1ZXMubmFtZU92ZXJyaWRlIHwgdHJ1bmMgNjMgfCB0cmltU3VmZml4ICItIiAtfX0Ke3stIGVuZCAtfX0KCnt7LyoKQ3JlYXRlIGEgZGVmYXVsdCBmdWxseSBxdWFsaWZpZWQgYXBwIG5hbWUuCldlIHRydW5jYXRlIGF0IDYzIGNoYXJzIGJlY2F1c2Ugc29tZSBLdWJlcm5ldGVzIG5hbWUgZmllbGRzIGFyZSBsaW1pdGVkIHRvIHRoaXMgKGJ5IHRoZSBETlMgbmFtaW5nIHNwZWMpLgoqL319Cnt7LSBkZWZpbmUgImZ1bGxuYW1lIiAtfX0Ke3stICRuYW1lIDo9IGRlZmF1bHQgLkNoYXJ0Lk5hbWUgLlZhbHVlcy5uYW1lT3ZlcnJpZGUgLX19Cnt7LSBwcmludGYgIiVzLSVzIiAuUmVsZWFzZS5OYW1lICRuYW1lIHwgdHJ1bmMgNjMgfCB0cmltU3VmZml4ICItIiAtfX0Ke3stIGVuZCAtfX0K
          name: templates/_helpers.tpl
        - data: >-
            YXBpVmVyc2lvbjogZXh0ZW5zaW9ucy92MWJldGExCmtpbmQ6IERlcGxveW1lbnQKbWV0YWRhdGE6CiAgbmFtZToge3sgLlZhbHVlcy5yZXNvdXJjZS5uYW1lIH19LWhlbGxvCiAgbGFiZWxzOgogICAgYXBwOiB7eyAuVmFsdWVzLnJlc291cmNlLm5hbWUgfX0taGVsbG8KICAgIGNoYXJ0OiB7eyAuQ2hhcnQuTmFtZSB9fS17eyAuQ2hhcnQuVmVyc2lvbiB8IHJlcGxhY2UgIisiICJfIiB9fQogICAgcmVsZWFzZToge3sgLlJlbGVhc2UuTmFtZSB9fQogICAgaGVyaXRhZ2U6IHt7IC5SZWxlYXNlLlNlcnZpY2UgfX0Kc3BlYzoKICByZXBsaWNhczoge3sgLlZhbHVlcy5yZXNvdXJjZS5zcGVjLnNpemUgfX0KICB0ZW1wbGF0ZToKICAgIG1ldGFkYXRhOgogICAgICBsYWJlbHM6CiAgICAgICAgcmVzb3VyY2U6IHt7IC5WYWx1ZXMucmVzb3VyY2UubmFtZSB9fQogICAgc3BlYzoKICAgICAgY29udGFpbmVyczoKICAgICAgLSBuYW1lOiBoZWxsbwogICAgICAgIGltYWdlOiB7eyAuVmFsdWVzLmltYWdlIH19CiAgICAgICAgcG9ydHM6CiAgICAgICAgLSBjb250YWluZXJQb3J0OiA4MAo=
          name: templates/deployment.yaml
      values:
        raw: |
          image: nginx
    config:
      raw: |
        resource:
          name: cool-app
          namespace: default
          spec:
            size: 3
    info:
      Description: Upgrade complete
      first_deployed:
        nanos: 971502171
        seconds:<PHONE_NUMBER>
      last_deployed:
        nanos: 891188724
        seconds:<PHONE_NUMBER>
      status:
        code: 1
    manifest: |-

      ---
      # Source: helloworld/templates/deployment.yaml
      apiVersion: extensions/v1beta1
      kind: Deployment
      metadata:
        labels:
          app: cool-app-hello
          chart: helloworld-0.2.0
          heritage: Tiller
          release: lostromos-cool-app
        name: cool-app-hello
        ownerReferences:
        - apiVersion: example.com/v1alpha1
          blockOwnerDeletion: true
          controller: true
          kind: ExampleApp
          name: cool-app
          uid: 5545f689-103c-11e8-a9d0-0a24face8626
      spec:
        replicas: 3
        template:
          metadata:
            labels:
              resource: cool-app
          spec:
            containers:
            - image: nginx
              name: hello
              ports:
              - containerPort: 80
    name: lostromos-cool-app
    namespace: default
    version: 169108

The CR status includes the release data for the helm chart. This allows us to completely encapsulate tiller - it doesn't need to write out configmaps or secrets to store the release information. It does mean that you can only "rollback" one release, but this seems to fit better with the operator model anyway.
We know this is a big change, thank your for taking the time to look at it.

Build failed due to golint timeout errors....rerunning now

We talked about this during the Helm Summit some, but the first thing I see that needs to change with regards to the status is namespacing them in some way. We have the concept of a helm prefix today, so that multiple instances of lostrómos can operate on the same custom resource. Status reporting will need to support this as well. For example in our use case with lostrómos we will have multiple versions deployed that watch the same set of custom resources.

For example in our use case with lostrómos we will have multiple versions deployed that watch the same set of custom resources
I believe this goes against our working definition of an Operator, but @ecordell or @ericavonb might be know more.

ping @ecordell or @ericavonb

I'm closing as there has been no update in over a month on this. Feel free to reopen in the future.


v1 Jurisdiction for forms

Is your feature request related to a problem? Please describe.
Feature request.
Describe the solution you'd like

[ ] Need to add one or many jurisdiction drop-downs and the user can choose what they want.
[ ] User can hit "+" and add more jurisdictions for overlapping situations (i.e. both CN & RISO in a given area).
[ ] Jurisdiction percentages should add to 100.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.

email from Val Miller: Jurisdictions have been a long standing thorn with IAPP which was partially solved by enabling the user to select the one - several jurisdictions and associated percentages (to a max of 100% of course) that the IP survey or treatment was covering/being applied to (Micheal W – this is done through eyeballing on site and sometimes adjusted through back in office review).
We CANNOT have a one-to-one relationship only with jurisdiction and force the user to duplicate entries because the infestation spreads through two or more jurisdictions.

IAPP data format:
jurisdictions: [
              {
                jurisdiction_code: survey.Jur1,
                percentage: survey.Jur1pct
              },
              {
                jurisdiction_code: survey.Jur2,
                percentage: survey.Jur2pct
              },
              {
                jurisdiction_code: survey.Jur3,
                percentage: survey.Jur3pct
              }
            ]
           ```


refactor(core): add predicate into cost model

The current rationale for my refactor: change the core API to match the refactor PR while making minimum changes to other crates.


Issue #1317: Kotlin - Add typed variants of Uni.awaitSuspending

Proposal to solve https://github.com/smallrye/smallrye-mutiny/issues/1317
@hantsy - your reference methods awaitSingle* extends a Publisher with potential numerous items, whereas Uni isn't a Publisher and emits at max one item.
That's why I don't consider "single" a good naming.
Happy to hear your opinion.

Check Mono extension and RxJava Maybe extension, they are a good match with Uni right?

you're right, will do.

Suspending this PR because of https://github.com/smallrye/smallrye-mutiny/issues/1325


Use local copy of video to speed up load times and remove tracking

I noticed some slow load times with all the scripts and bloat the Youtube embed player was fetching, and since the video file is relatively small, I think this is the best way to solve that (not to mention privacy benefits for end users).

Hi! Sorry for the late reply but I have to close this pull because the website is currently undergoing a rewrite. Feel free to make this PR on the spring-23-rewrite branch and I can merge it.

Couldn't you just stash this patch on a local branch and squash-merge after the rewrite?


Javascript: Smooth Page Scrolling - Add a "smooth-scroll" class

This could be super useful for things like navbars
http://codepen.io/IamManchanda/pen/VPZMba
Here is the simple javascript ( jquery ) code
$('a.smooth-scroll[href*="#"]:not([href="#"])').click(function() {
  if (location.pathname.replace(/^\//, '') == this.pathname.replace(/^\//, '') && location.hostname == this.hostname) {
    var target = $(this.hash);
    target = target.length ? target : $('[name=' + this.hash.slice(1) + ']');
    if (target.length) {
      $('html, body').animate({
        scrollTop: target.offset().top
      }, 1000);
      return false;
    }
  }
});

Source: CSS Tricks

I believe we have discussed this in the past and have settled on recommending other plugins for this. CC @mdo @cvrebert

I believe we have discussed this in the past and have settled on recommending other plugins for this. CC @mdo @cvrebert

@hnrch02 oh i see, is there any issue with the code ?
I am asking this because why would you need a plugin if you can do a thing in a 10 liner code

@hnrch02 oh i see, is there any issue with the code ?
I am asking this because why would you need a plugin if you can do a thing in a 10 liner code

I assumed you were suggesting that we add a plugin à la ScrollSpy or Tooltip for this functionality. Then we would need to write a lot more than 10 lines of code, since it would need extensive unit testing too 😄

I assumed you were suggesting that we add a plugin à la ScrollSpy or Tooltip for this functionality. Then we would need to write a lot more than 10 lines of code, since it would need extensive unit testing too 😄

nah nah only basic smooth scroll ... basic landing page functionality specifically for navbars
For example: https://blackrockdigital.github.io/startbootstrap-scrolling-nav/

nah nah only basic smooth scroll ... basic landing page functionality specifically for navbars
For example: https://blackrockdigital.github.io/startbootstrap-scrolling-nav/

Passing on this.

Passing on this.

Any specific Reasons @mdo.. Simple code ... No plugins

I'm not a huge fan of hijacking or modifying scrolling, don't feel it's a useful application of our time, doesn't help solve any glaring issues with the framework, etc. I'd sooner see us remove or simplify things than add more to the core.
Just because we can add something, doesn't mean we should.

Generally (depending on technique) messing with scrolling can also negatively impact things like hardware acceleration.
This is certainly something that would be more of a fit as a plugin/extension/addon, rather than core bootstrap.

Ohk no issue @mdo sir @patrickhlauke sir

But I have to say with most systems and mobile's being new and updated these days, I don't see  " that " much hardware acceleration issues
@patrickhlauke

@mdo @patrickhlauke it hurts me to say but the bootstrap team has ignored my relevant ideas
Relevant because this same feature, today morning i was able to integrate(merge) in an another framework succesfully and quite happily and sorry but bootstrap lacks behind when it comes to latest web .... specially javascript!
Sorry for being a bit sour ....  didn't meant to!


MSID: 88953 Version: 1 DOI: 2<IP_ADDRESS>2364

"msas": "Neuroscience"
"msid": "88953"
"preprintDoi": "10.1101/2<IP_ADDRESS>2364"
"Reviewed Preprint posted": "2023-09-27"
"Sent for peer review": "2023-05-31"
"Posted to bioRxiv": "2023-01-24" (link: "Go to bioRxiv": "https://www.biorxiv.org/content/10.1101/2<IP_ADDRESS>2364v2/")
[PLACE PDF URL HERE WHEN AVAILABLE] See step 7
Step 1. Inform bioRxiv
Who can help: @QueenKraken, @nlisgo, @scottaubrey

[x] Manuscript is in data hub index (https://data-hub-api.elifesciences.org/enhanced-preprints/docmaps/v1/by-publisher/elife/get-by-manuscript-id?manuscript_id=88953)

Step 2. Create preview of manuscript
Who can help: @fred-atherden, @nlisgo, @scottaubrey

[ ] Manuscript is available for preview (https://prod--epp.elifesciences.org/preview/88953/v1)

[ ] Trigger import for staging: https://prod--epp.elifesciences.org/import
[ ] Sync data folder with s3://prod-elife-epp-data/data
[ ] Trigger import for staging: https://staging--epp.elifesciences.org/import
[ ] Sync data folder with s3://staging-elife-epp-data/data
[ ] Pull request created to enhanced-preprint-data
[ ] Post link to pull request below

Pull request: [PLACE LINK TO PULL REQUEST HERE]
Detailed instructions: https://github.com/elifesciences/enhanced-preprints-data#add-a-manuscript
Step 3: Awaiting public reviews and QC
Who can help: Production team

[ ] QC passed or QC not needed

Who can help: Editorial team

[x] Public reviews available on sciety (https://sciety.org/articles/activity/10.1101/2<IP_ADDRESS>2364)
[x] Manuscript and editor details at the top of this issue (Supplied by Editorial team)
[ ] Reviews returned by EPP - https://prod--epp.elifesciences.org/api/reviewed-preprints/88953/v1/reviews

Example
"msas": "Genetics and Genomics", "Neuroscience"
"msid": "84628"
"version": "1"
"preprintDoi": "10.1101/2<IP_ADDRESS>4241"
"articleType": "Reviewed Preprint"
"status": "Published from the original preprint after peer review and assessment by eLife."

"Reviewed Preprint posted": "2023-01-02"
"Sent for peer review": "2022-10-28"
"Posted to bioRxiv": "2022-11-21" (link: "Go to bioRxiv": "https://www.biorxiv.org/content/10.1101/2<IP_ADDRESS>4241v1")

Editors:
Reviewing Editor
Michael B Eisen
University of California, Berkeley, United States
Senior Editor
Michael B Eisen
University of California, Berkeley, United States

Step 4: Modify manuscripts.json (no PDF)

[ ] Reviewed preprint is published to EPP (https://elifesciences.org/reviewed-preprints/88953)

[ ] Pull request created to enhanced-preprint-client
[ ] Pull request switched to "Ready for review"
[ ] Post link to pull request below

[ ] Request that a doi be registered

[ ] Post link to #enhanced-preprint comment below

Pull request: [PLACE LINK TO PULL REQUEST HERE]
#enhanced-preprint comment thread: [PLACE LINK TO COMMENT HERE]

Instructions to modify manuscripts.json

Visit: https://github.com/elifesciences/enhanced-preprints-client/actions/workflows/publish-manuscript.yaml
Click: Run workflow
Complete the form and click "Run workflow"
A successful run should result in a new pull request at https://github.com/elifesciences/enhanced-preprints-client/pulls
Open the pull request and click the "Ready for review" button to trigger tests
Once the tests pass and you are happy with the changes the PR can be merged

Example pull request: https://github.com/elifesciences/enhanced-preprints-client/pull/334/files
Once the pull request is merged in it should be available a few minutes later.

Request that a doi
Post the following in #enhanced-preprint:
@Fred can you register a doi for https://elifesciences.org/reviewed-preprints/88953

Step 5: Awaiting search reindex

[ ] Reviewed preprint is avaliable on journal homepage (https://elifesciences.org)

The search reindex is triggered once an hour. We need the reviewed preprint to be indexed as the search application serves the journal homepage.

Additional info
If needed, the jenkins pipeline to reindex search can be triggered sooner.
https://alfred.elifesciences.org/job/process/job/process-reindex-reviewed-preprints/

Step 6: Published! Request PDF generation

[ ] Post the link to the PDF url at the top of the issue

[ ] Post link to #enhanced-preprint comment below

#enhanced-preprint comment thread: [PLACE LINK TO COMMENT HERE]
Post the following to the #enhanced-preprint on slack:
@Ryan Dix-Peek please can you generate a PDF for https://elifesciences.org/reviewed-preprints/88953

Step 7: Introduce PDF to data folder and git repo

[ ] PDF is avaliable at https://github.com/elifesciences/enhanced-preprints-data/raw/master/data/88953/v1/88953-v1.pdf

Detailed instructions: https://github.com/elifesciences/enhanced-preprints-data#add-a-pdf
Step 8: Add PDF url to manuscripts.json

[ ] Reviewed preprint PDF is available for download (https://prod--epp.elifesciences.org/reviewed-preprints/88953)

[ ] Pull request created to enhanced-preprint-client
[ ] Pull request switched to "Ready for review"
[ ] Post link to pull request below

[PLACE LINK TO PULL REQUEST HERE]

Instructions to add PDF url to manuscripts.json

Visit: https://github.com/elifesciences/enhanced-preprints-client/actions/workflows/add-pdf-url-to-manuscript.yaml
Click: Run workflow
Complete the form and click "Run workflow"
A successful run should result in a new pull request at https://github.com/elifesciences/enhanced-preprints-client/pulls
Open the pull request and click the "Ready for review" button to trigger tests
Once the tests pass and you are happy with the changes the PR can be merged

Example pull request: https://github.com/elifesciences/enhanced-preprints-client/pull/397/files
Once the pull request is merged in it should be available a few minutes later.

Step 9: Done!

[ ] Kettle is on!

elocations missing from the reference list


SelectionArea: drag from side padding , support select a line.

Use case
when use SelectionArea, select from side padding. code is like:
SelectionArea(
  child: Container(
  decoration:BoxDecoration(color: Colors.white, borderRadius: BorderRadius.circular(6)),
  constraints: BoxConstraints(maxWidth: 500),
  padding: EdgeInsets.symmetric(horizontal: 12, vertical: 8),
  child: Text('this is line1\nthis is line2\nthis is line3\nthis is line4'),
  ),
),

this video show : drag from side padding , will select from start or end. can't select this line:
https://github.com/user-attachments/assets/7532a906-e13a-4dbb-954c-74f5efc92f6b
Proposal
I hope selection like this:
https://github.com/user-attachments/assets/cc22a32c-4232-4317-8982-8ed261e5d59e

@toly1994328
What platform are you seeing / running this on ?
Can you also check if your case is similar to https://github.com/flutter/flutter/issues/139890 ?

Platform is Windows. It's not similar，here SelectionArea with inner padding.
flutter doctor -v
Flutter assets will be downloaded from https://storage.flutter-io.cn. Make sure you trust this source!
[✓] Flutter (Channel stable, 3.24.3, on Microsoft Windows [版本 10.0.22631.4317], locale zh-CN)
• Flutter version 3.24.3 on channel stable at D:\SDK\Flutter\flutter3.24.0\flutter
• Upstream repository https://github.com/flutter/flutter.git
• Framework revision 2663184aa7 (4 weeks ago), 2024-09-11 16:27:48 -0500
• Engine revision 36335019a8
• Dart version 3.5.3
• DevTools version 2.37.3
• Pub download mirror https://pub.flutter-io.cn
• Flutter download mirror https://storage.flutter-io.cn
[✓] Windows Version (Installed version of Windows is version 10 or higher)
[!] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
• Android SDK at D:\SDK\Android
• Platform android-34, build-tools 34.0.0
• ANDROID_HOME = D:\SDK\Android
• Java binary at: D:\Soft\IDE\AndroidStudio2024\jbr\bin\java
• Java version OpenJDK Runtime Environment (build 17.0.11+0--11852314)
✗ Android license status unknown.
Run flutter doctor --android-licenses to accept the SDK licenses.
See https://flutter.dev/to/windows-android-setup for more details.
[✓] Chrome - develop for the web
• Chrome at C:\Program Files\Google\Chrome\Application\chrome.exe
[✓] Visual Studio - develop Windows apps (Visual Studio Community 2022 17.7.4)
• Visual Studio at D:\Soft\IDE\VS\Community
• Visual Studio Community 2022 version 17.7.34031.279
• Windows 10 SDK version 10.0.22621.0
[✓] Android Studio (version 2024.1)
• Android Studio at D:\Soft\IDE\AndroidStudio2024
• Flutter plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/9212-flutter
• Dart plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/6351-dart
• Java version OpenJDK Runtime Environment (build 17.0.11+0--11852314)
[✓] IntelliJ IDEA Ultimate Edition (version 2024.1)
• IntelliJ at D:\Soft\IDE\IntelliJ IDEA 2024.1.4
• Flutter plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/9212-flutter
• Dart plugin version 241.18808
[✓] Connected device (4 available)
• PDNT00 (mobile)   • 1650e3ec • android-arm64  • Android 12 (API 31)
• Windows (desktop) • windows  • windows-x64    • Microsoft Windows [版本 10.0.22631.4317]
• Chrome (web)      • chrome   • web-javascript • Google Chrome 129.0.6668.90
• Edge (web)        • edge     • web-javascript • Microsoft Edge 128.0.2739.63
[✓] Network resources
• All expected network resources are available.

Check if this recently fixed issue helps in your case or not.


Close connection on timeout

If a timeout occurs, the socket remains open. That's bad for CLI scripts, because they don't exit until all sockets are closed.
According to the socket.setTimeout(timeout[, callback]) docs:

When an idle timeout is triggered the socket will receive a 'timeout' event but the connection will not be severed. The user must manually end() or destroy() the socket.

I therefore suggest to destroy() the socket (end() leads to write after end errors in some cases):
 	socket.on 'timeout', =>
+		socket.destroy
 		done new Error 'lookup: timeout'

@schnittstabil Thanks, I will add socket.destroy().


Update UG

Fixes #101

Codecov Report

Merging #280 (705853d) into master (47e934c) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master     #280   +/-   ##
=========================================
  Coverage     66.27%   66.27%
  Complexity      837      837
=========================================
  Files           135      135
  Lines          2758     2758
  Branches        333      333
=========================================
  Hits           1828     1828
  Misses          815      815
  Partials        115      115

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 47e934c...705853d. Read the comment docs.


Writing tests for RefluxJS

I'm confused on testing RefluxJS. In my case, given that I have MessageStore and ActionStore, and the Store only listens to the Action, and no other store listens to the Action. Do I test the both the Action and Stores together or separately?
Might be an insignificant information, but I am using Jest.

This is a question originally posted at SO.

Unless you're doing some preEmit stuff on actions, it's kind of pointless to test them since all they do is pass on the arguments to anything listening on the action. There is little chance of regression and Reflux test suite should take care of that part.
It's more useful to test the stores and that they handle action invocations and other dependencies correctly (i.e. trigger with correct output).

Alright, thanks.  Should I unmock the actions which the stores listen to?

Unless you mock them on your own, you should deny Jest from mocking them.

Thanks @spoike and @jankuca :bow:


White spaces should not be removed in URLs

In case of a URL, spaces should not be removed, because they may be required, for an XML content for example:
	background-image: url('data:image/svg+xml;utf8,<svg xmlns="http://www.w3.org/2000/svg" width="10px" height="10px">...</svg>');````

I suggest to replace Line 99
````            token = token.replace(/\s+/g, "");````
with
````            token = token.replace(/\s+/g, " "); //  "" has been replaced with " " ````

Thanks for your report. I will investigate as soon as possible.


Add DevOps Tools shapes

Would be nice to add a shape collection of DevOps tools that are commonly used within organizations. The list of the tools may be found here for instance: https://stackshare.io/index/devops
Most of the icons are not available and need to be added manually. Most of the PNG files can be found and downloaded here: https://www.clipartmax.com/

They don't appear in the shape search field? Everything I've tried appears.

They don't appear in the shape search field? Everything I've tried appears.

Try to find such common tools like Jenkins, Artifactory, Gitlab, Ansible, terraform  ... etc. There is no results for them, but they are widely used.

OK, some I do, some I don't. If you don't get results for Jenkins and Gitlab I would double check your setup, the results should be the same for everyone

I would love to see this happening.

I would love to see this happening.


feat: add preprocessor to schema

So this is not ideal, since we do not capture the cases of
[preprocessor.cpp]
...
or
[preprocessor]
preprocessor.cpp = ...


TypeError: o3.replace is Not a Function - Project Not Running

Description:
I have been working on a project using itty-router-openapi, following the guide provided in the official documentation. The project was functioning correctly until recently. However, as of today, I am encountering an error that prevents the project from running.
Error Encountered
The specific error I am facing is:
TypeError: o3.replace is not a function
This error occurs when I try to run the project. I have not made any major changes to the codebase that might have led to this issue. It seems to have arisen spontaneously.

Steps to Reproduce

Set up a project following the steps in the itty-router-openapi guide.
Run the project.
Expected Behavior
The project should run without any errors, as it was doing previously.

Actual Behavior
The project fails to run, displaying the error message: TypeError: o3.replace is not a function.
Additional Information
It seems that this issue is new, as the project was working fine until today.
No significant changes were made to the codebase or the environment prior to the error occurring.
Screenshot

@G4brym Can you please check

Hey @rhutikcodes can you confirm what version of wrangler and itty-router-open you are running?
can you also post here your wrangler.toml?

name = "backend"
main = "src/index.ts"
compatibility_date = "2023-04-22"
"@cloudflare/itty-router-openapi": "^0.1.10", ( its breaking on all the versions)
if you delete your local node_modules and try it won't work

we are no longer offering support to itty-router-openapi v0.x.x, can you try to upgrade to the latest version v1.0.6
Here's a migration guide

Even on the lastest version, its not running
I followed this guide to create a fresh project, official documentation

i was able to replicate your issue locally, let will work on fixing it now

TLDR: The latest version of itty-router (the base library for this project) released about 10 hours ago broke some things, the quick fix for now is to force install the older version on your project with this command
npm install<EMAIL_ADDRESS>I will be releasing a new itty-router-openapi version today that locks this dependency instead of allowing patch updates

Hey @rhutikcodes i've just released a new version of itty-router-open that fixes the issue (v1.0.7)
Let me know if you find any more issues


symbol _ZdlPvm, version Qt_5 not defined in file libQt5Core.so.5

Hi. i download release 4.4 and follow your instructions. every thins is ok until open qt creator, it says:
"qtcreator: relocation error: /usr/lib/qtcreator/plugins/libQtcCppcheck.so: symbol _ZdlPvm, version Qt_5 not defined in file libQt5Core.so.5 with link time reference"
OS: Arch linux, 64bit,
Qt Creator: 4.4.1,
Qt: 5.9.2

Hi.
Prebuilt plugin is not binary compatible with arch's qt/qtcreator.
You should recompile it from source or use qtcreator from download.qt.io.
Duplicate of #22


Create a time series showing number of estimated vacancies in thousands

From here: https://www.ons.gov.uk/employmentandlabourmarket/peopleinwork/employmentandemployeetypes/bulletins/jobsandvacanciesintheuk/april2023
This is dependent on ticket #35
[ ] Create a vacancies index.njk page under src - dashboard - vacancies
[ ] Add line chart with number of vacancies in thousands per quarter
[ ] Add brief textual description of chart

Hey @taz-chiles - just wanted to check on this. My notes for the vacancies visualisations say that we agreed to develop two visualisations related to vacancies:

Estimated number of vacancies, quarterly.
Growth by sector

I just wanted to check both were covered by the above ticket - or given YFF noted the growth by sector is a secondary measure at present, whether we decided to deprioritise this at present until we've developed the pipeline & first visualisation set (or if we need additional clarity about the growth by sector stuff - which I'm happy to go and try to solve!)

Hi @ODILMichelle - thanks, yes I meant to add a second ticket for this visualisation - here's the ticket - #70
My thinking was that the growth by sector was a secondary measure and less of a priority, however it looks like they won't be too much trouble to visualise at the same time so I will take a look at them both!

@ODILMichelle @gilesdring @luke-strange first pass at vacancies data here:
https://data.youthfuturesfoundation.org/dashboard/vacancies/
Let me know your thoughts!


Rework the Contiki network address cache

@cheekyhalf This uses one of the fixes from your patch, lines 254-257. Otherwise it fails to bootstrap
The whole cache has been reworked to first parse the URI, then compare using Contiki implementation of network addresses. It compares them based on port & IP, and any URI comparison is gone at this point. This should solve the issue with cache overflowing when, for instance, doing observes which pass a slightly different URI to the application.

retest AWS CI (erbium)

@mtusnio you'll need the other line from the patch otherwise you'll end up with useCount=2 on the first call.

@cheekyhalf I don't have access to the patch at the moment, can you attach the diff with that line so that I can update the PR?

retest AWS CI (erbium)

retest AWS CI (libcoap)

retest AWS CI (libcoap)

retest AWS CI (libcoap)


Challenge 7: 	10001st Prime

By listing the first six prime numbers: 2, 3, 5, 7, 11, and 13, we can see that the 6th prime is 13.
What is the 10 001st prime number?

My program is more generic than the actual challenge. To answer the challenge simply use 10001 at runtime. The leg work is done in my library.

Updated due to the changed with the library function and the file 10001stPrime was changed to nthPrimeNumber to reflect its higher mutability.

Slight re-work due to the refactor.

Changed to use a brute force approach which is the only approach to give an answer in an acceptable time.


Everybody's working for the weekend.

This burndown is a lie:

The reason for this is that two of the remaining days are weekend days that don't show up on the idea line because the ideal line expects constant work every day, including weekend days.
One way to handle this is to include weekends on your burndown chart as flat lines as follows:

Note that this example also shows no change on actual or created on the weekends either, but this doesn't strike me as a reasonable approach for those two lines.

right. i agree.
how process this issue?

@hiphapis, so the "ideal" line is currently being created here:
https://github.com/appneta/burndown/blob/master/app/assets/javascripts/burndown/views/milestone-view.js#L98-L113
As you can see, it's quite naive, as it only generates a startDate and an endDate, so it ends up being a straight line.
Thus, I think the best way to change this to support weekends would be:

Create a new model called BurndownProgress.
Refactor the functionality from getIdealLine into BurndownProgress.
Update BurndownProgress to support "flatlining" while on weekends. Determining what days are weekends can be tricky, so it would be ideal to find a javascript library that can do this (i'm sure one exists)

Does that help?

Actually my thought is 'burndown' need setting page(or popup or embed or something)

choose compute type(by issues count, by label)
if you choose 'by label' then will set label rules (ex. "points: N", "Weight: N")
choose include weekend work (this issue https://github.com/appneta/burndown/issues/53)

PS.
i work startup CTO. mean is always i'm busy
my propose is not tiny.
therefore it's long journey


[boost-build/boost-uninstall] Update to 1.76.0

Others boost components has been upgraded to 1.76.0, however, we missed boost-build and boost-uninstall.
Remove patch 'fix_options.patch', since the release version 1.76.0 included the changes.
See https://github.com/boostorg/build/blob/develop/src/tools/msvc.jam

I assume the problem is still https://github.com/boostorg/build/issues/724. That was also the Problem in the upgrade PR: https://github.com/microsoft/vcpkg/pull/17335#issuecomment-821872323. I have no idea how to fix that, so it currently still use boost build 1.75.0, that works. To be honest I don't think that that will be fixed, maybe we should wait until boost fully supports CMake and than switch to CMake.

@autoantwort you may want to open a "new issue" over at https://github.com/bfgroup/b2 ref.: https://github.com/boostorg/build/issues/724#issuecomment-850859562 so that you may get a quicker response & you could reference your OG issue from https://github.com/boostorg/build/issues/724

Closing in favor of https://github.com/microsoft/vcpkg/pull/19556.


Are tests currently broken for wtforms?

Hi, are tests currently broken for wtforms?
I tried running the tests for wtforms with pytest. Some of the tests fail and some pass.
Here is a selection of the ones that I tested manually:
pytest -v tests/ # 5 ERRORS
pytest -v tests/test_csrf.py # 1 FAILED 8 PASSED
pytest -v tests/test_webob.py # PASSED
pytest -v tests/test_form.py # 5 FAILED 18 PASSED
pytest -v tests/test_i18n.py # 11 PASSED

You can use tox to run the tests instead of directly pytest. Then everything will be green.


Add Abstention tasks

Hello MTEB team !
In this PR, jointly with @hgissbkh , we propose the introduction of a new task to the MTEB leaderboard: Abstention.
The motivation is quite simple. Over the last years, Neural Retrieval has significantly improved upon heuristic-based IR systems. Yet, no model is perfect, and models often are unable to retrieve documents relevant to the user's query. This indicates abstention mechanisms would go a long way in improving the usability of such models, in order to improve confidence in retrieved results and let users have some control of the recall/precision tradeoff !
Going further, "simple" black-box abstention mechanisms (based on class logit distribution) have been shown to produce strong abstention baselines in classification tasks. In a recent work (https://arxiv.org/abs/2402.12997), we extend this to embedding models and show that using simple abstention heuristics common in the abstention litterature (score of best retrieved passage, score difference between best and second best retrieved passage, etc) we are able to get non trivial abstention mechanisms.  In our work (soon to be published at TMLR), we go further proposing more complex calibration based abstention mechanisms but we feel the MTEB is made to evaluate models and not mechanisms. As such, in this PR, we only implement 3 simple abstention heuristics that work out-of-the-box on retrieval and reranking tasks.
What we aim to assess with this PR is whether or not retrieval models (bi-encoder embedding models here) yield sufficiently calibrated (query, document) scores to be able to not only rank the documents and assess the best ones (retrieval task) but also given a ranking and the associated document scores, compute a "confidence score" which would enable abstention if too low. Better calibrated embedding models will perform better, and we feel this task opens up new and very interesting properties of embedding models !
While still new (and so not necessarily a task to add to the leaderboard right away), we feel having the codebase in MTEB is a nice first step. In this PR, we add the abstention task for 13 datasets spanning retrieval and reranking across 4 languages !
I'll further let @hgissbkh describe the metrics used in more detail !

[x] I have tested that the tasks runs with the mteb package.
[x] I have run the following models on the task (adding the results to the pr). These can be run using the mteb run -m {model_name} -t {task_name} command.

[x] sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2

[x] I have checked that the performance is neither trivial (both models gain close to perfect scores) nor random (both models gain close to random scores).
[x] I have filled out the metadata object in the dataset file (find documentation on it here).
[x] Run tests locally to make sure nothing is broken using make test.
[x] Run the formatter to format the code using make lint.
[x] I have added points for my submission to the points folder using the PR number as the filename (e.g. 438.jsonl).

Tagging @imenelydiaker for the review, don't hesitate in case you have any doubts or questions !
Cheers !

Tagging @imenelydiaker for the review, don't hesitate in case you have any doubts or questions ! Cheers !

This is very interesting thank you! I'll just leave one comment before reading the paper and try to review everything tomorrow.
Also adding other reviewers that may be interested @KennethEnevoldsen @Muennighoff @orionw

[Didn't review code yet, just read the paper]
So if I get this right, for a search query: first a retrieval step is performed, then you get similarity scores and then asses confidence scores to each score using a function like the max or std, then if your confidence is higher than a threshold you add the doc to the reranker pipeline, else you discard it. Is this right?
Some questions:

Is the main idea about discarding uncertain documents in a search/IR pipeline?
How does the linear model that asses confidence score generalize on other datasets? Or is it trained specifically for each dataset?
From my understanding htis works in a pipeline combining retrieval and reranking, so when applied to Retrieval only, what should happen after assessing confidence scores? It just discards less relevant documents? Or maybe I'm missing something here

Hippolyte will clear things up tmrw !
Rapidly, this is not an implementation of the paper at all, I just linked it for reference.
What we aim to assess with this PR is whether or not retrieval models (bi-encoder embedding models here) yield sufficiently calibrated (query, document) scores to be able to not only rank the documents and assess the best ones (retrieval task) but also given a ranking and the associated document scores, output a "confidence score" which would enable abstention if too low.
Here we are assessing model calibration through this abstention task by using very simple (but functioning) abstention heuristics.
The linked paper proposes more complex methods requiring calibration which we believe to be out of scope for MTEB !

In our paper, the linear confidence function is trained in-domain, i.e., for each specific dataset. However, the confidence functions we propose here are simple heuristics that don’t need to be trained (maximum score, standard deviation, highest score minus second highest score). For example, taking the maximum score has been proven to be a good proxy for how much a model is confident on a given classification instance (Hendrycks et al., 2016). We show in our paper that this observation transfers well to document ranking in general.

Really interesting task @hgissbkh!
I am still a little confused on the task definition, so I will explain what I understand:

The model does standard search and returns K documents with scores S_1, ..., S_K
Some heuristic is applied on top of that to remove any scores less than a standard deviation away (or perhaps a number threshold)
We then have some number of documents left that passed this heuristic

I guess my questions are:

How are you defining ground truth here? Are there annotations for what queries should be abstained from and which ones should return documents?
Can new models implement their own functions for abstention or do they use these heuristics the same? I think this may get rather complicated if models need to define a new function for calibration along with retrieval
Is there an evaluation of calibration in the metrics or just the documents that are returned?

I've typically seen this abstention done as a 2nd stage problem after the initial retriever is used, so it's taking me a bit to adjust to how this could be used for a first stage model.

Hi @orionw and thanks for your comment!
I will try to clarify things a bit.
The question we are trying to address with abstention is not to evaluate confidence at the document level but at the whole instance level. Let’s take your example again:

The model does standard search and returns the K most-relevant documents D_1, …, D_K with respect to query Q. Those documents have scores S_1 > … > S_K. Note that what I call an instance here is the tuple (Q, D_1, …., D_K) (the query and the K retrieved documents).
The confidence heuristic is then applied on the tuple of scores (S_1, …, S_K), not to each of them separately. For instance, we can take the maximum score as a confidence estimator (Hendrycks et al., 2016): c(S_1, …, S_K) = max(S_1, …, S_K).
Then, we decide if we return the retrieved documents or not, depending on whether c(S_1, …, S_K) is greater than a given threshold that controls abstention probability (the higher the threshold, the more we abstain overall). Put in a more formal way, say a retriever is a function R that takes a query Q, a document database DD, and a number of documents to return as input and that returns the top-K documents: R(Q, DD, K) = (D_1, …, D_K). Then, if c(S_1, …, S_K) < tau, R_abst(Q, DD, K) = "I'm not sure", otherwise, R_abst(Q, DD, K) = (D_1, …, D_K).

Now, regarding your questions more specifically:

How are you defining ground truth here? Are there annotations for what queries should be abstained from and which ones should return documents?

Good question! No, we do not need any additional annotations for abstention evaluation. We evaluate abstention quality by computing a normalized area under the metric-abstention curve. Let me explain:
Let’s take a test dataset T comprising of N instances (Q_1, D_1,1, …, D_1,K, Y_1), …, (Q_N, D_N,1, …, D_N,K, Y_N). In the vanilla case, we would simply evaluate the retrieval system by assessing the relevance of the K returned documents with respect to ground truth Y, for instance using NDCG@K. After doing it for each of the N instances, we can finally compute an average NDCG@K on T.
Now, let’s assume we want to keep only instances for which the confidence function is above a certain threshold tau. We would get a new dataset T_tau included in T (|T_tau| < |T|) for which we could also compute the average NDCG@K (hopefully greater than without abstention). Doing this for increasing values of tau, we would get an increasing curve. We finally compute the Area Under the Curve to assess overall abstention quality.
To get further details, you can have a look at Figure 2 and Section 4.3 of our paper (https://arxiv.org/abs/2402.12997).

Can new models implement their own functions for abstention or do they use these heuristics the same? I think this may get rather complicated if models need to define a new function for calibration along with retrieval

The heuristics we use are deliberately general and can be implemented on any models without any needs for adjustment. The three confidence functions we use are the max, the std and S_1-S_2 (highest score minus second highest score). These are common functions used for confidence estimation in the classification setting (Narayanan et al., 2012; Hendrycks et al., 2016; Pang et al., 2021; Gawlikowski et al., 2023). In our paper (Gisserot-Boukhlef et al., 2024), we show that those heuristics transfer well the to the IR setting.

Is there an evaluation of calibration in the metrics or just the documents that are returned?

I guess my response to question 1 answers this one too! :)
Hope this clarifies a bit and of course don’t hesitate if you have additional questions!

Hello,
Any thoughts on this @orionw ?

Thanks for explaining @hgissbkh and for the ping @ManuelFay! This is a novel idea and I'm still trying to parse the details of your statement.  Perhaps I'll have to give the paper a full read to understand.
I'm still a little worried about the automatic evaluation without gold data, as each embedding model will have a different calibration but I also admit I don't fully understand the example you gave.  If @imenelydiaker understands she can continue with the review, otherwise I'll need to take some time to work an example.
Thanks and sorry for the delay.

@orionw did you have a chance to look over an example? (I am going over PRs to get all datasets in)

Hey @orionw, I suggest we look at a concrete example!
Let’s assume we have a query Q and want to retrieve the top-5 documents using a retrieval system R. We retrieve those 5 documents, let’s say with the following scores: 0.9, 0.8, 0.7, 0.3, 0.2. If we choose the max as the confidence function (Hendrycks et al., 2016), then we get a confidence score of 0.9 (maximum of the 5 retrieved documents’ scores). Assume also that I have previously chosen a confidence threshold of 0.7. As 0.9 > 0.7, we decide to return the 5 retrieved documents. On the contrary, if we had set a confidence threshold of 0.95, we would have discarded the instance and returned an abstention message to the user.
If we extend this rationale to the whole dataset, we see that the number of abstained instances changes depending on the threshold we choose. And that’s how we construct AUC!

Assume we have the following test dataset, comprising of 3 instances (each instance comprises of a query, the 5 retrieved documents’ scores and the ground truth, 1 representing relevant documents and 0 representing irrelevant documents):
{(Q_1, [0.5, 0.4, 0.3, 0.2, 0.1], [0, 1, 0, 0, 0])
(Q_2, [0.7, 0.5, 0.3, 0.2, 0.2], [1, 0, 1, 0, 0])
(Q_3, [0.9, 0.8, 0.3, 0.3, 0.2], [1, 1, 0, 0, 0])}
We can first evaluate the 3 instances using NDCG@5: we get 0.63, 0.92, 1 respectively.
Then, let’s make the confidence threshold vary. If we set it equal to 0.4, no instance is discarded and the test NDCG is equal to (0.63 + 0.92 + 1) / 3 = 0.85. If we set it now to 0.6, the first instance is discarded (max score equal to 0.5) and the test NDCG becomes (0.92 + 1) / 2 = 0.96. Finally, setting the threshold to 0.8, the two first instances are discarded and the test NDCG becomes 1.
We get an increasing NDCG-abstention curve: (0%, 0.85), (33%, 0.92), (66%, 1).
We finally compute the area under this NDCG-abstention curve to evaluate abstention on the test dataset.

Thanks @hgissbkh, that is very helpful and I definitely see the appeal of the task.
A couple follow up questions: it seems like this is reducing the length of the returned list (since the least confident entries will be at the bottom).  So in essence, this is a metric computing the area under the nDCG@k score curve at various confidence scores -- is that correct?
Does that metric (nDCG@5 say) stay fixed or is it nDCG@length of the list? Looking at the paper it seems fixed?  So it's like taking the mAP score at various score thresholds.
I'm also trying to think about the edge cases here: if we have a retrieval system who's minimum score is 0.9 the nDCG score will be the same until the last confidence values, correct?
This seems like we're asking the models to have absolute scores (between 0-1) whereas perhaps the model does well but uses a relative range (0.9-1 for example). Is there a reason to prefer the one with the broader range?

Hey @hgissbkh and @ManuelFay, thank you for the detailed explanations. If abstention becomes a standard in RAG systems for example, this task would be very helpful I suppose.
Currently, I am failing to see maybe something obvious:

Is it correct to have a fixed threshold for all models? as @orionw mentionned it some models may use relative score ranges.
If a model gets higher abstention score than another one, what would this mean?

Help me undestand is I'm not getting this right please:
The way I see abstention is as a top-layer after retrieval (or any other task), so I'm failing to see is how is it an embedding evaluation task? It relies mostly on the scores of the task it is applied to (retrieval in your example).
According to my understanding, abstention scores will depend on initial retieval task scores and a fixed threshold. So there is a correlation between an embedder peformance on the initial task & absention scores, an embedder that is good at retrieval will be good at abstention. Is this correct?
Can we say the same about other evaluation tasks of MTEB? e.g., is there a correlation between classification and STS tasks or any other tasks?

Is it correct to have a fixed threshold for all models? as @orionw mentionned it some models may use relative score ranges.
---> No, there is no fixed threshold, we are computing an "area under the curve" score. Basically, you vary the threshold and see how much better it is than a baseline random policy. This is a classic metric for classification tasks requiring a threshold.
ROC
If a model gets higher abstention score than another one, what would this mean?
It would mean it is better calibrated. A "calibrated" retrieving model does not only need to rank documents in the correct order, but also to output score magnitudes that are coherent. This metric is not captured by ranking metrics at the moment !  Typically, we would like the "best" document for a given query to have a low score if it is not that relevant, and inversely.
So there is a correlation between an embedder peformance on the initial task & abstention scores, an embedder that is good at retrieval will be good at abstention. Is this correct?
We have found this to be mostly the case but this is not true by design. Again, since this is essentially a measure of calibration, a better calibrated model might not necessarily be a better retrieving model, this will depend on the loss function, regularization, retriever architecture, etc... However, a very bad retrieval model will have a hard time being correctly calibrated since it's inherently bad.
Can we say the same about other evaluation tasks of MTEB? e.g., is there a correlation between classification and STS tasks or any other tasks?
There definitely is a strong correlation between some tasks in MTEB.
To sum it up, what we are proposing here is a tractable way of measuring retriever calibration in a practical and useful setting.
Having said that, from this issue discussions, I feel the task is still a bit unclearly explained  and we probably were very bad at explaining the core concepts ! Thanks a ton for taking the time, it is super valuable feedback in all cases !

@orionw hadn't seen your comment about code.
In essence, the abstention task is a wrapper around other tasks (reranking, retrieval). It inherits all methods and properties or underlying tasks, essentially the only thing that changes is the associated scorer class. Since all tasks have their custom input format and methods, the wrapper just routes to the correct underlying task, retrieves the scores, and computes the abstention scores.
I decided to make abstention a separate task, in order to contain all modifications to a single directory and avoid having to make big breaking changes in the evaluation methods of other methods.  I put some thought into it and figured it should be treated as a task on it's own to guarantee consistency and allow for granular code modifications, rather than adding another set of metrics in the other tasks. I also feel it makes the task more readable.

Hi @orionw , regarding your follow-up questions:
-> So in essence, this is a metric computing the area under the nDCG@k score curve at various confidence scores -- is that correct?
At various confidence thresholds I would say, but you got the idea!
-> Does that metric (nDCG@5 say) stay fixed or is it nDCG@length of the list?
Yes it is fixed. First you choose your metric, say NDCG@5, and then you compute AUC based on this metric. But you can of course compute the AUC for any metric you want (NDCG@k, MAP…).
-> If we have a retrieval system who's minimum score is 0.9 the nDCG score will be the same until the last confidence values, correct?
We took care of this edge case in our implementation so that is flexible to any retrieval systems. We first have a look at all the NDCGs in the test set and then select the thresholds adequately. Taking my example from above, if the confidence scores are 0.9, 0.95 and 0.99 instead of 0.5, 0.7 and 0.9, we would simply choose the confidence thresholds differently and take for example 0.8, 0.92 and 0.96.
-> This seems like we're asking the models to have absolute scores (between 0-1) whereas perhaps the model does well but uses a relative range (0.9-1 for example).
I am not sure to get this one, sorry 😅

@ManuelFay, thanks, that makes sense. I think my suggestion would be to make the task classes themselves simpler, since we're just wrapping the other ones -- like if there was any way to wrap GerDaLIRSmall without having to redefine all those fields.  That way we don't have the same task repeated twice, if possible.

@hgissbkh thanks for the detailed response.

We first have a look at all the NDCGs in the test set and then select the thresholds adequately.

This was basically the question I had at the end, how do we determine these thresholds?
It seems tricky because on the one hand if you use a standard range (0-1) there may be some thresholds where nothing is removed and the scores penalize models which have smaller ranges (like a model that predicts between 0.9-1 always).  On the other hand, if you dynamically calculate it (e.g. 0.91, 0.92, ... 0.99) it doesn't seem like it measures confidence but rather just the normal ranking stat (e.g. a dynamic nDCG@k curve where you take every value of k and calculate the AUC).  Of these, the former seems more suited to calibration even if it ends up punishing the model for not fully using the range of similarity scores.
I could still be misunderstanding, so thanks for your patience :)

First you choose your metric, say NDCG@5, and then you compute AUC based on this metric.

Aren't metrics like NDCG@5 not well suited for this, as the top five documents won't change until you remove nearly all the documents?

Hi @orionw,
From what I understand, you are defending that to measure calibration, we should have a fixed range of thresholds that should not vary depending on the domain. By dynamically adjusting our thresholding (basically varying abstention rates from 0 to 1 - rather than the threshold from 0 to 1), we intended to obtain a clearer and more meaningful signal, that would often be squashed if computed on the 0 to 1 threshold interval for every domain.
We understand that this is an added complexity that complicates the readibility of a leaderboard like MTEB, and although we feel it makes the most sense, it definitely is an arbitrary choice.
Given the discussions in this PR, we do not intend to insist on merging this PR if you all feel it's not the right place for it at the moment.
In all cases, thanks for the precious feedback and  the great work !
Cheers,
Hippo and Manu

Hi @KennethEnevoldsen and many thanks for your remarks!
We have incorporated abstention as an evaluation metric rather than as a task in this new PR: . For the moment, we have implemented abstention metrics for Retrieval and Reranking only but they could also be relevant for all classification tasks.
We would be very happy to hear your opinion on this new implementation!

Wonderful @hgissbkh I will take a look - Will close this here as well