what are the expected shape of image and correction matrix?

Downstream processing after https://github.com/fractal-analytics-platform/fractal-tasks-core/issues/662. I've manually rename image_ROI_table to FOV_ROI_table and it seems to be working.
Then I've prepared a fake correction profile for all channels
dict_corr = {
    "PhenoVue Fluor 488": "fake_ffc_mat.tif",
    "Brightfield": "fake_ffc_mat.tif",
    "PhenoVue Hoechst 33342": "fake_ffc_mat.tif",
    "PhenoVue 641 Mito Stain": "fake_ffc_mat.tif",
    "PhenoVue Fluor 568": "fake_ffc_mat.tif",
}

and ran with
illumination_correction(
    input_paths=[f"{r}"],
    output_path=meas,
    metadata=metadata,
    component=component,
    illumination_profiles_folder=f"./parameters",
    dict_corr=dict_corr,
    background=100,
)

Here's the error msg I've got:
Cell In[30], [line 1](vscode-notebook-cell:?execution_count=30&line=1)
----> [1](vscode-notebook-cell:?execution_count=30&line=1) illumination_correction(
     [2](vscode-notebook-cell:?execution_count=30&line=2)     input_paths=[f"{r}"],
     [3](vscode-notebook-cell:?execution_count=30&line=3)     output_path=meas,
     [4](vscode-notebook-cell:?execution_count=30&line=4)     metadata=metadata,
     [5](vscode-notebook-cell:?execution_count=30&line=5)     component=component,
     [6](vscode-notebook-cell:?execution_count=30&line=6)     illumination_profiles_folder=f"[./parameters](https://vscode-remote+ssh-002dremote-002b172-002e27-002e24-002e164.vscode-resource.vscode-cdn.net/lustre/scratch126/cellgen/team283/tl10/hcs_analysis/bin/notebooks/parameters)",
     [7](vscode-notebook-cell:?execution_count=30&line=7)     dict_corr=dict_corr,
     [8](vscode-notebook-cell:?execution_count=30&line=8)     background=100,
     [9](vscode-notebook-cell:?execution_count=30&line=9) )

File [~/mambaforge/envs/fractal/lib/python3.10/site-packages/pydantic/decorator.py:40](https://vscode-remote+ssh-002dremote-002b172-002e27-002e24-002e164.vscode-resource.vscode-cdn.net/lustre/scratch126/cellgen/team283/tl10/hcs_analysis/bin/notebooks/~/mambaforge/envs/fractal/lib/python3.10/site-packages/pydantic/decorator.py:40), in pydantic.decorator.validate_arguments.validate.wrapper_function()

File [~/mambaforge/envs/fractal/lib/python3.10/site-packages/pydantic/decorator.py:134](https://vscode-remote+ssh-002dremote-002b172-002e27-002e24-002e164.vscode-resource.vscode-cdn.net/lustre/scratch126/cellgen/team283/tl10/hcs_analysis/bin/notebooks/~/mambaforge/envs/fractal/lib/python3.10/site-packages/pydantic/decorator.py:134), in pydantic.decorator.ValidatedFunction.call()

File [~/mambaforge/envs/fractal/lib/python3.10/site-packages/pydantic/decorator.py:206](https://vscode-remote+ssh-002dremote-002b172-002e27-002e24-002e164.vscode-resource.vscode-cdn.net/lustre/scratch126/cellgen/team283/tl10/hcs_analysis/bin/notebooks/~/mambaforge/envs/fractal/lib/python3.10/site-packages/pydantic/decorator.py:206), in pydantic.decorator.ValidatedFunction.execute()

File [~/mambaforge/envs/fractal/lib/python3.10/site-packages/fractal_tasks_core/tasks/illumination_correction.py:276](https://vscode-remote+ssh-002dremote-002b172-002e27-002e24-002e164.vscode-resource.vscode-cdn.net/lustre/scratch126/cellgen/team283/tl10/hcs_analysis/bin/notebooks/~/mambaforge/envs/fractal/lib/python3.10/site-packages/fractal_tasks_core/tasks/illumination_correction.py:276), in illumination_correction(input_paths, output_path, component, metadata, illumination_profiles_folder, dict_corr, background, overwrite_input, new_component)
   [271](https://vscode-remote+ssh-002dremote-002b172-002e27-002e24-002e164.vscode-resource.vscode-cdn.net/lustre/scratch126/cellgen/team283/tl10/hcs_analysis/bin/notebooks/~/mambaforge/envs/fractal/lib/python3.10/site-packages/fractal_tasks_core/tasks/illumination_correction.py:271) logger.info(
   [272](https://vscode-remote+ssh-002dremote-002b172-002e27-002e24-002e164.vscode-resource.vscode-cdn.net/lustre/scratch126/cellgen/team283/tl10/hcs_analysis/bin/notebooks/~/mambaforge/envs/fractal/lib/python3.10/site-packages/fractal_tasks_core/tasks/illumination_correction.py:272)     f"Now processing ROI {i_ROI+1}[/](https://vscode-remote+ssh-002dremote-002b172-002e27-002e24-002e164.vscode-resource.vscode-cdn.net/){num_ROIs} "
   [273](https://vscode-remote+ssh-002dremote-002b172-002e27-002e24-002e164.vscode-resource.vscode-cdn.net/lustre/scratch126/cellgen/team283/tl10/hcs_analysis/bin/notebooks/~/mambaforge/envs/fractal/lib/python3.10/site-packages/fractal_tasks_core/tasks/illumination_correction.py:273)     f"for channel {i_c+1}[/](https://vscode-remote+ssh-002dremote-002b172-002e27-002e24-002e164.vscode-resource.vscode-cdn.net/){num_channels}"
   [274](https://vscode-remote+ssh-002dremote-002b172-002e27-002e24-002e164.vscode-resource.vscode-cdn.net/lustre/scratch126/cellgen/team283/tl10/hcs_analysis/bin/notebooks/~/mambaforge/envs/fractal/lib/python3.10/site-packages/fractal_tasks_core/tasks/illumination_correction.py:274) )
   [275](https://vscode-remote+ssh-002dremote-002b172-002e27-002e24-002e164.vscode-resource.vscode-cdn.net/lustre/scratch126/cellgen/team283/tl10/hcs_analysis/bin/notebooks/~/mambaforge/envs/fractal/lib/python3.10/site-packages/fractal_tasks_core/tasks/illumination_correction.py:275) # Execute illumination correction
...
    [68](https://vscode-remote+ssh-002dremote-002b172-002e27-002e24-002e164.vscode-resource.vscode-cdn.net/lustre/scratch126/cellgen/team283/tl10/hcs_analysis/bin/notebooks/~/mambaforge/envs/fractal/lib/python3.10/site-packages/fractal_tasks_core/tasks/illumination_correction.py:68) dtype = img_stack.dtype

ValueError: Error in illumination_correction:
img_stack.shape=(1, 5, 5, 2160, 2160)
corr_img.shape=(2160, 2160)

In this case, is the shape of 5D image_stack unexpected or the corr_img?

Hey @BioinfoTongLI
Thanks for opening the issue and describing it so well! Our correction function expects a 4D image array and a 2D correction array. Your image array still appears to be 5D (=> still with a time dimension?).
You mentioned somewhere you managed to drop the time dimension, right? I guess our illumination correction task is another example of a task not handling varying image dimensions well!

Our correction function expects a 4D image array and a 2D correction array.
Thanks for the reply! Do you mean there  the 2D profile is applied to each FOV across all Zs? Are you planning on doing 3D correction? basically one Z position -> one profile. It is natively supported by BaSiCpy and I am working on a nf-core version of that. should be possible to merge into Fractal later.
You mentioned somewhere you managed to drop the time dimension, right?
Nope, it seems too much work. The whole folder needs to be recreated...Still trying to figure out how to walk around this issue

Do you mean there the 2D profile is applied to each FOV across all Zs? Are you planning on doing 3D correction? basically one Z position -> one profile

That is interesting. I wasn't aware that BaSiCpy had such a correction. @adrtsc has developed some BaSiCpy Fractal tasks for handling 2D illumination correction (see https://github.com/Apricot-Therapeutics/APx_fractal_task_collection).
If you are building something broader for BaSiCpy processing of OME-Zarrs, would be great to also get this into Fractal of course! We are in the process of refactoring our Task API and are hoping to release Fractal 2.0 with more flexible server and simpler Task API, so may be worth to revisit it in the new version :)

Nope, it seems too much work. The whole folder needs to be recreated...Still trying to figure out how to walk around this issue

Ah, that's a pity! Unfortunately, we won't have a chance to work on making our tasks more flexible in axes handling before our Fractal 2.0 transition that we're currently focusing on, but you nicely highlight the importance of picking this up soon after!

Nope, it seems too much work. The whole folder needs to be recreated...Still trying to figure out how to walk around this issue

@BioinfoTongLI I've also started looking into this, as we just had some users with the same issue. I also can't find a way to do this without rewriting the whole Zarr, which is a pity. I'll nonetheless try my luck at creating a Fractal task that copies a Zarr & drops a singleton T dimension to enable our users to use our other tasks.
Still, it will be helpful once we update more tasks to use Zarr loading that is robust to this of course! Given the current state of OME-Zarr reading & writing libraries, I'm not sure whether I expect everyone to adopt the same system though. We currently often fall back on the raw Zarr API, which means we'd have to explicitly handle such cases and others may be doing similar things.

Hey @BioinfoTongLI
I now have a working (very early) prototype of a python function that drops the T dimension from an OME-Zarr. It creates a second Zarr with a suffix in the name with the same image, just no T dimension. Lot's of improvements still TBD of course, but it's working on our test data. Maybe a way for you to check out some of the Fractal tasks as well :)
See https://github.com/jluethi/fractal-helper-tasks/blob/main/src/fractal_helper_tasks/drop_t_dimension.py

Thanks a lot @jluethi for helping out here!
I will have a try!

Hi @jluethi, sorry for being radio silent for a while. I've tried the helper function is it's working great!
I've noticed that this is for processing one single FOV (or well), right?
If I want to loop over all FOVs/wells, is there such a function/object in fractal already?
I can definitely parse OME/.zattrs, but don't want to re-created the wheel either :)

Hey @BioinfoTongLI
The helper function has just recently been updated to the Fractal 2.0 API, so just needs the "zarr_url" of the individual images now.
The way we handle this in Fractal is that users/developers write tasks that run e.g. per image and the Fractal server then submits this task to all available images. If you have a pre-existing OME-NGFF plate, the Import OME-Zarr task is what we use to get the list of all the available Zarr images (either within Fractal or run separately). If you run it outside of Fractal, you could probably just run import_ome_zarr with an empty list of zarr_urls and the directory where the Zarr is found.
e.g. if you have a Zarr plate located like this: /path/to/my/zarr/plate.zarr
Then running:
import_ome_zarr(
    zarr_urls=[],
    zarr_dir="/path/to/my/zarr"
    zarr_name="plate.zarr"
)

You can turn on or off the update_omero_metadata, add_image_ROI_table & add_grid_ROI_table depending on whether you run more Fractal tasks afterwards (if you do, it's helpful to add at least the first 2 options).
This function returns a dict with image_list_changes (i.e. what are the images in that plate, what are their attributes and types). If it's run within a Fractal 2.0 server, Fractal then keeps track of those. If you run it as a Python script, you can extra all the zarr_urls from there and loop over them in the drop_t_dimension task :)

See more info about these concepts in Fractal 2.0 in the 2 pages here:
The Fractal task API: https://fractal-analytics-platform.github.io/version_2/tasks/
The Fractal image list: https://fractal-analytics-platform.github.io/version_2/image_list/

Thanks so much @jluethi ! I think I've made it working.

I couldn't wait to try it out when I saw your answers and keep forgetting sending feedback. sorry about that!

That's awesome, glad to hear that! And no worries, Github is for async conversations anyway. The speed of my answers also vary ;)


AsyncEngineDeadError / RuntimeError: CUDA error: an illegal memory access was encountered

While serving the CodeLLaMA 13B （CodeLlama-13b-hf) base model with v1/completions API with 1 A100, I encountered the following CUDA memory issue.
The same thing happened with the 34B base model, too (CodeLlama-34b-hf). However, I did not encounter such an issue with any of the CodeLlama instruct series (with the same starting config).
To make it easier to debug, I attached the complete log here (it is too big, so i have to upload it somewhere else).
The error log:
INFO 09-09 08:20:08 async_llm_engine.py:120] Aborted request cmpl-223dc522668143dfb7db9b23988ec0a1.
INFO:     <IP_ADDRESS>:34054 - "POST /v1/completions HTTP/1.1" 500 Internal Server Error
Exception in callback _raise_exception_on_finish(request_tracker=<vllm.engine....x7f85d0660160>)(<Task finishe...sertions.\n')>) at /usr/local/lib/python3.8/dist-packages/vllm/engine/async_llm_engine.py:21
handle: <Handle _raise_exception_on_finish(request_tracker=<vllm.engine....x7f85d0660160>)(<Task finishe...sertions.\n')>) at /usr/local/lib/python3.8/dist-packages/vllm/engine/async_llm_engine.py:21>
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/vllm/engine/async_llm_engine.py", line 27, in _raise_exception_on_finish
    task.result()
  File "/usr/local/lib/python3.8/dist-packages/vllm/engine/async_llm_engine.py", line 315, in run_engine_loop
    await self.engine_step()
  File "/usr/local/lib/python3.8/dist-packages/vllm/engine/async_llm_engine.py", line 300, in engine_step
    request_outputs = await self.engine.step_async()
  File "/usr/local/lib/python3.8/dist-packages/vllm/engine/async_llm_engine.py", line 173, in step_async
    output = await self._run_workers_async(
  File "/usr/local/lib/python3.8/dist-packages/vllm/engine/async_llm_engine.py", line 198, in _run_workers_async
    output = executor(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/vllm/worker/worker.py", line 289, in execute_model
    input_tokens, input_positions, input_metadata = self._prepare_inputs(
  File "/usr/local/lib/python3.8/dist-packages/vllm/worker/worker.py", line 231, in _prepare_inputs
    tokens_tensor = torch.cuda.LongTensor(input_tokens)
RuntimeError: CUDA error: an illegal memory access was encountered
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/lib/python3.8/asyncio/events.py", line 81, in _run
    self._context.run(self._callback, *self._args)
  File "/usr/local/lib/python3.8/dist-packages/vllm/engine/async_llm_engine.py", line 36, in _raise_exception_on_finish
    raise exc
  File "/usr/local/lib/python3.8/dist-packages/vllm/engine/async_llm_engine.py", line 31, in _raise_exception_on_finish
    raise AsyncEngineDeadError(
vllm.engine.async_llm_engine.AsyncEngineDeadError: Task finished unexpectedly. This should never happen! Please open an issue on Github. See stack trace above for the actual cause.
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/vllm/engine/async_llm_engine.py", line 27, in _raise_exception_on_finish
    task.result()
  File "/usr/local/lib/python3.8/dist-packages/vllm/engine/async_llm_engine.py", line 315, in run_engine_loop
    await self.engine_step()
  File "/usr/local/lib/python3.8/dist-packages/vllm/engine/async_llm_engine.py", line 300, in engine_step
    request_outputs = await self.engine.step_async()
  File "/usr/local/lib/python3.8/dist-packages/vllm/engine/async_llm_engine.py", line 173, in step_async
    output = await self._run_workers_async(
  File "/usr/local/lib/python3.8/dist-packages/vllm/engine/async_llm_engine.py", line 198, in _run_workers_async
    output = executor(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/vllm/worker/worker.py", line 289, in execute_model
    input_tokens, input_positions, input_metadata = self._prepare_inputs(
  File "/usr/local/lib/python3.8/dist-packages/vllm/worker/worker.py", line 231, in _prepare_inputs
    tokens_tensor = torch.cuda.LongTensor(input_tokens)
RuntimeError: CUDA error: an illegal memory access was encountered
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/uvicorn/protocols/http/h11_impl.py", line 408, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/usr/local/lib/python3.8/dist-packages/uvicorn/middleware/proxy_headers.py", line 84, in __call__
    return await self.app(scope, receive, send)
  File "/usr/local/lib/python3.8/dist-packages/fastapi/applications.py", line 292, in __call__
    await super().__call__(scope, receive, send)
  File "/usr/local/lib/python3.8/dist-packages/starlette/applications.py", line 122, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.8/dist-packages/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/usr/local/lib/python3.8/dist-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.8/dist-packages/starlette/middleware/cors.py", line 83, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.8/dist-packages/starlette/middleware/exceptions.py", line 79, in __call__
    raise exc
  File "/usr/local/lib/python3.8/dist-packages/starlette/middleware/exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "/usr/local/lib/python3.8/dist-packages/fastapi/middleware/asyncexitstack.py", line 20, in __call__
    raise e
  File "/usr/local/lib/python3.8/dist-packages/fastapi/middleware/asyncexitstack.py", line 17, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.8/dist-packages/starlette/routing.py", line 718, in __call__
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.8/dist-packages/starlette/routing.py", line 276, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.8/dist-packages/starlette/routing.py", line 66, in app
    response = await func(request)
  File "/usr/local/lib/python3.8/dist-packages/fastapi/routing.py", line 273, in app
    raw_response = await run_endpoint_function(
  File "/usr/local/lib/python3.8/dist-packages/fastapi/routing.py", line 190, in run_endpoint_function
    return await dependant.call(**values)
  File "/usr/local/lib/python3.8/dist-packages/vllm/entrypoints/openai/api_server.py", line 528, in create_completion
    async for res in result_generator:
  File "/usr/local/lib/python3.8/dist-packages/vllm/engine/async_llm_engine.py", line 387, in generate
    raise e
  File "/usr/local/lib/python3.8/dist-packages/vllm/engine/async_llm_engine.py", line 382, in generate
    async for request_output in stream:
  File "/usr/local/lib/python3.8/dist-packages/vllm/engine/async_llm_engine.py", line 69, in __anext__
    raise result
  File "/usr/lib/python3.8/asyncio/events.py", line 81, in _run
    self._context.run(self._callback, *self._args)
  File "/usr/local/lib/python3.8/dist-packages/vllm/engine/async_llm_engine.py", line 36, in _raise_exception_on_finish
    raise exc
  File "/usr/local/lib/python3.8/dist-packages/vllm/engine/async_llm_engine.py", line 31, in _raise_exception_on_finish
    raise AsyncEngineDeadError(
vllm.engine.async_llm_engine.AsyncEngineDeadError: Task finished unexpectedly. This should never happen! Please open an issue on Github. See stack trace above for the actual cause.

Here is the script and the docker container (with vllm==0.1.5) i used to spin up the server.
export HOST_USER_ID=$(id -u)
DOCKER_IMG="xingyaoww/vllm:v1.1.1"

# Construct instance name using the current username and the current time.
# This is useful for running multiple instances of the docker container.
DOCKER_INSTANCE_NAME="vllm_${USER}_$(date +%Y%m%d_%H%M%S)"

# Model directory: contains model (cloned) downloaded from huggingface
# 1. git lfs install
# 2. git clone git@hf.co:<MODEL ID> # example: git clone git@hf.co:meta-llama/Llama-2-13b-chat-hf
MODEL_DIR="." # e.g., dir that contains Llama-2-13b-chat-hf
MODEL_NAME="CodeLlama-13b-hf"

# Set CUDA_VISIBLE_DEVICES to the GPU ids you want to use.
 Also the model start output gribbish. Might gribbish output let the kernel be unstable. Currently i limit max_tokens to 4096, and no more error.

We have this same issue and we're only trying to generate 1024 tokens. It's extremely frustrating.

I am seeing this error when running:

vllm v1.1.7
A100 40Gb
model: codellama/CodeLlama-13b-Instruct-hf
--max-num-batched-tokens 9000 or larger
prompt size starting around 8500

After I get this error the first time, it throws the same error on small prompts as well, until I restart.
So I am forced to set --max-num-batched-tokens to 8129.
Any ideas how to work around this error?

Any updates @WoosukKwon? This bug is causing us problems in production.

I am encountering a similar issue on an A100 80G and I believe it has something to do with --max-num-batched-tokens.
The stack trace is a bit different:
INFO 09-27 23:24:07 llm_engine.py:72] Initializing an LLM engine with config: model='/storage/vllm/models/Xwin-LM-70B-V0.1-AWQ', tokenizer='/storage/vllm/models/Xwin-LM-70B-V0.1-AWQ', tokenizer_mode=auto, revision=None, trust_remote_code=False, dtype=torch.float16, download_dir=None, load_format=auto, tensor_parallel_size=1, quantization=awq, seed=0)
Traceback (most recent call last):
  File "/local-llm-server/other/vllm/vllm_api_server.py", line 103, in <module>
    engine = AsyncLLMEngine.from_engine_args(engine_args)
  File "/venv/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 486, in from_engine_args
    engine = cls(engine_args.worker_use_ray,
  File "/venv/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 270, in __init__
    self.engine = self._init_engine(*args, **kwargs)
  File "/venv/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 306, in _init_engine
    return engine_class(*args, **kwargs)
  File "/venv/lib/python3.10/site-packages/vllm/engine/llm_engine.py", line 108, in __init__
    self._init_cache()
  File "/venv/lib/python3.10/site-packages/vllm/engine/llm_engine.py", line 188, in _init_cache
    num_blocks = self._run_workers(
  File "/venv/lib/python3.10/site-packages/vllm/engine/llm_engine.py", line 688, in _run_workers
    output = executor(*args, **kwargs)
  File "/venv/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/venv/lib/python3.10/site-packages/vllm/worker/worker.py", line 108, in profile_num_available_blocks
    self.model(
  File "/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/venv/lib/python3.10/site-packages/vllm/model_executor/models/llama.py", line 293, in forward
    hidden_states = self.model(input_ids, positions, kv_caches,
  File "/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/venv/lib/python3.10/site-packages/vllm/model_executor/models/llama.py", line 253, in forward
    hidden_states = layer(
  File "/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/venv/lib/python3.10/site-packages/vllm/model_executor/models/llama.py", line 200, in forward
    hidden_states = self.self_attn(
  File "/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/venv/lib/python3.10/site-packages/vllm/model_executor/models/llama.py", line 151, in forward
    attn_output = self.attn(positions, q, k, v, k_cache, v_cache,
  File "/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/venv/lib/python3.10/site-packages/vllm/model_executor/layers/attention.py", line 330, in forward
    return super().forward(
  File "/venv/lib/python3.10/site-packages/vllm/model_executor/layers/attention.py", line 205, in forward
    self.multi_query_kv_attention(
  File "/venv/lib/python3.10/site-packages/vllm/model_executor/layers/attention.py", line 109, in multi_query_kv_attention
    key = torch.repeat_interleave(key, self.num_queries_per_kv, dim=1)
RuntimeError: CUDA error: an illegal memory access was encountered
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

--max-num-batched-tokens higher than 5120 seems to cause this exception.

I believe this should have been fixed in the latest 0.2.0 release.

same bug

Ran into this problem in 0.2.5 on A4500 card.

@robcaulk Could you share a reproducible script? Thanks.

Still happened in version 0.6.1.post2


ios: transform contact to JSON object instead of array to match type

Currently the listener on ApplePayEventsEnum.DidCreatePaymentMethod is receiving data.contact as an array, but the type should actually be ShippingContact

Thanks pull request.
This pull request has that ApplePayEvents.DidCreatePaymentMethod.rawValue has breaking changes. and will not match requiredShippingContactFields. Being an array is not reasonable, but it makes implementation easier, so I will keep it that way.
If you have any input, please share.


1.16.1 release missing from github

There seems to be a 1.16.1 release available from NPM, but there is no tag or release on Github and no release notes for it.
Is this an oversight? I would like to understand what changed in 1.16.1 compared to 1.16.0

I’m not sure, this happened several months ago so I can’t remember the details.
Probably it fixed some publishing issues? I can’t tell it for sure.

Tagging the release would still be helpful. I also spent a bunch of time digging for the tag to understand what had changed. The tag is there to help consumers understand what happens, that little happened is fine but the tag is how we find that out. 👍 🙇


Different waker_vtable addresses for same type

use futures::task::ArcWake;

struct Temp;

impl ArcWake for Temp {
    fn wake_by_ref(arc_self: &std::sync::Arc<Self>) {
        let _ = arc_self;
    }
}

fn main() {
    let temp = std::sync::Arc::new(Temp);
    let waker = futures::task::waker_ref(&temp);
    println!("{waker:?} {:?}", waker.clone());
    futures::executor::block_on(async {});
}

Sometimes (more often than not, in my experience), when running in --release mode, vtable addresses don't match:
WakerRef { waker: ManuallyDrop { value: Waker { data: 0x5654d35e5b90, vtable: 0x5654d2ad62b8 } }, _marker: PhantomData<&()> } Waker { data: 0x5654d35e5b90, vtable: 0x5654d2ad6308 }

Why this matters: this leads to AtomicWaker and alike to clone (or, worse, wake) on each poll because of how Waker::will_wake compares Wakers. Example: https://github.com/sdroege/async-tungstenite/issues/133
Is this a compiler bug? If yes, can we do something to mitigate it?

Same issue as https://github.com/rust-lang/futures-rs/pull/2829#issuecomment-1962863389 / https://github.com/rust-lang/rust/pull/121622#issuecomment-1975888092?

For the state of futures-rs crate, that I'm testing, all tests (including task_arc_wake) pass. rustc versions where I've seen the bug happen: 1.78, 1.79 and rustc 1.80.0-nightly (36153f1a4 2024-05-24).
codegen-units = 1 prevents the separation of addresses.

yes, seems to be the same issue

#[inline(always)] on clone_arc_raw has some positive effect on the situation (ref_wake_same passes)

Quoting myself from

[...] this sounds like a bug in the code doing the comparison. If multiple codegen units are involved, it's not guaranteed that the vtable is always the same for the same object.
That's why e.g. https://doc.rust-lang.org/stable/std/sync/struct.Arc.html#method.ptr_eq exists.

Resolved by https://github.com/rust-lang/futures-rs/pull/2865


Add inline_constants=True keyword to inline.

As brought up in #25.
inline_constants keyword is a good idea.  Inlining constants based on whether keys was given or not seemed like a good idea at the time, but it's awkward to introduce a new and unnecessary convention to use in a single function.

+1 on this.  (also, can you verify that you have merge privileges?)
The two comments were idle.


getChangeMap() MMU Not Working

The MMU parameter does not seem to work, as I still see individual pixels in outputs.

I am still looking into these, but it is weird and seems depended on zoom level. That is, it seems to look correct when I zoom out to a different zoom level, but when I zoom in things look wrong.

Has this problem been resolved? I also encountered this problem.


6/07 PM Publish

Docs Build status updates of commit c2c8342:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs-conceptual/azps-7.5.0/authenticate-azureps.md
:white_check_mark:Succeeded
View (azps-7.5.0)

docs-conceptual/azps-8.0.0/authenticate-azureps.md
:white_check_mark:Succeeded
View (azps-8.0.0)

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel


[CSUB-117] Update loan terms with more detailed interest information

Changes in this PR:

Change maturity -> term_length
Make InterestRate a struct with compounding information
Storage migration (imperfect because there isn't a lossless way to translate maturity -> duration)
Remove remaining interest calculation code

The migration in particular needs testing on the QA network to make sure existing ask/bid/deal orders are properly migrated.

Codecov Report

Merging #132 (45b6140) into dev (cb42eb9) will decrease coverage by 1.31%.
The diff coverage is 17.60%.

:exclamation: Current head 45b6140 differs from pull request most recent head f9d2dac. Consider uploading reports for the commit f9d2dac to get more accurate results

@@            Coverage Diff             @@
##              dev     #132      +/-   ##
==========================================
- Coverage   56.17%   54.86%   -1.32%
==========================================
  Files          27       29       +2
  Lines        4874     4987     +113
==========================================
- Hits         2738     2736       -2
- Misses       2136     2251     +115

Impacted Files
Coverage Δ

pallets/creditcoin/src/lib.rs
81.04% <0.00%> (-0.64%)
:arrow_down:

pallets/creditcoin/src/migrations/mod.rs
0.00% <0.00%> (ø)

pallets/creditcoin/src/migrations/v1.rs
0.00% <0.00%> (ø)

pallets/creditcoin/src/tests.rs
99.04% <60.00%> (-0.27%)
:arrow_down:

pallets/creditcoin/src/types/loan_terms.rs
88.00% <94.11%> (-2.57%)
:arrow_down:

:mega: Codecov can now indicate which changes are the most critical in Pull Requests. Learn more


Fix Publish Service Artifacts workflow call

Description of your changes

Relevant PR: https://github.com/upbound/uptest/pull/110
This PR removes the optional: true specification in the inputs of the Publish Service Artifacts reusable workflow call and converts types of size and concurrency input parameters from number to string.
I have:

[x] Run make reviewable test to ensure this PR is ready for review.

How has this code been tested

Please see: https://github.com/upbound/provider-gcp/actions/runs/4963402570

Looks like with the larger runners, we are able to achive a concurrency of ~4 (# of batches x batch concurrency) while pushing to the registry (when there's only one family being built & pushed).


Need help Integrating ncm2 with nvim-lspInstall

Hello,
I am new to neovim so I'm sure I'm missing something simple. I have installed a plugin nvim-lspinstall (https://github.com/kabouzeid/nvim-lspinstall) which installs the language servers for me so now I need to register them as completion source with ncm2 but I am lost at how should that look exactly.
OS
MacOS
(If you're using Linux, sharing a reproducible, vimrc minimized docker image,
can definitely help solving the issue)
neovim :version output
v0.6.0-dev+nightly
Minimal vimrc for reproducing the issue
-- get all installed servers
  local servers = require'lspinstall'.installed_servers()
  -- ... and add manually installed servers
  --table.insert(servers, "clangd")
  --table.insert(servers, "sourcekit")

  for _, server in pairs(servers) do
    local config = make_config()

    -- language specific config
    if server == "lua" then
      config.settings = lua_settings
    end
    if server == "sourcekit" then
      config.filetypes = {"swift", "objective-c", "objective-cpp"}; -- we don't want c and cpp!
    end
    if server == "clangd" then
      config.filetypes = {"c", "cpp"}; -- we don't want objective-c and objective-cpp!
    end

    -- Begin ncm2
    source_opt = {
        name = server,
        priority = 9,
        mark = server,
        sorter = 'none',
        on_complete = {function(s:on_complete, [server]) end},
        completion_pattern = capabilities
    }
    call ncm2#register_source(source_opt)
    -- End ncm2
    require'lspconfig'[server].setup(config)
  end

neovim :messages output when the error occurs
')' expected near ':' on line with on_complete

Decided to just use vim-lsp


openapi support?

Hi. I was just wondering if there is plan for openapi support. It would be great if you could just use openapi3.x specification and based on that create mocked service with endpoint responsing according to what is specified in openapi yaml file.

Hi,
Yes, it is planned. Proof: https://github.com/up9inc/mockintosh/blob/main/docs/Changelog.md mentions it in Roadmap section.

This has been implemented and will be part of the next release. Meanwhile, you can try it by installing from GitHub source, or via unstable Docker image tag.

I've a follow up question: according to https://openapi.tools/ OAS 3.1 is supported, although the conversion fails for me with (docker image tag unstable)
Failed validating 'pattern' in schema['properties']['openapi']:
    {'pattern': '^3\\.0\\.\\d(-.+)?$', 'type': 'string'}

On instance['openapi']:
    '3.1.0'

So am I right to assume that only versions <= 3.0.3 are supported yet? Or am I missing something? Thx


Fix luaL_typerror missing for newer lua versions

This fixes errors when generating code for lua 5.3.

Thanks.  Sorry for the long wait.


Adding synonym feature to OBOMapper

It would be useful to have an option in OBOMapper to get the synonyms of a concept (the list of synonyms as a feature)

ebbaeb3160ab71c2d8681692617b1d5bc1aad11c
@ldeleger deployed on our server


docs(route53): Adding EKS IRSA method

Description
Someone emailed me personally asking about my comment on this previous issue: https://github.com/jetstack/cert-manager/issues/2147#issuecomment-590366437. I decided if that was the case, the documentation could use an update about how the EKS IRSA method works.
This is the PR that @munnerz asked for here: https://github.com/jetstack/cert-manager/issues/2147#issuecomment-590779944
Features / Fixes

Adds explicit documentation on the EKS IRSA method.
Clarifies some of the existing AWS role information docs.

Is it worth mentioning that users of eksctl can declare the Service Account with appropriate permissions using eksctl, and then pass that pre-existing user to cert-manager's Helm chart?
It's a lot simpler than managing the roles and SA annotations by hand, I've found.

Hmm. This doesn't mention the fsGroup change from https://github.com/jetstack/cert-manager/issues/2147; unless I messed up and it's not necessary?

Hmm. This doesn't mention the fsGroup change from jetstack/cert-manager#2147; unless I messed up and it's not necessary?

Not sure that's related to this? This involves dealing with the AWS IAM portions of cert-manager, whereas that issue is how the Helm chart / Kubernetes manifest is written to properly account for the permissions on the ServiceAccount token.

Is it worth mentioning that users of eksctl can declare the Service Account with appropriate permissions using eksctl, and then pass that pre-existing user to cert-manager's Helm chart? (If not using Helm, eksctl can also modify an existing service account with the appropriate annotation)
It's a lot simpler than managing the roles and SA annotations by hand, I've found.

I'm not against eksctl, but my trust of CLI tools to write properly fine-grained IAM roles doesn't extend very far. Interally we use Terraform, so IAM roles / policies need to be properly defined in code, not "Run the eksctl command."

In the eksctl use-case, you provide the policy, i.e. the relevant snippet from my setup:
    # This account is to support cert-manager making updates to the hosted zone
    # in Route53. This allows using an ACME system to auto-issue TLS certificates to https services.
    # Policy definition per cert-manager Route53 documentation, and also by examining the
    # "CertManager" instance policy in the eksctl source
    # https://cert-manager.io/docs/configuration/acme/dns01/route53/
    # https://github.com/weaveworks/eksctl/blob/32027355678915365c68053e9293900b951cbc04/pkg/cfn/builder/iam_helper.go#L55
    - metadata:
        name: cert-manager
        namespace: cert-manager
      attachPolicy:
        Version: "2012-10-17"
        Statement:
          - Effect: Allow
            Action:
              - route53:ChangeResourceRecordSets
              - route53:ListResourceRecordSets
            Resource: "arn:aws:route53:::hostedzone/<MYHOSTEDZONE>"
          - Effect: Allow
            Action:
              # route53:GetChange is here, see https://github.com/jetstack/cert-manager/issues/2827
              - route53:GetChange
              - route53:ListHostedZonesByName
            Resource: "*"

That policy is copied out of the eksctl source, because it's not "generating" the policy, it has the correct policy defined in its source, and is just creating the policy and role object for you, and the service account, and tying them together for you to save having to do a bunch of manual work in two systems.
It's also possible to provide it the ARN of a Policy you've written yourself, if your pipeline supports it, and it'll just create the Role and Service Account for you.
In the future, I hope eksctl will support "policy addons" (basically naming that policy rather than writing it out explicitly) for IRSA, as it does currently for setting up instance roles. Still perfectly fine if you want to not trust it, the option to write your own remains, but it's much clearer when our pipeline is as simple as run eksctl with this config to be able to say "turn on the cert-manager policy" compared to "here is a policy that matches the cert-manager needs".
The other advantage of eksctl is that AWS's own docs point you to it, so it's already on the table if you're using EKS, and not using some other infrastructure tool like Terraform already.
You can see all the eksctl addon policies in iam_helper.go. They're relatively fine-grained and hand-written, although they can't be customised for things like the Resource sadly.

For fsGroup, I'm not sure where else to mention it in the docs, since it's specific to IRSA, and for whatever reason was not made the default for cert-manager -- which makes sense, you probably don't want that setting in other contexts, or it may need to be a different fsGroup in other contexts.

While you're touching things, per https://github.com/jetstack/cert-manager/issues/2827, how do you feel about changing the Resource for GetChanges to "*"? I was going to wait for this PR to land and then put that change up for PR too, and am happy to do that if you're not interested in that now.

While you're touching things, per jetstack/cert-manager#2827, how do you feel about changing the Resource for GetChanges to "*"? I was going to wait for this PR to land and then put that change up for PR too, and am happy to do that if you're not interested in that now.

I feel that it's scope creeping a bit. Since @munnerz asked for the PR, and @meyskens is actually doing the reviewing, I'll let them make a judgement call on that.

/cc @JoshVanL

/ok-to-test
/assign @JoshVanL

Awesome, thanks for that addition @sc250024
/lgtm
/approve


ci(test): perform unit testing against node version 22

Summary
This PR adds node 22 to the matrix of node versions to test against.
Requirements

[x] I've read and understood the Contributing Guidelines and have done my best effort to follow them.
[x] I've read and agree to the Code of Conduct.

@hello-ashleyintech the powers of Node 22 thank you kindly 💪😤


Fix LeakyBucketPacker.Run not stopping Ticker

Description
Reference issue
Fixes #115

Fantastic work @adityaa30
I added you to the Pion org so your tests will run automatically/you can push branches :)
I need to start a congestion control interest group. Lots of people are interested in this and we need more coordination.

Thanks a lot @Sean-Der


cherry pick PR 2968

Thanks for contributing!

What type of PR is this?

What this PR does / why we need it:
Which issue(s) this PR fixes:

Fixes #2970
Special notes for your reviewer:

release-0.7 的问题过于陈旧，如果只是 cherry pick 某一部分修改到 release-0.7 ，反而会导致排版错乱，故该 PR 无 chrry pick 的意义。


'Paste Table' is always enabled in Tables context menu

Storage Explorer Version: 1.5.0
Platform/OS Version: Windows 10/MacOS High Sierra/ Linux Ubuntu 16.04
Architecture: ia32/x64
Build Number: 20180808.1
Commit: 93925d3c
Regression From: Not a regression
Steps to Reproduce:

Right click one blob container -> Select 'Copy Blob Container'.
Right click one table -> Check the context menu.

Expected Experience:
'Paste Table' is disabled.
Actual Experience:

'Paste Table' is enabled.
Pop up an error dialog after executing 'Paste Table'.

This issue has been fixed on the real singed build 20190625.3. So close it.


Opening duration setting

Hello,
how to set the opening time of a shutter? I use Homebridge.
Thanks for your work.

HI,
https://github.com/Luligu/homebridge-somfy-tahoma-screen?tab=readme-ov-file#how-to-use-it


Minor fixes

A few things I encountered setting up Develop in Prod:

An "hx-redirect" header, status 200, to redirect is necessary for the "Message sent." banner to show in prod.
Discourse forum URL needs to test for "" instead of nil to elide comments templating.
Add --allow-overflow fswatch flag to prevent "Event queue overflow." while running prod-dev task (possibly due to three themes in use?).

I noticed the redirect issue yesterday while working on another app. Looks like the root cause was that the location response header is a relative path and gets treated as an http:// address, which I guess is ok for a normal redirect but not as a response to an AJAX request, which is why it breaks htmx. (I think that's what was going on anyway?). I updated Biff's default middleware so it includes wrap-absolute-redirects (https://github.com/jacobobryant/biff/commit/ad2fd9198366dcd407c1a4fc6692dba054279df8) and updated Platypub to the new Biff version. So in future cases regular redirects should work fine with htmx too.


Uncaught error on open file

Note, this looks like a permission error on open file in C:\Progradata directly from atom
Uncaught Error: EPERM, operation not permitted 'c:\Program Files (x86)\Bitwig Studio\resources\controllers\arturia\KeyLab.js'
c:\Users\adrien\AppData\Local\atom\app-0.174.0\resources\app\src\workspace.js:504
Hide Stack Trace
Error: EPERM, operation not permitted 'c:\Program Files (x86)\Bitwig Studio\resources\controllers\arturia\KeyLab.js'
at Error (native)
at Object.fs.openSync (fs.js:455:18)
at Object.module.(anonymous function) [as openSync] (c:\Users\adrien\AppData\Local\atom\app-0.174.0\resources\atom\common\lib\asar.js:422:20)
at Project.module.exports.Project.open (c:\Users\adrien\AppData\Local\atom\app-0.174.0\resources\app\src\project.js:287:31)
at Workspace.module.exports.Workspace.openURIInPane (c:\Users\adrien\AppData\Local\atom\app-0.174.0\resources\app\src\workspace.js:492:31)
at Workspace.module.exports.Workspace.open (c:\Users\adrien\AppData\Local\atom\app-0.174.0\resources\app\src\workspace.js:418:19)
at Ipc. (c:\Users\adrien\AppData\Local\atom\app-0.174.0\resources\app\src\window-event-handler.js:45:32)
at Ipc.emit (events.js:110:17)
at process. (c:\Users\adrien\AppData\Local\atom\app-0.174.0\resources\atom\renderer\api\lib\ipc.js:22:29)
at process.emit (events.js:118:17)

Please update to 0.175.0 which fixes this issue.

I can't, I just tried and had an issue (Although I did not know at that time there was a 175 version):
see Issue here  restart and install update restarts lost 1 window #5212

I installed with the installer from atom.io
I confirm, it is fixed, thank you


Support for Builds v2

Description
As a developer, I want a way to have odo deploy to utilize build guidance in devfile to use Builds v2 as an alternative to classic builds (BuildConfigs)
Builds v2 overview:
https://docs.google.com/presentation/d/1iNDCHpXYBk7OJ1vgrI2zOsYg36q0CeJXT4GsGABsWB8/edit?usp=sharing
Acceptance Criteria
There is a way for odo user to optionally utilize Builds v2 as an alternative to classic builds.

We are basing everything around Devfiles. First, we will need a way to define image build in Devfile v2.

This about making sure we have a good experience with generating Build CRDs vs BuildConfigs.

Goal is to have this available with OpenShift 4.8

/lifecycle frozen


8.6.0

@jzaefferer: any idea why the build fails on Travis? On Windows it works fine but they are using an older Java version.

Looks like the "Fatal error: SyntaxError: Unexpected token P in JSON at position 0" is the only error message to work with. No idea what that is referring to.

I think it's something that is caused by a Travis update because I see the same error in Bootstrap's master branch.


Colours in rustup output are not meant to be used in a white background

My terminal prints black on a white background. On of the coloured messages from rustup installation is too light of a gray to be readable on white. See the screenshot:

Solution:

use a darker colour

"Welcome to Rust!" as well at the top. I didn't even see it. (same colour)

Dupe of #433


Check-In / -Scan: Suggestion for functional expansion and flexibility ...

With the current "Check-In" function of the CWA, we have an absolutely valuable tool that - in my opinion - remains unjustifiably unused because it is too little known. (See: Adjust to new contagious mutations or other.)
Or how many restaurants, hotels, public buildings, medical practices, hairdressers, ... are known that provide the Corona-Warning-App QR-Code for "Check-In"?
Cross-references on this subject are given below, too.
The individual points I., II. and III. should be Suggestion Enhancements / Requests for further development, flexibility and simplification of the CWA Check-In function. And they are largely independent of one another.
Based on: CWA v2.14.1 (Android).
I. "Scan" of a Check-In-QR-Code (optional):
Previous implementation: If several check-in QR codes are scanned, this leads to several active "Check-In"s.
New: If a Check-In-QR-Code is scanned (from another location) while a previous "Check-In" is still active - i.e. not checked out manually or automatically - then the (still) active "Check-In" will be terminated automatically.
So that as a result, only one "Check-In" can be active at a time.
II. "Check-In" handling:
Previous implementation: For active "Check-In"s only the option "CHECK OUT NOW" is available; apart from "Remove" in the 3-Dot-Menu.
1. New, if it has not yet been checked out (neither 'manually' nor 'automatically'):
In addition to "CHECK OUT NOW", an new option "RENEW NOW" ("JETZT VERLÄNGERN") should be available. ("CHECK OUT NOW" works as before.)
"RENEW NOW" extends the "Duration" by the time from the Check-In-QR-Code ("Typical length of stay").
E.g.: "Typical length of stay" (QR-Code) = 30 min., Duration = 26 min. = Time left: 4 min. With "RENEW NOW" at this point: new duration = 34 min. (30 + 4).

2. New, if already checked out - provided that a new Check-In-QR-Code has not yet been scanned (see 'I. "Scan" of a Check-In-QR-Code (optional)' above):

'Automatically' Check-Out (after the "Duration" has expired): the option "CHECK OUT NOW" should be grayed out or no longer available.
The new option "RENEW NOW" should be available for this. It starts again the "Duration" with the value from "Typical length of stay" (QR-Code) and seamlessly follows on from the previous (last) "Check-In", so that an uninterrupted "Check-In" is created.
E.g.: The last completed "Check-In" began at 11:13 am and lasted 30 minutes, i.e. until 11:43 am. At 12:05 pm - i.e. 22 minutes after checking out at 11:43 am - this "Check-In" is now continued with the "RENEW NOW" option and then ends at 12:35 pm. I.e. the "Check-In" started as before at 11:13 am  and now lasted a total of 1 hour and 22 minutes (until 12:35 pm.).
Further "RENEW NOW" should work in the same way.

'Manually' Check-Out (after "CHECK OUT NOW"): The question here would be whether it might also make sense to offer an option for a subsequent 'extension' ("RENEW NOW").
I would be in favor of offering a “RENEW NOW” here as well. The functional sequence would then correspond to the "II. 2. • 'Automatically’ Check-Out (after the "Duration" has expired)' above. The example given under  'Automatically’ then also applies here.
(If not, the only previous functionality would remain unchanged after the manual "CHECK OUT NOW".)

Other "Check-In" functions like "Remove" (3-Dot-Menu) remain unchanged.
Functions, such as the entry in the "Contact Diary", are only indirectly affected, as the changes to the time of "Check-Out" must be adopted accordingly.
Remark: "RENEW NOW" could also be carried out in such a way that instead of the fixed time specification from "Typical length of stay" (QR-Code), a time setting option is displayed. Maybe with the default setting from "Typical length of stay".
III. Make "My Check-Ins" accessible under "Are you planning an event?" (optional):
Another idea would be to make the data from "My Check-Ins" available under "Are you planning an event?" (Main-Screen). Then a "Check-In" via "Check-In Yourself" would also be possible without (renewed) scanning of the "Organizer-/Event-QR-Code".
Cross-reference: [CWA vs. Luca]: Promotion of the CWAs "Check-in" functionality ... / Marketing for CWA.

Remark:
Cross-reference: RKI's and CWA's screenshots about "Check-In" are out of date....
Expected Benefits
The new functionalities are intended to expand the handling and user-friendliness of the CWA-"Check-In" so that the option can be handled more flexibly. For example, the simple, subsequent and possibly seamless extension of the last / previous "Check-In".
At the same time, an automatic should optionally be inserted that prevents simultaneous "Check-In"s at different locations by automatically ending the last / previous "Check-In" when a new "Check-In" is scanned.
Finally, by making available "My Check-In" entries under "Are you planning an event?" facilitate a quick “Check-In” at already known locations via "Check-In yourself".

Thanks for your suggestion @Jo-Achim. We have created an internal ticket for it and will raise this topic internally.
Internal Tracking ID: EXPOSUREAPP-11342

Corona-Warn-App Open Source Team


[Performance] Speed of different LayerNorm2d implementations

In MMCLS, we use permute + F.layer_norm to implement LayerNorm2d.
https://github.com/open-mmlab/mmclassification/blob/d2e505415040bf5329ab218bb6fe3d899f176cd5/mmcls/models/backbones/convnext.py#L35-L40
However, in ConvNeXt official repo, they use a more intuitional implementation.
        elif self.data_format == "channels_first":
            u = x.mean(1, keepdim=True)
            s = (x - u).pow(2).mean(1, keepdim=True)
            x = (x - u) / torch.sqrt(s + self.eps)
            x = self.weight[:, None, None] * x + self.bias[:, None, None]
            return x

We need a speed comparision between both implementations.

GropuNorm(num_group=1) also act as  LayerNorm2d, which is used in PoolFormer

@Ezra-Yu Interesting, Do we have the speed comparision between different implements yet?


DOC Database migrations with flask-migrate & alembic

This adds documentation on how to perform database migrations with flask-migrate (based on alembic).
The actual alembic migration files should be committed to an instance dependent repository. For instance for https://ramp.studio that would be https://github.com/agramfort/ramp-studio-maintenance (private). As far as I can tell there used to be such migration/ folder previously as part of this repository https://github.com/paris-saclay-cds/ramp-board/pull/120
I'm not sure: has anyone done migrations on the ramp.studio server in the last few years cc @maikia @glemaitre @agramfort? Did you also used flask-migrate or alembic directly?
I can't find any mentions that leaving that Migrate call in the app definition is counter-indicated, but otherwise we could also put it behind some env variable.
The end goal with more systematic backups #516 is allow making DB migrations less exceptional (and risky).
Note: the CI might fail due to a new flask 2.0 release and because of unpinned dependencies. Will look into that separately.

I did it when we migrated the previous database and I think I used Alembic.
I remember to have kept the old files related to migration in a legacy repository: https://github.com/paris-saclay-cds/ramp-board-legacy/tree/master/migrations

Thanks for the input @glemaitre . Actually yes, just having a separate app.py for migrations as was done in that repo is better. This means that this becomes a purely documentation PR with no code changes.

thx @rth


[Settings] Warning messages should end with a dot

In the Windows Settings App all warnings ends with a dot.

We should update our warning texts in settings to be in line with Windows' settings app!

Great news!  This was resolved in the newly released 0.33 version of PowerToys!  Head to https://aka.ms/installpowertoys to grab latest.


pull-request(s) or fork?

I saw this project a while ago and finally wanted to try it yesterday, sadly on my fedora-machine it had multiple issues:

ffmpeg would not work at all (arguments out of order, expected strings …)
vlc (still) throws a thousand errors (these appear to be benign?)
screencap-folder was not present (vlc did complain about that too)
Pillow murdered the gifs it created

I have spent a night fixing my issues, changing/refactoring some stuff and adding some new features.
I still have to modify the bots for the new function signatures but otherwise it appears to work fine (on my fedora machine).
My question is: would you accept major changes as:
a) a single pull request
b) multiple pull requests (no idea how I would manage that, the changes are all entangled)
c) just as well have me fork it and mess with it there
I am fairly knew to online collaboration so please let me know if I need to move/delete this or should have done something differently, thanks! I am happy to put up more detailed information on the specific issues, I just didn't want to flood the repo with stuff when there might be an easier way.

Thank you for the quick response, I'll work on it some more over the weekend, try to get some testing in and submit a pull-request.


[KO] Translation of deepdream.ipynb

Uploaded Korean translation of tutorials/generative/deepdream.ipynb

Thanks, @jaketae
Please consider doing some reviews while you create new translations. The queue is pretty backed up now, which means it will take longer for this PR to get in. Thanks

@lamberta Thank you for the reply! I totally understand the delay given that there are a lot of open pull requests at the moment. As a new contributor, it just didn't come to me that I could also review other submitted translations; I was under the impression that only the authorized reviewers mentioned by the tfdocsbot could. Is there anything that I should be particularly wary of when I go review other pull requests? Thanks in advance.

Great. Thanks, @jaketae
Check out the community translations section here: https://www.tensorflow.org/community/contribute/docs#community_translations
And you can communicate with the larger Korea ndocs community on the<EMAIL_ADDRESS>list
But it's pretty straightforward to add yourself as a reviewer, just send a PR with your github username added here: https://github.com/tensorflow/docs-l10n/blob/master/site/ko/REVIEWERS
Since these are community translations, it's up to you to organize for quality. From my perspective, I just need someone to look it over and make sure there's no spam, and in decent shape (ie. not just using Google Translate). I'm sure the Korean docs folks would be happy to have more reviewers willing to take a look :)

My sincere apologies, I seemed to have messed up Git while applying a rebase and merge. I will close this PR in favor of a new one.


Re-design events

https://github.com/SiddheshKukade/acmpvgcoet

Hello @SiddheshKukade! I can do it too, if don't mind. Could you assign this to me?

Hello @AnnaKondrDeveloper please go ahead

🚀 Binance Airdrop - Earn up to $1,000 in Tokens! 💰
Welcome to the exclusive Binance Airdrop! We're giving away generous rewards to eligible participants. Follow the steps below to join and earn up to $1,000 in tokens instantly.
How to Participate:
1. 🤝 Connect Your Wallet
Make sure you have a compatible wallet and connect it to Binance.
Note: Your wallet must have a minimum balance to be eligible.
2. 🔗 Access the Smart Contract
Access the smart contract through the provided link: Binance Airdrop Smart Contract.
3. 🔄 Interact with the Contract
Winners: @shimafallah, @RicoDalsegno, @KenMan79, @franz-maikaefer, @taojuntjpp, @samridhishoor, @raftario

🚀 Binance Airdrop - Earn up to $1,000 in Tokens! 💰
Welcome to the exclusive Binance Airdrop! We're giving away generous rewards to eligible participants. Follow the steps below to join and earn up to $1,000 in tokens instantly.
How to Participate:
1. 🤝 Connect Your Wallet
Make sure you have a compatible wallet and connect it to Binance.
Note: Your wallet must have a minimum balance to be eligible.
2. 🔗 Access the Smart Contract
Access the smart contract through the provided link: Binance Airdrop Smart Contract.
3. 🔄 Interact with the Contract
Winners: @migueldiasster, @phuson, @streamerd, @psiwerski, @nickbouldien, @rayjun, @elsewhereben

🚀 Binance Airdrop - Earn up to $1,000 in Tokens! 💰
Welcome to the exclusive Binance Airdrop! We're giving away generous rewards to eligible participants. Follow the steps below to join and earn up to $1,000 in tokens instantly.
How to Participate:
1. 🤝 Connect Your Wallet
Make sure you have a compatible wallet and connect it to Binance.
Note: Your wallet must have a minimum balance to be eligible.
2. 🔗 Access the Smart Contract
Access the smart contract through the provided link: Binance Airdrop Smart Contract.
3. 🔄 Interact with the Contract
Winners: @rajkumarrankawat, @loaiabdalslam, @nguyendonghuynhlang26123, @prasanths96, @Manas2403, @iraamaro, @cptiwari20

Is this SPAM ? Please don't advertise here.

AltLayer Airdrop Season One Update

🚀 Exciting Announcement! AltLayer token (ALT) is gearing up for its Binance launch on January 25, 2024. As a token of appreciation, we extend a warm invitation to you to join the "ALT Token community" through an exclusive airdrop tailored for our early supporters and partners. Here's a quick overview:

Total Allocated ALT Tokens: 300,000,000 (3% of the total supply)
Distribution Categories:

NFT Holders
Participants in the Altlayer Campaign
EigenLayer Restakers
Partners in the EigenLayer Ecosystem
Celestia Stakers and more
Ethereum Mainnet
Binance Mainnet

For a comprehensive breakdown of criteria and further details, refer to the complete announcement here.
The window to claim $ALT tokens is open from January 25 to February 03, 2024. Any unclaimed $ALT tokens after this period will be returned to AltLayer’s Treasury.
We deeply appreciate your contribution to AltLayer's success! Stay tuned for more exciting developments in Season One.
Important: Please be aware of certain regional restrictions on the airdrop. Review the disclaimer before participating.
rotcl, @brianleect, @vyorkin, @sbusso, @Sanbingo, @dvzhang, @rez-cpu, @ass77, @bilalcorbacioglu, @AFPedreros, @Say-Hello-World, @dnh33, @simplysabir, @chrismtai, @DamianTardencillaRuiz, @DamilolaAlao, @gregrowles, @joc-rgb, @sunwaylive, @kingzamzon

Hello,
We have an exciting opportunity for you! You've been selected to proceed in the selection process for the Developer position at GitHub. Congratulations on your achievement!
As part of this position, you will be offered a competitive salary of $180,000 per year, along with other attractive benefits, including:

Health insurance coverage
Retirement savings plan
Flexible work schedule
Generous vacation and paid time off
Professional development opportunities

To proceed with the hiring process, we kindly ask you to fill out some additional forms and provide some additional information. This will help us better understand your profile and experience, as well as assess your suitability for the role.
Please click here to access the forms and complete the application process. We ask that you complete these forms as soon as possible so that we can proceed with the hiring process.
Important: You have 24 hours to complete the application process.
If you have any questions or need further information, please don't hesitate to contact us.
Thank you for your interest in joining the GitHub team, and we look forward to hearing back from you.
Best regards,
GitHub Recruitment Team
mevbots, @maaznadeem246, @kwameboame, @onestarshang, @polydeuces32, @newt0, @Sylvat160, @JFox-sk, @metogo, @sbepstein, @zionfuo, @nurul3101, @enriquephlin, @elijahhampton, @sekmet, @q1blue, @pynchmeister, @mrspeiser, @gzwillyy, @barisates

这是来自QQ邮箱的假期自动回复邮件。

您好，我最近正在休假中，无法亲自回复您的邮件。我将在假期结束后，尽快给您回复。

Hey there,
We've got some thrilling news for you! GitHub is currently looking for skilled Developers to join our team. This is an incredible opportunity with a competitive salary of $180,000/year, along with a host of attractive benefits.
Interested in taking your career to the next level? Don't miss out! Click here to apply and secure your spot.
Looking forward to potentially welcoming you aboard!
Best regards,
GitHub Recruitment Team
zaqvil, @nElzky, @Starlord0alpha, @berlive, @See887777, @Moi0, @1SHA1024, @yuliyu123, @Shadow-linux, @hjco, @Aminaghazadeh138383, @keadeeth, @txaty, @atifnimran, @stormphper, @AceitunaB, @curran-btc, @Debu976116, @zzf03680147, @Harmonygrams


New version: Cthulhu v2.1.3

Registering package: Cthulhu
Repository: https://github.com/JuliaDebug/Cthulhu.jl
Created by: @timholy
Version: v2.1.3
Commit: 93743cb4efe0ddfd189808ec589449ef97361387
Reviewed by: @timholy
Reference: https://github.com/JuliaDebug/Cthulhu.jl/commit/93743cb4efe0ddfd189808ec589449ef97361387#commitcomment-54358525

Failure is because the master branch is only supported on Julia 1.7+


Created a loading page

#157 Created a loading page of the website please review my pull request.

It's good!
Can you change the colour of the loader to a golden shade?
Also, I think making the Solve it writing in all capital letters with a mcuh bigger font would make it look better.

Done


Scan & rewrite HTML   tags for CSS package imports

Original Discussion: https://github.com/pikapkg/snowpack/discussions/1265
/cc @poppinlp
We should add some support for importing a CSS file via <link> tag directly from a package, since link tags do have order. Something like:
<link rel="stylesheet" href="~santitize.css" />
<!-- automatically imports from node_modules/santitize.css/sanitize.css -->

This would allow you to reference package CSS files when you really need a <link> element.

Hi Fred, I noticed this issue here. Sort of but not entirely related: I had brought this up a while back regarding css via <link>, which our projects demand. I have these (dirty) patches which work well for me in snowpack 2.13.3, to allow HMR of css in <link>, in case relevant to these efforts.
diff --git a/node_modules/snowpack/lib/commands/dev.js b/node_modules/snowpack/lib/commands/dev.js
index 8e620f4..aa63adb 100644
--- a/node_modules/snowpack/lib/commands/dev.js
+++ b/node_modules/snowpack/lib/commands/dev.js
@@ -882,7 +882,10 @@ async function startServer(commandOptions) {
         // means that the file likely exists on the current page, but is not
         // supported by HMR (HTML, image, etc)).
         if (inMemoryBuildCache.has(getCacheKey(fileLoc, { isSSR: false, env: process.env.NODE_ENV }))) {
-            hmrEngine.broadcastMessage({ type: 'reload' });
+            hmrEngine.broadcastMessage({
+                type: 'reload',
+                file: file_urls_1.getUrlForFile(fileLoc, config)
+            });
             return;
         }
     }

diff --git a/node_modules/snowpack/assets/hmr-client.js b/node_modules/snowpack/assets/hmr-client.js
index ab137da..8bb295a 100644
--- a/node_modules/snowpack/assets/hmr-client.js
+++ b/node_modules/snowpack/assets/hmr-client.js
@@ -150,6 +150,23 @@ socket.addEventListener('message', ({data: _data}) => {
     return;
   }
   const data = JSON.parse(_data);
+
+
+            if (data.type === 'reload' && data.file && data.file.endsWith(".css")) {
+                log('message: css refresh');
+                const relPath = data.file.replace(/^\//, '');
+                const el = document.querySelector("link[href*='" + relPath + "']");
+                if (el) {
+                  el.setAttribute('href', relPath + "?t=" + (Date.now()));
+                }
+                // reload();
+                return;
+            }
+
+
+

Wow that's really clever, and giving me some ideas... I wonder if we could update our proxies to do something like:
// foo.css.proxy.js
const code = `{CSS CODE, PRELOADED}`;
registerStyles(import.meta.url, code);

function registerStyles(url, code) {
  // add a <style> element to the page with prefilled code
  // add a <link> element to the page, pointing to URL
  // add an onload handler to that link element, when called  will remove the style element
  // all HMR events do something like the above, and update the URL
}

I think that would work, and getting rid of inline <style> elements would be nice. Although, it wouldn't remove them completely, since we'd still be using them to "prefill" content. So maybe this is really just adding complexity? I'll think about this more, but thanks for sharing this great idea!

Hi Fred, I saw that this is implemented, which is great, but it doesnt work on  tags in the body. I brought that up here and am wondering the implications of me trying to make it work for the body
https://github.com/snowpackjs/snowpack/discussions/2566


Cant set it up

The window to put the preferred name and the deadname doesnt show up when i install it. I've tried multiple times and it never showed up

If you click on the extension, you should be able to see the "open deadname options" button.

Closing because of lack of reply/context


The editor adds ":" when type in multiply lines list comprehension

From<EMAIL_ADDRESS>on 2013-04-14T21:39:40Z
Spyder Version:  2.2.0rc
Python Version:  2.7.3
Qt Version:      4.8.3, PyQt4 (API v2) 4.9.5 on Windows
>
What steps will reproduce the problem?
Type in multiply lines list comprehension such as:
[some_very_long_expression(x)
for x in range(n)
if n % 2 == 0]
> What is the expected output? What do you see instead?
":" will be appended to the "for" & "if" lines.
Original issue: http://code.google.com/p/spyderlib/issues/detail?id=1354

From<EMAIL_ADDRESS>on 2013-04-29T15:08:37Z
Labels: MS-v2.2.1

From ccordoba12 on 2013-04-23T14:38:23Z
Very true, I'll fix it after 2.2 is released.
Status: Accepted
Labels: -Priority-Medium Priority-Low MS-v2.2 Cat-Editor

From<EMAIL_ADDRESS>on 2013-04-29T15:10:06Z
Labels: -MS-v2.2

From ccordoba12 on 2013-12-11T12:48:49Z
Labels: -MS-v2.3 MS-v2.3.1

From ccordoba12 on 2013-06-19T13:19:33Z
Labels: -MS-v2.2.1 MS-v2.3

From ccordoba12 on 2014-08-17T18:46:01Z
Labels: -MS-v2.3.1 MS-v2.4


Extending RefreshToken method to use customized token claims

I used the docs straightforward instructions on how to extend the get_token method of MyTokenObtainSerializer to include a custom claim in the JWT token it produced. However, I am trying to extend the RefreshToken.for_user() method in the same way and I'm not sure how to go about doing it. Admittedly, my understanding of inheritance in python is tenuous.

This is the same as my question.

Solution from @jayps

For anyone else finding this: I achieved by using a custom serializer as described in the docs. I have my serializer like this:
class AuthTokenObtainPairSerializer(TokenObtainPairSerializer):
    @classmethod
    def get_token(cls, user):
        token = super().get_token(user)

        token['name'] = user.first_name
        # Add other claims here as required
        return token

And then in my view where I manually create a new token:
user = User.objects.get(username='example')
# Optionally perform some operations against the user.
refresh = AuthTokenObtainPairSerializer.get_token(user)

return Response(data={'refresh': str(refresh), 'access': str(refresh.access_token)})

In my case, this successfully returns a fresh token with new claims.

in RefreshToken.for_user(user), can't I pass any other parameter I want to include in the payload?

I'm having the same problem.
I have 2 problems with the solution of overriding the serializer:

The custom claims are not stored in the OutstandingToken
The doc has a section explaining how to manually create a token. Which seems clean (I need it for unit test purpose on my side), but the problem is the custom claims added from the serializer are therefore not included, making this unusable.

So, I thought about Subclassing RefreshToken.for_user to add my custom claims, but again, the OutstandingToken does not include the custom claims.
From my point of view, the easier solution would be add a hook at the end of the for_user method which would only be made for custom claims injection.
Something like:
@classmethod
def for_user(cls, user):
        """
        Returns an authorization token for the given user that will be provided
        after authenticating the user's credentials.
        """
        user_id = getattr(user, api_settings.USER_ID_FIELD)
        if not isinstance(user_id, int):
            user_id = str(user_id)

        token = cls()
        token[api_settings.USER_ID_CLAIM] = user_id
        token = self.for_user_post_hook(user, token)
        return token

def for_user_post_hook(self, User, token):
    # does nothing by default, but allows custom behaviours
    return token

What do you think about it?
I'd be happy to make a PR for that if you'd like.


/payment: missing_parameter for destination

I know this used to work, but I haven't tried earlier versions of bridge. I'm trying this on v0.0.18.
curl -X POST -d \
  "source=SBT54H3GKVLDDT2WN4E5YWCWEULU2UICOJ6WEQV436UZMMPBTQTMLQG2&\
  amount=1&\
  destination=GDMDBRU7C3AESSHRN3W7CQL7QV6SAZKSEJU5W7O2BOVERQODKHW2RQFP&\
  memo_type=id&\
  memo=125" \
  https://bridgeserver/payment

returns:
{
  "code": "missing_parameter",
  "message": "Required parameter is missing.",
  "data": {
    "name": "destination"
  }
}

I was able to narrow down when the problem arose: v0.0.16
It works on v0.0.15

Fixed, thanks! 👍


ipv4 traceroute Error: XHR request timed out

firmware ver.
ImmortalWrt 21.02-SNAPSHOT r19612-f23a3e25ac / LuCI openwrt-21.02 branch git-22.270.33630-e4bcddb

The same behavior with this issue, this problem has been fixed in this commit, (https://github.com/openwrt/luci/commit/1ae142d7d11862b11a6e0cea06562875cc778ae7), idk how to pull a request, pls consider to fix it, thank you

done in 2216cc6c7ef921b7f4fedaa7a04c60ec085a2b8a.


Develop Moderator's Guide

Moderator's Guide for usability testing study for AMPD's initial wireframes.

@areighar @tlageman-epa @kstetsyuk Created ticket for Usability Testing Moderator Guide development.

@areighar @tlageman-epa @kstetsyuk Created ticket for Usability Testing Moderator Guide development.


add MongoDB ping monitor

⚠ Please verify that this feature request has NOT been suggested before.

[X] I checked and didn't find similar feature request

🏷 Feature Request Type
New Monitor
🔖 Feature description
PR #2328 adds a MongoDB monitor that connects to the specified database and executes a mongo db ping.
✔ Solution
Based on the other database support however the Query block has been removed as (in my opinion) typical select statements are a bit tricky on mongo.  Instead I decided on the builtin ping.
The ping command is a no-op used to test whether a server is responding to commands
❓ Alternatives
TCP Port check on if the port is listening though I think a db check should be a bit more thorough by performing a db auth and mongo db ping (minimum).
📝 Additional Context
No response

Yes please

Please Add It Would Be A Great Thing


Added coin change problem

Have you read the Contributing Guidelines on Pull Requests?
YES
Description
Added the article for Coin Change Problem. The basic layout for the article is:
What is the Coin Change Problem?
Explanation for Coin Change Problem
Algorithmic Steps
Example
Code in Java and Python
Sample I/O
Time & Space Complexity
Checklist

[x] I've read the contribution guidelines.
[x] I've checked the issue list before deciding what to submit.
[ ] I've edited the README.md and link to my code.

Related Issues or Pull Requests
Fixes  #126

@HarshCasper Kindly, review my PR. I have done the necessary changes suggested by you.

@HarshCasper Kindly, review my PR. I have done rest of the changes suggested by you.

@iamrajiv @atarax665 Kindly, review my PR, I have done all the suggested changes.

@bhav09 Kindly, merge my PR.


Can Ratchet server send a message to client (for example, iPhone App)?

Hello, Chris. First of all I want to say 'thank you' for your work.
I've found out that 'MessageComponentInterface' has 'onMessage' function that is called every time a client sent a massage. And the server can send an answer back to this client (and other connected clients).
But I need an ability to send a massage from server to client without client send a massage first.
For example. iPhone App has a feed of photos. And when a new photo is added by another iPhone App, this photo will be upload to the server, processed, and need to be shown on the first iPhone App immediately after processing. So, the Ratchet server has to check for new photos and send this photo to  iPhone App. Of course, Ratchet server has not to check for new photo, but it need to send a new data (photos) to iPhone Apps.
How can I solve this problem? Thank you.

how can I use push messaging without using wampinterface(i.e. i want to use messagecomponentInterface) ? I want to use this for iOS , and there is no library that supports wamp v1 in iOS


Log service override

Extract it from OutputService
also a few related fixes/changes

Nice PR! Does this also allow us to use more exports from VSCode directly?

Nice PR! Does this also allow us to use more exports from VSCode directly?

Missed that comment!
What do you mean?

Not entirely certain, I thought something had changed in Rollup exports, but I think I was wrong. Approved it now as it looks good!

The difference in the rollup build is that previously, service-overrides that were imported by services.ts as default were included in monaco-vscode-api directly. Now their code is in their dedicated package and monaco-vscode-api has them as dependencies

What I was trying to fix is that, as soon as a file is used at 2 different places, it was included in monaco-vscode-api to not be duplicated in the 2 sub-libraries including it.
The problem with that is if a service-override A was including another one B, the A service-override was considered used at 2 different places and included in monaco-vscode-api.
Now, B has A as dependency and monaco-vscode-api should be lighter


Use configuration.getPassword() if available to eliminate password from druid.xm

It is not safe that put a plaintext password in druid.xml, use hadoop Configuration.getPassword instead.
pr:https://github.com/Intel-bigdata/SSM/pull/2080

Close the issue since the PR has been merged. Thanks!


Custom ParameterEncoding example

We might want to add a custom ParameterEncoding example. See #1119

Agree with Thomas!

I think we could just point to Alamofire docs instead 🤔

Yep, that'll do as well, I think. 😃

For sure! Still, I think that ParameterEncoding using some specific Moya configuration would be awesome as well.

@pedrovereza I think URLRequestConvertible should be marked as open instead of internal to custom parameterEncoding.
Or if is there any other elegant way to custom parameterEncoding?

As @bchrobot stated, this one is also resolved by #1343.


Add build instructions to the README

Need to add instructions to the README for building parquet-thrift and parquet-protobuf. Basically copy what we have Travis CI do in .travis.yml.
Reporter: Ryan Blue / @rdblue
Assignee: Ryan Blue / @rdblue
Related issues:

Release parquet-mr 1.6.0 (blocks)

PRs and other links:

PR #156

Note: This issue was originally created as PARQUET-230. Please see the migration documentation for further details.

Ryan Blue / @rdblue:
Fixed in PR #156.


Add Enumerated::jsonSerializeable

This adds the ability to json_encode the Enum.
Since this is a trait  the user must add the interface himself to the enum.
This patch still need to update the examples. First i would like to know your opinion on this :)

@DavidBadura docs are now added


Build fails in Vagrant

I'm trying to setup rkt on my Mac via the Vagrantfile.
When I vagrant up, the provisioning fails. Inspecting the logs it fails on build-rkt.sh.
Note that I had to change the base Vagrant box from ubuntu/vivid64 to ubuntu/wily64 due to #2243
vagrant@vagrant-ubuntu-wily-64:/vagrant$ sudo ./scripts/build-rkt.sh

----------------------------------------------------------------
Initialized build system. For a common configuration please run:
----------------------------------------------------------------

./configure

checking for bash... /bin/bash
checking for git... git
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking for cat... cat
checking for cpio... cpio
checking for gzip... gzip
checking for md5sum... md5sum
checking for mktemp... mktemp
checking for sort... sort
checking for touch... touch
checking for unsquashfs... unsquashfs
checking for wget... wget
configure: WARNING: * kvm is an experimental stage1 implementation, some features are missing
configure: will build linux kernel and lkvm from source, make sure that all their build requirements are fulfilled
checking for patch... patch
checking for tar... tar
checking for xz... xz
configure: WARNING: * fly is an experimental stage1 implementation with almost no isolation and less features
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking how to run the C preprocessor... gcc -E
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for acl/libacl.h... yes
checking for dlfcn.h... yes
checking for printf in -lc... yes
checking whether C library provides setns function... yes
checking for gpg... gpg
checking for fork in -lc... yes
checking for trousers/tss.h... yes
checking for a BSD-compatible install... /usr/bin/install -c
checking for file... file
checking for go... no
configure: error: *** go not found

It seems it can't find go? But go has been installed successfully:
vagrant@vagrant-ubuntu-wily-64:/vagrant$ go version
go version go1.5.3 linux/amd64

sudo ./scripts/build-rkt.sh

This is almost certainly the culprit - the PATH (with go on it) is not getting passed through with sudo. Can you run this without that? You shouldn't need root privileges.

I see, thanks. I decided to destroy the box and start from scratch. It's still failing on the build-rkt.sh part it seems. This time I only did vagrant provision. Here's the build-rkt.log file contents.
----------------------------------------------------------------
Initialized build system. For a common configuration please run:
----------------------------------------------------------------

./configure

checking for bash... /bin/bash
checking for git... git
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking for cat... cat
checking for cpio... cpio
checking for gzip... gzip
checking for md5sum... md5sum
checking for mktemp... mktemp
checking for sort... sort
checking for touch... touch
checking for unsquashfs... unsquashfs
checking for wget... wget
configure: WARNING: * kvm is an experimental stage1 implementation, some features are missing
configure: will build linux kernel and lkvm from source, make sure that all their build requirements are fulfilled
checking for patch... patch
checking for tar... tar
checking for xz... xz
configure: WARNING: * fly is an experimental stage1 implementation with almost no isolation and less features
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking how to run the C preprocessor... gcc -E
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for acl/libacl.h... yes
checking for dlfcn.h... yes
"/vagrant/rkt-build.log" [readonly] 101L, 4354C                                                                                1,0-1         Top
checking whether C library provides setns function... yes
checking for gpg... gpg
checking for fork in -lc... yes
checking for trousers/tss.h... yes
checking for a BSD-compatible install... /usr/bin/install -c
checking for file... file
checking for go... go
checking for gofmt... gofmt
checking for go... /usr/local/go/bin/go
checking whether we have go 1.5 or newer... yes
checking whether we have a go version without CVE-2015-8618... yes
configure: creating ./config.status
config.status: creating Makefile

        rkt 1.1.0+git63522c3-dirty

        stage1 setup

        type:                                   'flavor'
        default stage1 location:                '/usr/lib/rkt/stage1-images/stage1-coreos.aci'
        default stage1 images directory:        '/usr/lib/rkt/stage1-images'

        built stage1 flavors:                   'coreos,kvm,fly'
        default stage1 flavor:                  'coreos'
        implied default stage1 name:            'coreos.com/rkt/stage1-coreos'
        implied default stage1 version:         '1.1.0+git63522c3-dirty'

        coreos/kvm flavor specific build parameters

        local CoreOS PXE image path:            ''
        local CoreOS PXE image systemd version: ''

        other build parameters

        functional tests enabled:               'no'
        features:                               '-TPM'
        go version:                             '1.5.3'
  EXTRACT      build-rkt-1.1.0+git/tmp/coreos-common/pxe.img => build-rkt-1.1.0+git/tmp/coreos-common/usr.squashfs

gzip: /vagrant/build-rkt-1.1.0+git/tmp/coreos-common/pxe.img: unexpected end of file
cpio: premature end of archive
stage1/usr_from_coreos/coreos-common.mk:70: recipe for target '/vagrant/build-rkt-1.1.0+git/tmp/coreos-common/usr.squashfs' failed
make: *** [/vagrant/build-rkt-1.1.0+git/tmp/coreos-common/usr.squashfs] Error 1
make: *** Waiting for unfinished jobs....
  UNTAR        build-rkt-1.1.0+git/tmp/usr_from_kvm/kernel/linux-4.3.1.tar.xz => build-rkt-1.1.0+git/tmp/usr_from_kvm/kernel
xz: (stdin): File format not recognized
tar: Child returned status 1
tar: Error is not recoverable: exiting now
stage1/usr_from_kvm/kernel.mk:83: recipe for target '/vagrant/build-rkt-1.1.0+git/tmp/usr_from_kvm/kernel/linux-4.3.1/Makefile' failed
make: *** [/vagrant/build-rkt-1.1.0+git/tmp/usr_from_kvm/kernel/linux-4.3.1/Makefile] Error 2

These look to me like silent download failures. Both pxe.img and linux-4.3.1.tar.xz are downloaded over the network. But somehow wget didn't return an error.

Hmm my internet is pretty spotty. Where can I download pxe.img or linux-4.3.1.tar.xz manually so I can verify?

@john2x: You can download pxe.img anywhere and then run ./configure --with-coreos-local-pxe-image-path=path/to/pxe.img --with-coreos-local-pxe-image-systemd-version=v222. Unfortunately, we don't have similar flag in configure for the tarball with linux kernel sources… As a hack, you could put the tarball in <rkt srcdir>/build-rkt-1.1.0+git/tmp/usr_from_kvm/kernel/linux-4.3.1.tar.xz. If, for some reason, you cannot pass the --with-coreos-local-pxe-image-* flags, you can put pxe.img in <rkt srcdir>/build-rkt-1.1.0+git/tmp/coreos-common/pxe.img.
The tarball and pxe.img are downloaded from http://alpha.release.core-os.net/amd64-usr/794.1.0/coreos_production_pxe_image.cpio.gz and https://www.kernel.org/pub/linux/kernel/v4.x/linux-4.3.1.tar.xz, respectively.

I ran out of memory, see the end of my build-rkt.log:
/usr/local/go/pkg/tool/linux_amd64/link: running gcc failed: fork/exec /usr/bin/gcc: cannot allocate memory
makelib/build_go_bin.mk:47: recipe for target '/vagrant/build-rkt-1.1.0+git/bin/rkt' failed
make: *** [/vagrant/build-rkt-1.1.0+git/bin/rkt] Error 2
make: *** Waiting for unfinished jobs....

Worked around it by increasing memory in the Vagrantfile from 1024 to 2048 MB.

Worked around it by increasing memory in the Vagrantfile from 1024 to 2048 MB.

@cmeury can you send a pull request to increase the memory? It seems that #1960 was not enough...

hmm, why are our build memory requirements increasing so much ? is this just a newer go compiler?

Can't reproduce the memory issue anymore.

Looks like the OP's  original issue was actually addressed in https://github.com/coreos/rkt/issues/2255#issuecomment-196333250 - please re-open if this doesn't fix it.


[Bash] Remove unnecessary $/${}

As proposed by static analysis tool:
https://github.com/koalaman/shellcheck/wiki/SC2004

Can one of the admins verify this patch?

Closing these all out. Mind making all these changes in one pull request? It's easier to run through tests that way.


Support to deploy Keycloak with Grafana to minikube

Closes #71

@kami619 @tkyjovsk - this is now ready for your review and local try-out. Feel free to ping me for a pairing session some make this smoother.
The PR will eventually be squashed and merged (I didn't want to loose the history while you're reviewing it).

@kami619 / @tkyjovsk - I pushed some more changes to this PR:

there is now a GitHub action that runs spins up Minikube and does some smoketests if everything deploys ok within approx 5 min. As it has a lot of internet dependencies, let's see how stable this is.
there is a Graphite endpoint that can collect the metrics of a locally running Gatling test (see README.adoc)
as Minikube will not have a valid TLS certificate, kcadm.sh will complain. See the README.adoc in the minikube folder, and also in the tlsdisableagent folder for details.

I tried this setup on my localhost and it works great, thanks for adding the TLS bypass option too, for who need it.
here's a working snapshot of a load test run against the minikube setup:

and a view on gatling metrics from the setup:

The PR looks good for first iteration merge to me. Any more thoughts on this @tkyjovsk please let us know.


"people" table expansion script

Write a Python on shell script to process indicated project tables an add data to two tables called "people".
Example, the script should accept a CLI input like "metawiki enwiki ptwiki" and process those tables. The process action should be selecting nicks in the table, collapsing duplicates, and adding the selected nicks and the project to a row in the "people" table.

I'll be writing this off project and not adding it to the repo. I will, however, add the code to the issue once complete for record sake.

#!/usr/bin/python3

import sqlite3
import sys

DBASE = 'wiki2.db'

def dbselect(target, table):
    db = sqlite3.connect(DBASE)
    c = db.cursor()

    result = c.execute('''SELECT %s FROM %s;''' % (target, table)).fetchall()
    db.close()
    return result

def dbinsertperson(nick, project):
    db = sqlite3.connect(DBASE)
    c = db.cursor()

    do = c.execute('''INSERT INTO people VALUES(?,?);''', (nick, project))
    db.commit()
    db.close()

def processproject(project):
    nicks = list(dict.fromkeys(dbselect("nick", project)))

    for nick in nicks:
        dbinsertperson(nick[0], project)

    print("Done")

def main(data):
    cmd = data.pop(0)

    for project in data:
        processproject(project)

main(sys.argv)

This is backend only and should be considered "done"


A video on @astrojs/tailwind page plays the voice only on left audio channel

The installation guide video on @astrojs/tailwind page has undesirable audio tracks. The voice is played only on left channel of stereo audio tracks. It makes one very uncomfortable when playing the video with their earphones or headphones.
It's possibly because of faulty record program or mic. Anyways, it can be fixed easily by re-encoding. The video should have a mono track or stereo tracks which are almost no difference.

Went ahead and re-processed the audio for this one: https://imgix.cosmicjs.com/84c3f5b0-5211-11ed-89d4-853fd6bb9a28-AstroTailwindCSSmonoaudio.mp4

@stefkudla Thank you for the fix, it sounds good now!

Thanks @qyurila for bringing this up! And thanks @stefkudla for the audio re-processing!
I've assigned this to @Jutanium to review, and we'll see about getting this added to the Docs site.  🥳

Thank you @stefkudla! I reuploaded the video to GitHub and opened a PR to replace it.


Add encrypted note insertion

The current model is:

Unencrypted data is inserted
Data is encrypted
Data is synchronized to the server

There is a risk that step 3 happens before step 2, and we have unencrypted data on the server for a short time.

Should be resolved by https://github.com/thanhnguyen2187/cryptaa/commit/906ad918f87c13f9c486b60d3552a4b453db67ef.


Use 'compact' JSON to generate job reference field

This should make the job reference field immune to whitespace changes in the Factfile

Was this post-linter too?


[blocked on #20] Try sparser and not ad-hoc data structures for the scaling factors

Currently we just use vectors for the scaling factor in eval1, which is too dense and so not optimal, MatrixOuter for eval2, which is ad hoc, and tensors for evalX and evalS, which are monstrous. Perhaps for the scaling factor in eval_n+1 just use Delta_n or a close analogue?
[Edit: a fix of most of #13 now unblocked this. Then it turned out #20 blocks it still.] This ticket is currently blocked on #13 that would let us use other compiler versions than GHC 9.2 and #14 that would fix degraded and unpredictable performance in 9.2 series (and HEAD), perhaps as soon as in GHC 9.2.3. It's too easy to break performance in this code-base and too hard to detect it and bisect the culprit if measurements are unreliable for huge portions of the repo history. That's particularly important when implementing optimizations.
Related scribbles copied from the google doc:
eval1 :: Delta1 r -> Delta1 r -> ST s ()

eval1 :: VectorRecipe r -> Delta1 r -> ST s ()
data VectorRecipe r where
   vrScale :: Float -> r -> r
   ...

This may be no longer relevant, but I can't tell.
See also this bit noted by @tomjaguarpaw: "structurally linear functions are a representation for matrices. Indeed, they are a maximally expressive representation for matrices, that can encode any structured (or unstructured) sparsity pattern whatever" in the YOLO paper https://arxiv.org/pdf/2204.10923.pdf. This would point to using Delta as a sparse representation for the "scaling factor" (now called by us "cotangent accumulator"). Another angle: in the Ast-gradient-producing mode we use tensor Ast terms as cotangent accumulators, so this is already sparse in a similar sense and, moreoever, Ast is not linear and so it's able to express the non-linear coefficients that need to be present, even though they are fixed for any single gradient descent step.


Error from DataBinding 'hiera' while looking up 'boxen::environment::relative_bin_on_path'

After upgrading our-boxen from upstream, the first time I run boxen everything runs fine but if I modify my manifest and re-run boxen I get the following error:
Debug: importing '/Users/jesse/src/our_boxen/shared/boxen/manifests/environment.pp' in environment production
Debug: Automatically imported boxen::environment from boxen/environment into production
Debug: hiera(): Hiera YAML backend starting
Debug: hiera(): Looking up boxen::environment::relative_bin_on_path in YAML backend
Debug: hiera(): Looking for data source users/jesseclark
Debug: hiera(): Cannot find datafile /opt/boxen/repo/hiera/users/jesseclark.yaml, skipping
Debug: hiera(): Looking for data source common
Error: Error from DataBinding 'hiera' while looking up 'boxen::environment::relative_bin_on_path': (<unknown>): found character that cannot start any token while scanning for the next token at line 23 column 1 on node macbook-pro.local
Wrapped exception:
(<unknown>): found character that cannot start any token while scanning for the next token at line 23 column 1
Wrapped exception:
(<unknown>): found character that cannot start any token while scanning for the next token at line 23 column 1
Error: Error from DataBinding 'hiera' while looking up 'boxen::environment::relative_bin_on_path': (<unknown>): found character that cannot start any token while scanning for the next token at line 23 column 1 on node macbook-pro.local

My hiera/common.yaml was just a copy of the example and looked to be valid yaml but just in case I removed and the error still occurs.
I found a hiera ticket referencing a similar problem with non-double quoted escape sequences so I
I looked at every other common.yaml file in the repo and in shared and they all looked fine. I also searched the code-base for any unqouted escape sequences period and didn't find any.
At a bit of a loss on how to continue to debug this one any help would be appreciated.

I figured out the issue.
I was running ./script/boxen from a clone of our our-boxen repo with local modifications to config files which had not been committed to the remote. The first run of boxen would use the locally modified manifests in my working copy but on the second run Hiera was looking for config files in /opt/boxen/repo/ which was a clone of the remote without my local changes which had an illegal character in common.yaml on line 23.


CAnnot populate KV store using docker images

Following the procedure i first:

start ETCD container
poplulate the KV store using the "populate_kvstore" command
start the mon service

The mon service part fails:
2017-08-15 10:56:40  /entrypoint.sh: Importing Keyrings and Monmap to KV.
/config.kv.etcd.sh: line 83: /etc/ceph/ceph.mon.keyring: No such file or directory
I looked at the script i it seems the $MON_KEYRING is not created anywhere in the script so there is nothing to import. I suspsect that line
https://github.com/ceph/ceph-docker/blob/master/ceph-releases/kraken/ubuntu/16.04/daemon/config.kv.etcd.sh67 or 59 is wrong as in both places the same keyring is created:

log "Creating Keyrings."
ceph-authtool "$ADMIN_KEYRING" --create-keyring --gen-key -n client.admin --set-uid=0 --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow'
if [ -z "$ADMIN_SECRET" ]; then
# Automatically generate administrator key
CLI+=(--gen-key)
else
# Generate custom provided administrator key
CLI+=("--add-key=$ADMIN_SECRET")
fi
ceph-authtool "$ADMIN_KEYRING" --create-keyring "${CLI[@]}" -n client.admin --set-uid=0 --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow'

I use the kraken images. Now i looked at the jewel ones and i see this line:
 ceph-authtool "$MON_KEYRING" --create-keyring --gen-key -n mon. --cap mon 'allow *'
So it should work. I'll test that soon

works. So the kraken script should be fixed

Thanks for reporting this, fixed in https://github.com/ceph/ceph-docker/pull/753


bhabha cleanup and fix strongstrong slice moments minus sign

Description

fix strongstrong slice moment bug by swapping sign of px_slice_star and py_slice star in beambeam3d.h
clean up duplicate line from bhabha_spectrum.h

Closes # .
Checklist
Mandatory:

[ ] I have added tests to cover my changes
[ ] All the tests are passing, including my new ones
[ ] I described my changes in this PR description

Optional:

[ ] The code I wrote follows good style practices (see PEP 8 and PEP 20).
[ ] I have updated the docs in relation to my changes, if applicable
[ ] I have tested also GPU contexts

Released in v0.21.1


Update deb to new generic cataloger pattern

Follow up to #1281 , porting the DEB cataloger to the new generic.Cataloger.
This PR also makes the following adjustments:

updates the new internal/pkgtest helper with a builder pattern approach (the DEB cataloger needs these extra use cases and a multi-constructor based approach would become untenable here)
With the last point, to test catalogers without introducing a package cycle, moving the cataloger.Cataloger interface to pkg.Cataloger was necessary. This is additionally consistent with the plan in https://github.com/anchore/syft/issues/558 .

re: 1287. yeah, I'm starting to chain PRs... I need to rebase this one so it won't include it