Create fragments for questions on the review page

Problem
On the North Star review page, the applicant can see all the answers to their questions. This is a new way of rendering questions. This means we need new fragments for:
Mocks in https://github.com/civiform/civiform/issues/7935
Consider enums: https://github.com/civiform/civiform/issues/8795
Done when
New fragments exist for all question types:

[ ] address
[ ] checkbox
[ ] currency
[ ] date
[ ] dropdown
[ ] email
[ ] file
[ ] id
[ ] name
[ ] number_input
[ ] phone
[ ] radio_button
[ ] static_text (maybe not needed?)
[ ] text

Following ApplicantProgramSummaryView.java, we can accomplish this with potentially just 2 fragments:

[ ] File upload
[ ] Everything else


[Bug] Streaming output error of tool calling has still not been resolved.

I used the hermes_tool_parser.py as tool-parser-plugin and registered the parser as hermes_patched, but still have the same problem.
Already referred to #9874 #10395 #10398
Traceback (most recent call last):
  File "/app/hermes_tool_parser.py", line 228, in extract_tool_calls_streaming
    function_name: Union[str, None] = current_tool_call.get("name")
                                      ^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'get'
Error trying to handle streaming tool call.
Traceback (most recent call last):
  File "/app/hermes_tool_parser.py", line 292, in extract_tool_calls_streaming
    args_delta_start_loc = cur_arguments_json.index(delta_text) \
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: substring not found

Here is how I start vllm service with the latest package:
python3 -m vllm.entrypoints.openai.api_server \
--model /app/Qwen2.5-72B-Instruct-AWQ \
--port 7415 \
--tensor-parallel-size 2 \
--gpu-memory-utilization 0.95 \
--max-model-len 64000 \
--enforce-eager \
--disable_custom_all_reduce \
--enable-auto-tool-choice \
--tool-parser-plugin /app/hermes_tool_parser.py \
--tool-call-parser hermes_patched  \
--chat-template /app/qwen.jinja

I also tried using Docker image v0.6.3.post1 v0.6.4 v0.6.4.post1
Originally posted by @Sala8888 in https://github.com/vllm-project/vllm/pull/10398#issuecomment-2493504273

I got the same problem,
ERROR 11-23 01:34:01 hermes_tool_parser.py:338] Error trying to handle streaming tool call.
ERROR 11-23 01:34:01 hermes_tool_parser.py:338] Traceback (most recent call last):
ERROR 11-23 01:34:01 hermes_tool_parser.py:338]   File "/home/sycamore/.conda/envs/llm_env/lib/python3.10/site-packages/vllm/entrypoints/openai/tool_parsers/hermes_tool_parser.py", line 227, in extract_tool_calls_streaming
ERROR 11-23 01:34:01 hermes_tool_parser.py:338]     function_name: Union[str, None] = current_tool_call.get("name")
ERROR 11-23 01:34:01 hermes_tool_parser.py:338] AttributeError: 'NoneType' object has no attribute 'get'
ERROR 11-23 01:34:01 hermes_tool_parser.py:338] Error trying to handle streaming tool call.
ERROR 11-23 01:34:01 hermes_tool_parser.py:338] Traceback (most recent call last):
ERROR 11-23 01:34:01 hermes_tool_parser.py:338]   File "/home/sycamore/.conda/envs/llm_env/lib/python3.10/site-packages/vllm/entrypoints/openai/tool_parsers/hermes_tool_parser.py", line 291, in extract_tool_calls_streaming
ERROR 11-23 01:34:01 hermes_tool_parser.py:338]     args_delta_start_loc = cur_arguments_json.index(delta_text)
ERROR 11-23 01:34:01 hermes_tool_parser.py:338] ValueError: substring not found
ERROR 11-23 01:34:02 hermes_tool_parser.py:338] Error trying to handle streaming tool call.
ERROR 11-23 01:34:02 hermes_tool_parser.py:338] Traceback (most recent call last):
ERROR 11-23 01:34:02 hermes_tool_parser.py:338]   File "/home/sycamore/.conda/envs/llm_env/lib/python3.10/site-packages/vllm/entrypoints/openai/tool_parsers/hermes_tool_parser.py", line 291, in extract_tool_calls_streaming
ERROR 11-23 01:34:02 hermes_tool_parser.py:338]     args_delta_start_loc = cur_arguments_json.index(delta_text)
ERROR 11-23 01:34:02 hermes_tool_parser.py:338] ValueError: substring not found
besides,  when i print the tool_calls function.arguments in strem mode like:
for chunk in create:
    try:
        print(chunk.choices[0].delta.tool_calls[0].function.arguments,end="")
    except Exception as e:
        pass

the output like :
None{"args":"entity": ""\u5e7f\u4e1c\u641c\u4e00\u641c\u79d1\u6280\u6709\u9650\u516c\u53f8, "func": ""get_company_funding{"args": {"entity": "\u5e7f\u4e1c\u641c\u4e00\u641c\u79d1\u6280\u6709\u9650\u516c\u53f8"}, "func": "get_company_funding"}
It is very unfriendly for me to parse json

cc @K-Mistele

Supplementary note.
Before using a new parser, VLLM displayed the tool calling arguments in the content of response instead of tool_call.
At the same time, the vllm server also reported the same error.
Here are some examples of response:
現在我將查詢 DB 中的所有 controller 信息。
<tool_call>
{"name": "text_to_sql", "arguments": {"retrieve_steps": "查全部的controller", "columns": "*", "analysis_method": "summary the data and get max", "db_name": "21", "plot": false, "return_sql": false}}}
</tool_call>

請稍等，我會盡快完成分析並提供給您。

<tool_response>
{"name": "voice_to_text", "arguments": {"AudioPath": "["/path/example.mp3"]", "KeywordsPath": "None", "language": "zh"}
</tool_response>

When I modify the code using the above method, VLLM directly returns the error.
Hope this problem can be solved as soon as possible, thank you!

@cedonley  @joennlae
Thank you for your fix, but the problem is still not solved.
I have read [Bugfix] Multiple fixes to tool streaming when using auto tool choice. and [Bugfix] fix race condition that leads to wrong order of token returned and installed the latest version of vllm using the instructions from here:
pip install https://vllm-wheels.s3.us-west-2.amazonaws.com/nightly/vllm-1.0.0.dev-cp38-abi3-manylinux1_x86_64.whl

The version is 0.6.4.post2.dev202+ge25810ae
But vllm server still has the same error:
INFO 12-02 12:13:00 engine.py:267] Added request chatcmpl-906ae28088d14caca3e3a355ac0a3036.
INFO 12-02 12:13:00 metrics.py:460] Avg prompt throughput: 171.5 tokens/s, Avg generation throughput: 4.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
ERROR 12-02 12:13:02 hermes_tool_parser.py:337] Error trying to handle streaming tool call.
ERROR 12-02 12:13:02 hermes_tool_parser.py:337] Traceback (most recent call last):
ERROR 12-02 12:13:02 hermes_tool_parser.py:337]   File "/opt/conda/lib/python3.10/site-packages/vllm/entrypoints/openai/tool_parsers/hermes_tool_parser.py", line 228, in extract_tool_calls_streaming
ERROR 12-02 12:13:02 hermes_tool_parser.py:337]     function_name: Union[str, None] = current_tool_call.get("name")
ERROR 12-02 12:13:02 hermes_tool_parser.py:337] AttributeError: 'NoneType' object has no attribute 'get'
INFO 12-02 12:13:05 metrics.py:460] Avg prompt throughput: 296.4 tokens/s, Avg generation throughput: 15.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
INFO 12-02 12:13:20 metrics.py:460] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 8.4 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

Did I do something wrong? Or is this bug not resolved yet?

The PR is not yet merged, but based on the error in the logs, I believe it may be resolved once the PR is merged, as the error raised is one of several that I resolved in my commits.

Thanks for your reply, I will wait for the PR to be merged!

I got the same problem,
ERROR 11-23 01:34:01 hermes_tool_parser.py:338] Error trying to handle streaming tool call. ERROR 11-23 01:34:01 hermes_tool_parser.py:338] Traceback (most recent call last): ERROR 11-23 01:34:01 hermes_tool_parser.py:338] File "/home/sycamore/.conda/envs/llm_env/lib/python3.10/site-packages/vllm/entrypoints/openai/tool_parsers/hermes_tool_parser.py", line 227, in extract_tool_calls_streaming ERROR 11-23 01:34:01 hermes_tool_parser.py:338] function_name: Union[str, None] = current_tool_call.get("name") ERROR 11-23 01:34:01 hermes_tool_parser.py:338] AttributeError: 'NoneType' object has no attribute 'get' ERROR 11-23 01:34:01 hermes_tool_parser.py:338] Error trying to handle streaming tool call. ERROR 11-23 01:34:01 hermes_tool_parser.py:338] Traceback (most recent call last): ERROR 11-23 01:34:01 hermes_tool_parser.py:338] File "/home/sycamore/.conda/envs/llm_env/lib/python3.10/site-packages/vllm/entrypoints/openai/tool_parsers/hermes_tool_parser.py", line 291, in extract_tool_calls_streaming ERROR 11-23 01:34:01 hermes_tool_parser.py:338] args_delta_start_loc = cur_arguments_json.index(delta_text) ERROR 11-23 01:34:01 hermes_tool_parser.py:338] ValueError: substring not found ERROR 11-23 01:34:02 hermes_tool_parser.py:338] Error trying to handle streaming tool call. ERROR 11-23 01:34:02 hermes_tool_parser.py:338] Traceback (most recent call last): ERROR 11-23 01:34:02 hermes_tool_parser.py:338] File "/home/sycamore/.conda/envs/llm_env/lib/python3.10/site-packages/vllm/entrypoints/openai/tool_parsers/hermes_tool_parser.py", line 291, in extract_tool_calls_streaming ERROR 11-23 01:34:02 hermes_tool_parser.py:338] args_delta_start_loc = cur_arguments_json.index(delta_text) ERROR 11-23 01:34:02 hermes_tool_parser.py:338] ValueError: substring not found
besides, when i print the tool_calls function.arguments in strem mode like:
for chunk in create:
    try:
        print(chunk.choices[0].delta.tool_calls[0].function.arguments,end="")
    except Exception as e:
        pass

the output like :
None{"args":"entity": ""\u5e7f\u4e1c\u641c\u4e00\u641c\u79d1\u6280\u6709\u9650\u516c\u53f8, "func": ""get_company_funding{"args": {"entity": "\u5e7f\u4e1c\u641c\u4e00\u641c\u79d1\u6280\u6709\u9650\u516c\u53f8"}, "func": "get_company_funding"}
It is very unfriendly for me to parse json

This:
None{"args":"entity": ""\u5e7f\u4e1c\u641c\u4e00\u641c\u79d1\u6280\u6709\u9650\u516c\u53f8, "func": ""get_company_funding{"args": {"entity": "\u5e7f\u4e1c\u641c\u4e00\u641c\u79d1\u6280\u6709\u9650\u516c\u53f8"}, "func": "get_company_funding"}

Doesn't look like the model is trying to generate a valid tool call; the structure is off. Possibly a chat template issue, or a precision loss with the AWQ Quant?
Can you share your chat template and the tools you're passing to the model? Hard to debug without these.

Hi @K-Mistele They are not printing the full tool call, only the arguments. The issues are (outlined in my PR mentioned earlier) is:

We're extensively using json.loads/dumps() calls, which explains why the model returns UTF-8 without the tool parsing, but is encoding non-ascii characters within the tool call. This isn't a showstopper except that we're mixing and matching ensure_ascii=False w/ the default (True) in various parsers, which breaks non-ascii arguments at times given that the "diffs" won't match (note the extra quotes and the missing {'s that invalidate this JSON...this is from the diffs getting misaligned).

The first argument looks to be "args": {"entity". The arg name is very short is very short, and likely runs into the issue mentioned in my PR where the chunks come back as '"ar' then 'gs": {"ent'. What happens in this case before my fix is that the first argument can be corrupted. I'm not 100% sure if that's what is happening to Sala8888, but it's very common prior to my fix when the first argument is short...

What's worse is you can see the full (and correct) arguments get repeated after the first occurrence  (right after the first appearance of get_company_funding in the string. This is because the first instance was corrupted from what was in the "expected" string, so the diff calculation is completely wrong, causing the full arguments to be returned in the "closing" part of the tool parser loop.

Looking forward to this being merged, also blocked by this. Thanks @cedonley !

@cedonley @joennlae Thank you for your fix, but the problem is still not solved.
I have read [Bugfix] Multiple fixes to tool streaming when using auto tool choice. and [Bugfix] fix race condition that leads to wrong order of token returned and installed the latest version of vllm using the instructions from here:
pip install https://vllm-wheels.s3.us-west-2.amazonaws.com/nightly/vllm-1.0.0.dev-cp38-abi3-manylinux1_x86_64.whl

The version is 0.6.4.post2.dev202+ge25810ae
But vllm server still has the same error:
INFO 12-02 12:13:00 engine.py:267] Added request chatcmpl-906ae28088d14caca3e3a355ac0a3036.
INFO 12-02 12:13:00 metrics.py:460] Avg prompt throughput: 171.5 tokens/s, Avg generation throughput: 4.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
ERROR 12-02 12:13:02 hermes_tool_parser.py:337] Error trying to handle streaming tool call.
ERROR 12-02 12:13:02 hermes_tool_parser.py:337] Traceback (most recent call last):
ERROR 12-02 12:13:02 hermes_tool_parser.py:337]   File "/opt/conda/lib/python3.10/site-packages/vllm/entrypoints/openai/tool_parsers/hermes_tool_parser.py", line 228, in extract_tool_calls_streaming
ERROR 12-02 12:13:02 hermes_tool_parser.py:337]     function_name: Union[str, None] = current_tool_call.get("name")
ERROR 12-02 12:13:02 hermes_tool_parser.py:337] AttributeError: 'NoneType' object has no attribute 'get'
INFO 12-02 12:13:05 metrics.py:460] Avg prompt throughput: 296.4 tokens/s, Avg generation throughput: 15.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
INFO 12-02 12:13:20 metrics.py:460] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 8.4 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

Did I do something wrong? Or is this bug not resolved yet?

fyi it is likely that the version of vLLM that you pulled from the nightly build did not contain these fixes, as both the linked PRs are still open and have not been merged into main. You might try pulling the PRs themselves and checking that way.

I think it makes the most sense to either test with those, or wait for them to be merged and then test with the nightly build, before trying to debug other issues here since otherwise we could be either (a) debugging an issue that has already been solved, or (b) trying to debug a compound issue which would be quite difficult

@cedonley @joennlae I used the branch you provided to build vllm, but still got the same error.
The branch I use: cedonley:fix_toolstream_truncate, 44ai-labs:fix-racecondition-generation
Error messages:
ERROR 12-04 14:00:42 hermes_tool_parser.py:337] Error trying to handle streaming tool call.
ERROR 12-04 14:00:42 hermes_tool_parser.py:337] Traceback (most recent call last):
ERROR 12-04 14:00:42 hermes_tool_parser.py:337]   File "/app/vllm/vllm/entrypoints/openai/tool_parsers/hermes_tool_parser.py", line 228, in extract_tool_calls_streaming
ERROR 12-04 14:00:42 hermes_tool_parser.py:337]     function_name: Union[str, None] = current_tool_call.get("name")
ERROR 12-04 14:00:42 hermes_tool_parser.py:337]                                       ^^^^^^^^^^^^^^^^^^^^^
ERROR 12-04 14:00:42 hermes_tool_parser.py:337] AttributeError: 'NoneType' object has no attribute 'get'

In addition, vllm may still send back a response after an error occurs. I used the same code to retrieve tool_calls information. The format and content retrieved in v0.6.3post1 were correct. However, in the branch you provided, there was a problem with the retrieved format, causing json decode errors.
Retrieval results in v0.6.3post1:
{'tool_calls': [{'index': 0, 'id': 'chatcmpl-tool-8882ac18eed149139d5f950e4c01390e', 'type': 'function', 'function': {'name': 'voice_to_text', 'arguments': '{"AudioPath": "http://example.com/audio.mp3", "language": "zh", "function_one_name_called": true}'}}]}

Retrieval results in branch:
{'tool_calls': [{'index': 0, 'id': 'chatcmpl-tool-d462cd97e6c940f2be6c33ee015f8eab', 'type': 'function', 'function': {'name': 'voice_to_text', 'arguments': ' true}'}}]}

Hope you can solve this problem, thank you!

Here is another related error.
I originally only added the tool message (role: tool) in messages after getting the tool's response, and then entered the next iteration of  LLM, so the messages contained: system-user-tool
In order to let LLM know the history of tool calls, I added an assistant message containing tool_calls information before A, so the messages contained: system-user-assistant-tool.
Example:
[
    {
        "role": "system",
        "content": "You are a helpful assistant."
    },
    {
        "role": "user",
        "content": "幫我把音檔轉成逐字稿。檔名: http://example.com/audio.mp3"
    },
    {
        "role": "assistant",
        "tool_calls": [
            {
                "id": "chatcmpl-tool-cb9685ce207a4bcfa461eafea3d6e801",
                "function": {
                    "arguments": "{\"AudioPath\": \"http://example.com/audio.mp3\", \"language\": \"zh\", \"function_one_name_called\": true}",
                    "name": "voice_to_text"
                },
                "type": "function"
            }
        ]
    },
    {
        "role": "tool",
        "tool_call_id": "5097b499-6da1-4343-ae3c-4134b660e065",
        "name": "voice_to_text",
        "content": "Testing"
    }
]

It works when stream=False, but when stream=True the error occurs as follows:
ERROR 12-04 07:30:15 serving_chat.py:156] Error in applying chat template from request
ERROR 12-04 07:30:15 serving_chat.py:156] Traceback (most recent call last):
ERROR 12-04 07:30:15 serving_chat.py:156]   File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/openai/serving_chat.py", line 124, in create_chat_completion
ERROR 12-04 07:30:15 serving_chat.py:156]     conversation, mm_data_future = parse_chat_messages_futures(
ERROR 12-04 07:30:15 serving_chat.py:156]                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 12-04 07:30:15 serving_chat.py:156]   File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/chat_utils.py", line 529, in parse_chat_messages_futures
ERROR 12-04 07:30:15 serving_chat.py:156]     sub_messages = _parse_chat_message_content(msg, mm_tracker)
ERROR 12-04 07:30:15 serving_chat.py:156]                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 12-04 07:30:15 serving_chat.py:156]   File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/chat_utils.py", line 475, in _parse_chat_message_content
ERROR 12-04 07:30:15 serving_chat.py:156]     result_msg["tool_calls"] = list(parsed_msg["tool_calls"])
ERROR 12-04 07:30:15 serving_chat.py:156]                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 12-04 07:30:15 serving_chat.py:156] pydantic_core._pydantic_core.ValidationError: 1 validation error for ValidatorIterator
ERROR 12-04 07:30:15 serving_chat.py:156] 0.index
ERROR 12-04 07:30:15 serving_chat.py:156]   Extra inputs are not permitted [type=extra_forbidden, input_value=0, input_type=int]
ERROR 12-04 07:30:15 serving_chat.py:156]     For further information visit https://errors.pydantic.dev/2.9/v/extra_forbidden
INFO:     <IP_ADDRESS>:39236 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request

This error occurs in v0.6.3post1

I've written a short script that tries to replicate your tool call. Note that you need to add a final "assistant" response to your example messages that shows the assistant's typical reply to the tool information.
gist with python test script
I start my server with the following arguments:
python -m vllm.entrypoints.openai.api_server --model /ai/models/Qwen2.5-72B-Instruct-AWQ --host <IP_ADDRESS> --port 5005 --served-model-name qwen2.5-large --disable-log-requests --kv-cache-dtype auto --enable-auto-tool-choice --tool-call-parser hermes -tp 2 --gpu-memory-utilization 0.95 --distributed-executor-backend ray --enable-prefix-caching  --enable-chunked-prefill

The script outputs both the stream=False followed by a separate stream=True call for the same request:
❯ python test_vllm_bug10831.py
stream=False results:
[ChatCompletionMessageToolCall(id='chatcmpl-tool-66fe6d006e4f484aa27a32c0c3078d06', function=Function(arguments='{"AudioPath": "https://example.com/real_audio.mp3", "language": "zh", "function_one_name_called": true}', name='voice_to_text'), type='function')]

streamed tool call id: chatcmpl-tool-1b5f49b78bfe4095bdcf4788c1ecb307
streamed tool call name: voice_to_text
streamed tool call arguments: {"AudioPath": "https://example.com/real_audio.mp3", "language": "zh", "function_one_name_called": true}

As you see above, I'm not seeing any issues when using with my PR.
If you don't get this output from the script, try to run the script when you start vLLM with debug logs enabled and provide the relevant debug lines.
export VLLM_LOGGING_LEVEL=DEBUG


(enhancement) Travis CI

Allowing this project's build to be checked against the Travis CI service would be a great enhancement to prove its stability and reliability.
Would just need the travis.yml file pushed up.

Whenever Encode gets integrated alongside its unit tests, it'll be here, ready and waiting.


Step 4: Icons not showing

Hello,
I just started with the project and am stuck at step 4, the icons do not show.
Here is the error am getting:
Failed to load resource: the server responded with a status of 404 (Not Found)
ic_add_white_24px.svg
Failed to load resource: the server responded with a status of 404 (Not Found)
wind.png
Failed to load resource: the server responded with a status of 404 (Not Found)
partly-cloudy.png
Failed to load resource: the server responded with a status of 404 (Not Found)
thunderstorm.png
Failed to load resource: the server responded with a status of 404 (Not Found)
Request URL:http://<IP_ADDRESS>:8887/images/ic_refresh_white_24px.svg
Request Method:GET
Status Code:404 Not Found
Remote Address:<IP_ADDRESS>:8887
Is there a fix for this?

Solved the issue.
In my css: styles/inline.css.
It was
background-image: url("/images/clear.png");
and i simply changed it to
background-image: url("../images/clear.png");

Please update the documentation as newbies may not figure this out. As of today, it still shows the incorrect URL for those images:


Correction for raw affiliation School of Social Work, King’s University College at Western University, London, ON, Canada

Correction needed for raw affiliation School of Social Work, King’s University College at Western University, London, ON, Canada
raw_affiliation_name: School of Social Work, King’s University College at Western University, London, ON, Canada
new_rors: ;02grkyz14
previous_rors: 048f5b183;02grkyz14
works_examples: W4366527641;W4387497932
contact: 62b401bb28595e8e52b70d006a143e36:c3142d9648899053 @ uwo.ca

This issue was accepted and ingested by the OpenAlex team on 2024-12-21. The new affiliations should be visible within the next 7 days.


Issues when using two different config files

Hey,
I'm using Laravel Mix to compile SASS with Tailwind. I have two different areas of the site (website and admin area) and I want to have to different style sheets for each area. During compilation I get @apply errors due to colors not being found. It appears that Tailwind only uses the first configuration file passed to it:
mix.sass('resources/assets/site/sass/site.scss', 'public/css')
    .options({
        processCssUrls: false,
        postCss: [
            tailwindcss('./tailwind-site.js')
        ]
    });

mix.sass('resources/assets/admin/sass/admin.scss', 'public/css/')
    .options({
        processCssUrls: false,
        postCss: [
            tailwindcss('./tailwind-admin.js')
        ]
    });

In this case this admin.scss produces an error '@apply' cannot be used with '.bg-white' because '.bg-white' either cannot be found... as the site config does not contain white in the colors object, even though the admin config does.
I'd like to use two config files in this case to keep my CSS small for the user facing site but I'm less concerned about file size for the admin area.
Note; flipping the build order of these gets rid of the issue. I have also tried requiring Tailwind into two different constants but it did not make a difference.

I had the same problem. Solved it by using two webpack-config files. Can post my setup if that’s interesting?

@nivv sounds like a decent work around, would be good to see how you've done it, are you using Mix or standalone?

See previous discussion here: https://github.com/tailwindcss/tailwindcss/issues/261
Currently it's a limitation of Mix and nothing we can do in Tailwind. You could do it with a custom Webpack setup for sure, but I haven't personally had a chance to put together an example.
Closing as not a Tailwind issue but feel free to continue the conversation.

Adam,
Any chance you could give me some direction on how you might pull this off with two configurations in webpack? I have two VueJS apps that need to lean on two different configs and would like to get them in the same build chain.... killing myself with webpack here.


Modular Mask Generation

This is for internal use only; if you'd like to open an issue or request a new feature, please open a bug or enhancement issue
Instructions
This document should be filled out prior to embarking on any project that will take more than a couple hours to complete. The goal is to make sure that everyone is on the same page for the functionality and requirements of new features. Therefore, it's important that this is detailed enough to catch any misunderstandings beforehand. For larger projects, it can be useful to first give a high-level sketch, and then go back and fill in the details. For smaller ones, filling the entire thing out at once can be sufficient.
Relevant background
There are two plotting overlays which share many of the same functionalities:

plot_utils.plot_pixel_cell_cluster_overlay
plot_utils.plot_neighborhood_cluster_result

It would be beneficial to generalize the plotting for pixel, cell, and neighborhood clusters both for maintenance, and on the user's end.
In addition, we can expand the customization efforts for generating the cluster overlays:

Allow users to provide a color mask they like

Add an option to supply a continuous gradient as well instead of categorical assignment. Color the segmentation mask by the intensity of the continuous variable.

An option to display or hide the legend

The legend should account for gradients within the categorical labels.

Make sure the changes work with the Image Stitching Notebook.
For extensibility reasons, it should be flexible enough to support future clustering tasks

Design overview
Create a user facing wrapper function which handles input validation (paths, fovs + data array), and cluster information.  Then
Code mockup
import alpineer

def plot_cluster_overlay(
    cluster_xr: xr.DataArray,
    cluster_col: str,
    fovs: List[str],
    assignment_type: Optional[str] = "categorical",
    cmap_name: Union[str, matplotlib.colors.Colormap],
    cell_ids: List[str],
    cluster_id_mapping: Optional[Union[pathlib.Path, str, pd.DataFrame]] = None,
    legend: bool,
    fig_size: Tuple[int, int],
    save_dir: Union[pathlib.Path, str],
    display_image: Optional[str],
    ) -> None

    # Input validation
    alpineer.misc_utils.verify_in_list(fovs, cluster_xr.fovs.values)
    alpineer.misc_utils.verify_in_list(assigmnemt_type, ["categorical", "continuous"])

    if assignment_type == "continuous" and type(cmap_name) in [Sequential, Sequential2, Cyclic]:
        # define the colormap, add black for empty slide
        mycols = cm.get_cmap(cmap_name, k).colors
        mycols = np.vstack(([0, 0, 0, 1], mycols))
        cmap = colors.ListedColormap(mycols)
        bounds = [i-0.5 for i in np.linspace(0, k+1, k+2)]
        norm = colors.BoundaryNorm(bounds, cmap.N)

        # check if `cmap` is a Sequential, Sequential2 or Cyclic colormap, not a Qualitative one
    if assignment_type == "continuous" and cmap_name == Qualitative:
        # Notify the user that a continuous colormap cannot be constructed from a Qualitative one.

    if type(cluster_id_to_name_path) in (pathlib.Path, str):
        # Load and process the cluster_id_to_name csv file
        cluster_id_to_name = pd.read_csv(cluster_id_to_name_path)
        misc_utils.verify_same_elements(
            cluster_mapping_cols=cluster_id_to_name.columns.values,
            required_cols=['metacluster', 'mc_name']
        )

        # Subset on metacluster and mc_name
        metacluster_id_to_name = cluster_id_to_name[['metacluster', 'mc_name']].copy()

        # take only the unique pairs
        metacluster_id_to_name = metacluster_id_to_name.drop_duplicates()

        # sort by metacluster id ascending, this will help when making the colormap
        metacluster_id_to_name = metacluster_id_to_name.sort_values(by='metacluster')

        # assert the metacluster index in colors matches with the ids in metacluster_id_to_name
        misc_utils.verify_same_elements(
            metacluster_colors_ids=list(metacluster_colors.keys()),
            metacluster_mapping_ids=metacluster_id_to_name['metacluster'].values
        )

        # use metacluster_colors to add the colors to metacluster_id_to_name
        metacluster_id_to_name['color'] = metacluster_id_to_name['metacluster'].map(
            metacluster_colors
        )
    ...

    for fov in fovs:
        cluster_mask = cluster_xr[cluster_xr[fov_col] == fov].values

        # Create the figure

        if legend:
            # Add the legend
            if cluster_id_to_name_path:
                # Add the custom legend cluster names

        # Plotting
        ...

        if save_dir:
            misc_utils.save_figure(save_dir, f"{fov}.png)

Required inputs
Currently, the parameters for the two functions are as follows:
def plot_neighborhood_cluster_result(
    img_xr,
    fovs,
    k,
    save_dir=None,
    cmap_name='tab20',
    fov_col='fovs',
    figsize=(10, 10)):
    """Takes an xarray containing labeled images and displays them.
    Args:
        img_xr (xarray.DataArray):
            xarray containing labeled cell objects.
        fovs (list):
            list of fovs to display.
        k (int):
            number of clusters (neighborhoods)
        save_dir (str):
            If provided, the image will be saved to this location.
        cmap_name (str):
            Cmap to use for the image that will be displayed.
        fov_col (str):
            column with the fovs names in `img_xr`.
        figsize (tuple):
            Size of the image that will be displayed.
    """

and
def plot_pixel_cell_cluster_overlay(img_xr,
    fovs,
    cluster_id_to_name_path,
    metacluster_colors,
    save_dir=None,
    fov_col='fovs',
    figsize=(10, 10)):
    """Overlays the pixel and cell clusters on an image

    Args:
        img_xr (xarray.DataArray):
            xarray containing labeled pixel or cell clusters
        fovs (list):
            list of fovs to display
        cluster_id_to_name_path (str):
            a path to a CSV identifying the pixel/cell cluster to manually-defined name mapping
            this is output by the remapping visualization found in `metacluster_remap_gui`
        metacluster_colors (dict):
            maps each metacluster id to a color
        save_dir (str):
            If provided, the image will be saved to this location.
        fov_col (str):
            column with the fovs names in `img_xr`.
        figsize (tuple):
            Size of the image that will be displayed.
    """

We can merge the two to get:

cluster_xr: An xarray.DataArray storing the labeled clusters
cluster_col: The xarray.DataArray axis which will used for gathering the clusters
fovs: A List[str] for the FOVs to display
save_dir: A pathlib.Path or str directory to save the clustering masks / overlays to
cmap: Either a pre-existing colormap found on the docs, which can be verified in matplotlib.colormaps. This can also be a pre-existing colormap from external libraries such as CMasher.
legend: A bool, flagging whether the legend should be shown or not
figsize: A tuple, the size of the image which will be displayed
cell_ids: A List[str], a list of cell IDs so the ID to color mapping can stay the same for any image of a particular dataset (good if a cell doesn't exist in a particular image)
cluster_id_mapping: A pathlib.Path or str pointing to a csv file generated by metacluster_remap_gui, or any cluster ID to a user given "name". This should also support data frames.
assignment_type: A str, either "categorical" or "continuous", used to color the segmentation masks.
display_image: A str, either "figure" or "tiff". The user can specify if they want the colored tiff itself, or a Matplotlib styled plot.

Output files

Clustering masks / overlays have an option to be saved. They can be saved as Matplotlib figures, or as colored tiff files.
Will need to edit the Pixie pipeline to add an option for returning the cluster ID to name data frames.

Timeline
Give a rough estimate for how long you think the project will take. In general, it's better to be too conservative rather than too optimistic.

[ ] A couple days
[x] A week
[ ] Multiple weeks. For large projects, make sure to agree on a plan that isn't just a single monster PR at the end.

Estimated date when a fully implemented version will be ready for review: 2/24
Estimated date when the finalized project will be merged in: 2/28

@ngreenwald Is there anything else to consider?

A couple things:

Sometimes the user will want to plot one image at a time. Since not all images will have all cell IDs, different colors get assigned to different cells in each image. We'll likely need to take in a list of all possible cell IDs, so that the mapping is the same each time
the cluster_id_to_name_path would be better if it was more general. Being able to pass in the df directly, and with more general names, will allow this to work in different contexts. This may require tweaking how Pixie saves this information, or a loading function that swaps the names
For returning the labeled image directly: This should be an option for the user. Sometimes, rather than a figure, you want the actual tiff image, but colored by the colormap. i.e.

new_colormap = cmap()....
labeled_img = load_from_dir....
colored_tiff = new_colormap(labeled_img)

@ngreenwald Added those modifications in.

Looks good!


fixed iptables disabled recipe

Description
flush (deletes) iptables rules
Issues Resolved
disable recipe not deletes old rules
Check List

[ ] All tests pass. See https://github.com/chef-cookbooks/community_cookbook_documentation/blob/master/TESTING.MD
[ ] New functionality includes testing.
[ ] New functionality has been documented in the README if applicable
[ ] All commits have been signed for the Developer Certificate of Origin. See https://github.com/chef-cookbooks/community_cookbook_documentation/blob/master/CONTRIBUTING.MD

This will run on every Chef run. You'll probably want to use a notifies from the service stop to ensure this dosn't do that. Also you'll need to sign this commit for DCO before we can merge it.

You are right, added the notifies thing.
I cant add it to the service stop because it has "only_if" statement.
So it runs only on rhel
I added it to the folder deletion.

Looks good. Thanks


git up not working in Ubuntu

Ran the gem install and when I run the git up command I get:
/usr/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:36:in gem_original_require': /var/lib/gems/1.8/gems/posix-spawn-0.3.9/lib/posix/spawn.rb:243: syntax error, unexpected tSYMBEG, expecting tAMPER (SyntaxError)           ::Kernel::exec(*argv, :close_others=>false)                                  ^ /var/lib/gems/1.8/gems/posix-spawn-0.3.9/lib/posix/spawn.rb:274: syntax error, unexpected tIDENTIFIER, expecting tAMPER ...(*system_command_prefixes, cmd, :out => w, r => :close)                               ^ /var/lib/gems/1.8/gems/posix-spawn-0.3.9/lib/posix/spawn.rb:274: syntax error, unexpected tASSOC, expecting tCOLON2 or '[' or '.' ...command_prefixes, cmd, :out => w, r => :close)                               ^ /var/lib/gems/1.8/gems/posix-spawn-0.3.9/lib/posix/spawn.rb:274: syntax error, unexpected tASSOC, expecting '=' ...prefixes, cmd, :out => w, r => :close)                               ^ /var/lib/gems/1.8/gems/posix-spawn-0.3.9/lib/posix/spawn.rb:526: syntax error, unexpected ',', expecting ']'         [*system_command_prefixes, args[0]]                                   ^ /var/lib/gems/1.8/gems/posix-spawn-0.3.9/lib/posix/spawn.rb:526: syntax error, unexpected ']', expecting kEND         from /usr/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:36:in require'
from /var/lib/gems/1.8/gems/posix-spawn-0.3.9/lib/posix-spawn.rb:1
from /usr/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:36:in gem_original_require'         from /usr/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:36:in require'
from /var/lib/gems/1.8/gems/grit-2.5.0/lib/grit/git.rb:2
from /usr/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:36:in gem_original_require'         from /usr/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:36:in require'
from /var/lib/gems/1.8/gems/grit-2.5.0/lib/grit.rb:34
from /usr/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:36:in gem_original_require'         from /usr/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:36:in require'
from /var/lib/gems/1.8/gems/git-up-0.5.12/lib/git-up.rb:2
from /usr/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:36:in gem_original_require'         from /usr/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:36:in require'
from /var/lib/gems/1.8/gems/git-up-0.5.12/bin/git-up:3
from /usr/local/bin/git-up:19:in `load'
from /usr/local/bin/git-up:19

It didn't even get installed in my case.
[zzzz@q2s3 tt_dev]$ gem install git-up

fWARNING:  Installing to ~/.gem since /usr/lib64/ruby/gems/1.8 and
          /usr/bin aren't both writable.
WARNING:  You don't have /home/zzzz/.gem/ruby/1.8/bin in your PATH,
          gem executables will not run.
Building native extensions.  This could take a while...
Successfully installed colored-1.2
Successfully installed posix-spawn-0.3.9
Successfully installed mime-types-1.25.1
Successfully installed diff-lcs-1.2.5
Successfully installed grit-2.5.0
Successfully installed git-up-0.5.12
6 gems installed
Installing ri documentation for colored-1.2...
Installing ri documentation for posix-spawn-0.3.9...
Installing ri documentation for mime-types-1.25.1...
Installing ri documentation for diff-lcs-1.2.5...
Installing ri documentation for grit-2.5.0...
Installing ri documentation for git-up-0.5.12...
Installing RDoc documentation for colored-1.2...
Installing RDoc documentation for posix-spawn-0.3.9...
Installing RDoc documentation for mime-types-1.25.1...
Installing RDoc documentation for diff-lcs-1.2.5...
Installing RDoc documentation for grit-2.5.0...
Installing RDoc documentation for git-up-0.5.12...
[zzzz@q2s3 tt_dev]$
[zzzz@q2s3 tt_dev]$ git up
git: 'up' is not a git command. See 'git --help'.

Did you mean one of these?
        pull
        push

@cppexpert I think you need to add your ruby gems bin directory to your PATH.
$ export PATH="$PATH:~/.gem/ruby/1.8/bin"

Put that in your ~/.bashrc file if you don't want to run it on start.

Forgot to mention: I'm also seeing this issue with @jamespsterling's traceback  :smiley:
Reproduced with formatting here:
/usr/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:36:in `gem_original_require': /var/lib/gems/1.8/gems/posix-spawn-0.3.9/lib/posix/spawn.rb:243: syntax error, unexpected tSYMBEG, expecting tAMPER (SyntaxError)
          ::Kernel::exec(*argv, :close_others=>false)
                                 ^
/var/lib/gems/1.8/gems/posix-spawn-0.3.9/lib/posix/spawn.rb:274: syntax error, unexpected tIDENTIFIER, expecting tAMPER
...(*system_command_prefixes, cmd, :out => w, r => :close)
                              ^
/var/lib/gems/1.8/gems/posix-spawn-0.3.9/lib/posix/spawn.rb:274: syntax error, unexpected tASSOC, expecting tCOLON2 or '[' or '.'
...command_prefixes, cmd, :out => w, r => :close)
                              ^
/var/lib/gems/1.8/gems/posix-spawn-0.3.9/lib/posix/spawn.rb:274: syntax error, unexpected tASSOC, expecting '='
...prefixes, cmd, :out => w, r => :close)
                              ^
/var/lib/gems/1.8/gems/posix-spawn-0.3.9/lib/posix/spawn.rb:526: syntax error, unexpected ',', expecting ']'
        [*system_command_prefixes, args[0]]
                                  ^
/var/lib/gems/1.8/gems/posix-spawn-0.3.9/lib/posix/spawn.rb:526: syntax error, unexpected ']', expecting kEND
	from /usr/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:36:in `require'
	from /var/lib/gems/1.8/gems/posix-spawn-0.3.9/lib/posix-spawn.rb:1
	from /usr/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:36:in `gem_original_require'
	from /usr/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:36:in `require'
	from /var/lib/gems/1.8/gems/grit-2.5.0/lib/grit/git.rb:2
	from /usr/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:36:in `gem_original_require'
	from /usr/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:36:in `require'
	from /var/lib/gems/1.8/gems/grit-2.5.0/lib/grit.rb:34
	from /usr/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:36:in `gem_original_require'
	from /usr/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:36:in `require'
	from /var/lib/gems/1.8/gems/git-up-0.5.12/lib/git-up.rb:2
	from /usr/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:36:in `gem_original_require'
	from /usr/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:36:in `require'
	from /var/lib/gems/1.8/gems/git-up-0.5.12/bin/git-up:3
	from /usr/local/bin/git-up:23:in `load'
	from /usr/local/bin/git-up:23

@gepoch I added to my path just like you wrote, but my ruby gems directory does not have a bin directory.
$ echo $PATH
/home/amedee/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/home/amedee/.rvm/bin:/home/amedee/cov-analysis/bin:/home/amedee/.rvm/bin:~/.gem/ruby/1.8/bin

$ ls ~/.gem/ruby/1.8/
cache

@amedee yeah.. I'm not much of a ruby person, actually..
I eventually ended up switching to the python port, which works perfectly for me.

@gepoch  I uninstalled ruby and I am currently installing it as described on https://coolaj86.com/articles/installing-ruby-on-ubuntu-12-04.html
With one change:
curl -L https://get.rvm.io | bash -s stable

should be
curl -L https://get.rvm.io | bash -s stable --ruby --auto-dotfiles


Generates squeaks

When i try the sample program , it generates squeaks instead of the sound. I am using an external USB speaker to play.

I have just pushed a bit update to naudiodon that may be worth a try. There are also parameters for portAudio that can affect this.
Closing for now.


Disapatch timer do not stop.

Hi, do you see some behaviours when using a dispatchTimer inside one view? In the NavigateFrom function, I stop the timer but the timer still running and calling is tick function.

Hi,
sorry but I don't understand what's the issue you're experiencing. Can you please post some code and what result you're expecting? Thanks.

Closing this since it hasn't been active for more than a year.


remove functions deprecated in v0.22.0

R.wrap was deprecated in #1818 after discussion in #1387.

:cow2:

!
what hapened to my cow emoji?

Every time I go reaching for this @davidchambers, I'm coming to you to ask, "Now how do I do this one without wrap"  :smile:
@buzzdecafe:
Why, has you cow emoji been mooved?

has you[r] cow emoji been mooved?

it's been changed into some cartoon-y thing; not the rugged, gritty, real-world cow i loved. going forward with 🐫

Are you referring to the cow's appearance in Gmail by chance, @buzzdecafe? GitHub recently switched from using images to using actual emoji in email messages, and Gmail uses a cartoonish emoji set. :cow2: still looks good on GitHub!
Also, @benperez may have a thing or two to say about your use of camels. ;)

hmmm, today it is back to the cow noir i prefer. Maybe O:dromedary_camel: ?


AttributeError: 'Track' object has no attribute 'last_detection_confidence'

Thanks For Nice Work . I am testing my videos on the pre-trained weights .  Thank you

Looks like no object has been detected. Did you solve this?

Yeah , I solve this problem as follow
1- Go to "ACAM_Demo-master/object_detection/object_detector.py" and Open it
2- On line 167
Replace
detection_conf = track.last_detection_confidence

With
detection_conf = 0

This issue Will be resolved.


Setup repository to use Rust dev container

Right now if you launch a GitHub Codespace for this repo, you get the general default dev container, which doesn't include a Rust toolchain. This is a little inconvenient, and it would be nice to be able to launch a Rust stable container for doing dev work in Codespaces. I'm not super familiar with Codespaces configuration, so if anyone else is interested in tackling this, it would be helpful!

The container should also have cbindgen

Done! Fixed by #87 and #88. We now have a devcontainer will all the dependencies installed for us.


[Windows] Null Safety Migration tool 404 error

Dart SDK version: 2.16.1 (stable) (Tue Feb 8 12:02:33 2022 +0100) on "windows_x64"
Windows
Edge

Error occurred when trying to migrate old Flutter project. Make the migration tool correctly respecting Windows' case insensitive paths.
Steps to reproduce: https://github.com/flutter/flutter/issues/100134#issue-1169639718
Others: https://github.com/microsoft/vscode/issues/145195

As of 1c7fe7196dd7a542f7f91cd6ea2ac81e1a13146c, the null safety migration tool has been removed from active development and retired. No further work on the tool is planned.
If you still need help, or you believe this issue has been closed in error, please feel free to reopen.


cannot read environment variable after build

i have a react app ,  i build the app with command: npm run build
if we start the app with development code by running ENV=production npm run start it can read the process.env.ENV variable beacause i'm adding this plugins to webpack dev config
   new webpack.DefinePlugin({
      'process.env':{
        'ENV': JSON.stringify(process.env.ENV),
      }
    }),

i also add the script above to webpack prod config,
but if i try this command after build ENV=prod serve -s build
it cannot read the environment variable
how to fix this?

Sorry, we do not provide support once you've ejected!
You can try to ask this question on Stack Overflow.


New models don't get picked up by flask-migrate

I'm unable to add any new models to the migrations. Here's an example model I added for testing:
The folder structure is the same as the user module
{{cookiecutter.app_name}}/{{cookiecutter.app_name}}/example/
__init__.py
models.py
The example/models.py :
from flaskapp.database import Column, Model, SurrogatePK, db

class Example(SurrogatePK, Model):
    __tablename__ =  'examples'

I also added an import to app.py for whatever it's worth
from flaskapp import commands, public, user, scraper, example
I run flask db init and then flask db migrate but it only ever detects the users and roles models.
I've tried a variety of techniques to get new models defined, but only new models defined in user/models.py ever get detected when I run a migration.
Any help would be greatly appreciated. Thanks!

Figured out it was an import issue.
Adding from .models import Example to the __init__.py solved my problem. This allowed the app.py to find my models with this import from flaskapp import example. I still don't know how the user models gets found. They only get imported by the forms.py and unit tests.

This is indeed confusing. User model gets picked up by Flask-Migrate because it gets imported in public/views.py, which in turn in imported in blueprint's __init__.py.
So it basically works by coincidence. The right solution, as you have correctly figured out, is to also import the models (and not just views) in __init__.py for each blueprint (this is only required for Flask-Migrate, nothing else). Also, rather than importing each specific model, you can import all models using:
from .models import *


IP Whitelisting for API server in CAPO

User Story

As a customer I want to use IP whitelisting on the Kubernetes API server so that I can be shure that only specified network ranges have access to it.

TODO

[x] Create  upstream issue
[ ] Implement whitelisting

Let's wait a few more days once we've got a response from the issue reporter (if he's still interested by implementing this change or if we could take over the implementation part)

will take care of the upstream implementation and added first thoughts into the upstream issue:
https://github.com/kubernetes-sigs/cluster-api-provider-openstack/issues/1045#issuecomment-1106488475

@bavarianbidi is this done then?

@bavarianbidi is this done then?

It's already implemented and merged but there isn't a CAPO release yet ( since months).
If we wouldn't create an own release we have to wait for upstream.


[Feature Request] Use custom brand icons

only if you wish, but I created some thinq icons for LG items over at custom brand icons https://github.com/elax46/custom-brand-icons#lg
you can link to this repo if you wish for icons for devices
welcome to link to the LG items and to discuss us adding more

I do not understand how I should use this icons inside the integration. Could you better explain?

you can pull the icons from the git, but credit the git custom brand icons if you dont mind


Images in top of columns for bootstrap 5

Change for bootstrap 5, images in top column

For images in columns bootstrap 5

Hi,
Thanks for this PR, but unfortunately this will cause a breaking change with Bootstrap 4.
I wonder if we can keep both classes, ie float-right and float-end to get it to work against 4 and 5.
I don't have time to test this at the moment, so I will keep this PR open.
Thanks,
Ivan


Add automated checks for two LGTM rule.

Both Docker and opencontainers projects have the same set of rules for merging. We should add some checks to Gordon so that it harasses maintainers who merge code without it having two LGTMs (as well as adding a comment while there are still LGTMs missing).
Preferably Gordon would also check that the isn't any self-LGTMs (opencontainers/project-template#13).
This allows us to do more management of our respective projects without needing to use a proprietary tool (PullApprove).

So two feature requests:

Require 2 LGTM (or allow this to be configurable)
Don't accept self-LGTM

These sound reasonable to me. Do you want to submit a patch for not accepting self-lgtm? Devs can always "merge --force" so it shouldn't block people on tiny projects.
For requiring multiple lgtm, we should work out a way to make this configurable. I'll spend some time thinking about how to do this. One option might be to have a .gordon config file to make this configurable. Suggestions are welcome.


Default colors look horrible in CodeSpaces

See https://github.com/PowerShell/GraphicalTools/issues/111
This may be an issue that needs to be fixed in CodeSpaces, but it's also possible the defaults gui.cs uses could be tweaked to at least make it readable without screwing it up for other consoles.

We need to figure this out as CodeSpaces is becoming more real and Terminal.Gui apps are basically unusable w/in it now.

A workaround is to use the Dark+ theme (some others look ok too).

This appears to have been fixed by codespaces:


⚠ ClamAV has degraded performance

In af20dfe, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 213 ms

Resolved: ClamAV performance has improved in 83b2ef2.


If enter Chinese characters, garbled characters are displayed

If there is no plan to localize the interface, can you consider the editable area? Currently, Chinese characters are garbled.

Godot v3 (an old version of the engine behind Arrow) has a bunch of limitations when it comes to non-Latin scripts, including lack of proper RTL support, and as far as I remember, not being able to fallback to or load system fonts.  Arrow  itself has few problems too. One is a limited font being used both for icons (UI in general) and inputs which was a decision far from ideal. Thankfully Godot v4 has a great language support, but aside from the port being very time-consuming, Arrow should also be fixed in ways to say there will be a perfect language support.
Anyway! These are problems that should be addressed eventually and fundamentally . They've been bothering me for a long time and I have had plans to fix them; but I can not guarantee anything happening soon.
Meanwhile, changing default fonts may help as a temporary solution.
Find a font suitable for your job and replace the current ones with that. You will need to change text-icon of some UI buttons too; otherwise they would look weird or not show up at all.
Please inform me of the results and if changing the font helped.
I'll keep this issue open as a subject to be properly addressed in  mid or long-term.
Have a nice time


Mark tracks as favorties

add favorite icons to the popup
favorite icons can be toggled on and off
list of favorites is saved in local storage

Thank you for this! I'll try this out this weekend.


Switch to alternative beacon message with middle button long press

Hi,
I would love to have the ability to switch to an alternative message (and icon) when holding the middle button.
For example, I'm on a SOTA activation and when returning back I want another beacon message.
Or to switch between pedestrian and car, etc...
Since we have OneButton already included we could use attachLongPressStop to trigger between messages.
We could also have an array of messages (and icons) in the json config file and you would simply select the right message that currently suits your preference by holding the middle button.
Does this make any sense?
Waiting for your replies.
BTW I'm willing to help with the code.

I agree with you @tekk.
As a user I'd like to have an option for choosing another callsign/SSID, symbol/overlay and message. Maybe this could be definable in a profiles array or object in the tracker.json or one tracker_{SSID}.json for each profile.

I have implemented required changes in #47
Please review and let me know.
73 OM7TEK

implemented in #47


[card] ENS support

Is your feature request related to a problem? Please describe.
xpubs are soooo ugly.. and eth addresses for that matter.
Describe the solution you'd like
Resolve ENS names. 2 places we juggle ugly strings: sending to some xpub and cashing out to some eth address.
Cashing out to an eth address is easy bc the public ENS resolver can resolve address records pointing to an on-chain address.
Sending to an xpub derived from an ens name is a little trickier. An EIP will be submitted to either add an xpub-type record an xpub key to the text record. This will take a while, in the meantime we can hijack the description key of an ens name's text record.
When it's all done, I should be able to cashout to bohendo.eth (by pulling the address from bohendo.eth's address record) or send from another daicard to bohendo.eth (by retrieving an xpub from the description key from bohendo.eth's text-record).
Obviously, this will require a bit of internal verification (eg the data in the text-record's description key needs to be a valid xpub) and end-user verification (eg display the derived xpub/address in small-but-readable text at the bottom of the input field so tech-savy users can sanity check that their name was resolve to the correct value.
Once completed, we can point Metamask & other projects to this solution as a reference implementation for their own app's ENS support

Need to update to use the new key that Bruno suggested:
VND.CONNEXT: ${xpub}|${chainId}

Done, closing


FEATURE: Using @reduxjs/toolkit in mobile codebase

Description
This PR is dedicated to use Redux Toolkit in favor of Redux. It'll help with DX as well as eliminating some current performance issues.
Resolves #480
Type of change

New feature (non-breaking change which adds functionality)

You really wanted this change haha
I was thinking of doing something like:
https://github.com/ZibanPirate/trove/tree/main/src/redux
Where we have slices in one file, in the same dir as store configuration as well as our homemade subscription function.
And we have action creators in different dir (thunks) where they can be tree shaked and fetched separately from main bundle in the chunk of whatever page required it.
There are some black magic typing there, but it's essentially just helping mapping the slices into one state interface, that can also be used for our own useSliceSelector
I may have gone overboard with my own approach, so I would like to see how you gonna approach this, and have a collaborative approach

I will try to make it as similar as the old codebase, however in this new structure the actions and reducers are merged in slices folder, so each slice folder will contains its specific actions. It's a good idea to use RTK Query in this case, but I'll start with using extra reducers as thunk actions and probably later on we'll switch to RTK Query which I think it's the best approach.
You can find here my approaches in RTK Query:
https://github.com/Omar-Belghaouti/blogs-full-stack/tree/main/frontend/src/store
https://github.com/Omar-Belghaouti/blogs-full-stack/tree/main/frontend/src/store
As said above, for this PR I'll work on it without RTK Query however it'll have similar structure. And you can work in the web codebase in the meantime I work in the mobile codebase.

@ZibanPirate checkout the new performance for the contribution filters. Now with RTK we can store array data in a normalized form with the help of entity adapters, I was actually planning to normalize it manually here 😅 Thanks to RTK it's much easier to normalize data which guarantees performance for arrays data.

I know there are many tasks in this PR or should I say it looks like it 😅

moved src/redux to src/store, which didn't help the diff, and made it a bit harder to spot the changes.

For this I had to rename the redux folder to store so there will no ambiguity when importing stuff when using relative imports like we talked about it here.

refactored the state, and introduced new properties like status.

The new status property is more generalized than the old refreshing property. In this version we can definitely know what is happening in the current state (if it's refreshing or not + if there is an error). So you can think of it as it's holding 3 states of the current slice which are idle (refreshing: false), loading (refreshing: true) and error if there was an error when fetching. The reason it was defined as it is was because of the cases in the extraReducers property in the slice, those cases like pending, fulfilled and rejected are defined out of the box for all async thunks which is more suitable in this case.

an attempt to "fix" the contributions filters lag.

To be honest I wasn't trying to fix that lag, it was fixed just by implementing the array states as entity adapters that's all. The UI code as you may noticed is not changed at all expect the parts of selecting the data which I thought it's good to have multiple selectors that shows within the code what state are we selecting.
Now, replying to your comments:

Let’s make a clear difference between Actions (that we define in the slice object), and Action Creators (aka Async Actions aka Thunks). meaning that the slice shouldn't know about any Thunk, and updating the status should be moved from the slice .addCase(fetchContributions.pending, into the Thunk body, and let the Thunk handle the status changes (which again is out of this PR scope to add status in the first place). and finally move these thunks outside the state files so they could be tree-shaked (not that needed for mobile code, but better have consistency with web).

As said above, those cases are predefined and the status property is more suitable for it. I believe there is a state that isn't playing major factor during production which is error but I thought it'd definitely be helpful during debugging as it holds the error.message if any bug in that situation is occurred. Also, it is recommended to put it this way, I mean to fetch data in the createAsyncThunk and only set the data in extraReducers otherwise we would have the same old getState and dispatch which is not ergonomic especially when we have all the capabilities given from @reduxjs/toolkit.

Can we have just one selector? it's not like we have any optimization out of writing a selector for each specific path from the state. It's just too much code I think.

For this I thought that it is more readable to see which state we are selecting within the code (ex: useSelector(selectArticles)). I am not sure if you are saying that we have one selector for the global state or for each one of the slices (that mostly holds the data, status and error). If it's the latter case I will export one selector for each slice so we can have one useSelector statement in the UI.
So I hope this clears things up. And let's see your approach on the web and do some Discussions afterwards.

moved src/redux to src/store, which didn't help the diff, and made it a bit harder to spot the changes.

For this I had to rename the redux folder to store so there will no ambiguity when importing stuff when using relative imports like we talked about it here.

hmm fair, but my only request is to temporarily put the folder name back to src/redux in this PR, and we renamed it in a follow-up PR 🙏, I tried adding comments but they seemed out of context because of the diff. Once you rename it, I wil put all my comments for all the tasks in this PR 😄 (and i hope you will have fun addressing them 😈 😅)

Oh I will 😅, I'll rename it back


Correct vertical alignment of FormControl validation icon

Summary
Nudged the icon up 2px
What should reviewers focus on?

Does it look better?

Supporting resources (related issues, external links, etc):

https://github.com/primer/brand/issues/650

Contributor checklist:

[x] All new and existing CI checks pass
[x] Tests prove that the feature works and covers both happy and unhappy paths
[x] Any drop in coverage, breaking changes or regressions have been documented above
[x] New visual snapshots have been generated / updated for any UI changes
[x] All developer debugging and non-functional logging has been removed
[x] Related issues have been referenced in the PR description

Reviewer checklist:

[ ] Check that pull request and proposed changes adhere to our contribution guidelines and code of conduct
[ ] Check that tests prove the feature works and covers both happy and unhappy paths
[ ] Check that there aren't other open Pull Requests for the same update/change

Screenshots:

Please try to provide before and after screenshots or videos

 Before  After

Thanks @rezrah. I'll modify the alignment on a per-icon basis and nudge the check slightly higher

That's updated, good spot @rezrah.
How did you view it with the grid lines out of interest?

Look much better, thanks @joshfarrant.
I think this regression occurred because we don't have dedicated stories for the validation states that are tracked in our VRT suite. If possible, could you please add some stories under FormControl enabling the validationStatus props for both success and error states. We'll then be able to see visual diffs in future.

Good suggestion, thanks @rezrah
That's added, although the changes are lost somewhere in the vrt spec files updates.
I'll rebase and get rid of those changes once https://github.com/primer/brand/pull/670 is in


Respond should send back to sender rather than all windows

https://github.com/jsonnull/electron-trpc/blob/e536ba607b03d907c30298986289112913463e03/packages/electron-trpc/src/main/createIPCHandler.ts#L30
the above line should be respond: (response) => _event.sender.send(ELECTRON_TRPC_CHANNEL, response) rather than send to all windows
otherwise it would have problem when using with multiple windows and there is duplicate id used and this happens frequently

Thanks for raising this issue, and for your PR!
I've merged, added a small fix and some tests, and this is now out in<EMAIL_ADDRESS>Please note that I updated the type of createContext from your PR to createContext({ event }) so that it's a little more flexible if more options are available to context later.

https://github.com/jsonnull/electron-trpc/blob/08aae7f93bb655e8f9fcfc88c2496510c068527e/packages/electron-trpc/src/main/createIPCHandler.ts#L49
@jsonnull this line forgot to update, it should be createContext: (opts: CreateContextOptions) => Awaitable<inferRouterContext<TRouter>>; 😅

@eslym so right! Thanks for catching that :D
I pushed an update, will be out in 0.4.1 momentarily.


Host seperate docs for stable version

Hi,
I was caught rather off-guard when I tried to use an example which I had used yesterday, but commits in the last 24 hours caused incompatibilities as the example from the documentation pages had been updated (rightly so) for a newer version of the API of this module.
I can't help but feel that it would be nice if it was easier to browse documentation relevant to the stable (i.e. tagged and released) version of this module.
A workaround would be to clone the gh-pages branch at the relevant commit I suppose.

Nice, but the script will destroy any untracked files in the working dir


Review

Pull from fraizy22 home assistant integration repository the review branch in the new development branch (based on main branch)

Pull from fraizy22 home assistant integration repository the review branch in the new development branch (based on main branch)


fix: Initialise _checked in GuppyModule

Closes #489

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 91.99%. Comparing base (e900c96) to head (500d15b).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #491   +/-   ##
=======================================
  Coverage   91.99%   91.99%
=======================================
  Files          59       59
  Lines        5921     5922    +1
=======================================
+ Hits         5447     5448    +1
  Misses        474      474

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Add tests for ListBucketClass

Proposed Changes

Add tests for list bucket class

Fixes #363

Can you please remove the controller-gen in your projects bin folder and re-run make generate.
Can you please run go mod tidy to cleanup the go.sum diffs.
Is there a reason why we moved the BucketClass creation into each test? I would have kept it in the suite_test.go.

For point

Addressed.
Done go mod tidy and make test. It doesn't show any go.sum diffs.
To test scenario to check empty BucketClass list.


admin datatbles not showing HTML properly

Hello
i have installed CMS XAMP , after installation all pages where datatable is shown like Theme Management are showing encoded HTML

Did you composer update?

yes i did it, but no change

Which WebEd version are you using?

its 3.1.56
i am using php 7.0.4

Please check the file in
vendor/sgsoft-studio/themes-management/src/Http/DataTables/ThemesListDataTable.php at line 72.
The column thumbnail should be set as raw column.
->rawColumns(['description', 'actions', 'thumbnail'])

Please run these commands:
composer clear-cache
composer update

Hope it works.

issue was due to bug in yajra/laravel-datatables-oracle v7.9.6  it is fixed in yajra/laravel-datatables-oracle v7.9.7
Thanks

Ok thanks.


ModuleNotFoundError: No module named 'softlearning.utils'

I am trying to install your package for a student and I am running into the following problem:
@(mbpo) Singularity> mbpo run_local examples.development --config=examples.config.halfcheetah.0
Traceback (most recent call last):
  File "/usr/local/bin/miniconda/envs/mbpo/bin/mbpo", line 11, in <module>
    load_entry_point('mbpo==0.0.1', 'console_scripts', 'mbpo')()
  File "/usr/local/bin/miniconda/envs/mbpo/lib/python3.6/site-packages/mbpo-0.0.1-py3.6.egg/softlearning/scripts/console_scripts.py", line 202, in main
  File "/usr/local/bin/miniconda/envs/mbpo/lib/python3.6/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/bin/miniconda/envs/mbpo/lib/python3.6/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/usr/local/bin/miniconda/envs/mbpo/lib/python3.6/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/bin/miniconda/envs/mbpo/lib/python3.6/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/bin/miniconda/envs/mbpo/lib/python3.6/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/bin/miniconda/envs/mbpo/lib/python3.6/site-packages/mbpo-0.0.1-py3.6.egg/softlearning/scripts/console_scripts.py", line 71, in run_example_local_cmd
  File "/usr/local/bin/miniconda/envs/mbpo/lib/python3.6/site-packages/mbpo-0.0.1-py3.6.egg/examples/instrument.py", line 205, in run_example_local
  File "/usr/local/bin/miniconda/envs/mbpo/lib/python3.6/site-packages/mbpo-0.0.1-py3.6.egg/examples/development/__init__.py", line 35, in get_parser
  File "/usr/local/bin/miniconda/envs/mbpo/lib/python3.6/site-packages/mbpo-0.0.1-py3.6.egg/examples/utils.py", line 8, in <module>
  File "/usr/local/bin/miniconda/envs/mbpo/lib/python3.6/site-packages/mbpo-0.0.1-py3.6.egg/softlearning/algorithms/__init__.py", line 1, in <module>
  File "/usr/local/bin/miniconda/envs/mbpo/lib/python3.6/site-packages/mbpo-0.0.1-py3.6.egg/softlearning/algorithms/sql.py", line 8, in <module>
  File "/usr/local/bin/miniconda/envs/mbpo/lib/python3.6/site-packages/mbpo-0.0.1-py3.6.egg/softlearning/algorithms/rl_algorithm.py", line 12, in <module>
  File "/usr/local/bin/miniconda/envs/mbpo/lib/python3.6/site-packages/mbpo-0.0.1-py3.6.egg/softlearning/samplers/__init__.py", line 4, in <module>
  File "/usr/local/bin/miniconda/envs/mbpo/lib/python3.6/site-packages/mbpo-0.0.1-py3.6.egg/softlearning/samplers/remote_sampler.py", line 10, in <module>
  File "/usr/local/bin/miniconda/envs/mbpo/lib/python3.6/site-packages/mbpo-0.0.1-py3.6.egg/softlearning/samplers/utils.py", line 5, in <module>
  File "/usr/local/bin/miniconda/envs/mbpo/lib/python3.6/site-packages/mbpo-0.0.1-py3.6.egg/softlearning/replay_pools/__init__.py", line 4, in <module>
  File "/usr/local/bin/miniconda/envs/mbpo/lib/python3.6/site-packages/mbpo-0.0.1-py3.6.egg/softlearning/replay_pools/trajectory_replay_pool.py", line 8, in <module>
ModuleNotFoundError: No module named 'softlearning.utils'

I have mujoco and mujoco_py installed within a Singularity container. I installed your package by modifying tensorflow-gpu to tensorflow in the environment/requirements.txt file. Any suggestions?

If you have also installed softlearning in the same python environment, it is possible it is looking for utils there instead of in the softlearning folder. To see if this is the case, you can check where your softlearning import is coming from:
python -c "import softlearning; print(softlearning.__path__)"

If it is finding something outside of the mbpo repo, then the easiest solution would be to install mbpo in a separate environment.

I made an assumption the setup.py file would put this all into my environment. I need to execute mbpo from within the repository.


The triangle move but shake in D3D12HelloConstBuffers.

I run the sample D3D12HelloConstBuffers. I can see a triangle move from left to right.
But the triangle shake randomly.
I tried on two computer with GTX960 and GTX 1650, get the same issue.

Same issue.

It seems to be a bug in the debug layer? I built it with release and now the stutter is gone.
With it enabled it somehow doesn't present some frames.

You should see if you have uninitialized variables. You can enable the debug layer in your release build to see if that's really related or not.

If I turn the debug layer on I see lots of INFO Create/Destroy ID3D12GraphicsCommandList and the sample stutters a lot in Release. With debug layer off those messages are gone and there is no stutter.

OK, then it's not a bug. It's what happens when you have lots of debug output from any program. There are mechanisms for suppressing specific messages or whole categories of messages from the debug layer.
Which INFO messages are you getting in particular?

[8020] D3D12 INFO: Create ID3D12GraphicsCommandList: Addr=0x000002446BE26780, ExtRef=1, IntRef=0 [ STATE_CREATION INFO #560: CREATE_COMMANDLIST12] [8020] D3D12 INFO: Destroy ID3D12GraphicsCommandList: Name="unnamed", Addr=0x000002446BCAF3A0 [ STATE_CREATION INFO #586: DESTROY_COMMANDLIST12] [8020] D3D12 INFO: Create ID3D12GraphicsCommandList: Addr=0x000002446BCAF3A0, ExtRef=1, IntRef=0 [ STATE_CREATION INFO #560: CREATE_COMMANDLIST12] [8020] D3D12 INFO: Destroy ID3D12GraphicsCommandList: Name="unnamed", Addr=0x000002446BE26780 [ STATE_CREATION INFO #586: DESTROY_COMMANDLIST12]
Like that, repeated (I'm guessing every frame). And I realized that of course those messages won't show up if I have the debug layer off, because it's off...
What's weird is that I'm getting stutter from this Microsoft's own graphics sample.
If I use some 3rd party FPS counter overlay, it disappears the same frames that the triangle stutters.
Not sure if it's a driver issue or what. I have a GTX 1070 with latest drivers.

It's unusual to create/destroy command-lists every frame (i.e. it's a heavy-weight operation), hence why it's an INFO message. You should create them once, and then reuse them in general.
In fact, if I build & run the latest version of D3D12HelloConstBuffers using Windows 10 (17763), I don't see any INFO messages in that sample at all... I'll upgrade to 18362 and see if it repros.

Yes, I'm on 18362

Two things:

The INFO messages for creates and destroys are due to the debug layer virtualization of command queues. See ID3D12Debug1::SetEnableSynchronizedCommandQueueValidation.
INFO messages are disabled by default. You'd need to either explicitly enable them from ID3D12InfoQueue's filter APIs, or by using the DirectX Control Panel.

#ifndef NDEBUG
    // Configure debug device (if active).
    ComPtr<ID3D12InfoQueue> d3dInfoQueue;
    if (SUCCEEDED(m_d3dDevice.As(&d3dInfoQueue)))
    {
#ifdef _DEBUG
        d3dInfoQueue->SetBreakOnSeverity(D3D12_MESSAGE_SEVERITY_CORRUPTION, true);
        d3dInfoQueue->SetBreakOnSeverity(D3D12_MESSAGE_SEVERITY_ERROR, true);
#endif
        D3D12_MESSAGE_ID hide[] =
        {
            D3D12_MESSAGE_ID_CREATE_COMMANDLIST12,
            D3D12_MESSAGE_ID_DESTROY_COMMANDLIST12,
        };
        D3D12_INFO_QUEUE_FILTER filter = {};
        filter.DenyList.NumIDs = _countof(hide);
        filter.DenyList.pIDList = hide;
        d3dInfoQueue->AddStorageFilterEntries(&filter);
    }
#endif

The release is Ok. The issue only happen in debug.


Support non-blocking MailboxReceivedOperator

Currently MailboxReceivedOperator

does a busy wait on all mailbox
always pick the next mailbox to check in exact order instead of round-robin pick or random

propose to create a non-blocking mailbox receive operator
several thoughts:

consolidate multiple ArrayBlockingQueue in each observer into just sharing a single queue thus the mailbox receive operator waits for signals from the blocking queue offer
create a signaling mechanism to info mailbox receive operator thread to start reading directly from the notifying observer

also the same mechanism should also work for the local passthrough mailbox design see #9484

I think we should consider our thread utilization end-to-end. Specifically, in our multistage engine if we consider all the operators between a MailboxReceiveOperator and a MailboxSendOperator as a single "stage" we need to make sure that the stage only runs when there's work to be done and that the threads that are doing that work are efficiently shared between potentially many concurrent queries.
While investigating #9563 I discovered that in order to really get non-blocking behavior we need the entire stage to be non-blocking (e.g. the entire stage should be deferred while there's no work to do, so that the thread pool running it can be freed up to schedule another task)
Ideally, I'd suggest something like:

This gives us a few improvements over the existing architecture.

the obvious advantage is that it's reactive and will only stage execution when there's work to be done (we don't block threads/work pools unless work is being done)
it allows us to control memory and thread utilization across multiple queries that are running on a single node, while applying backpressure on servers if we aren't able to keep up (e.g. we're doing some complex aggregation on the intermediate stage nodes)
I'm not totally sure this is possible with GRPC the way it's written today, but if we could figure out a way to reuse byte buffers from incoming MailboxContent we could likely be more efficient in terms of GC
Debug-ability and observability are pretty awesome if this is done correctly because you can easily introspect each part of the pipeline (data was received, a task was scheduled, a task completed the data that was passed to it, a task is idle)

We could certainly do this in stages - commenting on your specific considerations:

consolidate multiple ArrayBlockingQueue in each observer into just sharing a single queue thus the mailbox receive operator waits for signals from the blocking queue offer

I think this is exactly right - I'd suggest we even consolidate it across queries (and then use a simple in-memory scheduler to forward the right data to the right tasks) but that's probably a bigger change.

create a signaling mechanism to info mailbox receive operator thread to start reading directly from the notifying observer

also +1, but the subtle difference with what I suggest here is that it doesn't signal to the mailbox receive operator specifically, but rather the executor that schedules the receiver operator in first place. basically, there would be no mailbox receive operator thread allocated unless there's work to be done.
Happy to start prototyping some of this stuff up :)

May be this is also a good time / opportunity to rethink from the operator level. Here are some thoughts...

Think of operator chain as a pipeline and data can be pumped up / down in / out of operators.
Some operators can produce / output data (technically data blocks / record batches) upon input / consume. Example PROJECT.
Some operators need to consume everything before produce / output data. Example GROUP BY

Operators have a state machine something along the lines .....

SETUP / INIT
CAN_CONSUME
CAN_CONSUME_FROM_LEFT
CAN_CONSUME_FROM_RIGHT
CAN_PRODUCE
BLOCKED
DONE

Operator abstraction with sub-interfaces

Producer (e.g API produceData, outputData)
SingleConsumer (e.g API consumeData, noMoreToConsume). Can also be a Producer
DualConsumer (e.g HashJoin). Can also be a Producer

Pipe:

An abstraction representing a collection of operators to enable pumping data between operators
provides method pump()
Works with source / sink concept for upstream / downstream
pump() can return
-- DONE essentially also setting noMoreToConsume on sink because upstream is done producing
-- PUMPED / IN_PROGRESS (upstream / source has output batch for downstream / sink to consume)
-- UPSTREAM_NOT_READY if sink is CAN_CONSUME, but source is not ready to produce
-- DOWNSTREAM_NOT_READY if sink itself is blocked or is holding batches that it is yet to produce / output and thus can't consume more (yet) from upstream

During execution planning when physical operators are created is when we setup the Pipeline.

we need to make sure that the stage only runs when there's work to be done and that the threads that are doing that work are efficiently shared between potentially many concurrent queries

+1

Yes, I think we need executors at the inner stage level to execute the pipelines inside the Stage.
Each such executor will execute a pipeline inside a single thread. It calls pump() on the pipeline (described above).
It has the capability to be scheduled, rescheduled, yield etc.
Through the state machine, we need to know when work is available
We need to wake up this executor when work is available to be done for the pipeline

I like the idea to make operator a state machine. It is more extensible. If an operator needs to be retried, timed out or checkpoint in the future, we can easily extend that.
Since we are thinking about the architecture point of view, we should consider isolation/priority in the thread pool. Otherwise, it will easily exhaust the server resources.
This sounds like a fit for actor model? :)

Had some a quick discussion with @walterddr and wanted to jot down those thoughts here so I don't forget:

there's two orthogonal design considerations: there's parallelism within a pipe/stage and there's scheduling across different pipe/stages (potentially across queries as well). The comment that I had posted is mostly regarding the latter. I think @siddharthteotia's comment is mostly about the former (though it has good thoughts on both).
implementing backpressure via GRPC on a per-mailbox level is possible, but we need to take care to make sure that the backpressure is piped all the way back to the leaf server execution itself. If you look at QueryRunner, there's currently no way to apply backpressure as it executes the entire leaf node requests before creating/sending data in the MailboxSendOperator. This is possible, but just requires some work
"Some operators need to consume everything before produce / output data. Example GROUP BY", for now I'll refer to these operators as stateful as opposed to stateless operators (SORT and the broadcast part of HASH_JOIN fall into this category as well).
I think it's important that all pipes/stages support partial execution - basically you can schedule a pipeline and it will do all the work that it can do and then terminate, even if it doesn't see an EOS block or produce any data (in the case of stateful operators, stateless ones will always produce data). The partial state is maintained so that when it is rescheduled (when another block is available) it can continue when it left off.

Some thoughts on @siddharthteotia's comments specifically:

Yes, I think we need executors at the inner stage level to execute the pipelines inside the Stage.

I'm a bit confused about the terminology here. Perhaps we can standardize on "operator" as a single unit of work, "pipeline" as the smallest schedule-able set of operators and "stage" as a complete remote-receive-to-remote-send set of pipelines. Today "pipeline" is always equivalent to a stage. I think that was how you were using that terminology.
With these definitions, I think we want executors to be independent of the number of stages/pipelines that are currently running on a multistage intermediate server. That might make QoS (quality of service) and thread pool management a little difficult - IMO one fixed thread pool and a priority-aware scheduler can get us pretty far so long as each pipeline can be scheduled independently and we have (see below) a mechanism to split pipelines.

Operators have a state machine something along the lines

I really like this idea, this will also help us in the case where we want to split stages into additional local stages - and we can leverage the work @ankitsultana was working on in #9484 to increase parallelism without needing to introduce a parallel processing framework within a single stage/task.

Adding notes from offline discussion with @walterddr @61yao @siddharthteotia. Feel free to add additional details if I missed anything.
Design notes:

the shared buffer itself should have a notion of fairness to make sure that it doesn't fill up with data all from a single query
the operator chain task pool should wake up on either (a) new incoming data for the corresponding mailbox or (b) a new operator chain is registered for execution to make sure that there is no race between registering a new operator chain and receiving data from an upstream sending node
the shared buffer should have configurable limits, ideally on both data size as well as number of blocks

Discussion around out-of-scope considerations:

retrying failed operator chains/queries (note: cascading a failure is in scope and should leverage Query Preemption)
implementing parallelism within an operator-chain
implementing pipelining or partition-level parallelism of operator chains
for v1, the operator chain scheduler will be round-robin. it should be pluggable to support priority scheduling that can ensure fairness but those implementations are out of scope

Next steps, @agavra to come up with a PEP document with more implementation and design details after some prototyping.

Putting a link to a design doc here: https://docs.google.com/document/d/1XAMHAlhFbINvX-kK1ANlzbRz4_RkS0map4qhqs1yDtE
I'll schedule a design review, if anyone following this ticket would like to be a part of it just ping me on the OSS Slack with your email and I can add you.


Set BasicProperties delivery mode for BasicPublish

Hi.
I’d like to understand why RabbitMQDeliveryMode only enables persistence for the exchange and not for both the published message and the exchange.
In this scenario, it seems like there’s a chance of message loss, which I’m concerned about.
This is my first time working with this project, and I haven't found any related issues so far.

@Andre-Meira It apears this was lost during refactoring the library.

Hi @ArieGato.
I'm not sure how this property was handled before the library was refactored, but I think it would be useful to have a configuration based on RabbitMQ’s IBasicProperties to set message parameters like:

DeliveryMode
Expiration
Priority
CorrelationId

I’d be glad to help with any improvements on this.

thanks for the offer, but I have an idea how to solve this. For this I'm waiting for RabbitMQ client 7.x. That will also solve #211.

Ok, thanks @ArieGato.
I'm waiting for the resolution.

@Andre-Meira The issue with persistence has been fixed in v7.0.3
Setting the other properties of IBasicProperties will come in v8.

The issue has been successfully resolved.
Tks...


Error when running account-followers.feed.example.ts

Hello.
when I'm trying to run "account-followers.feed.example.ts", I receive this error:

It refers to this line which I edited:
const { IgApiClient } = require('../src');
How can I solve this?

I just read the comments on #1146 issue recently.
I think it could solve this problem to.
If it doesn't I let you know guys.

TypeScript files (.ts) needs to be compiled into JavaScript (.js).

You can also run them using ts-node which is already installed after running npm i.

I think that including some .js examples would be useful for many users, no?

Typescript syntax is very similar to regular JS so I don't think there's an example needed.


Fixes Dropbox authentication

Cool, thanks!
We need to focus on the async flow readability a bit. Generally, we have these instruments:

this is specified during runtime based on how the method gets called. Anything executed as a callback will have incorrect this.
func.bind(stuff) will create a bound version of func that will always have this == stuff (independently on how func gets called)
generally speaking, nested callbacks are best solved using Promises. We would have to use a Promise polyfill for older browsers, though.

Let me refactor the current solution using proper binding (without explicit closures) first; I will consider adding Promises later.

Refactored explicit us to .bind(this ) in https://github.com/ondras/wwwsqldesigner/commit/9ff7bfe0ff62fa43beb590cb37c836f4e77f2ce9.


issue about span job to other node

If there are too many jobs to run ,  how the system(openwhisk on kubernetes ) span jobs to other node ?
kubenetes span openwhisk components or job to other node?
If a new job come, openwhisk restart a docker container to run the job, right?  how does openwhisk allocate and free resource? resource is managed by openwhisk or by kubernetes ? and when do kubernetes span openwhisk components or  job to other node?  how to load balance?
Anyone who can help me?

The invoker manages a pool of containers (say in a pod) and the load balancer (part of the controller) distributes load to invokers. If you add new invokers they register with the load balancer so it can use them.
The reason for the custom container manager is for promoting container reuse and cut latency of allocating containers.

@rabbah Thanks a lot
It's mean that we can increase the number of invoker to parallel the job threads. The pool of containers is managed by invoker . Actually the container resource is managed by docker engine,right? and the docker engine can manage the resource of this node(kubernetes slave node) it run on.
Can the invoker allocate resource on other node(kubernetes slave node) ? I think no
I want to know how to span job to other node when resource is not enough on one node .
Span openwhisk components and job  or just job to other node ? Span is managed by openwhisk or by kubernetes?

@rabbah thanks
The container which invoker managed is allocated in invoker container or  on the localhost? The pool of containers is running  in the invoker container or on the host?

I think this issue can be closed.  Can someone with the right privileges please close it?


CO - Send OTC Results to TEST

CO is interested in review what OTC data ReportStream has available for them. Please send available OTC data to CO's TEST / staging endpoint for their review.

@brick-green-agile6 will generate a sample for OTC for CO, and submit to CO by 12/3

@anshulkumar-usds I am unclear about what is needed here. Should I open up all OTC to a test feed for CO?

1/23: Anshul checking with CO on what's needed for OTC results

CO does not have staffing to address this for the next two weeks. RS needs to circle back with CO in early/mid Feb

5/5: CO got back to us that they are willing to test the data. They requested we send this data to their TEST env for their evaluation.
@brick-green-agile6 perhaps something for next sprint?

Deprioritized for this sprint.


Refactor/refactor favicon

I had to play around with my suggestions locally to get my recommendation right. I thought you might appreciate me just pushing the changes... but feel free to disregard.
The changes

Move the icons out of uploads and in to images/logos
Change the favicon path to the fav icon svg instead of the logo png
Merge in main
Revert testing changes

Also we should remove the old favicon (Prima Mesa P), see latest commit.

Closed, works but didn't fix the root issue (upload folder not passed through correctly), see #98 for the full discussion and final PR.


fix: hide duplicate curriculum units

Description

Hide duplicate units without examboard
Simplify unit visualiser tests
Add comments and re-order units tab code for clarity

How to test

Go to https://deploy-preview-2101--oak-web-application.netlify.thenational.academy/teachers/curriculum/english-secondary-aqa/units
Scroll to Year 10
You should see only one unit titled "Modern text: first study" with 2 unit options

Screenshots
How it used to look (delete if n/a):

How it should now look:

Checklist

[x] Added or updated tests where appropriate
[x] Manually tested across browsers / devices
[x] Considered impact on accessibility
[x] Design sign-off
[x] Approved by product owner
[x] Does this PR update a package with a breaking change

:tada: This PR is included in version 1.303.1 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Fkey ACL change attempts fail with 500 errors

The easiest way to reproduce this is probably to just do:
table = model.table(sname, tname)
table.apply()

on a table with foreign keys.
This isn't a high priority -- I don't actually need to change any foreign key ACLs. This is just something I stumbled across.

Please provide more information.  I have working projects that apply changes at table, column, and fkey nodes of a model while reconfiguring a catalog...

Closing this old issue for lack of detail and ability to reproduce. If you encounter problems like this again, please open a new issue and try to include error/stack information from the server logs.


There are issues when I use my own datasets

Excuse me, when I use my own datasets(I change the original data to my own data, and the path remains the same), there are some bugs in Pycharm, look at the link

Could you please tell me what should I do

Excuse me, when I use my own datasets(I change the original data to my own data, and the path remains the same), there are some bugs in Pycharm, look at the link

Could you please tell me what should I do

Hey, I faced this problem too. And I find that this problem occurs when we run one dataset first (like the example dataset) and then change to another. All you need to do is to delete the 'results' folder ( 'tf_ner/models/lstm_crf/results') before we change the dataset. Or you can pull this repository again before you change dataset, but be sure to save important files.
Hope that it's still helpful. This is a really late reply tho 😇


Nested initialization list in struct crash fix

fixes: #4489
Problem:

Previously, TopStruct var = {} would crash if a nested member also had a initExpr assigned to {}. This crash is due to a particular ordering of how {} resolves its own type blindly setting it as an invalid type as a place holder.

Solution:

only set the init list type when invalid (if we checkTerm twice on the same object, this won't break the code)

Note: removed the overhaul to default initialization since it breaks Falcor and likely needs more time dedicated to it, the code history will be kept in this branch (in the mean-time it is best to fix the current regression with a more simple fix).

Why do we double check the expr in the first place?

This is not isolated behavior, If we have two AssignExpr that share the same ->right member, we will run checkTerm on the ->right member twice. This only breaks visitInitializerListExpr because visitInitializerListExpr is dependent on being called only once.
Why this crash happens now?
The behavior is showing up now due to minor ordering changes in AST building.
Previous Logic:

Slang did not ensureDecl(memberVariable, DeclCheckState::DefinitionChecked) when checking a structDecl, we only auto generate __init for structDecl.
Naturally ensureDecl() on the auto generated __init for structDecl. This causes a check for AssignExpr (contains initExpr). This assigns place-holder invalid type to initExpr from call to visitInitializerListExpr
Slang naturally runs ensureDecl(memberVariable, DeclCheckState::DefinitionChecked)
Now we call checkVarDeclCommon to check the memberVariable. This runs CheckTerm() on our initExpr as well. This re-assigns place-holder invalid type from call to visitInitializerListExpr. We now run coerce to change the initExpr type to something valid.

New Logic:

Slang calls ensureDecl(memberVariable, DeclCheckState::DefinitionChecked) when checking a structDecl
Now we call checkVarDeclCommon to check the memberVariable. This runs CheckTerm() on our initExpr. This assigns place-holder invalid type from call to visitInitializerListExpr. We now run coerce to change the initExpr type to something valid.
we auto generate __init for structDecl now.
Naturally ensureDecl() on the auto generated __init for structDecl. This causes a check for AssignExpr (contains initExpr). This re-assigns place-holder invalid type to initExpr from call to visitInitializerListExpr

We were double checking the initExpr before. Code working was pure chance on how Slang was parsing structDecls.

Can we stop doing that?

The solution is to not use initExpr directly ever (clone the AST node before using) since we will only have 1 instance of each initExpr in that case. This is a hack though, visitInitializerListExpr is fundamentally unstable with Slang AST.

Shouldn't we exit immediately if we can determine the expr is already checked?

Shouldn't we exit immediately if we can determine the expr is already checked?

I don't know if this would be a problem (we may modify arg list)?

An expr should never need to be checked twice. Once it is checked it should be in its final form and not be modified elsewhere.

@ArielG-NV
Does the gfx-smoke test pass?
If yes, can you enable the gfx-smoke test and re-run all the checks?

Does the gfx-smoke test pass?
If yes, can you enable the gfx-smoke test and re-run all the checks?

It does pass, I can (and will) enable the test.

ok, can you please enable the test in this PR?

ok, can you please enable the test in this PR?

we never enabled gfx-smoke since gfx has problems with CI

ok, can you please enable the test in this PR?

we never enabled gfx-smoke since gfx has problems with CI, here should be an issue open for this already.

Do you know what is the issue? And do we have issue tracking this?

Do you know what is the issue? And do we have issue tracking this?

https://github.com/shader-slang/slang/issues/3176


Embed placement.Storage in placement.Service

placement.Service used to expose Delete(), SetPlacement and Placement(), which are also exposed from placement.Storage interface, now that we also need to expose SetProto() and Proto(), this pr made placement.Storage embedded in placement.Service interface so that future updates in the placement.Storage interface are available through placement.Service as well.
this pr also moved Dryrun logic to placement.Storage.
@xichen2020 @prateek @jeromefroe

Coverage decreased (-0.1%) to 84.026% when pulling f1f3976a0b3f5449b22c6e353a9da89b594dc586 on chao-service into 5c461717e3d6dcaa613628fecc0dc209d0697f26 on master.

Coverage decreased (-0.1%) to 84.026% when pulling b89a89e834b258488b823cf3fedc00c404affb99 on chao-service into 5c461717e3d6dcaa613628fecc0dc209d0697f26 on master.