[Kernel] Fullgraph and opcheck tests

Add opcheck tests for all remaining ops and add some more models (and an inductor check) to the torch.compile fullgraph tests.
The new tests uncovered a few issues (schema problems, missing meta-function, etc.) which have been fixed.
The selective_scan_fwd op had a weird schema/behavior.  It returned one of the input tensors but also modified it in place.  This confusing pytorch so I removed that tensor x from the outputs and left it as modified in place.
The only ops that don't have any sort of opcheck test are:

advance_step_flashattn
advance_step_flashinfer

cc @SageMoore , @ProExpertProg , @youkaichao , @tlrmchlsmth , @LucasWilkinson

 PR Checklist (Click to Expand)
Thank you for your contribution to vLLM! Before submitting the pull request, please ensure the PR meets the following criteria. This helps vLLM maintain the code quality and improve the efficiency of the review process.
PR Title and Classification
Only specific types of PRs will be reviewed. The PR title is prefixed appropriately to indicate the type of change. Please use one of the following:

[Bugfix] for bug fixes.
[CI/Build] for build or continuous integration improvements.
[Doc] for documentation fixes and improvements.
[Model] for adding a new model or improving an existing model. Model name should appear in the title.
[Frontend] For changes on the vLLM frontend (e.g., OpenAI API server, LLM class, etc.)
[Kernel] for changes affecting CUDA kernels or other compute kernels.
[Core] for changes in the core vLLM logic (e.g., LLMEngine, AsyncLLMEngine, Scheduler, etc.)
[Hardware][Vendor] for hardware-specific changes. Vendor name should appear in the prefix (e.g., [Hardware][AMD]).
[Misc] for PRs that do not fit the above categories. Please use this sparingly.

Note: If the PR spans more than one category, please include all relevant prefixes.
Code Quality
The PR need to meet the following code quality standards:

We adhere to Google Python style guide and Google C++ style guide.
Pass all linter checks. Please use format.sh to format your code.
The code need to be well-documented to ensure future contributors can easily understand the code.
Include sufficient tests to ensure the project to stay correct and robust. This includes both unit tests and integration tests.
Please add documentation to docs/source/ if the PR modifies the user-facing behaviors of vLLM. It helps vLLM user understand and utilize the new features or changes.

Adding or changing kernels
Each custom kernel needs a schema and one or more implementations to be registered with PyTorch.

Make sure custom ops are registered following PyTorch guidelines: Custom C++ and CUDA Operators and The Custom Operators Manual
Custom operations that return Tensors require meta-functions. Meta-functions should be implemented and registered in python so that dynamic dims can be handled automatically. See above documents for a description of meta-functions.
Use torch.libary.opcheck() to test the function registration and meta-function for any registered ops.  See tests/kernels for examples.
When changing the C++ signature of an existing op, the schema must be updated to reflect the changes.
If a new custom type is needed, see the following document: Custom Class Support in PT2.

Notes for Large Changes
Please keep the changes as concise as possible. For major architectural changes (>500 LOC excluding kernel/data/config/test), we would expect a GitHub issue (RFC) discussing the technical design and justification. Otherwise, we will tag it with rfc-required and might not go through the PR.
What to Expect for the Reviews
The goal of the vLLM team is to be a transparent reviewing machine. We would like to make the review process transparent and efficient and make sure no contributor feel confused or frustrated. However, the vLLM team is small, so we need to prioritize some PRs over others. Here is what you can expect from the review process:

 After the PR is submitted, the PR will be assigned to a reviewer. Every reviewer will pick up the PRs based on their expertise and availability.
 After the PR is assigned, the reviewer will provide status update every 2-3 days. If the PR is not reviewed within 7 days, please feel free to ping the reviewer or the vLLM team.
 After the review, the reviewer will put an  action-required label on the PR if there are changes required. The contributor should address the comments and ping the reviewer to re-review the PR.
 Please respond to all comments within a reasonable time frame. If a comment isn't clear or you disagree with a suggestion, feel free to ask for clarification or discuss the suggestion.

Thank You
 Finally, thank you for taking the time to read these guidelines and for your interest in contributing to vLLM. Your contributions make vLLM a great tool for everyone!

made a small comment otherwise LGTM


Replace custom splash screen

Add compat library for native splash screen and completely remove the custom one.

Closed by a4483788325f12b35dd48e6aeef7a9f1f55bebae


coprocessor: support builtin aggregation function bit_and, 'bit_or, bit_xor.

The related modification in tipb is : https://github.com/pingcap/tipb/pull/67

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Please fix the CI

@breeswish @winkyao PTAL

@AndreMouche to make ci work,  https://github.com/pingcap/tipb/pull/67 must be checked in first.

@hanfei1991 PTAL

Should be merge after #2503 because some other tipb changes works with it.

@AndreMouche @hanfei1991 @winkyao  PTAL

@AndreMouche I'm also working on the tidb part. It's in progress.

Thanks, I've got it. We would like to review this PR after related PRs merged.

@AndreMouche  the related PRs have been merged. I've revert the changes in Cargo.lock to fix dependency for tipb. PTAL.
BTW: make test and make works well on my laptop, but ci/circleci fails when build rocksDB. Do you know the reason?

coprocessor/select is not used for tidb.....
Please implement it in coprocessor/dag, haha...

@hanfei1991 @breeswish PATL

Please fix the conflicts @spongedu

/run-all-test

/ok-to-test

/run-all-tests


[Feature] Ability to extract files simultaneously

Fix or Feature to #10

Going forward, I wonder whether it would be better to generate a unique ID for each instance rather than use the file name. The plan is to eventually include partial results of the waveform data in the progress so that eventually you might not need the file.

This looks great! thank you guys for the great work.
@ryanheise if you think this is resource-intensive what do you think about adding a max number of concurrent jobs so that people could limit the number of files being extracted?
if this PR doesn't get merged, do you think we can at least throw an error if a job tries to run while another one is running?
I myself will be implementing a queue interface but I think an exception thrown is much needed.
I will be happy to help. :)

@ryanheise if you think this is resource-intensive what do you think about adding a max number of concurrent jobs so that people could limit the number of files being extracted?

That is something that can/should be implemented by the app. (I like to keep things simple)

if this PR doesn't get merged, do you think we can at least throw an error if a job tries to run while another one is running? I myself will be implementing a queue interface but I think an exception thrown is much needed. I will be happy to help. :)

At this stage my plan is to eventually merge this PR.
Before I do that, I'm just thinking about whether we want to have a unique ID for each instance generated by uuid or just use the file path as the unique ID.  (Again just thinking of simplicity.)

Before I do that, I'm just thinking about whether we want to have a unique ID for each instance generated by uuid or just use the file path as the unique ID.  (Again just thinking of simplicity.)

I think we should avoid the path because of duplication. If the same widget is rendered twice this would cause an error. Where using uuid even if there are duplicates the waveform would be extracted.

I would consider it a misuse of the plugin to simultaneously write to the same path, so the path should be unique among the simultaneous requests.

It is a misuse, but I still think UUID would be the better approach since it's not prone to as many problems as the path approach down the line. And it would stop people from opening issues when they misuse the plugin, instead, it would just make their apps a little bit slower.

This problem is not a a real problem since it can be documented and an exception can be thrown if the same output file is being used in two simultaneous requests. You mentioned there are more problems, though?

I'll add to the above that using a UUID does not actually solve the problem of using the same output file in simultaneous requests - an exception would be needed.

This PR has now been merged and released, and I have done some minor cleanup and removed the uuid key in favour of the output filename. Thanks, @AlexSmirnov9107 for the contribution.


Fix Proxy start up throw  NullPointerException

The issue tracker is used for bug reporting purposes ONLY whereas feature request needs to follow the RIP process. To avoid unnecessary duplication, please check whether there is a previous issue before filing a new one.
It is recommended to start a discussion thread in the mailing lists in cases of discussing your deployment plan, API clarification, and other non-bug-reporting issues.
We welcome any friendly suggestions, bug fixes, collaboration, and other improvements.
Please ensure that your bug report is clear and self-contained. Otherwise, it would take additional rounds of communication, thus more time, to understand the problem itself.
Generally, fixing an issue goes through the following steps:

Understand the issue reported;
Reproduce the unexpected behavior locally;
Perform root cause analysis to identify the underlying problem;
Create test cases to cover the identified problem;
Work out a solution to rectify the behavior and make the newly created test cases pass;
Make a pull request and go through peer review;

As a result, it would be very helpful yet challenging if you could provide an isolated project reproducing your reported issue. Anyway, please ensure your issue report is informative enough for the community to pick up. At a minimum, include the following hints:
BUG REPORT

Please describe the issue you observed:

when i execute command:
nohup sh bin/mqbroker -n localhost:9876 --enable-proxy &

Please tell us about your environment:

Other information (e.g. detailed explanation, logs, related issues, suggestions on how to fix, etc):

FEATURE REQUEST

Please describe the feature you are requesting.

Provide any additional detail on your proposed use case for this feature.

Indicate the importance of this issue to you (blocker, must-have, should-have, nice-to-have). Are you currently using any workarounds to address this issue?

If there are some sub-tasks involved, use -[] for each sub-task and create a corresponding issue to map to the sub-task:

sub-task1-issue-number: sub-task1 description here,
sub-task2-issue-number: sub-task2 description here,
...

This is a bug about proxy metrics. We should check config metricsExporterType before init metrics component in proxy local mode. Will you submit a pr to fix it?

@ShadowySpirits  yes I will submit a pr to fix it


HLG not included in SDR as HDR Format

https://github.com/TRaSH-/Guides/blob/master/docs/json/sonarr/cf/sdr.json
https://github.com/TRaSH-/Guides/blob/master/docs/json/radarr/cf/sdr.json
      "name": "HDR Formats",
      "implementation": "ReleaseTitleSpecification",
      "negate": true,
      "required": false,
      "fields": {
        "value": "\\bHDR(\\b|\\d)|\\b(dv|dovi|dolby[ .]?vision)\\b|\\b(FraMeSToR|HQMUX|SiCFoI)\\b|\\b(PQ)\\b"

Doesn't include HLG. Thus when run on item with HLG it gets scored as SDR. Adding HLG allows for proper HLG scoring.
\\bHDR(\\b|\\d)|\\b(dv|dovi|dolby[ .]?vision)\\b|\\b(FraMeSToR|HQMUX|SiCFoI)\\b|\\b(PQ)\\b|\\b(HLG)\\b

Thnx for reporting will be fixed as soon as PR #1169 has been merged


[CARBONDATA-2258] Separate visible and invisible segments info into two files to reduce the size of tablestatus file.

The size of the tablestatus file is getting larger, there are many places will scan this file and it will impact the performance of reading this file.
According to the discussion on thread, it can append the
invisible segment list to the file called 'tablestatus.history' when execute
command 'CLEAN FILES FOR TABLE' (in method 'SegmentStatusManager.deleteLoadsAndUpdateMetadata') every time, separate visible and invisible segments into two files(tablestatus file and tablestatus.history file).
Be sure to do all of the following checklist to help us incorporate
your contribution quickly and easily:

[ ] Any interfaces changed?

[ ] Any backward compatibility impacted?

[ ] Document update required?

[ ] Testing done
Please provide details on
- Whether new unit test cases have been added or why no new tests are required?
- How it is tested? Please attach test report.
- Is it a performance related change? Please attach the performance test report.
- Any additional information to help reviewers in testing this change.

[ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

Build Failed with Spark 2.2.1, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder/3283/

SDV Build Fail , Please check CI http://<IP_ADDRESS>:8080/job/ApacheSDVTests/4017/

retest this please

Build Failed with Spark 2.2.1, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder/3288/

Build Failed  with Spark 2.1.0, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder1/4516/

Build Failed with Spark 2.2.1, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder/3303/

Build Failed  with Spark 2.1.0, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder1/4531/

retest sdv please.

SDV Build Success , Please check CI http://<IP_ADDRESS>:8080/job/ApacheSDVTests/4031/

Build Failed with Spark 2.2.1, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder/3308/

Build Failed  with Spark 2.1.0, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder1/4536/

Build Success with Spark 2.1.0, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder1/4542/

SDV Build Success , Please check CI http://<IP_ADDRESS>:8080/job/ApacheSDVTests/4035/

retest this please

retest sdv please

Build Failed with Spark 2.2.1, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder/3318/

Build Success with Spark 2.1.0, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder1/4545/

SDV Build Success , Please check CI http://<IP_ADDRESS>:8080/job/ApacheSDVTests/4040/

SDV Build Success , Please check CI http://<IP_ADDRESS>:8080/job/ApacheSDVTests/4043/

SDV Build Success , Please check CI http://<IP_ADDRESS>:8080/job/ApacheSDVTests/4044/

retest this please

Build Success with Spark 2.1.0, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder1/4549/

Build Success with Spark 2.2.1, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder/3322/

@jackylk @ravipesala  please help to review, thanks

Build Success with Spark 2.2.1, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder/3339/

Build Success with Spark 2.1.0, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder1/4565/

SDV Build Success , Please check CI http://<IP_ADDRESS>:8080/job/ApacheSDVTests/4061/

LGTM


cjk search experience is too bad, the highlight position is wrong.

I think this is highly related to #790.

Hello @jet10000! Sorry for the delay, we are currently working hard on the new search engine that should fix this issue! Is it possible to have the dataset and your query? Or at least a sample of the dataset to reproduce the error.

My bad, this issue should be already fixed with the integration of the new tokenizer thanks to this PR -> https://github.com/meilisearch/MeiliSearch/pull/1091
If it's not, please feel free to re-open this issue


Subtitles are not included in the final video

Bug description
If the original video had a subtitle stream, it is not included in the edited video. I believe this should be the case, since there is the option -sn which should explicitly disable subtitle output. However, even with this flag there are no subtitles.
What's your platform?
OS: Darwin 23.4.0 arm64
Python: 3.12.4
PyAV: 12.3.0
Auto-Editor: 24w30a

What command did you use
auto-editor video.mp4 --no-open

What properties does your input video have?
video.mp4:

recommendedTimebase: 25/1
video:

track 0:

codec: h264
fps: 25
resolution: 640x360
aspect ratio: 16:9
pixel aspect ratio: 1
duration: 5272.48
pix fmt: yuv420p
timebase: 1/12800
bitrate: 267032
lang: und

audio:

track 0:

codec: aac
samplerate: 48000
channels: 1
duration: 5272.405
bitrate: 64971
lang: und

subtitle:

track 0:

codec: mov_text
lang: eng

container:

duration: 5272.48
bitrate: 338162

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'video.mp4':
Metadata:
major_brand     : isom
minor_version   : 512
compatible_brands: isomiso2avc1mp41
encoder         : Lavf61.1.100
Duration: 01:27:52.48, start: 0.000000, bitrate: 338 kb/s
Stream #0:00x1: Video: h264 (High) (avc1 / 0x31637661), yuv420p(progressive), 640x360 [SAR 1:1 DAR 16:9], 267 kb/s, 25 fps, 25 tbr, 12800 tbn (default)
Metadata:
handler_name    : VideoHandler
vendor_id       : [0][0][0][0]
Stream #0:10x2: Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, mono, fltp, 64 kb/s (default)
Metadata:
handler_name    : SoundHandler
vendor_id       : [0][0][0][0]
Stream #0:20x3: Subtitle: mov_text (tx3g / 0x67337874), 0 kb/s (default)
Metadata:
handler_name    : SubtitleHandler
Comments
No response

25.0.1 should fix this


Deprecations, disables, deletions for 3.4.0

Do not merge until the next tag will be 3.4.0.

Review period ended.


Removed unnecessary code in Libraries/Text/Text.js

I don't think it's necessary to use spread properties (prevState) in getDerivedStateFromProps
Test Plan:

[x] Check npm run flow
[x] Check npm run flow-check-ios
[x] Check npm run flow-check-android

Release Notes:
[GENERAL] [ENHANCEMENT] [Libraries/Text] - remove unnecessary code

I don't feel confident in the behavior of getDerivedStateFromProps to know what the return value is for this. The docs just say:

It should return an object to update the state, or null to update nothing.

It isn't clear to me whether this is a partial object similar to what you can use in setState or if it needs to be the complete object as would be used to set state in the constructor.
Do you have any resources that make it clear which it is?

@TheSavior

Text already has a process of initializing state. (https://github.com/facebook/react-native/blob/v0.57.4/Libraries/Text/Text.js#L103)

In getDerivedStateFromProps, only responseHandlers are changed when the value of props changes or it is touchable from the beginning. (https://github.com/facebook/react-native/blob/v0.57.4/Libraries/Text/Text.js#L110)

I'm sure getDerivedStateFromProps only needs to return the value of the change, not all values of the state. This is because previous and new values of state are merged internally. You can see it in the code below.
function applyDerivedStateFromProps(workInProgress, ctor, getDerivedStateFromProps, nextProps) {
....
var partialState = getDerivedStateFromProps(nextProps, prevState);
var memoizedState = partialState === null || partialState === undefined ? prevState : _extends({}, prevState, partialState);
....
}

Hi @ifsnow, is this part of the larger effort under #22100? I plan to combine those entries in the changelog to a single entry, so the changelog section in this PR is n/a.

@turnrye I agree with your plan to combine those entries. Thank you.


ZCPU: Removed unnecessary compiling

Programs which include 5 different libraries and are >2000 lines of code need ~5 seconds to compile. The code gets then 3 times compiled:

When validating
When closing editor
When placing ZCPU
That's really annoying if you make a small change. It's also pretty useless to compile a program 2-3 times.
I fixed it by only letting it compile when validating.

Is code automatically validated before placing?

It gets compiled when placing

Does this change make it so the CPU tool uploads whatever was last compiled, not what's currently open in the editor?

It uploads the last compiled program. You can only compile with the validator button in the editor, which is the aim of this patch. If you upload the program while compiling, it will upload the last compiled (before current compiling) program.

Can you make it so uploading will compile if the code has been changed since it was last compiled?
I'm guessing the answer is "no since the editor doesn't keep track of that"

so keep the last (hash of the) source that went into compilation and compare?

Also, if the player starts a compile and then tries to upload, the upload should wait for the compile to finish before starting, so you can't accidentally upload stale code that way.

What about disabling uploading while compiling?

you'd also need to stop compilation once there's a change, otherwise you end up uploading code that finished compilation after the last editor change, but started compiling before it.

@TomyLobo In the long term I'd like to make a new more sane mostly-compatible language and rewrite the entire parser and compiler. As of yesterday I even have internet at home, too.

sounds good

but rather ambitious

@TomyLobo HL-ZASM is slow because of the intentional throttling.
I can only speculate why it's intentionally throttled as much as it is, but the problem fixed in this commit (part of PR #1004) might've been part of it - it causes an incredible amount of slowdown on large files. The throttling is broadly adjustable by the wire_cpu_compile_speed convar.
It would probably be better to throttle the compiler based on CPU time instead - say, to allow 10ms per frame. (How does E2 determine the amount of CPU time used?)

Because of the (apparent need) for this throttling, the compiler's written in a weird incremental way. I'd much rather see it written like a normal compiler and then use coroutine.yield() to do throttling as necessary, which would easily let us run the compiler until some amount of time has passed.

This got useless since the large compile times got fixed through #1004.


PipelineRollout Tests

Fixes #20
Modifications
Updated the suite_test.go file to be able to setup an environment to test PipelineRollout reconciliation logic using Numaflow CRDs. Also, added test cases to pipelinerollout_controller_test.go file to test create, update, and delete of a PipelineRollout resource.
Verification
Issued make test command and verify all tests passed.

This is very elegant....but I'm not sure if I see any cases of things going wrong? should that be added to a subsequent PR? (at least the case of invalid yaml, not sure what else off the top of my head)

It should be noted this unit test doesn't really cover things on an individual function level, but maybe this is really the only good way to test Reconcile(). But it makes me wonder if we want to include individual function tests for some functions that are strictly related to data processing now or in the future. Any functions that are trivial it's probably not worth doing, but I'm not sure if any of our functions are non-trivial.

yes, this is validation check in CRD not webhook, right? looks good then.

@juliev0 do you want to take another look? Otherwise I will merge this. Thanks!

@juliev0 do you want to take another look? Otherwise I will merge this. Thanks!

please do, thanks!


Files are stuck in Queued state.

Hi!
My files are stuck in Queued (attached sreenshot) state (video, pdf, mp3, zip, etc.), but images (png,jpeg) are uploading without any issues. Anyone else experiencing same problems?

My package.json:
  "dependencies": {
    "@sanity/base": "^2.17.2",
    "@sanity/cli": "^2.17.1",
    "@sanity/components": "^2.14.0",
    "@sanity/core": "^2.17.2",
    "@sanity/default-layout": "^2.17.2",
    "@sanity/default-login": "^2.17.2",
    "@sanity/desk-tool": "^2.17.2",
    "@sanity/google-maps-input": "^2.18.0",
    "@sanity/orderable-document-list": "^0.0.9",
    "@sanity/production-preview": "^2.15.0",
    "@sanity/vision": "^2.17.2",
    "prop-types": "^15.7",
    "react": "^17.0",
    "react-dom": "^17.0",
    "react-icons": "^4.2.0",
    "react-is": "^18.1.0",
    "sanity-plugin-color-picker": "^1.0.3",
    "sanity-plugin-iframe-pane": "^1.0.6",
    "sanity-plugin-media": "^1.4.8",
    "styled-components": "^5.3.5"
  }

If anyone can help, thanks in advance!

After trying out all versions of the plugin, here is an update:
Versions where everything works okay ✅
1.4.0
1.4.1
1.4.2
1.4.3
1.4.4

Versions where files are stuck in QUEUE ❌
1.4.8
1.4.7
1.4.5

Thanks for the detailed report @Igor178! This should be resolved in 1.4.9.
The regression was brought about in this commit – please do shout if you come across any other issues.


After clicking Reset, reads still loaded in variant service

Reset only appears to work on parameters.  Is that by design?

Fixed in https://github.com/PATRIC3/p3_web/pull/301


nerdctl run -d  -m  failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process (memory.memsw.limit_in_bytes: no such file or directory: unknown)

Description
nerdctl run -d  -m  xx  failed
but   nerdctl run -d    xx  ok
Steps to reproduce the issue
1.run in root user
2. test nerdctl run container and create container
3. # nerdctl run -d  --name nginx -p 80:80 nginx:alpine
1f67345beb5852bf5a88df336ea5810f2dee60e2c63c776e38f358e6cfa5950c
4 .#nerdctl   version
Client:
Version:       v1.4.0
OS/Arch:       linux/amd64
Git commit:    7e8114a82da342cdbec9a518c5c6a1cce58105e9
buildctl:
Version:      v0.11.6
GitCommit:    2951a28cd7085eb18979b1f710678623d94ed578
Server:
containerd:
Version:      v1.7.1
GitCommit:    1677a17964311325ed1c31e2c0a3589ce6d5c30d
runc:
Version:      1.1.7
GitCommit:    v1.1.7-0-g860f061b
lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 20.04 LTS
Release:        20.04
Codename:       focal
uname  -r
5.4.0-29-generic
apt search  libseccomp
libseccomp-dev/focal-updates,focal-security,now 2.5.1-1ubuntu1~20.04.2 amd64 [installed]
Describe the results you received and expected
#nerdctl run -d  -m   100m --name nginx -p 80:80 nginx:alpine
docker.io/library/nginx:alpine:                                                   resolved       |++++++++++++++++++++++++++++++++++++++|
index-sha256:b35f5bceca1f97a5cf7346a6964f8fc2ae9ffdabb94ea97f9ae50bb802f3b557:    done           |++++++++++++++++++++++++++++++++++++++|
manifest-sha256:2d4efe74ef541248b0a70838c557de04509d1115dec6bfc21ad0d66e41574a8a: done           |++++++++++++++++++++++++++++++++++++++|
config-sha256:4937520ae206c8969734d9a659fc1e6594d9b22b9340bf0796defbea0c92dd02:   done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:4db1b89c0bd13344176ddce2d093b9da2ae58336823ffed2009a7ea4b62d2a95:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:768e67c521a97f2acf0382a9750c4d024fc1e541e22bab2dec1aad36703278f1:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:9f05b0cc5f6e8010689a6331bad9ca02c62caa226b7501a64d50dcca0847dcdb:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:bd338968799fef766509223449d72392692f1f56802da9059ae3f0965c2885e2:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:4c5efdb87c4a2350cc1c2781a80a4d3e895447007d9d8eac1e743bf80dd75c84:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:6a107772494d184e0fddf5d99c877e2fa8d07d1d47b714c17b7d20eba1da01c6:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:c8794a7158bff7f518985e76c590029ccc6b4c0f6e66e82952c3476c095225c9:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:8de2a93581dcb1cc62dd7b6e1620bc8095befe0acb9161d5f053a9719e145678:    done           |++++++++++++++++++++++++++++++++++++++|
elapsed: 5.0 s                                                                    total:  16.2 M (3.2 MiB/s)
FATA[0006] failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error setting cgroup config for procHooks process: open /sys/fs/cgroup/memory/default/efb56c11b4a2000da2dfb381ce8cbfcf47085c0ac7f6478d1956b696894ca170/memory.memsw.limit_in_bytes: no such file or directory: unknown
What version of nerdctl are you using?
nerdctl-full-1.4.0-linux-amd64.tar.gz
Are you using a variant of nerdctl? (e.g., Rancher Desktop)
None
Host information
No response

Adding swapaccount=1 to kernel cmdline may work

@AkihiroSuda is there something tbd here?


Float division by zero

Hello, I have a set of vcf files and I am trying to use your software to check how many groups I have in my data. It is single-cell data and what I want to do exactly is compare my genotype data in each cell to every donor and determine what donor they come from.
I'm copying the issue:
Traceback (most recent call last):
File "/home/ciro/bin/NGSCheckMate/ncm.py", line 1461, in
classifying()
File "/home/ciro/bin/NGSCheckMate/ncm.py", line 905, in classifying
distance = pearson_def(vecA, vecB)
File "/home/ciro/bin/NGSCheckMate/ncm.py", line 59, in pearson_def
return diffprod / math.sqrt(xdiff2 * ydiff2)
ZeroDivisionError: float division by zero
Thank you very much,
Ciro

Hi again,
Now I called my SNPs with Freebayes but still find another error:
Traceback (most recent call last):
File "/home/ciro/bin/NGSCheckMate/ncm.py", line 1460, in
createDataSetFromDir(base_dir,bedFile)
File "/home/ciro/bin/NGSCheckMate/ncm.py", line 220, in createDataSetFromDir
real_depth[file] = depth[file] / float(real_count[file])
ZeroDivisionError: float division by zero
Let me know if you need any further details.

Hello, Mr. Ciro,
I think there are two possibilities about errors which you mentioned.

the number of genotyped loci is too small so that there may be few or nothing SNPs which we are using for calculating a correlation.
for the two sample comparison, there may be no overlapped loci between two samples, when the number of genotypes loci is too small even they have a few SNPs which we used for calculating a correlation.

Both of cases are due to the very low coverage (Calculating an average coverage of 21,067 SNPs), We could not apply NGSCheckMate to those samples, so we should remove those samples from the input set now.
Thank you,
Best,
SJ

Hello,
Is there a way to
Thank you for the answer,
Ciro


BUG: if datafiles have the same stem only one of them is uploaded

Ref. description of https://github.com/equinor/fmu-sumo-sim2sumo/pull/98

#98 merged


Run on Windows

When I run cargo test, I get the following error:
   Compiling snappy-sys v0.1.0
   Compiling serde_json v1.0.60
   Compiling clap v2.33.3
   Compiling miow v0.3.6
   Compiling proc-macro-crate v0.1.5
   Compiling parking_lot v0.11.1
   Compiling dirs-next v1.0.2
   Compiling crossbeam-epoch v0.9.1
error: failed to run custom build command for `snappy-sys v0.1.0`

Caused by:
  process didn't exit successfully: `C:\Users\Jasper\CLionProjects\scylla-rust-driver\target\debug\build\snappy-sys-42b9f30b7e2a9a5e\build-script-build` (exit code: 101)
  --- stdout
  cargo:rerun-if-env-changed=SNAPPY_NO_PKG_CONFIG
  cargo:rerun-if-env-changed=PKG_CONFIG
  cargo:rerun-if-env-changed=SNAPPY_STATIC
  cargo:rerun-if-env-changed=SNAPPY_DYNAMIC
  cargo:rerun-if-env-changed=PKG_CONFIG_ALL_STATIC
  cargo:rerun-if-env-changed=PKG_CONFIG_ALL_DYNAMIC
  cargo:rerun-if-env-changed=PKG_CONFIG_PATH_x86_64-pc-windows-msvc
  cargo:rerun-if-env-changed=PKG_CONFIG_PATH_x86_64_pc_windows_msvc
  cargo:rerun-if-env-changed=HOST_PKG_CONFIG_PATH
  cargo:rerun-if-env-changed=PKG_CONFIG_PATH
  cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR_x86_64-pc-windows-msvc
  cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR_x86_64_pc_windows_msvc
  cargo:rerun-if-env-changed=HOST_PKG_CONFIG_LIBDIR
  cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR
  cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_x86_64-pc-windows-msvc
  cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_x86_64_pc_windows_msvc
  cargo:rerun-if-env-changed=HOST_PKG_CONFIG_SYSROOT_DIR
  cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR
  OPT_LEVEL = Some("0")
  TARGET = Some("x86_64-pc-windows-msvc")
  HOST = Some("x86_64-pc-windows-msvc")
  TARGET = Some("x86_64-pc-windows-msvc")
  TARGET = Some("x86_64-pc-windows-msvc")
  HOST = Some("x86_64-pc-windows-msvc")
  CC_x86_64-pc-windows-msvc = None
  CC_x86_64_pc_windows_msvc = None
  HOST_CC = None
  CC = None
  TARGET = Some("x86_64-pc-windows-msvc")
  HOST = Some("x86_64-pc-windows-msvc")
  CFLAGS_x86_64-pc-windows-msvc = None
  CFLAGS_x86_64_pc_windows_msvc = None
  HOST_CFLAGS = None
  CFLAGS = None
  DEBUG = Some("true")
  running: "./autogen.sh"

  --- stderr
  thread 'main' panicked at 'failed to run: The system cannot find the file specified. (os error 2)', C:\Users\Jasper\.cargo\registry\src\github.com-1ecc6299db9ec823\snappy-sys-0.1.0\build.rs:99:19
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
error: build failed

What libraries must I install to be able to run the tests? Ofcourse I have a ScyllaDB running, but I am wondering what steps are needed to make it work. I don't see anything in the README.

Hm, looks like our snappy-sys dependency is causing issues in non-linux environment... Perhaps we should just drop it for now, since it's only needed for specific compression configuration, which is not mandatory. @piodul what do you think?
If we decide to drop the package, the problem should disappear on its own. snappy-sys seems to be unmaintained for a while, so it might be a good idea to find a replacement for it eventually.

Cargo supports features to allow optional dependencies, we could use them to make snappy optional.

@Jasperav @psarna There is a crate with a pure rust implementation of the snappy algorithm called snap. According to the benchmarks done by its creator, it offers comparable performance to the reference C++ library.
I'll submit a PR in a second which changes the crate that we use for snappy compression to "snap". I tested it on my Windows installation, and now the driver seems to compile and pass all unit tests (it didn't before due to the error reported in issue's description).

Wow this was quick! I can verify it works, thanks!


token auto refresh in Client + SSR

Hi everyone! This is some issues + improvements for Authentication with supabase.
Issues:
This works but it isn’t ideal for the following reasons:

Supabase stores the refresh token in localstorage which makes it vulnerable to XSS. This is done so the supabase-js client library can refresh the token on the users behalf.
There’s no refresh mechanism on a SSR request.
It doesn’t follow the best practices for JWTs on frontend clients.

Potential improvements:
Any improvement will likely make Supbase Auth more difficult to use, so what I’m proposing would be opt-in for users that want to improve their auth security.

Allow configuration of the refresh endpoint so we can proxy the request to Supabase ourselves. This would allow us to set a refresh token in a cookie and remove it from localstorage. There would need to be some extra security work in order to make sure a client wasn’t sending the request directly to a Supabase auth endpoint.
Expose a function to generate the cookie using just the token. This would save us having to generate Express-style request/response objects just to get the cookie.

Note:

I garbed these information from this blog, You can read it:
https://www.ashleyconnor.co.uk/2021/06/24/authenticated-server-rendered-pages-with-sveltekit-and-supabase#issues

I was going to make a similar thread. It is surprising to me that the solution is storing the JWT in local storage. I would really like to see an improvement to reduce security risks. For me, this is a deal breaker.

Hey @zakaria-chahboun thank you for the feedback. Would you mind providing more information regarding your security concerns on these topics.

Supabase stores the refresh token in localstorage which makes it vulnerable to XSS. This is done so the supabase-js client library can refresh the token on the users behalf.

Local storage is equivalent (and more secure) than storing tokens in cookies. HTTP-Only cookies are not an option since Supabase Auth targets a wide variety of apps: web, mobile, mobile-web (React Native), desktop, etc.

There’s no refresh mechanism on a SSR request.

We've since published a guide on server-side rendering to help you understand the flow and how to keep it secure.

It doesn’t follow the best practices for JWTs on frontend clients.

Would you mind pointing to practices we're not implementing?

Allow configuration of the refresh endpoint so we can proxy the request to Supabase ourselves. This would allow us to set a refresh token in a cookie and remove it from localstorage. There would need to be some extra security work in order to make sure a client wasn’t sending the request directly to a Supabase auth endpoint.

You can do this by controlling the Redirect URL allow list and implementing your own SPA that would set cookies if you need to. Be advised that this is going to cause a lot of edge case problems in modern-style apps with respect to refresh tokens.

Expose a function to generate the cookie using just the token. This would save us having to generate Express-style request/response objects just to get the cookie.

We are going to be working on PKCE authorization flows soon which should give you more control over the flow of JWT and refresh tokens.
(I'll close the issue for now since it's been inactive for a while, but do feel free to continue the discussion or re-open the issue.)


Support compound primary keys, as introduced in Rails 7.1

Rails 7.1 introduced support for compound or composite keys :
class CreateProducts < ActiveRecord::Migration[7.1]
  def change
    create_table :products, primary_key: [:store_id, :sku] do |t|
      t.integer :store_id
      t.string :sku
      t.text :description
    end
  end
end

For such tables, model.id returns an array of the ids which make up the primary key.
This pull request adds support for this.  Previously, it would cause the hashids gem to error because it would be called with @hashids.encode( token, [ 1, 2 ] ).

ArgumentError: Unknown migration version "7.1"; expected one of "4.2", "5.0", "5.1", "5.2", "6.0"

Do you have any suggestions on how best to hide the new tests and supporting code from Rails versions earlier than?  There shouldn't be any changes to prefixed_ids itself, just the inside dummy Rails app used by the tests.
Maybe https://github.com/flazz/semver is the best way, and wrapping the migrations/tests by a conditional based on the Rails version?

Yeah I would just check the Rails version around the tests. That's how I do it in other gems after I first saw Devise doing that.
https://github.com/search?q=repo%3Aheartcombo%2Fdevise+rails71_and_up%3F&type=code

Untested against Rails <7.1, will see whether the workflows work now.

This will only work if all the composite keys are Integers, right?

This will only work if all the composite keys are Integers, right?

Yeah, because I think that's what hashids requires :
class Hashids
  ...
  def encode(*numbers)
    numbers.flatten! if numbers.length == 1

    numbers.map! { |n| Integer(n) } # raises if conversion fails
    ...
  end
  ...
end

I don't think there is anything in this gem which precludes it from working with strings or mixed types.

I also noticed hashids has been renamed to sqids so I'm going to make another PR to migrate to that.

I also noticed hashids has been renamed to sqids so I'm going to make another PR to migrate to that.

Nevermind, sqids is not backwards compatible.

Released in 1.8.0. Thanks for the feature @carldr!


BusyIndicator Package Error

Just downloaded and seem to be getting same error as #3; see image below for reference

Closing this as on subsequent rebuild error failed to be raised


Too many queries cause block importing to crash.

Summary
If the indexer is receiving many requests it's possible to hit the Postgres client limit.
I reproduced this with a docker environment which was sending lots of queries to indexer over and over again.
indexer-postgres | 2021-06-01 16:36:04.546 UTC [823] FATAL:  sorry, too many clients already
indexer       | {"error":"error committing block, CommitBlock(): pq: sorry, too many clients already","level":"error","msg":"ImportDecodedBlock 9","time":"2021-06-01T16:36:04Z"}

Scope/Requirements
Make sure we can recover gracefully from Postgres failures during block import.
Urgency/Relative Priority

Resolved by #554, I can no longer reproduce this


check that downloaded version of oc command is being run by the installer

The installer downloads the correct version of oc, however it may run a different version of oc if it appears in the PATH before the downloaded and installed version, so it should check that the version it's running is the one that is required
e.g.
wget https://github.com...../oc-$VERSION.tgz
MYVERSION=`oc version`
if not matching("$MYVERSION", "$VERSION") then
   error "there is a different version of oc installed in your environment, please remove the old version"
   error "$(which oc)"
"

or maybe even simply
if $(which oc) != "/usr/local/bin/oc"

Yes agree this sounds like a good idea.
Another option would be to prompt the user during the installation and ask where they want to put the oc tool (with a default location of /usr/local/bin) then use that location for each of our following actions.
ping @aidenkeating

@maleck13 Yep I think your suggestion makes sense. Will assign this to myself.

I don't mind if the installer asks the user, however after downloading we should still check to ensure that the correct version is being run


Fix race condition in external compaction commit

When an external compaction commits and the tablet is not present, it deletes the marker.   There is a race condition where if a tablet was recently unloaded, it would delete the final state marker resulting in the work of an external compaction being lost and probably redone later.

I think tablet servers should not delete the external compaction final state markers when they do not know about the external compaction.  They should only delete these markers when they are hosting the tablet related to the compaction and the tablet knows about the compaction.
For final state markers that point reference external compactions that no longer exists or tablets that no longer exist, the coordinator could periodically look for these and clean them up.  This periodic detection would be similar to what the dead compaction detection is currently doing, but would be looking for the inverse.  The periodic algorithm could do the following.

Scan the final states markers in metadata table (~ecomp section) and store them in a set w/ UUID (compaction id) as key.
Scan the metadata tablets section looking for compactions tablets think are running and remove them from the set.
Delete any final state markers in the set since these reference tablets and/or external compactions that no longer exists.

@keith-turner - You moved this to Done, but it's still open. Is this good to close?


chore: replace json-parse-helpfulerror with parse-json

The json-parse-helpfulerror package uses jju which uses a license that is not
OSI approved.
Fixes #65.

Thanks, @hansl!

Published in 2.1.3.


ImportError: cannot import name RaopCallbacks

hello all:
i install this software and try to play a sample.mp3
it reports that:ImportError:
Traceback (most recent call last):
File "run.py", line 16, in
from shairplay import initialize_shairplay, shutdown_shairplay, RaopCallbacks
ImportError: cannot import name RaopCallbacks
plz tell me how to handle it?

I'm getting the same error..

Are you sure that the install script finished successfully?  When I have seen this error in the past, it was due to install script failing before the shairplay library install completed.
The other item to check is that you are using the venv as described in wiki.
run > source BUILD/venv/bin/activate after the install script completes.

Same error, using venv and install finished.


Notebook cell drag area should be larger

Open my-work.github-issues
Try to drag some cells around, notice how the Drag area is only a bit around the Expando. I think the drag area should be everything right of the title as well.
This would make it more consistent and make the feature more discoverable.
I suggest to try to behave like Split View views regardgin the drag area.

There was a bug in the sizing of the drag handle. Fixed now. I want to keep it just that area on the left so it can be consistent between markdown and code cells and not take too much space

@roblourens should we add bug label and milestone so I get this verified or it is good?


Error while executing pipeline ingest in local

Hi,
I  am trying to run the mlflow pipeline. Currently facing an issue with p.run("ingest") in my local machine using the jupyter notebook.
The following error is occurring:
FileNotFoundError: [WinError 2] The system cannot find the file specified.
Appreciate your help to resolve this problem.
Below is the stack trace:
FileNotFoundError                         Traceback (most recent call last)
Input In [6], in <cell line: 1>()
----> 1 p.run("ingest")
File ~\AppData\Roaming\Python\Python39\site-packages\mlflow\pipelines\regression\v1\pipeline.py:337, in RegressionPipeline.run(self, step)
272 def run(self, step: str = None) -> None:
273     """
274     Runs the full pipeline or a particular pipeline step, producing outputs and displaying a
275     summary of results upon completion. Step outputs are cached from previous executions, and
(...)
335         regression_pipeline.run()
336     """
--> 337     return super().run(step=step)
File ~\AppData\Roaming\Python\Python39\site-packages\mlflow\pipelines\pipeline.py:75, in _BasePipeline.run(self, step)
72 # TODO Record performance here.
73 # Always resolve the steps to load latest step modules before execution.
74 self._steps = self._resolve_pipeline_steps()
---> 75 last_executed_step = run_pipeline_step(
76     self._pipeline_root_path,
77     self._steps,
78     # Runs the last step of the pipeline if no step is specified.
79     self._get_step(step) if step else self._steps[-1],
80 )
82 self.inspect(last_executed_step.name)
84 # Verify that the step execution succeeded and throw if it didn't.
File ~\AppData\Roaming\Python\Python39\site-packages\mlflow\pipelines\utils\execution.py:71, in run_pipeline_step(pipeline_root_path, pipeline_steps, target_step)
69     make_env.update(step.environment)
70 # Use Make to run the target step and all of its dependencies
---> 71 _run_make(
72     execution_directory_path=execution_dir_path,
73     rule_name=target_step.name,
74     extra_env=make_env,
75 )
77 # Identify the last step that was executed, excluding steps that are downstream of the
78 # specified target step
79 last_executed_step = pipeline_steps[0]
File ~\AppData\Roaming\Python\Python39\site-packages\mlflow\pipelines\utils\execution.py:278, in _run_make(execution_directory_path, rule_name, extra_env)
267 def _run_make(execution_directory_path, rule_name: str, extra_env: Dict[str, str]) -> None:
268     """
269     Runs the specified pipeline rule with Make. This method assumes that a Makefile named Makefile
270     exists in the specified execution directory.
(...)
276     :param rule_name: The name of the Make rule to run.
277     """
--> 278     _exec_cmd(
279         ["make", "-s", "-f", "Makefile", rule_name],
280         capture_output=False,
281         stream_output=True,
282         synchronous=True,
283         throw_on_error=False,
284         cwd=execution_directory_path,
285         extra_env=extra_env,
286     )
File ~\AppData\Roaming\Python\Python39\site-packages\mlflow\utils\process.py:94, in _exec_cmd(cmd, throw_on_error, extra_env, capture_output, synchronous, stream_output, **kwargs)
88     elif stream_output:
89         # Redirect stderr to stdout in order to combine the streams for unified printing to
90         # sys.stdout, as documented in
91         # https://docs.python.org/3/library/subprocess.html#subprocess.run
92         kwargs["stderr"] = subprocess.STDOUT
---> 94 process = subprocess.Popen(
95     cmd,
96     env=env,
97     text=True,
98     **kwargs,
99 )
100 if not synchronous:
101     return process
File ~\anaconda3\envs\py39\lib\subprocess.py:951, in Popen.init(self, args, bufsize, executable, stdin, stdout, stderr, preexec_fn, close_fds, shell, cwd, env, universal_newlines, startupinfo, creationflags, restore_signals, start_new_session, pass_fds, user, group, extra_groups, encoding, errors, text, umask)
947         if self.text_mode:
948             self.stderr = io.TextIOWrapper(self.stderr,
949                     encoding=encoding, errors=errors)
--> 951     self._execute_child(args, executable, preexec_fn, close_fds,
952                         pass_fds, cwd, env,
953                         startupinfo, creationflags, shell,
954                         p2cread, p2cwrite,
955                         c2pread, c2pwrite,
956                         errread, errwrite,
957                         restore_signals,
958                         gid, gids, uid, umask,
959                         start_new_session)
960 except:
961     # Cleanup if the child failed starting.
962     for f in filter(None, (self.stdin, self.stdout, self.stderr)):
File ~\anaconda3\envs\py39\lib\subprocess.py:1420, in Popen._execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, unused_restore_signals, unused_gid, unused_gids, unused_uid, unused_umask, unused_start_new_session)
1418 # Start the process
1419 try:
-> 1420     hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
1421                              # no special security
1422                              None, None,
1423                              int(not close_fds),
1424                              creationflags,
1425                              env,
1426                              cwd,
1427                              startupinfo)
1428 finally:
1429     # Child is launched. Close the parent's copy of those pipe
1430     # handles that only the child should have open.  You need
(...)
1433     # pipe will not close when the child process exits and the
1434     # ReadFile will hang.
1435     self._close_pipe_fds(p2cread, p2cwrite,
1436                          c2pread, c2pwrite,
1437                          errread, errwrite)
FileNotFoundError: [WinError 2] The system cannot find the file specified

Hi @parvez-golam, Thank you for reporting the issue. I think there was a similar issue that was reported earlier. It seems like installing make solves this issue. Can you give that a try? Let us know if you find more issues. Thank you.

Thank you @sunishsheth2009 ! It's working after installation of make.


Implementation of int.result block that is like a temporary index...

...with new types of methods that allow for query/index-style probing rather than just an iterator.
Once this is in place, the bind join algorithms should be re-implemented to leverage this new class (in particular, if the input block is of this new type, use it to probe with each solmap from the server response; if the input block is not of this type, use this type when populating the data structure for the request to the server).

Now that we have the implementations of SolutionMappingsIndex in place, this TODO here can be attempted.


Variant value is missing in some configs on version 1.11.1

Hello, after we updated the sdk to the latest version 1.11.1, we started to receive null for some config variants, looks like the expected value is assigned to the key field instead.
For example:
Variant(value=null, payload=null, expKey=null, key=off, metadata={default=true})

Note: The issue is not reproducible on 1.10.1, we'll rollback for now.

Hi @kevinguitar, many thanks for raising this issue. It seems that local evaluation is returning a default variant in this case. To better assist on this issue, it would be helpful if you could let us know:

What variant was expected instead
What configuration (e.g. user_id, device_id) was when evaluation was called
The link to your flag if possible

Please feel free to respond either on this thread, or via email to<EMAIL_ADDRESS>

@tyiuhc  I've emailed all the details to you, thanks for helping!

For those who are experiencing the same situation:
In pre-1.11 versions, the "fetch()" network call will get a response with only variants for flags where the user is included; in v1.11, if the user is not included in the flag, a default variant is returned.
In all versions, when "variant()" is called, if the user is not included in the flag, a default variant is returned. In pre-1.11, this is "Variant(value=null, payload=null, expKey=null)", in 1.11, this is "Variant(value=null, payload=null, expKey=null, key=off, metadata={default=true})".


notifications should indicate if they refer to an unlinkable entity

For example, creating a comment will generate a notification, but if the user then deletes the comment it becomes impossible to link to directly. Requires a server update, will create an issue for a 'deleted' flag on notifications.

This doesn't appear to be working. Amber deleted a comment on EID1445 but there is no indication on my notifications page.

I believe the entities that affect the 'linkability' of a notification are the primary entities - genes, variants, evidence, etc. and not comments. So deleting the comment won't toggle the notification to unlinkable, but deleting the actual entity to which the comment is attached will.
@acoffman is this the case?

Yeah, looks like there's a hole in the logic there. I'll get it fixed!

Added a server issue, closing.

Awesome. Thanks guys!


[Design Component] Fix Tab component

https://adevinta.slack.com/archives/C059UCP6B63/p1695795084926589

Published to Design OPS team
Next step: @lauragautheron will update the KeyScreen file.


[question] Dynamic loading of koto modules

A previous question was about dynamic loading of modules written in Rust. However, I need something similar for Koto modules. I want my application to download Koto modules based on a dependency specification and make them available to the Koto script executed by the application, preferably without having to import them in the script itself (something like a dynamic prelude). Is that possible somehow?

Hi @jasal82, yes that's possible, but with the restriction that your host application and the module libraries must be compiled using the same Rust toolchain so that you avoid potential issues with ABI incompatibilities.
Given that your application is going to be responsible for downloading the modules, then it can also be responsible for building them, and as long as the right toolchain is used then all should be well.
The module libraries can then be dynamically loaded (using libloading or similar), and assuming they expose a make_module function (or something), then the resulting modules can be added to the Koto prelude at runtime.
Let me know if you have any questions, I'll be interested to hear how you get on!

Hi @irh, I think you misunderstood the question. It was not about precompiled module libraries. I want to be able to define functions and modules in Koto scripts and add them to the prelude of my interpreter so that scripts executed on that interpreter can use these functions without having to import anything.

I see, I took 'Koto modules' to mean modules for Koto that have been implemented in Rust, which is where the compilation challenges came from.
For modules that are implemented in Koto, the host application would need to:

Compile and run the module script (e.g. using Koto::compile_and_run()).
Place the resulting dependency module's exports map in the prelude (e.g. Koto::prelude().add_map()).

I think this should work? Let me know if you run into any problems.

Sounds straightforward, I'll give it a try.

This works fine:
let mut koto = Koto::new();

let mut koto_core_loader = Koto::new();
let core = fs::read_to_string("core.koto")?;
koto_core_loader.compile_and_run(&core).map_err(koto_error_to_anyhow).context("Error while compiling core")?;

let prelude = koto.prelude();
prelude.add_map("core", koto_core_loader.exports().clone());

Now I can extract it into a function and load the Koto scripts specified in my dependency list. Just need to figure out how to handle interdependencies...

OK great, let me know if you run into any problems, and I'll be keen to see what you come up with.


Bug:

Terms

[X] I have read the guidelines for Contributing to Roots Projects
[X] This request is not a duplicate of an existing issue
[X] I have read the docs and followed them (if applicable)
[X] I have seached the Roots Discourse for answers and followed them (if applicable)
[X] This is not a personal support request that should be posted on the Roots Discourse community

Description
I'm yarn installing the following javascript carousel package from: https://pawelgrzybek.github.io/siema/#installation and its failing if include is placed in any other view above app.blade.php eg. content-page.blade.php
I have also tried this with a different javascript package called embla carousel: https://www.embla-carousel.com/get-started/module/ - and similar thing happens when @include is placed anywhere above app.blade.
I have tried with fresh install of bud sage this morning and also bedrock with acorn installed.
Steps To Reproduce
Install javascript carousel package from: https://pawelgrzybek.github.io/siema/#installation and my app.js is as follows.
import {domReady} from '@roots/sage/client';
import Siema from 'siema';

/**
 * app.main
 */
const main = async (err) => {
  if (err) {
    // handle hmr errors
    console.error(err);
  }
};

// fire up Siema Carousel
const mySiema = new Siema({
  selector: '.siema',
});

All installs correctly, and if I add the blade include ...
@include('partials.sliders.siema')

Hi, I'm slide 1

Hi, I'm slide 2

Hi, I'm slide 3

to app.blade.php:
@include('sections.header')

    @yield('content')

  @include('partials.sliders.siema')

  @hasSection('sidebar')

      @yield('sidebar')

  @endif

@include('sections.footer')

the carousel works as expected with no issues or errors.
But if I try to move the blade include into any other view eg.
partials > content-page.blade:
@php(the_content())

@include('partials.sliders.siema')

{!! wp_link_pages(['echo' => 0, 'before' => '
' . __('Pages:', 'sage'), 'after' => '
']) !!}

then it stops working and throws an error in console:
Uncaught Error: Something wrong with your selector 😭
Expected Behavior
Carousel should load as its does if loaded into app.blade.php view. Any other view above app.blade doesn't work and throws console error.
Actual Behavior
Carousel doesn't work and throws console error if include is place anywhere above app.blade.php:
Uncaught Error: Something wrong with your selector 😭
Relevant Log Output
app.b4cc08.js:1 Uncaught Error: Something wrong with your selector 😭
    at new e (app.b4cc08.js:1:2010)
    at Object.683 (app.b4cc08.js:1:535)
    at t ((index):615:134)
    at t (app.b4cc08.js:1:13546)
    at app.b4cc08.js:1:13556
    at n ((index):615:1038)
    at app.b4cc08.js:1:51
e @ app.b4cc08.js:1
683 @ app.b4cc08.js:1
t @ (index):615
t @ app.b4cc08.js:1
(anonymous) @ app.b4cc08.js:1
n @ (index):615
(anonymous) @ app.b4cc08.js:1

Versions
Sage 10.1.2. Bedrock 1.19.2. MacOS BigSur 11.6.2

hi @fabianwurk 👋
i'm not seeing a bug in sage based on this report, which looks like a support request. per our guidelines, please only use this issue tracker to report bugs in sage
come visit us at https://discourse.roots.io/ for support or questions


Fail on Download Metadata for the game Time Clickers

Check for existing issue
Please use issue search in the repository if the bug is something really obvious there's a high change it was already reported.
Describe the bug
A clear and concise description of what the bug is.
If you try to download Metadata on a game trough the official store it doesn't work
This doesn't work on the free steam game Time Clickers
To Reproduce
If it's not clear from the description how to reproduce the issue please add clear steps to reproduce the behavior.
Try to download metadata from Official Store on the game time clickers
Screenshots
If applicable, add screenshots to help explain your problem.
https://media.discordapp.net/attachments/864521430070722560/883407123969622016/unknown.png?width=883&height=498
Diagnostics ID
Please generate diagnostics package from "About Playnite" menu and attach identifier you received. If the issue is a crash bug, you can generate diag. package directly from crash dialog.
278cde2e-52d0-46f0-ba61-6b24f791f10d

Fixed in master.


way to modify each page render's html content

The current implementation of the hook is limited in terms of being able to update the html's head or body tags with required
meta or anything.
This is required for OG Images, adding dynamic styles and if I wish to add in CSS post processing etc.
Over the top of my head, the 2 options as solutions are

New onRender hook that gets the final HTML and you can modify it as needed.
Giving the existing Writer hook a way to return back additions to the head and body tags.

By default handled with hooks and file level hooks, the modifications are stacked in the order of the hooks executed. (Add in Docs)


Getting a weak-vtables warning with clang on ubuntu 22.04

Description
I noticed that when ever I try to compile with clang ( 11 - 15 as I have installed) on with my normal warning flags on linux then I always ends up with a warning from the exceptions header about weak-vtables, I do know that some would say I have alot of warning flags activated but that is because that is what I saw some time ago that fmtlib uses and chose to use it myself also to catch errors and posible bad practices from my side
I do use CPM-cmake to pull in libraries purely because that is able to store them in a local cache so I don't have to pull the libraries over and over for each project and it does use the github release
Reproduction steps
Tested with clang 11, 12, 13, 14, 15 on ubuntu jammy (22.04)
The shell command is taken from cmake log
/usr/bin/clang++-15  -I/home/buildclient/.cache/CPM/nlohmann_json/b3708972f6694fe462e4112e47aa04f10d2390b4/nlohmann_json/include -g -Werror -Wall -Wextra -pedantic -Wconversion -Wundef -Wdeprecated -Wweak-vtables -Wshadow -Wno-gnu-zero-variadic-macro-arguments -Wzero-as-null-pointer-constant -std=gnu++17 -MD -MT CMakeFiles/tanja84dk_testproject.dir/src/main.cpp.o -MF CMakeFiles/tanja84dk_testproject.dir/src/main.cpp.o.d -o CMakeFiles/tanja84dk_testproject.dir/src/main.cpp.o -c /buildtest/Cpp-Tanja84dk-DockerClient/src/main.cpp

I'm using CPM-cmake to pull in libraries and in essence its just pulling the 3.11.2 release from the release tab by tag
cmake_minimum_required(VERSION 3.16)
project(tanja84dk_testproject LANGUAGES CXX VERSION 0.1.0
)

list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/cmake")

include(cmake/CPM.cmake)

include(CheckCXXCompilerFlag)

if (CMAKE_CXX_COMPILER_ID MATCHES "GNU")
  set(PEDANTIC_COMPILE_FLAGS -pedantic-errors -Wall -Wextra -pedantic
      -Wold-style-cast -Wundef
      -Wredundant-decls -Wwrite-strings -Wpointer-arith
      -Wcast-qual -Wformat=2 -Wmissing-include-dirs
      -Wcast-align
      -Wctor-dtor-privacy -Wdisabled-optimization
      -Winvalid-pch -Woverloaded-virtual
      -Wconversion -Wundef
      -Wno-ctor-dtor-privacy -Wno-format-nonliteral)
  if (NOT CMAKE_CXX_COMPILER_VERSION VERSION_LESS 4.6)
      set(PEDANTIC_COMPILE_FLAGS ${PEDANTIC_COMPILE_FLAGS}
         -Wno-dangling-else -Wno-unused-local-typedefs)
  endif ()
  if (NOT CMAKE_CXX_COMPILER_VERSION VERSION_LESS 5.0)
      set(PEDANTIC_COMPILE_FLAGS ${PEDANTIC_COMPILE_FLAGS} -Wdouble-promotion
          -Wtrampolines -Wzero-as-null-pointer-constant -Wuseless-cast
          -Wvector-operation-performance -Wsized-deallocation -Wshadow)
  endif ()
  if (NOT CMAKE_CXX_COMPILER_VERSION VERSION_LESS 6.0)
      set(PEDANTIC_COMPILE_FLAGS ${PEDANTIC_COMPILE_FLAGS} -Wshift-overflow=2
          -Wnull-dereference -Wduplicated-cond)
  endif ()
  set(WERROR_FLAG -Werror)
endif ()

if (CMAKE_CXX_COMPILER_ID MATCHES "Clang")
  set(PEDANTIC_COMPILE_FLAGS -Wall -Wextra -pedantic -Wconversion -Wundef
    -Wdeprecated -Wweak-vtables -Wshadow
    -Wno-gnu-zero-variadic-macro-arguments)
  check_cxx_compiler_flag(-Wzero-as-null-pointer-constant HAS_NULLPTR_WARNING)
  if (HAS_NULLPTR_WARNING)
    set(PEDANTIC_COMPILE_FLAGS ${PEDANTIC_COMPILE_FLAGS} -Wzero-as-null-pointer-constant)
  endif ()
  set(WERROR_FLAG -Werror)
endif ()

if (MSVC)
  set(PEDANTIC_COMPILE_FLAGS /W4)
  set(WERROR_FLAG /WX)
endif ()

set(tanja84dk_testproject_SOURCES
    src/main.cpp
)

set(tanja84dk_testproject_LIBRARIES
    nlohmann_json
)

CPMAddPackage(
    NAME            nlohmann_json
    GIT_REPOSITORY  https://github.com/nlohmann/json
    VERSION         3.11.2
    GIT_TAG         v3.11.2
)

add_executable(tanja84dk_testproject ${tanja84dk_testproject_SOURCES})

set_target_properties(tanja84dk_testproject
    PROPERTIES
        ARCHIVE_OUTPUT_DIRECTORY "${CMAKE_BINARY_DIR}/lib"
        LIBRARY_OUTPUT_DIRECTORY "${CMAKE_BINARY_DIR}/lib"
        RUNTIME_OUTPUT_DIRECTORY "${CMAKE_BINARY_DIR}/bin"
        CXX_STANDARD 17
    FOLDER
        Application
)

target_link_libraries(tanja84dk_testproject PRIVATE ${tanja84dk_testproject_LIBRARIES})

target_compile_options(tanja84dk_testproject
    PRIVATE
        ${WERROR_FLAG}
        ${PEDANTIC_COMPILE_FLAGS})

target_compile_features(tanja84dk_testproject
    PRIVATE
        cxx_std_17
)

### Expected vs. actual results

Expected it to build successfully specially because I didn't do anything specially in the small test but it warned/errored with the weak-vtables in the include/nlohmann/detail/exceptions.hpp

### Minimal code example

```Shell
#include <iostream>
#include <nlohmann/json.hpp>

int main() { return 0; }

Error messages
[main] Building folder: Cpp-Tanja84dk-DockerClient
[build] Starting build
[proc] Executing command: /usr/bin/cmake --build /buildtest/Cpp-Tanja84dk-DockerClient/build --config Debug --target all --
[build] [1/2  50% :: 2.195] Building CXX object CMakeFiles/tanja84dk_testproject.dir/src/main.cpp.o
[build] FAILED: CMakeFiles/tanja84dk_testproject.dir/src/main.cpp.o
[build] /usr/bin/clang++-15  -I/home/buildclient/.cache/CPM/nlohmann_json/b3708972f6694fe462e4112e47aa04f10d2390b4/nlohmann_json/include -g -Werror -Wall -Wextra -pedantic -Wconversion -Wundef -Wdeprecated -Wweak-vtables -Wshadow -Wno-gnu-zero-variadic-macro-arguments -Wzero-as-null-pointer-constant -std=gnu++17 -MD -MT CMakeFiles/tanja84dk_testproject.dir/src/main.cpp.o -MF CMakeFiles/tanja84dk_testproject.dir/src/main.cpp.o.d -o CMakeFiles/tanja84dk_testproject.dir/src/main.cpp.o -c /buildtest/Cpp-Tanja84dk-DockerClient/src/main.cpp
[build] In file included from /buildtest/Cpp-Tanja84dk-DockerClient/src/main.cpp:2:
[build] In file included from /home/buildclient/.cache/CPM/nlohmann_json/b3708972f6694fe462e4112e47aa04f10d2390b4/nlohmann_json/include/nlohmann/json.hpp:35:
[build] In file included from /home/buildclient/.cache/CPM/nlohmann_json/b3708972f6694fe462e4112e47aa04f10d2390b4/nlohmann_json/include/nlohmann/adl_serializer.hpp:14:
[build] In file included from /home/buildclient/.cache/CPM/nlohmann_json/b3708972f6694fe462e4112e47aa04f10d2390b4/nlohmann_json/include/nlohmann/detail/conversions/from_json.hpp:23:
[build] /home/buildclient/.cache/CPM/nlohmann_json/b3708972f6694fe462e4112e47aa04f10d2390b4/nlohmann_json/include/nlohmann/detail/exceptions.hpp:36:7: error: 'exception' has no out-of-line virtual method definitions; its vtable will be emitted in every translation unit [-Werror,-Wweak-vtables]
[build] class exception : public std::exception
[build]       ^
[build] /home/buildclient/.cache/CPM/nlohmann_json/b3708972f6694fe462e4112e47aa04f10d2390b4/nlohmann_json/include/nlohmann/detail/exceptions.hpp:134:7: error: 'parse_error' has no out-of-line virtual method definitions; its vtable will be emitted in every translation unit [-Werror,-Wweak-vtables]
[build] class parse_error : public exception
[build]       ^
[build] /home/buildclient/.cache/CPM/nlohmann_json/b3708972f6694fe462e4112e47aa04f10d2390b4/nlohmann_json/include/nlohmann/detail/exceptions.hpp:187:7: error: 'invalid_iterator' has no out-of-line virtual method definitions; its vtable will be emitted in every translation unit [-Werror,-Wweak-vtables]
[build] class invalid_iterator : public exception
[build]       ^
[build] /home/buildclient/.cache/CPM/nlohmann_json/b3708972f6694fe462e4112e47aa04f10d2390b4/nlohmann_json/include/nlohmann/detail/exceptions.hpp:205:7: error: 'type_error' has no out-of-line virtual method definitions; its vtable will be emitted in every translation unit [-Werror,-Wweak-vtables]
[build] class type_error : public exception
[build]       ^
[build] /home/buildclient/.cache/CPM/nlohmann_json/b3708972f6694fe462e4112e47aa04f10d2390b4/nlohmann_json/include/nlohmann/detail/exceptions.hpp:222:7: error: 'out_of_range' has no out-of-line virtual method definitions; its vtable will be emitted in every translation unit [-Werror,-Wweak-vtables]
[build] class out_of_range : public exception
[build]       ^
[build] /home/buildclient/.cache/CPM/nlohmann_json/b3708972f6694fe462e4112e47aa04f10d2390b4/nlohmann_json/include/nlohmann/detail/exceptions.hpp:239:7: error: 'other_error' has no out-of-line virtual method definitions; its vtable will be emitted in every translation unit [-Werror,-Wweak-vtables]
[build] class other_error : public exception
[build]       ^
[build] 6 errors generated.
[build] ninja: build stopped: subcommand failed.
[proc] The command: /usr/bin/cmake --build /buildtest/Cpp-Tanja84dk-DockerClient/build --config Debug --target all -- exited with code: 1
[driver] Build completed: 00:00:02.244
[build] Build finished with exit code 1

Compiler and operating system
Clang 11, 12, 13, 14, 15 on ubuntu 22.04
Library version
3.11.2
Validation

[X] The bug also occurs if the latest version from the develop branch is used.
[ ] I can successfully compile and run the unit tests.

I am facing the same issue (my project has to have -Wweak-vtables enabled), and it seems like a low effort to fix it? std::exception has a virtual destructor, which is the reason for this warning. All that is needed is to declare an explicit destructor in each of the exception classes and add out-of-line default definitions like "exception::~exception() = default;" alongside.

Indeed we compile with -Wno-weak-vtables in the CI. I will have a look.

Ok, I'm getting to the boundaries of my C++ knowledge.
I can fix the -Wweak-vtables warning by adding
~exception() override;

to the class definition and add
exception::~exception() = default;

outside the class. This makes Clang happy, but Clang-Tidy complains:
error: function '~exception' defined in a header file; function definitions in header files can lead to ODR violations [misc-definitions-in-headers,-warnings-as-errors]

I am note sure how to fix this, because the library is header-only.
Furthermore, GCC complains (-Wdeprecated-copy-dtor):
implicitly-declared 'parse_error::parse_error(const parse_error&)' is deprecated [-Werror=deprecated-copy-dtor]
note: because 'parse_error' has user-provided 'parse_error::~parse_error()'

I guess this can be fixed by just adding a copy constructor and a copy assignment operator.

I think maybe you should add inline before the definition.  I've never tried that with a out of line defaulted function before.

Oh, yeah, if there wasn't a destructor there before and you add one, it suppresses the other of the special 5.

Okay, I've done some reading and some thinking, and I don't think it's possible to fix this in a header-only library.  I mean that we MIGHT be able to silence the warning, but it will still result in the exact same behavior everywhere.  There is no implementation file where we can define any virtual function.  All the virtual functions will still be defined in every translation unit that includes the header.
The best we can do is to forcefully silence the warning, and I'm not sure that we should actually do that.  It is not possible to be 100% warning free, and this particular warning is only enabled explicitly or by using -Weverything, which even the clang devs say shouldn't be used.  https://quuxplusone.github.io/blog/2018/12/06/dont-use-weverything/

I added a diagnostic push to suppress the warning in #4500. PTAL.


Rendering fails on first statement.

Rendering fails on first request (test 2), only in the first statement.

"vue": "3.5.7" (composition api)
"pinia": "2.2.2"

test 1

<template>

</template>

test 2

<template>

</template>

test 3

<template>

</template>

Looks like you are using @iconify/iconify package. It works by swapping elements to render SVG. That doesn't work well with various frameworks, so it should not be used when you are using Vue.
Furthermore, @iconify/iconify package has been deprecated.
For Vue there is @iconify/vue or, even better solution that works perfectly with Nuxt, iconify-icon package.
Closing this as not fixable because it is a known issue that cannot be fixed and package is deprecated.

I'm using @iconify/tailwind and the repository indicated on NPM is github.com/iconify/iconify.

Oh I see. Saw span syntax, assumed it was old package.
If it fails to render, it means CSS for icon is missing, which means Tailwind didn't register that as class name when parsing your Vue project. Add class name for icon to whitelist in tailwind config.

Thanks!
safelist: [{ pattern: /tabler--.+/ }]


Fix computation of mean

Used to compute the mean time of a batch size, then devide by the
sum of the batch sizes: therefore the more iteration you did, the
fastest the micro-bench appeared to be (the reported number was
roughly #iterations times to small).

CI fixed. I can merge when reviewed.


Downloading artifacts from the "Workflow" page fails on minikube

Artifact download from the "Workflow" page (where each step of the workflow shows up as a circle) fails on minikube. Looks like the browser is given a url by axops which is something like  http://axartifactmanager.axsys:9892/.... which doesn't resolve.
On AWS, the browser gets a url for the S3 bucket, something like:
https://ax-shri-test-us-west-2.s3.amazonaws.com/shri.test.eng.applatix.net/...
However, the "Artifacts" tab works just fine on minikube as well as AWS.
It seems that axops has two different endpoints for getting artifacts. Only one of them might have been updated recently.

@alexmt this may be a UI issue


Add condition to delete drawing

What this PR does / why we need it?
the error occurs when another user deletes the shape while the user is working on it.
Add conditions to delete shapes.
The conditions to be added are as follows.

The shape's isEditing value must be true.
The user can exit while editing, so check if the user is currently connected.

Any background context you want to provide?
What are the relevant tickets?

Fixes #77
Checklist

[x] Added relevant tests or not required
[x] Didn't break anything

I would talk about this and register again


[Azure.Monitor.Query] Update Swaggers and regenerate code with renames

Contributing to the Azure SDK
Please see our CONTRIBUTING.md if you are not familiar with contributing to this repository or have questions.
For specific information about pull request etiquette and best practices, see this section.

API change check
APIView has identified API level changes in this PR and created following API reviews.
Azure.Monitor.Query


build: expose all icons

Motivation
Expose all icons added in #51.
Changes

export icons
refactor components and icons to dedicated lib entries

Thanks for the review! I published a new version to npm 😃


[Web] Flutter beta branch switch from stable with find some program

some widgets can‘t find property

Hi @zhuyonghao
Please restart your IDE and try again
If the problem persists, Can you please provide your flutter doctor -v your flutter run --verbose and a complete reproducible minimal code sample
Thank you

Thank you， Switch to Dev, then switch back to Beta, and it gets better


Node CLI Won't Accept Multiple Key Types

This issue is being forked from substrate-developer-hub/tutorials#21. @DemiMarie-parity suggested that this tutorial should instruct users to pass keys to the Substrate node by way of CLI parameters. However, the tutorial in question requires that users add two keys to the node, but as far as I can tell the command line parameters can only be used to add a single key (because of the relationship of the --node-key-file and --node-key-type parameters). I ran this interpretation by Demi and she concurred and suggested I open this issue.

The flags you mentioned are for your node's libp2p key. It is the key responsible for generating the node id. It is different than the session keys that the tutorial inserts via the author_insertKey rpc. RPC is the correct way to insert session keys.

Thanks for that context, Joshy :+1: In light of what @DemiMarie-parity said, I wonder if the node should support CLI parameters for the session keys. Reopening this issue after a brief offline discussion with Joshy.

Almost all slashing in other networks results from operators miss-handling keys.  We designed this architecture with the controller key certifying the session keys partially so that operators only certified the public session keys, while nodes created and managed the private session keys entirely automatically.  There are various reasons why people want to do other things sometimes, but we should've a simplified work flow in which operators never really touch those private session keys directly.

To elaborate on what @burdges stated:

Session keys are ephemeral by design.  They should not (and do not need to) be backed up, for example.  Losing a session key can only happen when a node is offline, and when it does happen, the correct response is to generate a new session key and sign it with the controller key.
Controller keys should be stored semi-offline in a hardware device, where they are protected from being leaked.  Using them should generally require manual intervention, but this should be possible on short notice.
Using cURL to make an RPC call is an anti-pattern and very fragile.  A much better approach would be to have a command-line tool that handles the communication between the hardware wallet (which stores the secret controller keys) and the node itself.  Such a tool is already under development by me and @kirushik.

I also think that changing the CLI in this situation is the wrong thing to do. But we should at least all understand the context being discussed. Dan's question arose in the context of the "Start a private network" tutorial. Specifically the Add keys to keystore setion.
So while everything @burdges and @DemiMarie-parity said is correct in general, it is mostly not addressing @danforbes concern. It specifically misses two important points.

This is not about rotating session keys it's about inserting keys to author the first few blocks of a new chain.
Not all nodes use the staking pallet and its controller configuration. The node template, which the tutorial in question is about, does not use either. It is simple Proof of Authority.

@JoshOrndorff I still don't understand what the exact question is? :D

The question is: what capabilities currently exist within Substrate to obviate the security concern that @DemiMarie-parity brought up; how can these capabilities be composed to address Demi's concern?

The point is, that besides genesis you never need insertKey. You are always advised to use rotateKeys, as this is much easier.
You can also insert the data directly into the keystore, if you like.
You can also make curl read the data from a file before sending it or use subkey to send the key.
But we are talking about a tutorial, that should be easy, aka showing a curl command. It should not be over complicated because of some very unlikely attack.
Given that this is an issue that will not be solved by Substrate, I close it.


Finalization rule 3 is unreachable

It's impossible to have 0b111 without triggering Rule 2. Observe the bit position of previous_epoch - 1,

Get left 1 justified
In the next epoch the position become previous_justified.
In the epoch to trigger rule 3, it trigger rule 2 first.

  j
0b10
  p
  j
0b111

Rule 2: Set state.finalized_epoch = state.previous_justified_epoch if (state.justification_bitfield >> 1) % 4 == 0b11 and state.previous_justified_epoch == previous_epoch - 1.
Rule 3: Set state.finalized_epoch = state.justified_epoch if (state.justification_bitfield >> 0) % 8 == 0b111 and state.justified_epoch == previous_epoch - 1.
Related:

https://github.com/ethereum/trinity/pull/264/files#r255382122
https://github.com/ethereum/eth2.0-specs/issues/410

Rule 1 might also have some issue, it can only finalize the epoch that was already finalized by Rule 2.

Suppose that at the end of epoch N+2, epoch N+1 is justified but epoch N+2 is not. Then, at the end of epoch N+3, epochs N+1, N+2 and N+3 are all justified, because of messages that were not yet included in the chain before.

That case will trigger rule 2 before 3. because N+1 is now previous justified epoch and the bitfield in rule 3 is a special case of the bitfield in rule 2

Ok, try this:

In epoch N+1, JE is N, prev JE is N-1, and not enough messages get in to do anything
In epoch N+2, JE is N, prev JE is N, and enough messages from the previous epoch get in to justify N+1. N+1 now becomes the JE. Not enough messages from epoch N+2 itself get in to justify N+2.
In epoch N+3, LJE is N+1, prev LJE is N, and enough messages get in to justify epochs N+2 and N+3.

Rule (2) does not get triggered because prev LJE is NOT previous_epoch - 1, but rule (3) does get triggered because LJE is previous_epoch - 1, the previous epoch is justified, and the current epoch is justified.

Alright, finally find the problem. Some case could trigger two rules at the same time.
I made a wrong assumption and implement the spec wrong. The wrong implementation returns the finalized epoch when a rule is triggered, while in the correct one every rule must be checked.
Run a simulation in Trinity shows that:

In epoch N+2, Rule 2 would finalize N.
In epoch N+3, Rule 1 would first to finalize N again, then Rule 3 finalize N+1.

A quick look into the different codebases.

Prysm has the same style as my implementation, need a head up for this bug. cc @terenc3t.
The Lighthouse is safe, cc @paulhauner.

Thank you v for the example ❤

Thanks @hwwhww :)
I've drew up some diagrams a while ago and noticed the same thing about double-execution.
Something that I haven't quite got my head across is why rules #1,3 require there to be two justified epochs following the finalized epoch, whilst #2,4 only require one.

The meta-rule behind all four rules is that if there are two epochs B[1] and B[n], where both are justified and B[n] was justified using B[1] as a source, and all intermediate epochs B[2] ... B[n-1] are all also justified (no matter what source), then B[1] can be finalized.
Rules 1 and 3 cover the n=3 case (the difference between the two is that rule 1 assumes that the current epoch is n+1, and rule 3 assumes it's n),and rules 2 and 4 cover the n=2 case.
The reason why any rule except rule 4 (the only rule in the original FFG paper) is necessary is to cover the possibility that it takes an entire epoch for attestations to get included in the chain, and so epoch 1 can't get justified until the chain has already moved on to epoch 2.

The meta-rule behind all four rules is that if there are two epochs B[1] and B[n], where both are justified and B[n] was justified using B[1] as a source, and all intermediate epochs B[2] ... B[n-1] are all also justified (no matter what source), then B[1] can be finalized.
Rules 1 and 3 cover the n=3 case (the difference between the two is that rule 1 assumes that the current epoch is n+1, and rule 3 assumes it's n),and rules 2 and 4 cover the n=2 case.
The reason why any rule except rule 4 (the only rule in the original FFG paper) is necessary is to cover the possibility that it takes an entire epoch for attestations to get included in the chain, and so epoch 1 can't get justified until the chain has already moved on to epoch 2.

So my understanding of the FFG paper was that justification can happen to any past epoch as well. But we only allow attestations to the current and previous epoch. So I guess that is why we have those two cases, whereas I would have expected that we need to check all past epochs for justification/finalisation?
Is there a description of the exact consensus algorithm that is used?

We have these limited cases because of a few reasons:

We only allow attestations from previous and current epoch to be included on chain
We only allow attestations with the expected justified epoch wrt the state at the time of attestation to be included on chain
We eagerly try to process the current epoch attestations even though not all slot attestations have been given an entire epoch to be included

The combination of these three attributes create the 4 possible cases possible to satisfy the general rule described here https://github.com/ethereum/eth2.0-specs/issues/611#issuecomment-463896109
If we allowed 3 epochs of inclusions rather than two, we would need to cover the n=4 case (or change the state a bit to handle things more generally).
There is a WIP academic paper describing this more general purpose finality rule and it's instantiation in a PoS chain similar to our beacon chain. Will share when there's a draft ready

OK, thanks :) This helps me understand what's going on.


Chef client version 12.4

We have an issue at the moment where we can't build due to having an older version of chef client (12.4).
This means we can't upgrade to the latest iptables package (3.x.x) to allow this line to be run:
https://github.com/IshentRas/cookbook-openshift3/blob/827d517a0c0793ab3718b61dcbe8223fabc3aa63/recipes/common.rb#L11
our version of iptables (1.0.0) expects 'content' instead of 'lines' here.
Unfortunately an upgrade is not likely soon because a plan needs to be made for cookbooks that are not back-compatible.
I'm not sure if there is a way to run different code based on the client version in chef. A quick search seemed to suggest this was not 'the chef way'.
Do you have a strong view on this Will? One option might be to pin iptables to 1.0.0 rather than >= 1.0.0 but this may not be what you want.

Maybe this helps...
http://stackoverflow.com/questions/22833743/extract-cookbook-name-and-version-during-chef-client-run
I will investigate.

OK, I have repro'd (with ShutIt of course):
https://github.com/ianmiell/shutit-openshift-cluster/commit/ffff7a1141f909fbc2fe6c89c7536b3b6c7d38a3
and then shown that the error can be 'avoided' with, eg:
log run_context.cookbook_collection['iptables'].version
if run_context.cookbook_collection['iptables'].version == '3.0.0'
  iptables_rule 'firewall_jump_rule' do
    lines '-A INPUT -j OS_FIREWALL_ALLOW'
  end
end

so do you have any advice on how to implement (eg with template or some other method?) for the pre-3.0.0 version, before I write a PR?

@ianmiell I personally do not have any preferences about how to implement it. Just make a condition on the version..?

OKIDOKE!

Resolved.


Change license to MIT-0

MIT seems to require attribution, which isn't intentional by my part.

I'll close this for now as I am unsure if this change is actually needed; might re-open for a future update. Feel free to drop suggestions in this regard.


IGNITE-7660: Refactor LSQR algorithm

Refactor for the issue ignite-7660.

PR is related to resolved JIRA issue. Feel free to reopen if PR is still useful.


feat: display instructlab sessions, with mock instructlab manager

What does this PR do?
display instructlab sessions, with mock instructlab manager
Screenshot / video of UI
https://github.com/user-attachments/assets/4e95c681-a8f8-4735-9473-d58d23848979
What issues does this PR fix or reference?
Fixes #1568
Fixes #1616
How to test this PR?
You need to set an hidden setting, in ~/.local/share/containers/podman-desktop/configuration/settings.json
(you must completely stop Podman Desktop - not only close the window -, edit the config file, then restart Podman Desktop for the change to be taken into consideration)
  "ai-lab.experimentalTuning": true,

LGTM.
Could we get navigation and tooltip from the model column to model details ?

That should be ok now

Looks better to me. I expected also the same navigation on target model but could be a follow up PR

As the target model will exist only when the instructlab session will be finished, I prefered not to add it for now.


[10 Different Sheets] change image link to point at Roll20 repo

Changes / Comments

change several sheets that had image link pointing to the creator's repo so they now point at the images' location in the Roll20 repo. Tested the unsure cases

Roll20 Requests
Comments are very helpful for reviewing the code changes. Please answer the relevant questions below in your comment.

[x] Does the pull request title have the sheet name(s)? Include each sheet name.
[x] Is this a bug fix?
[ ] Does this add functional enhancements (new features or extending existing features) ?
[ ] Does this add or change functional aesthetics (such as layout or color scheme) ?
[ ] If changing or removing attributes, what steps have you taken, if any, to preserve player data ?
[ ] If this is a new sheet, did you follow Building Character Sheets standards ?

If you do not know English. Please leave a comment in your native language.

Hey @Anduh I know it's a pain to do but we need each sheet to have it's own pull request. We started enforcing it in the past month to make it easier to revert any single sheet commits in case of issues. Sorry about the headache.

We started enforcing it in the past month to make it easier to revert any single sheet commits in case of issues.

As I said to Olivia when she did the first two weeks of the pulls, it would be great if you set out a statement if you make a change in policy or other similar things. But yeah this is fair, I was assuming having separate PRs for this trivial stuff needed to be split, but at least I had separate commits for each.
I'll re-submit them at some point, and the sheet author community about the stricter enforcement of "one sheet per request"-policy.

@Anduh I went ahead and merged this. I generally recommend breaking different sheet updates into different branches even for scenarios like this. Its a best practice because if for some reason one of these fails and we need to revert the change its going to revert all of them instead of just one update.

There's something wrong with the image files as Roll20 shows A LOT of broken images for the All for One sheet.

Yep, I see the problem now, I'll submit a fix right away. Roll20 does the updates weekly on Tuesdays, so that's the when the fix gets applied.


fix(oas): use random id if no title in the oas

Instead of a panic insert a random UUID as the name.

Codecov Report
Patch coverage: 50.00% and project coverage change: -0.27 :warning:

Comparison is base (46625eb) 69.37% compared to head (915d48f) 69.10%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #53      +/-   ##
==========================================
- Coverage   69.37%   69.10%   -0.27%
==========================================
  Files          11       11
  Lines        1564     1573       +9
==========================================
+ Hits         1085     1087       +2
- Misses        418      424       +6
- Partials       61       62       +1

Impacted Files
Coverage Δ

convertoas3/oas3.go
77.58% <50.00%> (-0.38%)
:arrow_down:

... and 1 file with indirect coverage changes

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Add option to select which variables are displayed in histogram

This is optional, but could be a nice feature.
For the report builder tool, it would be nice to include an option select which variables have histograms in the report, since the relative importance of the variables changes throughout the season.
Could be nice to do this for appendix tables too.

Closed in #136.


Daily build with full benchmarks [HPPC-20]

Issue: HPPC-20 (migrated from JIRA), created by Stanisław Osiński (@stanislawosinski), resolved Mar 08 2010

Comment by Stanisław Osiński (@stanislawosinski) (migrated from JIRA)
Done: http://builds.carrot2.org/browse/HPPC-BENCHMARK, triggered after the head build completes successfully.

Comment by Stanisław Osiński (@stanislawosinski) (migrated from JIRA)
I can set up an agent on codex, which should give us more stable results.

Comment by Dawid Weiss (@dweiss) (migrated from JIRA)
One thing to worry about it background activity during this benchmark run. The load on ophelia has become a significant noise factor.


issue with btc domain name

hi
i just paid for a btc.domain name and my transaction id is 0x3bfe1f07cb10802edea2e7da8c7a02c0b08b190473dd0d51b1284654bd5b3524. can anyone help why there is error and i cant proceed with step 2 for registration. thanks

Yes, it went thru. Just wondering why it is not 2 stx but total about 6 stx used for the transaction. Gas fee is almost 2 stx each leg? I plan to buy another btc domain but hold back for now  Next qn, I would like to stake my stx via xverse . wondering how I can do it ? it seems to connect to hiro and not xverse   Sent from Mail for Windows From: dartmanSent: Wednesday, 5 April 2023 5:36 pmTo: hirosystems/explorerCc: Behappy55; AuthorSubject: Re: [hirosystems/explorer] issue with btc domain name (Issue #1084) Hi Behappy55 - I can see both transactions (pre-order and name-register) went thru successfully. Nice.—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID<EMAIL_ADDRESS>