Allow disabling/enabling checks via rules

Codecov Report
Attention: Patch coverage is 81.81818% with 8 lines in your changes missing coverage. Please review.

Project coverage is 94.77%.

Files with missing lines
Patch %
Lines

internal/config/parsed_rule.go
80.00%
3 Missing and 2 partials :warning:

internal/config/rule.go
62.50%
2 Missing and 1 partial :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1171      +/-   ##
==========================================
- Coverage   94.83%   94.77%   -0.07%
==========================================
  Files         104      104
  Lines       12179    12204      +25
==========================================
+ Hits        11550    11566      +16
- Misses        480      486       +6
- Partials      149      152       +3

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Unsynced Ethereum client and get_root_deployment_height/0

OMG.Eth.RootChain implements get_root_deployment_height/0 which fails if the Ethereum client is not fully synced (but responsive!).
This means that the function call will be returning {:error, :wrong_contract_address} and throw the Monitored children into restart spree, until the Eth client responds with a correct height.
I propose we solve this by an early check and a raised alarm (before the system fully boots all children, or completely async and letting a few crashes occur). We would check whether we can get the deployment height, if we cannot and the call is returning {:ok, nil}, there's a possibility that the client is still syncing and we need to schedule the check for later.
The question that is open - how do we differentiate between a client still syncing and wrong config input for the contract_addr and txhash_contract?

I see a couple of items here:
1/ we can handle the {:ok, nil} response with sth less confusing than :wrong_contract_address for a good start
2/ note, we don't have to by fully synced to work - it is enough to be "synced enough" to see the receipt of the contract deploying tx. Here we're so "unsynced" we don't even see the relevant contract yet. Unless this is of course the child chain, which is more stringent on that matter.

how do we differentiate between a client still syncing and wrong config input for the contract_addr and txhash_contract?

If we have a way to not suffer from the pointless restarts, we can also treat these equally, assuming that not seeing the contract at all is sth like a fatal condition.
Btw, why are we actually having the restart spree at all - shouldn't it be done after a few attempts (I remember we bumped the max_restarts, but wasn't this obsoleted by proper geth connectivity handling?)

Handling on that level (the implementation of get_root_deployment_height/0) wouldn't be too beneficial really, because EthereumEventListener, BlockGetter, BlockQueue expect a tuple with an integer.
Yes indeed, the client needs to be synced up to the contract deployment height. Still it can take quite a while before you're there when starting a clean watcher deployment, for example.

The new geth connectivity checks just that - whether the network and the client is responding (similar to a ping but we fetch last height).
If we have a way to not suffer from the pointless restarts, we can also treat these equally, assuming that not seeing the contract at all is sth like a fatal condition.
If the contract with the address specified in the config is really deployed, then it's not temporary fatal condition. Until the client syncs to the contract height. But if the contract doesn't even exists, then the condition would never recover. At best we can do probably is handle that (or integrate Etherscan to check validity?)

in (1.) I'm just suggesting getting rid of the misleading error message.
In general, I'm supportive of solving the problem fully and elegantly.
But the question remains - why is the restart spree happening at all, if it should give up after 3 attempts (the default)? If the application just refused to start, with an understandable error message and log, I think we would be much better off already.
It would perhaps be too strong for cases where we're just syncing, but it would at least be adequate for cases of misconfiguration. And telling these two cases apart is in fact hard, without tricks like etherscan etc.

The restart don't stop because the restart strategy was replaced by custom monitoring.  This is not related to ethereum connectivity where we would gracefully handle the exits because there's an alarm raised.
I suggest we do the following:
if get_root_deployment_height/0 is unable to return a result we understand ({:ok, <some number>}, we raise an alarm that contains a general description, something like:
Ethereum client syncing in progress or contract address miss configuration. We then prevent restarts, but periodically check get_root_deployment_height/0. If we get an answer, we clear the alarm and restart all children. Wdyt?

restart don't stop because the restart strategy was replaced by custom monitoring

oh, I thought the monitoring "inherited" the Supervisor's behavior. I'd assume that there are errors, on which we do want to stop after all.

I suggest we do the following:...

sounds good. If it is a quick fix, we can just apply this, but if it's not - I still opt to amend the misleading error message & restart spree first.


Broker Redirect: Option 1

Cache response in broker, and gives to embedded app on next API invocation with the same parameters

Closing in favor of #2428


feat: add resume option

Which issue(s) this PR fixes:

Fixes #225
Main features:

Add --resume option
Move get_deployment to queries.py
Introduced new Enum DeploymentTypeEnum
Also applied the non resolved comments from https://github.com/TOSIT-IO/tdp-lib/pull/244

Additional comments:

To make clear that you license your contribution under the Apache License Version 2.0, January 2004 and that you give permission to TOSIT, you have to acknowledge this by using the following check-box.

[x] I hereby declare this contribution to be licensed under the Apache License Version 2.0, January 2004.
[x] I hereby agree to grant TOSIT a copyright license to use my contributions.

Closed in favor of https://github.com/TOSIT-IO/tdp-lib/pull/258


hold cuda11.2 from upgrading to cuda12.2

This PR fixes an issue where apt-get upgrade was upgrading CUDA from 11.2 to 12.2, causing mismatches between libcublas 11.2, libcudnn8, and other GPU libraries. This led to GPU-related errors in production.
Changes:
Held back library libcudnn8 to prevent it from upgrading to CUDA 12.2.

I tested the resulting image and it works fine.


sorted import statements

after the namespace change, the import statements were no longer sorted.

Also, with the namespace change and indentation correction, I observed that some of the multi-line statements were collapsed to a single long line. Was that intentional? For e.g.: https://github.com/apache/incubator-myriad/blob/master/myriad-scheduler/src/main/java/org/apache/myriad/scheduler/fgs/YarnNodeCapacityManager.java#L215

While you are at it, could you also please take care of adding explicit imports for statements like this: https://github.com/apache/incubator-myriad/blob/master/myriad-scheduler/src/main/java/org/apache/myriad/scheduler/fgs/YarnNodeCapacityManager.java#L70

We really need to come up with a formatter for IDEA and Eclipse. I've been
tweaking mine so that I can just reformat files and everything is good.
There are still some quirks. The checkstyles is set to 250 columns, so I
complied with that and it looks like it unwrapped things.
On Thu, Oct 29, 2015 at 11:51 AM, Santosh Marella<EMAIL_ADDRESS>wrote:

Also, with the namespace change and indentation correction, I observed
that some of the multi-line statements were collapsed to a single long
line. Was that intentional? For e.g.:
https://github.com/apache/incubator-myriad/blob/master/myriad-scheduler/src/main/java/org/apache/myriad/scheduler/fgs/YarnNodeCapacityManager.java#L215
—
Reply to this email directly or view it on GitHub
https://github.com/apache/incubator-myriad/pull/26#issuecomment-152222439
.

Fixed fully qualified class names. Want to make a call on the line wrapping?
On Thu, Oct 29, 2015 at 12:12 PM, Jim Klucar<EMAIL_ADDRESS>wrote:

We really need to come up with a formatter for IDEA and Eclipse. I've been
tweaking mine so that I can just reformat files and everything is good.
There are still some quirks. The checkstyles is set to 250 columns, so I
complied with that and it looks like it unwrapped things.
On Thu, Oct 29, 2015 at 11:51 AM, Santosh Marella <
<EMAIL_ADDRESS>wrote:

Also, with the namespace change and indentation correction, I observed
that some of the multi-line statements were collapsed to a single long
line. Was that intentional? For e.g.:
https://github.com/apache/incubator-myriad/blob/master/myriad-scheduler/src/main/java/org/apache/myriad/scheduler/fgs/YarnNodeCapacityManager.java#L215
—
Reply to this email directly or view it on GitHub
https://github.com/apache/incubator-myriad/pull/26#issuecomment-152222439
.

Thanks @klucar . Regarding the line wrapping, I guess we need to agree on a reasonable line length. How about 120?

Well 80 is the old punch card standard, and 132 is the "widescreen"
terminal standard. So I'd say 132 to keep up with outdated requirements.
On Thu, Oct 29, 2015 at 4:58 PM, Santosh Marella<EMAIL_ADDRESS>wrote:

Thanks @klucar https://github.com/klucar . Regarding the line wrapping,
I guess we need to agree on a reasonable line length. How about 120?
—
Reply to this email directly or view it on GitHub
https://github.com/apache/incubator-myriad/pull/26#issuecomment-152323160
.

+1. Go for it! I'll change my IDE to use 132.
Santosh
On Thu, Oct 29, 2015 at 2:06 PM, Jim Klucar<EMAIL_ADDRESS>wrote:

Well 80 is the old punch card standard, and 132 is the "widescreen"
terminal standard. So I'd say 132 to keep up with outdated requirements.
On Thu, Oct 29, 2015 at 4:58 PM, Santosh Marella<EMAIL_ADDRESS>
wrote:

Thanks @klucar https://github.com/klucar . Regarding the line
wrapping,
I guess we need to agree on a reasonable line length. How about 120?
—
Reply to this email directly or view it on GitHub
<
https://github.com/apache/incubator-myriad/pull/26#issuecomment-152323160>
.

—
Reply to this email directly or view it on GitHub
https://github.com/apache/incubator-myriad/pull/26#issuecomment-152326857
.

I think line wrapping is the only thing left on this PR. @klucar - I can do the line wrapping before merging. Let me know.

Go for it
On Tuesday, November 3, 2015, Santosh Marella<EMAIL_ADDRESS>wrote:

I think line wrapping is the only thing left on this PR. @klucar
https://github.com/klucar - I can do the line wrapping before merging.
Let me know.
—
Reply to this email directly or view it on GitHub
https://github.com/apache/incubator-myriad/pull/26#issuecomment-153411233
.


Add verification of llama model access in first_finetune_tutorial.rst

Details in https://github.com/pytorch/torchtune/issues/638. Add more instructions to improve the user experience on model accessibility.
Context

...

Changelog

...

Test plan

....

Sorry but seems like lint is complaining. mind running:
pre-commit run --all-files

and then pushing a commit?


[Image captioning] pack_packed wrong lengths

In the Image Captioning tutorial
in the DecoderRNN:
def forward(self, features, captions, lengths):
  embeddings = self.embed(captions)
  embeddings = torch.cat((features.unsqueeze(1), embeddings), 1)
  packed = pack_padded_sequence(embeddings, lengths, batch_first=True)
  hiddens, _ = self.lstm(packed)
  outputs = self.linear(hiddens[0])
  return outputs

shouldn't the lenghts inside pack_padded be lenghts+1 to take into account the increased length due to the features added with cat?
e.g. (assuming numbers are captions, e is the embedding, f are the features and batch_size = 4)
if
embeds:
e(126)  e(1214)  e(14)    e(4033)
e(126)  e(6)     e(84)    e(4033)
e(126)  e(3002)  e(4033)  e(0)
e(126)  e(3002)  e(4033)  e(0)

has lengths = [4,4,3,3]
then
embeds_cat:
f_0  e(126)  e(1214)  e(14)    e(4033)
f_1  e(126)  e(6)     e(84)    e(4033)
f_2  e(126)  e(3002)  e(4033)  e(0)
f_3  e(126)  e(3002)  e(4033)  e(0)

should have lengths = [5,5,4,4], right?

@LapoFrati i was thinking the same thing as I went through this code...why wouldn't you want to use last word in the caption as input?

@ghwatson understand the  actually can be ignored. However, in this code implementation after torch.cat, the sequence will change from " token1 token2 ... tokenk" to "feature  token1 token2 ..tokenk". The first step will be the "feature" vector and it's always used to predict  only? From my understanding, the "feature" vector would make more sense if it works as the starting state of LSTM.

Using the "feature vector" you have a training process that looks like:

step 0: features + zeroes  -> caption_0 + state_0
step 1: embed(caption_0) + state_0 -> caption_1 + state_1
...
step n: embed(caption_n-1) + state_n-1 -> <end>

where zeroes is the default value if no state is provided.
@sth4k if you use the conv output as a "starting state" and manually provide it to the LSTM, what would you use as first input to the LSTM? A vector of constant values? I'm not an expert but I think both approaches should work in principle.

Using the "feature vector" you have a training process that looks like:

step 0: features + zeroes  -> caption_0 + state_0
step 1: embed(caption_0) + state_0 -> caption_1 + state_1
...
step n: embed(caption_n-1) + state_n-1 ->

where zeroes is the default value if no state is provided.
@sth4k if you use the conv output as a "starting state" and manually provide it to the LSTM, what would you use as first input to the LSTM? A vector of constant values? I'm not an expert but I think both approaches should work in principle.

Hi LapoFrati, thanks for your reply. Somehow github eats my word <s.o.s> which is the starting of sentence token (in your case caption_0). so you can see, features + zeros will always be used to predict caption_0 which is basically a starting token (similar to eos). Does it make sense?


Merge with gip-recia/uPortal-web-components

/cc @jgribonvald

Thanks Christian for the move.
This is working !

The change rename the dist file to esco.js (and others like that) and so you can't find anymore the js as the js name isn't the same as the project (it's like that I understood it) !

Good catch, patch incoming


Serving resources ending with slash not possible

It is not possible to serve resources which are ending on a slash. E.g.
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix foaf: <http://xmlns.com/foaf/0.1/> .

<http://example.org/> rdf:type foaf:Person .
<http://example.org/> foaf:name "Example" .

A file .html will be generated. Which can't be served, at least with jekyll serve.

We should switch to this concept: https://jekyllrb.com/docs/pages/#named-folders-containing-index-html-files
Do you agree?

Yes, that sounds good.
(just for later reference:)
For distinguishing between the actual resource, the HTML resource and potentially an RDF resource (cf. https://www.w3.org/TR/cooluris/#semweb) still some kind of proxy with redirects and content negotiation can be used in front of the static pages.

Moved to https://github.com/white-gecko/jekyll-rdf/issues/24


"Typo" in documentation for range of midi channel

This is a bit of a smart ass "issue" but in "https://github.com/robbert-vdh/nih-plug/blob/master/src/midi.rs" the channel of NoteEvent (For messages that have a channel) is declared as:
    /// The note's channel, from 0 to 16.
    channel: u8,

It should probably read 0 to 15 (4-Bit) if it is zero based or 1-16. From my experience it is zero based.
https://www.cs.cmu.edu/~music/cmsip/readings/MIDI tutorial for programmers.html

It's 0..16. The to is exclusive here. I'll probably rewrite all of these comments at some point to use Rust's range syntax at some point.

Thanks for the response! Should I close the issue?

Can keep it open, I'll need change this at some point. 'x to y' means that y is exclusive to most people (as opposed to 'x through y'), but since that's apparently still ambiguous it's probably best to just use Rust range syntax here.

I tried a couple things but all the alternatives just read worse. So instead I added a note on what from x to y means and I changed all floating point ranges to be in the standard [x, y] interval notation.

Hi, thank you for the fixes. I hate to say that now it is wrong in another place :-/, because in the next line where the note value is defined, it says it goes from 0 to 127. But in this case it 127 should be included. If I may make a suggestion:
pub enum NoteEvent {
    /// A note on event, available on [`MidiConfig::Basic`] and up.
    NoteOn {
        timing: u32,
        /// A unique identifier for this note, if available. Using this to refer to a note is
        /// required when allowing overlapping voices for CLAP plugins.
        voice_id: Option<i32>,
        /// The note's channel, with values from 0 to 15 inclusive.
        channel: u8,
        /// The note's MIDI key number, with values from 0 to 127 inclusive.
        note: u8,
        /// The note's velocity, in the range `[0, 1]`. Some plugin APIs may allow higher precision
        /// than the 127 levels available in MIDI.
        velocity: f32,
    },
   //...
}

I think it is better to keep it simple and unambigious. Also it is might be better to keep it in one place, because if I look it up in the reference I will usually not look at the beginning of the file or read the full text. A reference is not meant to be read completely but to look up specific things. Also if you hover it in any IDE it will not show the explanation in the beginning.
If you agree I can try to make a PR if you like.


Subset selection noting working for jobs

Feature to add:
-See that an input file is 'Select'
-Read particles.star from Select directory
-Find the Extract/{folder ID}
-Then see if it exists in .aws_relion

Fixed issue with commit:
https://github.com/leschzinerlab/AWS/commit/e515d4b90fe1e1e14a36356be0ac8e12d59863c5


GD BAK Generator

Replicate all gd-param tables onto a secondary system that is connected to an MSSQL Server. The secondary system can then provide MSSQL Server backup file on demand.
Perhaps we should do this via some REST API.

PUT the entire data model
GET the BAK of the current data model

Open Question:

How to not include auxiliary tables with the backup (Apache Causeway's Secman)?

General purpose table replicator implemented.


Respect the startup info state initially passed to wt via ShellExecute

⚠ I'm targeting #13811 -> #13570 with this, because I want to have all my startup HWND hijinks on one continuous branch, rather than 100 things to merge and mess with each other. ⚠
This is just the SHOWDEFAULT bit from #12979. This seems to also work now, but I'm PR'ing it separately so it can be a separate revert from #13811, if it is problematic.
Closes #9053
This doesn't fix defterm-initiated connections, that'll be a fourth PR in this chain.

These merge-down stacks will need individual rebasing or merging in of main (or the appropriate parent branch) to trigger a full CI run after the lower PR merges.

We may need a --minimized flag or something, to pass to the monarch...

oh it was so much easier than that


CVE-2023-45857 on the axios module used in the package

We are seeing a CVE on the module.
The CVE is: CVE-2023-45857
Vulnerability scan summary for package-lock.json
	# of Packages scanned: 561
	# of Vulnerabilities found: 1
	Vulnerability found: Cross-site Request Forgery (CSRF) - SNYK-JS-AXIOS-6032459
		URL: https://security.snyk.io/vuln/SNYK-JS-AXIOS-6032459
		Package: axios
		Version: 1.4.0
		Introduced By: ibm-cloud-sdk-core:4.1.2 --> axios:1.4.0
		Severity: high
		Description: axios is a promise-based HTTP client for the browser and Node.js.

			Affected versions of this package are vulnerable to Cross-site Request Forgery (CSRF) due to inserting the X-XSRF-TOKEN header using the secret XSRF-TOKEN cookie value in all requests to any server when the XSRF-TOKEN0 cookie is available, and the withCredentials setting is turned on. If a malicious user manages to obtain this value, it can potentially lead to the XSRF defence mechanism bypass.
		Updated at: 2023-10-25T07:24:40.920085Z
		Codes: [CVE-2023-45857]

@luiof what type of scanner found that vulnerability?   I'm curious because our CRA (Code Risk Analysis) scanning job did not find this vulnerability.

cc: @ricellis

The issue in axios for this seems to suggest that it requires withCredentials: true to be used. AFAIK that hasn't been used here since v4 release with the axios 1.x upgrade. Ofc it would be prudent to upgrade axios as soon as a fixed version is available anyway.

Axios 1.6.0 is available now, I made #255 for the bump.

:tada: This issue has been resolved in version 4.1.4 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


fix(olHelpers): pass extractStyles to ol.format.KML

extractStyles is a parameter for ol.format.KML, and not ol.source.Vector. This fixes also the bug described in #268.
Linked to #268

Thanks a lot for contributing!


Globally overrides console.warn causing errors if string not passed

The global override of console.warn assumes the argument passed is a string but this isn't necessarily the case. It crashes my whole app since I have log statements with different argument types
svelte-markdown.js?v=b129818a:53 Uncaught (in promise) TypeError: message.includes is not a function
    at console.warn (svelte-markdown.js?v=b129818a:53:17)
    at updateAttribute (ComfyProperties.svelte?t=1685400662407:2268:11)
    at Array.change_handler (ComfyProperties.svelte?t=1685400662407:2385:38)
    at Textbox.change_handler (ComfyProperties.svelte?t=1685400662407:1564:36)
    at chunk-PUOMPBAM.js?v=a23c27b8:1014:12
    at Array.forEach (<anonymous>)
    at chunk-PUOMPBAM.js?v=a23c27b8:1013:25
    at handle_change (Textbox.svelte:761:3)
    at $$self.$$.update (Textbox.svelte:987:7)
    at update (chunk-PUOMPBAM.js?v=a23c27b8:1106:8)

Can you show a reduced example that triggers this error?


Empty (or invalid) settings.json yields unhelpful error message

Something wiped out my settings.json and left an empty file there (no idea what happened), but now every time I run the CLI, for simple commands, I get the message:
$ turso db list
Error: could not read local settings

This is very problematic because it doesn't say where that file lives (could vary by platform), what the problem is with it, or how to recover.  Ideally the output should name the file, the error, and make a recommendation to delete the file if it got corrupted.

I found that it's also the case that using -c on the command line can cause this error to appear if the argument wasn't valid in some way, for example, specifying a file instead of a folder.  This case could be improved as well.


Add libiconv

If compile libxml2 without iconv support, that will lead to errors like
0009:err:msxml:doparse Unsupported encoding gb2312
and inability to handle such documents.

Libiconv, got it! Thanks!

Hey @wahaha2019, I've added the libiconv build and made sure Wine is using it. Maybe you can test or send me a problematic document so I can test. Thanks for reporting it, it revealed some bugs :)

Thank you. I want test, then build with:
DOCKER_BUILDKIT=0 docker build --build-arg BUILD_WITH_LTO=n --build-arg BUILD_JOBS=2 --build-arg PLATFORM=x86-64 --build-arg PREFIX=$HOME/.local -t static-wine32:latest . 2>&1 | tee build.log
but build failed:
Sending build context to Docker daemon  30.84MB
Step 1/23 : FROM ubuntu:22.04
22.04: Pulling from library/ubuntu
aece8493d397: Pulling fs layer
aece8493d397: Verifying Checksum
aece8493d397: Download complete
aece8493d397: Pull complete
Digest: sha256:2b7412e6465c3c7fc5bb21d3e6f1917c167358449fecac8176c6e496e5c1f05f
Status: Downloaded newer image for ubuntu:22.04
---> e4c58958181a
Step 2/23 : SHELL ["/bin/bash", "-c"]
---> Running in ea249f6444f5
Removing intermediate container ea249f6444f5
---> 8dd5e532518c
Step 3/23 : RUN dpkg --add-architecture i386 &&     apt update &&     apt upgrade -y &&     DEBIAN_FRONTEND=noninteractive apt install -y build-essential pkg-config         gcc-multilib
g++-multilib gcc-mingw-w64 libcrypt1-dev:i386 flex bison         python3 python3-pip wget git ninja-build gperf automake         autoconf-archive libtool autopoint gettext nasm glslang
-tools &&     pip3 install mako jinja2 jinja2-cli &&     wget -q https://github.com/Kitware/CMake/releases/download/v3.27.5/cmake-3.27.5-linux-x86_64.tar.gz -P $HOME &&     tar xf $HOME
/cmake-*-linux-x86_64.tar.gz -C /usr --strip-components=1 &&     cmake --version &&     git clone --depth 1 --branch 1.2.1 https://github.com/mesonbuild/meson.git "$HOME/meson" &&     e
cho "#!/bin/sh" > /usr/bin/meson &&     echo "python3 "$HOME/meson/meson.py" $@" > /usr/bin/meson &&     chmod +x /usr/bin/meson && meson --version &&     DEBIAN_FRONTEND=noninteract
ive apt install -y nano xvfb x11-apps         imagemagick &&     echo "#!/bin/sh" > /usr/bin/startx &&     echo "Xvfb "$DISPLAY" -screen 0 1200x800x24 &" >> /usr/bin/startx &&     ec
ho >> /usr/bin/startx &&     chmod +x /usr/bin/startx
---> Running in 8e43769804eb
WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

E: Problem executing scripts APT::Update::Post-Invoke 'rm -f /var/cache/apt/archives/.deb /var/cache/apt/archives/partial/.deb /var/cache/apt/.bin || true'
E: Sub-process returned an error code
The command '/bin/bash -c dpkg --add-architecture i386 &&     apt update &&     apt upgrade -y &&     DEBIAN_FRONTEND=noninteractive apt install -y build-essential pkg-config         gc
c-multilib g++-multilib gcc-mingw-w64 libcrypt1-dev:i386 flex bison         python3 python3-pip wget git ninja-build gperf automake         autoconf-archive libtool autopoint gettext na
sm glslang-tools &&     pip3 install mako jinja2 jinja2-cli &&     wget -q https://github.com/Kitware/CMake/releases/download/v3.27.5/cmake-3.27.5-linux-x86_64.tar.gz -P $HOME &&     ta
r xf $HOME/cmake--linux-x86_64.tar.gz -C /usr --strip-components=1 &&     cmake --version &&     git clone --depth 1 --branch 1.2.1 https://github.com/mesonbuild/meson.git "$HOME/meson
" &&     echo "#!/bin/sh" > /usr/bin/meson &&     echo "python3 "$HOME/meson/meson.py" $@" > /usr/bin/meson &&     chmod +x /usr/bin/meson && meson --version &&     DEBIAN_FRONTEND=n
oninteractive apt install -y nano xvfb x11-apps         imagemagick &&     echo "#!/bin/sh" > /usr/bin/startx &&     echo "Xvfb "$DISPLAY" -screen 0 1200x800x24 &" >> /usr/bin/startx
&&     echo >> /usr/bin/startx &&     chmod +x /usr/bin/startx' returned a non-zero code: 100

When I built at 2023-11-02 with the Ubuntu 20.04 Docker file via this command:
DOCKER_BUILDKIT=0 docker build --build-arg BUILD_JOBS=2 --build-arg PLATFORM=x86-64 --build-arg PREFIX=$HOME/.local -t static-wine32:latest . 2>&1 | tee build.log
it exit ok, but the final tar.gz file is too small. The latest content with errors of the build.log:
gcc -m32 -o dlls/winex11.drv/winex11.so -shared -Wl,-Bsymbolic -Wl,-soname,winex11.so -Wl,-z,defs dlls/winex11.drv/bitblt.o
dlls/winex11.drv/brush.o dlls/winex11.drv/clipboard.o dlls/winex11.drv/desktop.o
dlls/winex11.drv/display.o dlls/winex11.drv/event.o dlls/winex11.drv/graphics.o
dlls/winex11.drv/init.o dlls/winex11.drv/keyboard.o dlls/winex11.drv/mouse.o
dlls/winex11.drv/opengl.o dlls/winex11.drv/palette.o dlls/winex11.drv/pen.o
dlls/winex11.drv/vulkan.o dlls/winex11.drv/window.o dlls/winex11.drv/wintab.o
dlls/winex11.drv/x11drv_main.o dlls/winex11.drv/xim.o dlls/winex11.drv/xinerama.o
dlls/winex11.drv/xrandr.o dlls/winex11.drv/xrender.o dlls/winex11.drv/xvidmode.o
dlls/ntdll/ntdll.so dlls/win32u/win32u.so -L/usr/local/lib -lX11 -lpthread -lxcb -lXau
-L/usr/local/lib -lXext -lX11 -lpthread -lxcb -lXau -ldl -L/usr/local/lib -lXrender -lX11
-lpthread -lxcb -lXau -ldl -L/usr/local/lib -lXrandr -lXext -lXrender -lX11 -lpthread -lxcb -lXau
-ldl -L/usr/local/lib -lXxf86vm -lXext -lX11 -lpthread -lxcb -lXau -L/usr/local/lib -lXinerama
-lXext -lX11 -lpthread -lxcb -lXau -L/usr/local/lib -lXcursor -lXrender -lXfixes -lX11 -lpthread
-lxcb -lXau -ldl -L/usr/local/lib -lXi -lXext -lXfixes -lX11 -lpthread -lxcb -lXau
-L/usr/local/lib -lXcomposite -lXfixes -lX11 -lpthread -lxcb -lXau -L/usr/local/lib
-L/usr/local/lib/dri -L/usr/local/lib -lvulkan -Wl,--whole-archive -lvulkan_radeon -lvulkan_intel
-lvulkan_intel_hasvk -lvulkan_runtime -lvulkan_util -lvulkan_wsi -Wl,--no-whole-archive
-ldrm_amdgpu -lGL -ldl -lm -pthread /usr/lib/gcc/x86_64-linux-gnu/9/32/libstdc++.a -lgallium_dri
-lpthread -lvulkan -lOSMesa -pthread /usr/lib/gcc/x86_64-linux-gnu/9/32/libstdc++.a -lm -ldl
-lglapi -lpthread -lunwind -llzma -lz -lxcb-dri3 -lxcb-present -lxcb-randr -lxcb-sync -lxshmfence
-lxcb-xfixes -lxcb-render -lxcb-shape -lexpat -lm -ldrm_radeon -pthread -lelf -lz -lzstd -pthread
-ldrm_nouveau -pthread -ldrm_intel -pthread -ldrm -lm -lpciaccess -lXxf86vm -lxcb-glx -lX11-xcb
-lxcb-dri2 -lXext -lXfixes -lX11 -lpthread -lxcb-shm -lxcb -lXau -ludev -lrt -pthread -lcap
-L/usr/local/lib -lOSMesa -pthread /usr/lib/gcc/x86_64-linux-gnu/9/32/libstdc++.a -lm -ldl -lglapi
-lpthread -lz -lzstd -pthread -lunwind -llzma -lz -ldrm -lm -lm -ldl -lgstreamer-full-1.0
-m32 -march=x86-64 -flto -flto-partition=one
^[[91mlto-wrapper: warning: Extra option to -Xassembler: --noexecstack, dropping all -Xassembler and -Wa options.
gcc: fatal error: Killed signal terminated program lto1
compilation terminated.
lto-wrapper: fatal error: gcc returned 1 exit status
compilation terminated.
/usr/bin/ld: error: lto-wrapper failed
^[[0m^[[91mcollect2: error: ld returned 1 exit status
^[[0m^[[91mmake: *** [Makefile:120551: dlls/winex11.drv/winex11.so] Error 1
^[[0m^[[91mmake: *** Waiting for unfinished jobs....
^[[0m^[[91mlto-wrapper: warning: Extra option to -Xassembler: --noexecstack, dropping all -Xassembler and -Wa options.
../subprojects/fdk-aac-2.0.2/libMpegTPDec/src/tpdec_latm.h:126:3: warning: type 'struct LATM_LAYER_INFO' violates the C++ One Definition Rule [-Wodr]
../subprojects/fdk-aac-2.0.2/libMpegTPEnc/src/tpenc_latm.h:130:3: note: a different type is defined in another translation unit
../subprojects/fdk-aac-2.0.2/libMpegTPDec/src/tpdec_latm.h:122:8: note: the first difference of corresponding definitions is field 'm_frameLengthType'
../subprojects/fdk-aac-2.0.2/libMpegTPEnc/src/tpenc_latm.h:126:14: note: a field with different name is defined in another translation unit
../subprojects/fdk-aac-2.0.2/libMpegTPDec/src/tpdec_adts.h:141:8: warning: type 'struct STRUCT_ADTS' violates the C++ One Definition Rule [-Wodr]
../subprojects/fdk-aac-2.0.2/libMpegTPEnc/src/tpenc_adts.h:133:3: note: a different type is defined in another translation unit
../subprojects/fdk-aac-2.0.2/libMpegTPDec/src/tpdec_adts.h:142:18: note: the first difference of corresponding definitions is field 'bs'
../subprojects/fdk-aac-2.0.2/libMpegTPEnc/src/tpenc_adts.h:111:7: note: a field with different name is defined in another translation unit
../subprojects/fdk-aac-2.0.2/libSBRenc/include/../src/bit_sbr.h:131:8: warning: type 'struct SBR_HEADER_DATA' violates the C++ One Definition Rule [-Wodr]
../subprojects/fdk-aac-2.0.2/libSBRdec/src/env_extr.h:295:3: note: a different type is defined in another translation unit
../subprojects/fdk-aac-2.0.2/libSBRenc/include/../src/bit_sbr.h:132:11: note: the first difference of corresponding definitions is field 'sbr_amp_res'
../subprojects/fdk-aac-2.0.2/libSBRdec/src/env_extr.h:271:3: note: a field with different name is defined in another translation unit
../subprojects/fdk-aac-2.0.2/libSBRenc/include/../src/ton_corr.h:131:3: warning: type 'struct PATCH_PARAM' violates the C++ One Definition Rule [-Wodr]
../subprojects/fdk-aac-2.0.2/libSBRdec/src/lpp_tran.h:187:3: note: a different type is defined in another translation unit
../subprojects/fdk-aac-2.0.2/libSBRenc/include/../src/ton_corr.h:120:7: note: the first difference of corresponding definitions is field 'sourceStartBand'
../subprojects/fdk-aac-2.0.2/libSBRdec/src/lpp_tran.h:174:9: note: a field of same name but different type is defined in another translation unit
../subprojects/fdk-aac-2.0.2/libSBRenc/include/../src/ton_corr.h:131:3: note: type 'INT' should match type 'UCHAR'
../subprojects/fdk-aac-2.0.2/libSBRenc/include/../src/sbr.h:129:16: warning: type 'struct SBR_CHANNEL' violates the C++ One Definition Rule [-Wodr]
../subprojects/fdk-aac-2.0.2/libSBRdec/src/sbr_dec.h:173:3: note: a different type is defined in another translation unit
../subprojects/fdk-aac-2.0.2/libSBRenc/include/../src/sbr.h:130:22: note: the first difference of corresponding definitions is field 'hEnvChannel'
../subprojects/fdk-aac-2.0.2/libSBRdec/src/sbr_dec.h:170:18: note: a field with different name is defined in another translation unit
../subprojects/fdk-aac-2.0.2/libSACenc/src/sacenc_bitstream.h:241:3: warning: type 'struct LOSSLESSDATA' violates the C++ One Definition Rule [-Wodr]
../subprojects/fdk-aac-2.0.2/libSACdec/src/sac_dec.h:201:3: note: a different type is defined in another translation unit
../subprojects/fdk-aac-2.0.2/libSACenc/src/sacenc_bitstream.h:236:9: note: the first difference of corresponding definitions is field 'bsXXXDataMode'
../subprojects/fdk-aac-2.0.2/libSACdec/src/sac_dec.h:196:9: note: a field of same name but different type is defined in another translation unit
../subprojects/fdk-aac-2.0.2/libSACenc/src/sacenc_bitstream.h:241:3: note: array types have different bounds
../subprojects/fdk-aac-2.0.2/libFDK/include/nlc_dec.h:129:3: warning: type 'DATA_TYPE' violates the C++ One Definition Rule [-Wodr]
../subprojects/fdk-aac-2.0.2/libSACenc/src/sacenc_nlc_enc.h:120:3: note: an enum with mismatching number of values is defined in another translation unit
pcm_local.h:383:6: warning: type of 'snd1_pcm_set_appl_ptr' does not match original declaration [-Wlto-type-mismatch]
pcm.c:7864:6: note: type mismatch in parameter 4
pcm.c:7864:6: note: 'snd1_pcm_set_appl_ptr' was previously declared here
pcm.c:7864:6: note: code may be misoptimized unless '-fno-strict-aliasing' is used
pcm_local.h:382:6: warning: type of 'snd1_pcm_set_hw_ptr' does not match original declaration [-Wlto-type-mismatch]
pcm.c:7857:6: note: type mismatch in parameter 4
pcm.c:7857:6: note: 'snd1_pcm_set_hw_ptr' was previously declared here
pcm.c:7864:6: note: code may be misoptimized unless '-fno-strict-aliasing' is used
pcm_local.h:382:6: warning: type of 'snd1_pcm_set_hw_ptr' does not match original declaration [-Wlto-type-mismatch]
pcm.c:7857:6: note: type mismatch in parameter 4
pcm.c:7857:6: note: 'snd1_pcm_set_hw_ptr' was previously declared here
pcm.c:7857:6: note: code may be misoptimized unless '-fno-strict-aliasing' is used
../gobject/gsignal.c: In function 'signal_emit_unlocked_R':
../gobject/gsignal.c:3885: warning: argument to 'alloca' may be too large [-Walloca-larger-than=]
gcc: fatal error: Killed signal terminated program lto1
compilation terminated.
lto-wrapper: fatal error: gcc returned 1 exit status
compilation terminated.
/usr/bin/ld: error: lto-wrapper failed
collect2: error: ld returned 1 exit status
^[[0m^[[91mmake: *** [Makefile:117917: dlls/winegstreamer/winegstreamer.so] Error 1
^[[0m./
./lib/
./lib/wine/
./lib/wine/i386-windows/
./lib/wine/i386-windows/sfc.dll
./lib/wine/i386-windows/wmi.dll
./lib/wine/i386-windows/stdole2.tlb
./lib/wine/i386-windows/lz32.dll
./lib/wine/i386-windows/tzres.dll
./lib/wine/i386-windows/security.dll
./lib/wine/i386-windows/light.msstyles
./lib/wine/i386-windows/usp10.dll
./lib/wine/i386-windows/apisetschema.dll
./lib/wine/i386-windows/shdoclc.dll
./lib/wine/i386-windows/icmp.dll
./lib/wine/i386-windows/stdole32.tlb
./lib/wine/i386-windows/msimsg.dll
./lib/wine/i386-windows/normaliz.dll
./lib/wine/i386-windows/mferror.dll
./share/
./share/applications/
./share/applications/wine.desktop
./share/wine/
./share/wine/wine.inf
./share/wine/nls/
./share/wine/nls/c_10029.nls
./share/wine/nls/c_1252.nls
./share/wine/nls/c_28599.nls
./share/wine/nls/c_720.nls
./share/wine/nls/sortdefault.nls
./share/wine/nls/c_10082.nls
./share/wine/nls/c_865.nls
./share/wine/nls/normnfkc.nls
./share/wine/nls/l_intl.nls
./share/wine/nls/c_10004.nls
./share/wine/nls/c_1251.nls
./share/wine/nls/c_1255.nls
./share/wine/nls/c_936.nls
./share/wine/nls/c_10002.nls
./share/wine/nls/c_28592.nls
./share/wine/nls/normidna.nls
./share/wine/nls/c_875.nls
./share/wine/nls/c_855.nls
./share/wine/nls/c_10007.nls
./share/wine/nls/c_28595.nls
./share/wine/nls/c_28597.nls
./share/wine/nls/c_708.nls
./share/wine/nls/c_037.nls
./share/wine/nls/c_950.nls
./share/wine/nls/c_869.nls
./share/wine/nls/c_1254.nls
./share/wine/nls/c_1258.nls
./share/wine/nls/c_437.nls
./share/wine/nls/c_10079.nls
./share/wine/nls/c_20949.nls
./share/wine/nls/c_28603.nls
./share/wine/nls/c_10017.nls
./share/wine/nls/c_737.nls
./share/wine/nls/c_28598.nls
./share/wine/nls/c_1026.nls
./share/wine/nls/c_10008.nls
./share/wine/nls/c_10003.nls
./share/wine/nls/normnfkd.nls
./share/wine/nls/c_775.nls
./share/wine/nls/c_10001.nls
./share/wine/nls/c_20866.nls
./share/wine/nls/c_857.nls
./share/wine/nls/c_28596.nls
./share/wine/nls/c_28605.nls
./share/wine/nls/c_874.nls
./share/wine/nls/c_850.nls
./share/wine/nls/c_10081.nls
./share/wine/nls/c_10010.nls
./share/wine/nls/c_10006.nls
./share/wine/nls/c_932.nls
./share/wine/nls/normnfc.nls
./share/wine/nls/c_500.nls
./share/wine/nls/c_20127.nls
./share/wine/nls/c_1253.nls
./share/wine/nls/c_863.nls
./share/wine/nls/c_21866.nls
./share/wine/nls/c_28593.nls
./share/wine/nls/c_1250.nls
./share/wine/nls/c_1257.nls
./share/wine/nls/normnfd.nls
./share/wine/nls/c_10021.nls
./share/wine/nls/c_852.nls
./share/wine/nls/c_10005.nls
./share/wine/nls/c_949.nls
./share/wine/nls/c_864.nls
./share/wine/nls/c_861.nls
./share/wine/nls/c_866.nls
./share/wine/nls/c_1361.nls
./share/wine/nls/c_20932.nls
./share/wine/nls/c_860.nls
./share/wine/nls/c_862.nls
./share/wine/nls/locale.nls
./share/wine/nls/c_10000.nls
./share/wine/nls/c_28594.nls
./share/wine/nls/c_1256.nls
./share/wine/nls/c_28591.nls
./share/wine/fonts/
./share/wine/fonts/coure.fon
./share/wine/fonts/sserifeg.fon
./share/wine/fonts/ssef1257.fon
./share/wine/fonts/couree.fon
./share/wine/fonts/ssef874.fon
./share/wine/fonts/webdings.ttf
./share/wine/fonts/svgasys.fon
./share/wine/fonts/coue1257.fon
./share/wine/fonts/sserifet.fon
./share/wine/fonts/coureg.fon
./share/wine/fonts/smallet.fon
./share/wine/fonts/sserifft.fon
./share/wine/fonts/jsmalle.fon
./share/wine/fonts/vgasysg.fon
./share/wine/fonts/sserifee.fon
./share/wine/fonts/sserifer.fon
./share/wine/fonts/ssee1256.fon
./share/wine/fonts/ssef1256.fon
./share/wine/fonts/vgasys.fon
./share/wine/fonts/vgasyse.fon
./share/wine/fonts/tahoma.ttf
./share/wine/fonts/vgas1256.fon
./share/wine/fonts/ssef1255.fon
./share/wine/fonts/vgas1257.fon
./share/wine/fonts/courer.fon
./share/wine/fonts/wingding.ttf
./share/wine/fonts/vgas1255.fon
./share/wine/fonts/sserife.fon
./share/wine/fonts/vgas874.fon
./share/wine/fonts/ssee874.fon
./share/wine/fonts/coue1255.fon
./share/wine/fonts/sseriffe.fon
./share/wine/fonts/coue1256.fon
./share/wine/fonts/cvgasys.fon
./share/wine/fonts/ssee1255.fon
./share/wine/fonts/smaller.fon
./share/wine/fonts/jvgafix.fon
./share/wine/fonts/ssee1257.fon
./share/wine/fonts/vgafix.fon
./share/wine/fonts/sseriffr.fon
./share/wine/fonts/smallee.fon
./share/wine/fonts/smalleg.fon
./share/wine/fonts/smalle.fon
./share/wine/fonts/smae1255.fon
./share/wine/fonts/vgasyst.fon
./share/wine/fonts/hvgasys.fon
./share/wine/fonts/smae1256.fon
./share/wine/fonts/sseriffg.fon
./share/wine/fonts/marlett.ttf
./share/wine/fonts/vgasysr.fon
./share/wine/fonts/sseriff.fon
./share/wine/fonts/symbol.ttf
./share/wine/fonts/couret.fon
./share/wine/fonts/smae1257.fon
./share/wine/fonts/tahomabd.ttf
./share/wine/fonts/jvgasys.fon
./share/man/
./share/man/pl.UTF-8/
./share/man/pl.UTF-8/man1/
./share/man/pl.UTF-8/man1/wine.1
./share/man/man1/
./share/man/man1/wine.1
./share/man/de.UTF-8/
./share/man/de.UTF-8/man1/
./share/man/de.UTF-8/man1/wine.1
./share/man/fr.UTF-8/
./share/man/fr.UTF-8/man1/
./share/man/fr.UTF-8/man1/wine.1
./bin/
./bin/wine-preloader
./bin/wine
make uninstall
/build
Removing intermediate container 86e2d73d69c8
---> d4ff917914ec
Successfully built d4ff917914ec
Successfully tagged static-wine32:latest

Hey @wahaha2019, thanks for the input, can you tell me your Docker version? I found some info online it might be releated to an outdated Docker.

Just to let you know I'm working on reverting the base image to Ubuntu 20.04 so it should work with older Docker versions.

No, don't worry for now, it should work with older versions too.

I upgraded Docker to version 24.0.7, build afdd53b according to this link: https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-on-ubuntu-18-04

OK, this is good but let me fix the build errors that came in after I added libiconv. I found out the unpatched Wine itself doesn't support iconv because it uses a version of libxml2 with less features.

Now, I'm building with Docker version 24.0.7, that will take about 13 hours, because the host machine is a low end VPS with only 1G memory.

You can stop the build, it won't work currently :) Let me finish the patch.

OK, I'll wait for your result.

May be this is helpful to you: https://www.winehq.org/pipermail/wine-devel/2019-August/149379.html

This is already in Wine/static-wine32 (https://github.com/MIvanchev/wine/blob/static-dependencies/dlls/msxml3/main.c) but for some reason it doesn't work for you... But it also doesn't matter, it's important that libxml2 is compiled with all features and I'm grateful to you for finding that out!!

@wahaha2019, I committed some changes that I've been working on in the last few days. I reverted the image to Ubuntu 20:04 and added some code to build newer versions of the build tools. It turns out it's very hard to make Wine use a more modern libxml2 so instead, before building, you can download this file to
dependencies/patches before starting the build.

Thank you for the helpful work!
My VPS machine is a low end one, it has only 7.5GB free disk space. How many disk space will a success build need?

I think around ~4GB, last time I checked the static-wine32 Docker image was about 4.2GB.


Synch

Description

#199 incl. #505
#489
#488
#463
#499
#357

Pre-review checks
Please ensure to do as many of the following checks as possible, before asking for committer review:

[x] DEPENDENCIES are up-to-date. Dash license tool. Committers can open IP issues for restricted libs.
[x] Copyright and license header are present on all affected files

https://gitlab.eclipse.org/eclipsefdn/emo-team/iplab/-/issues/14259


Fix error in setting up number of threads

The setup code figures out whether to do anything, based on whether any arguments have changed from the last call.  The threads argument is a little tricky to handle because 0 is reinterpreted as the number of threads available overall.  Therefore, we should compare the number of requested threads last time vs. this time, not the actual number of threads.

Codecov Report

Merging #26 into master will increase coverage by <.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master      #26      +/-   ##
==========================================
+ Coverage   94.72%   94.72%   +<.01%
==========================================
  Files          39       39
  Lines        3506     3507       +1
==========================================
+ Hits         3321     3322       +1
  Misses        185      185

Impacted Files
Coverage Δ

src/helpme.h
97% <100%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2612f77...cdae653. Read the comment docs.


BGI data was not suitable for version 2.0.7

STAR
--genomeDir ./celescope/reference
--readFilesIn read_2.fq.gz read_1.fq.gz
--readFilesCommand zcat
--soloCBwhitelist ./scopeV2.2.1/bclist ./scopeV2.2.1/bclist ./scopeV2.2.1/bclist
--soloCellFilter EmptyDrops_CR 3000 0.99 10 45000 90000 500 0.01 20000 0.001 10000
--outFileNamePrefix .
--runThreadN 10
--clip3pAdapterSeq AAAAAAAAAAAA
--outFilterMatchNmin 50
--soloFeatures Gene GeneFull_Ex50pAS
--outSAMattributes NH HI nM AS CR UR CB UB GX GN
--soloType CB_UMI_Complex
--soloCBposition 0_0_0_7 0_24_0_31 0_48_0_55
--soloUMIposition 0_57_0_68 --soloUMIlen 12
--soloCBmatchWLtype 1MM
--outSAMtype BAM SortedByCoordinate
--soloCellReadStats Standard
--soloBarcodeReadLength 0
When I used 2.0.7 to analysis BGI data (STARsolo step shown up), mapping and counting were finished normally. But, an error  "IndexError: sequence length is not enough in R1 read" raised when exec 'celescope.tools.starsolo.get_Q30_cb_UMI'.
I think it was related to the para 'soloCBposition' and 'soloUMIposition ' (the barcode sequence is 68bp, and the library was built with singleron kit v1), but I'm not sure. So I need your help.
Thank you!

https://github.com/singleron-RD/CeleScope/blob/master/doc/chemistry.md
singleron kit v1 corresponds to chemistry scopeV2.2.1, which requires at least 69bp R1. If you have 68bp R1, the last 1bp UMI will be thrown away.
change -soloUMIposition 0_57_0_68 --soloUMIlen 12 to -soloUMIposition 0_57_0_67 --soloUMIlen 11 should work.

https://github.com/singleron-RD/CeleScope/blob/master/doc/chemistry.md singleron kit v1 corresponds to chemistry scopeV2.2.1, which requires at least 69bp R1. If you have 68bp R1, the last 1bp UMI will be thrown away. change -soloUMIposition 0_57_0_68 --soloUMIlen 12 to -soloUMIposition 0_57_0_67 --soloUMIlen 11 should work.

In multi_rna, use '--chemistry customized --pattern C8L16C8L16C8L1U11T18 --whitelist "./scopeV2.2.1/bclist ./scopeV2.2.1/bclist ./scopeV2.2.1/bclist"

Thanks for your advice! I found the linker in scopeV2.2.1 was "TCGGTGACAGCCATATCGTAGTCAGAAGCTGAC", and the last base "C" was not sequenced in BGI data. So, the "L1" in '--pattern C8L16C8L16C8L1U11T18' was not exact I think.

https://github.com/singleron-RD/CeleScope/blob/master/doc/chemistry.md singleron kit v1 corresponds to chemistry scopeV2.2.1, which requires at least 69bp R1. If you have 68bp R1, the last 1bp UMI will be thrown away. change -soloUMIposition 0_57_0_68 --soloUMIlen 12 to -soloUMIposition 0_57_0_67 --soloUMIlen 11 should work.

In multi_rna, use '--chemistry customized --pattern C8L16C8L16C8L1U11T18 --whitelist "./scopeV2.2.1/bclist ./scopeV2.2.1/bclist ./scopeV2.2.1/bclist"

Thanks for your advice! I found the linker in scopeV2.2.1 was "TCGGTGACAGCCATATCGTAGTCAGAAGCTGAC", and the last base "C" was not sequenced in BGI data. So, the "L1" in '--pattern C8L16C8L16C8L1U11T18' was not exact I think.

if so，just remove the L1 and use U12

该邮件从移动设备发送
--------------原始邮件--------------
发件人："biozhao77 @.>;
发送时间：2024年3月5日(星期二) 晚上9:19
收件人："singleron-RD/CeleScope" @.>;
抄送："周义其 @.>;"Comment @.>;
主题：Re: [singleron-RD/CeleScope] BGI data was not suitable for version 2.0.7 (Issue #276)
https://github.com/singleron-RD/CeleScope/blob/master/doc/chemistry.md singleron kit v1 corresponds to chemistry scopeV2.2.1, which requires at least 69bp R1. If you have 68bp R1, the last 1bp UMI will be thrown away. change -soloUMIposition 0_57_0_68 --soloUMIlen 12 to -soloUMIposition 0_57_0_67 --soloUMIlen 11 should work.
In multi_rna, use '--chemistry customized --pattern C8L16C8L16C8L1U11T18 --whitelist "./scopeV2.2.1/bclist ./scopeV2.2.1/bclist ./scopeV2.2.1/bclist"
Thanks for your advice! I found the linker in scopeV2.2.1 was "TCGGTGACAGCCATATCGTAGTCAGAAGCTGAC", and the last base "C" was not sequenced in BGI data. So, the "L1" in '--pattern C8L16C8L16C8L1U11T18' was not exact I think.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you commented.Message ID: @.***>

if so，just remove the L1 and use U12
…
---------- 该邮件从移动设备发送
--------------原始邮件-------------- 发件人："biozhao77 @.>; 发送时间：2024年3月5日(星期二) 晚上9:19 收件人："singleron-RD/CeleScope" @.>; 抄送："周义其 @.>;"Comment @.>; 主题：Re: [singleron-RD/CeleScope] BGI data was not suitable for version 2.0.7 (Issue #276) ----------------------------------- https://github.com/singleron-RD/CeleScope/blob/master/doc/chemistry.md singleron kit v1 corresponds to chemistry scopeV2.2.1, which requires at least 69bp R1. If you have 68bp R1, the last 1bp UMI will be thrown away. change -soloUMIposition 0_57_0_68 --soloUMIlen 12 to -soloUMIposition 0_57_0_67 --soloUMIlen 11 should work. In multi_rna, use '--chemistry customized --pattern C8L16C8L16C8L1U11T18 --whitelist "./scopeV2.2.1/bclist ./scopeV2.2.1/bclist ./scopeV2.2.1/bclist" Thanks for your advice! I found the linker in scopeV2.2.1 was "TCGGTGACAGCCATATCGTAGTCAGAAGCTGAC", and the last base "C" was not sequenced in BGI data. So, the "L1" in '--pattern C8L16C8L16C8L1U11T18' was not exact I think. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

I'm trying. But I have another question, if the last base "C" will be thrown when L1 was removed? Because I noticed that the linker was 33bp including "C".

The lastest pipeline actually does NOT use the sequence of the linker. It just use the --pattern to locate the barcode segments and check if the barcode segments match the whitelist or not.


Not working (5.5)

I don't understand why cors are not working in my case.
I have installed package, and implemented it as follows:
Kernel.php
 protected $middleware = [
        \Illuminate\Foundation\Http\Middleware\CheckForMaintenanceMode::class,
        \Illuminate\Foundation\Http\Middleware\ValidatePostSize::class,
        \App\Http\Middleware\TrimStrings::class,
        \Illuminate\Foundation\Http\Middleware\ConvertEmptyStringsToNull::class,
        \App\Http\Middleware\TrustProxies::class,
        \Barryvdh\Cors\HandleCors::class,
    ];

    protected $routeMiddleware = [
        'auth' => \Illuminate\Auth\Middleware\Authenticate::class,
        'auth.basic' => \Illuminate\Auth\Middleware\AuthenticateWithBasicAuth::class,
        'bindings' => \Illuminate\Routing\Middleware\SubstituteBindings::class,
        'can' => \Illuminate\Auth\Middleware\Authorize::class,
        'guest' => \App\Http\Middleware\RedirectIfAuthenticated::class,
        'throttle' => \Illuminate\Routing\Middleware\ThrottleRequests::class,
        'cors' => \Barryvdh\Cors\HandleCors::class,
    ];

API routing
Route::group(["middleware" => "cors"], function () {
    Route::resource("entity", "EntityController", ["except" => ["edit", "create"]]);
    Route::resource("instance", "InstanceController", ["except" => ["edit", "create"]]);
});

Config
'supportsCredentials' => false,
    'allowedOrigins' => ['*'],
    'allowedHeaders' => ['*'],
    'allowedMethods' => ['GET, POST, PUT, PATCH, DELETE, OPTIONS, HEAD'],
    'exposedHeaders' => [],
    'maxAge' => 0,

I still get 405 and responses with:
access-control-allow-origin →*
access-control-expose-headers →*
allow →GET, HEAD
cache-control →no-cache, private

Did I miss something ?
Thanks a lot

This is probably related to the same issue I reported #240 - using dev-master as opposed to the latest tag appears to fix this.

try 'allowedMethods' => ['*'] instead 'allowedMethods' => ['GET, POST, PUT, PATCH, DELETE, OPTIONS, HEAD']

Shouldn't your allowedMethods array be ['GET', 'POST', 'PUT', 'PATCH', 'DELETE', 'OPTIONS', 'HEAD'],?

Please check with 0.10.1 and re-open if you still have this.


Add Calculate to math-parser-benchmark-project

It is time to check how well does Calculate performs against the most popular C++ math expressions parsers out there. The math-parser-benchmark-project from the Github user @ArashPartow is the appropiate place to do so.
The objective is to achieve 100% accuracy. A pull request will be opened to polish some of Calculate's internals to achieve it.
The project has also been forked to add Calculate to the tests.

¡To celebrate the end of this year Calculate has been added to the math-parser-benchmark-project finally!
And the results aren't too bad :
Benchmark settings:
  - Expressions File is "bench_expr_all.txt"
  - Reference parser is ExprTk
  - Iterations per expression: 10000
  - Number of expressions: 210
  - Number of excessive failures: 4
  - Release build
  - Compiled with MSVC Version 1916
  - IEEE 754 (IEC 559) is Available
  - 32-bit build

Scores:
  #     Parser                  Type            Points   Score   Failures
  -----------------------------------------------------------------------
  00    ExprTk                  double            2021     100     0
  01    FParser 4.5             double            1564      56     2
  02    atmsp 1.0.4             double            1515      49     4
  03    muparser 2.2.4          double            1499      49     0
  04    TinyExpr                double            1123      41     3
  05    MTParser                double            1077      40     9
  06    MathExpr                double             962      30    12
  07    Calculate 2.1.1rc10     double             884      26     0
  08    Lepton                  double             457      10     4
  09    muparserx               double             243       6     0

Happy new year to everyone! 🎉 🎉 🎉


Not compatible with "endroid/qrcode:2.*"

Change the package version to "endroid/qrcode": "1.9.*" in composer.json
After update get:
"PHP message: PHP Fatal error:  Uncaught Error: Call to undefined method Endroid\QrCode\QrCode::setImageType() in /htdocs/vendor/acelaya/zf2-acqrcode/src/Service/QrCodeService.php:68

This module depends on endroid/qr-code ^1.6
Removing a public method introduces a BC break and should be reserved for major versions.
This should be reported to that library if they meant to fulfill SemVer. Otherwise, I'll try to fix it in a new version.

if u use endroid/qr-code ^1.6 this package will be updated to the latest version - 2.*
use instead 1.6.* or 1.9.* or else to keep 1.* version of endroid/qr-code
version 2.* doesn't compatible with 1.*

That's not correct. ^1.6 is the same as >=1.6 and <2.0.
In fact you are using v1.9, which should be backward compatible.
--
Alejandro Celaya Alastrué
El 16 may. 2017 3:47 p. m., "Siomkin Alexander"<EMAIL_ADDRESS>escribió:

if u use endroid/qr-code ^1.6 this package will be updated to the latest
version - 2.*
use instead 1.6.* or 1.9.* or else to keep 1.* version of endroid/qr-code
version 2.* doesn't compatible with 1.*
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/acelaya/ZF2-AcQrCode/issues/9#issuecomment-301787513,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACl-ZLlxIV78KERWSSLCwkaHaeshxUSwks5r6ajzgaJpZM4NcbcO
.


Archiving MorPhiC data

Kamino closed and cloned this issue to morphic-bio/data-ingestion-development


Generate project files for VS2017

This patch generates VS project files for the default version of Visual Studio that is installed on the build machine. It uses SCons functions to determine what the default version of VS/VC is. This means that SCons 2.5.* will not detect a VS2017 installation and fall back to VS2015. SCons 3.0 alpha generates projects for VS2017 if it is installed. (More precisely, if VS2017 is installed in such a way that it is recognized as the default, which should typically be the case even if both VS2015 and VS2017 are available).
Whereas previously the value of the -j flag was hardcoded to 2, the VS build now uses the same number of threads as the SCons run that generated the solution files.
Another change to the previous behavior is that the path to the batch file that sets up the VS environment is now generated when the SConstruct file is run; previously it queried the value of the environment variable VCInstallDir when building Godot. I don't think this change matters, but it might impact people who have strange VS2015 setups.
I've also refactored the code a bit to avoid the extremely long and repetitive strings in the source and fixed (what I believe was) a minor bug: one of the command lines used & instead of ^& in one place, and I could see no real reason for this difference.

This looks fine, but I wonder if we could move this logic out of SConstruct and put it instead in methods.py? That would help reduce the bloat in the main script (arguably adding more bloat in methods.py, but that's less problematic IMO).

No problem, pushed an updated version.

Actually I was thinking more to more everything after if (env['vsproj']) == "yes": to its own method, like methods.generate_vs_project(env).. If that's possible though, I haven't checked what it would entail.

Ah, I see. I didn't want to change too much of the existing build file but I can try to see how it would work out.

OK, everything (except for an old comment) is in methods.py now.

Removed the comment, as requested in the review.

SCons 2.5.1 generates the Visual Studio 2015 files although 2017 is installed. It is supposed to be just imported successfully. When I open the project though, there's 641 errors of "cannot open source file"s. I see they are not included into the project when Project -> Show all files is hit


Rofi auto focus not work after upgrade to hyprland version 0.37.1

Hyprland Version
 System/Version info
Hyprland, built from branch HEAD at commit c5e28ebcfe00a510922779b2c568cfa52a317445  (props: bump ver 0.37.1).
Date: Sat Mar 16 14:51:49 2024
Tag: v0.37.1

flags: (if any)

System Information:
System name: Linux
Node name: void
Release: 6.6.22_1
Version: #1 SMP PREEMPT_DYNAMIC Fri Mar 15 23:56:15 UTC 2024

GPU information:
01:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 470/480/570/570X/580/580X/590] [1002:67df] (rev e7) (prog-if 00 [VGA controller])

os-release: NAME="Void"
ID="void"
PRETTY_NAME="Void Linux"
HOME_URL="https://voidlinux.org/"
DOCUMENTATION_URL="https://docs.voidlinux.org/"
LOGO="void-logo"
ANSI_COLOR="0;38;2;71;128;97"

DISTRIB_ID="void"

plugins:

Bug or Regression?
Bug
Description
Hey i newly upgraded my hyprland version to 0.37.1 and eveything works perfectly even the steam freeze bug fixed at this version but rofi menu will not auto focus on launch until i hover my cursor on it. here is this video :
https://github.com/hyprwm/Hyprland/assets/89181937/c48025b1-0c48-4ad2-8330-084ae9c3e7f0
os : void linux
i installed all of required packages for hyprland :
hyprland-protocols xorg-server-xwayland xdg-desktop-portal-hyprland xdg-desktop-portal xdg-utils wayland wayland-protocols xdg-desktop-portal-wlr xdg-desktop-portal-gtk
How to reproduce
my hyprland configs :
.dots-hyprland
Crash reports, logs, images, videos
No response

Have you tried the wayland fork of rofi? The fork never had issues on Hyprland for me.
I just tried plain rofi out of curiosity. It doesn't use fractional scale, has blur rendering artifacts at borders and corners and has the focus issue you described. I'd strongly recommend checking out the fork.

won't be fixed because it's xwayland shenanigans

Have you tried the wayland fork of rofi? The fork never had issues on Hyprland for me.
I just tried plain rofi out of curiosity. It doesn't use fractional scale, has blur rendering artifacts at borders and corners and has the focus issue you described. I'd strongly recommend checking out the fork.

I had the same issue and confirm that moving to the Wayland fork of Rofi fixed the issue.

Fixed it with rofi by adding the windowrule
windowrulev2 = stayfocused,class:(Rofi)

Not fixed. Window is focused but not activated for keyboard input. Adding
windowrulev2 = forceinput,class:(Rofi)
brings no change. @vaxerski  any idea?
The rofi wayland fork brings more side effects with waybar. so want to stick with rofi.

hyprland will refuse focus to OR surfaces that don't explicitly allow it. Input sniffing (like in rofi's case) is exploiting Xorg and won't be supported.

Thanks for your feedback.

Thanks. Hit this issue as well after upgrading to latest Hyprland package in arch repo (0.37.1-1, not using aur's git version).
Moved to the lbonn fork, works great, looks slightly better, and feels snappier (I've been aware that it's running on x11 but been too lazy to replace it)
@mylinuxforwork - not sure what waybar side effects you're getting - but maybe it's time to fix those instead? If you're running a Wayland setup, it deserves to have it's launcher native Wayland too :stuck_out_tongue_winking_eye:
(btw thanks @vaxerski - love Hyprland and all the good work you're putting out :metal: )

I moved to Wayland rofi as well and fixed the issues. Could solve all waybar sideeffects in my dotfiles.

If you're using extensions like rofi-emoji, then you might need also to rebuild them if switching to a fork of rofi.
In my case, using rofi-emoji-git did the trick.

Thanks for your feedback.

A quick fix if someone needs it. You can manually set focus to the rofi window after a short delay with hyprctl dispatcher.
Example: bind = [superkey], [key], exec, [rofi script path or command] & sleep 0.2; hyprctl dispatch focuswindow "^(Rofi)"

As I wait for the few X-only menu programs I use to be updated to support Wayland, this does the job of dealing with those Xwayland shenanigans. This is a bit hacky and adds a bit of a delay before the program gets focused, but it's better than having to manually focus it each time. Thanks!
Also note that the delay 0.2 might be changed to be higher if you have a slower drive/the program takes longer to launch on your system.

Instead of creating a delay, you can just use the focuswindow disbatcher:
<rofi_keybind>, focus, window, title:(rofi), floating
I'm also using this window rule: windowrulev2 = stayfocused, class:(Rofi)
This seemed to work for me

Instead of creating a delay, you can just use the focuswindow disbatcher:
<rofi_keybind>, focus, window, title:(rofi), floating
I'm also using this window rule: windowrulev2 = stayfocused, class:(Rofi)
This seemed to work for me

This does not seem to work properly for me without a delay; My guess is that I implemented this improperly (I am still not that used to Hyprland).
What does the line with the bind in question look like in your configuration?

Ok, getting rid of the second keybinding, just use @Atemo-C 's windowrulev2

Hello, I take the liberty of reopening this post just long enough to post a solution that should work for everyone, because the previous ones don't work for me either.
# Se but waiting for rofi to open then trigger focus
$focusRofi = & while [ "$(hyprctl clients | grep "class: Rofi")x" == "x" ]; do continue; done; hyprctl dispatch focuswindow "^(Rofi)"

# Add variable behind each rofi execution
bind = $mainMod, D, exec, rofi -show drun $focusRofi
bind = $mainMod, M, exec, rofi-beats $focusRofi
bind = $mainMod, V, exec, rofi-vm $focusRofi
bind = $mainMod, W, exec, rofi-wpa $focusRofi
bind = $mainMod, B, exec, rofi-bluetooth $focusRofi

Sorry for the English, I'm using a translator.


Feature/implement get profile

Should be simple enough.

Note this should not be merged before #20 !

Tnx, @c-castillo I'm still waiting for andres to finish #3 then it should be easy to add!


Update installation.md

Added information on how to install cloudflared for Linux machines.

Hi @abracchi-tw not sure what conflicts I need to resolve here?


Extract vertex welding and checks from b2PolygonShape::Set into their own function

Currently, when creating a polygon shape via b2PolygonShape::Set, box2d will perform vertex welding for vertices that are closer than 0.5 * b2_linearSlop, and assert that the number of vertices is >= 3, and that edge lengths are > b2_epsilon. There is currently no way for the user code to determine if a set of vertices will be changed and/or asserted on by box2d without duplicating the entire logic in b2PolygonShape::Set. Note that vertex welding might make a polygon invalid that wasn't before.
This is problematic in my case, because my game generates polygons dynamically, so degenerate polygons might occur during normal execution. If they occur, I want my game to know about them and handle the error. I do not want box2d to assert or silently fail.
Silently modifying the polygon might also lead to other issues, such as an independent rendering representation not matching the actual polygon in box2d, because box2d performed vertex welding. To prevent this, one would need to read back the vertices after calling b2PolygonShape::Set from the b2PoygonShape, which seems strange.
In general, I think it would be helpful to make changes and errors of polygon creation visible and obvious to the caller. To avoid having to replicate (and track changes to) the logic inside b2PolygonShape::Set, I propose extracting vertex welding and all checks from b2PolygonShape::Set into their own function, which takes vertices as input and returns the vertices after welding, or an error, if the polygon was degenerate. The caller would then be able to take the modified vertices and know that box2d will not alter them or fail when calling b2PolygonShape::Set, or handle the error on their side, with more context.
For backwards compatibility and to prevent unnecessary work, one could add a parameter like "bool verticesAreGuaranteedToBeValid" to b2PolygonShape::Set, which, if true, would skip the check inside b2PolygonShape::Set. That way, people who are interested in handling the transformations/errors on their side can do so and set this bool to true, and all others set it to false and still just assert, when the check fails in b2PolygonShape::Set.

Duplicate of #671


Automatically invite people to github.com/orgs/finos/people when CLA covered

In order to maximise the amount of people signed on https://github.com/orgs/finos/people , while enforcing CLA signature, metadata-tool can embed the logic of inviting GitHub users as soon as they are covered by CLA.
Code is already implemented on https://github.com/finos/metadata-tool/pull/53 and delivers the command invite-clas-to-finos-org.
Tasks to deliver:

[ ] Email<EMAIL_ADDRESS>to inform about the new feature (ie, as soon as CLA is signed, people automatically get invited to https://github.com/orgs/finos/people , therefore they may receive an email shortly. Anyone can ask<EMAIL_ADDRESS>whether they're covered or not by a FINOS CLA
[ ] Add invite-clas-to-finos-org to the nightly build (and enable API call in code)
[ ] Monitor execution and check pending invitations on https://github.com/orgs/finos/people

Related to #114

Announce email being created by @mcleo-d and @maoo in the following restricted Google Doc ...

https://docs.google.com/document/d/1QRzcDFKRXesFTeLZ6hUUGSKlUI8ff-SnRQv9lwCKCHQ/edit?usp=sharing


[feature] {{ watch_url }} variable should handle URLs with Jinja variables

Version and OS
I'm using the latest commit, https://github.com/dgtlmoon/changedetection.io/commit/8536af0845b74918d1b2439d30ac4d5b3e59ec94
Is your feature request related to a problem? Please describe.
According to https://github.com/dgtlmoon/changedetection.io/wiki/Handling-variables-in-the-watched-URL, I'm able to add Jinja variables to the URL to be monitored. That works fine, but if I use {{ watch_url }} in the notifications, it displays the un-formatted string (as in with the Jinja syntax)
Describe the solution you'd like
{{ watch_url }} contains the URL used in the last change. This can be tricky, as we can't simply parse the Jinja variables again, as they're mostly used for dates (and can return a different value than the URL scraped).
Describe the use-case and give concrete real-world examples
Any monitor that uses Jinja2 templating in the URL and wants to link to the monitored URL would benefit from this.

That works fine, but if I use {{ watch_url }} in the notifications, it displays the un-formatted string (as in with the Jinja syntax)

its not clear if you're talking about the notification "URL" or the notification "body"
can you clarify?

aaah you mean when {{ watch_url }} is a URL that has jinja2 templates

aaah you mean when {{ watch_url }} is a URL that has jinja2 markup

Exactly. I tried only in the notification body, but I imagine the results would be the same?

{{ watch_url }} works in the notification body and URL for a long time now, are you sure you used {{ and not { ?

Hi @dgtlmoon, I just tried it again to be sure and the problem exists. The URL I'm monitoring is https://www.gov.br/planalto/pt-br/acompanhe-o-planalto/agenda-do-presidente-da-republica-lula/agenda-do-presidente-da-republica/{% now 'America/Sao_Paulo', '%Y' %}-{% now 'America/Sao_Paulo', '%m' %}-{% now 'America/Sao_Paulo', '%d' %}. I'm running the notifications on Telegram and my notification is:
Notification Title: Alteração em {{ watch_url }}
Notification Body:
{{ watch_url }}
{{ diff_url }}
---

{{ current_snapshot }}

In both cases, the watch URL is displayed with the jinja2 constructs (i.e., literally https://www.gov.br/planalto/pt-br/acompanhe-o-planalto/agenda-do-presidente-da-republica-lula/agenda-do-presidente-da-republica/{% now 'America/Sao_Paulo', '%Y' %}-{% now 'America/Sao_Paulo', '%m' %}-{% now 'America/Sao_Paulo', '%d' %}). See the following screenshot:

I have the same problem. I would like to have the real URL that had been checked in my notifications.


BluFi handle GATT packet has buffer overflow (IDFGH-3021)

INSTRUCTIONS
In the process of BluFi's network configurationg through ble, there is an overflow when processing the write ATT command at characteristic = 0xFF01. The specific problem is in the btc_blufi_recv_handler function of blufi_prf.c. First ,parse the incoming ATT data data [0] and data [1] as the length of memery which apply from heap. And then copy the memory data starting from data [2] to the applyed memory, causing a buffer overflow
DETAILS

Here, if the fc parameter is 0x10, the overflow part can be entered. Total_len is parsed from data. Here, control data [0] = 1, data [1] = 0, so hdr-> data [0] | (((uint16_t) hdr-> data [1]) << 8) == 1, that is, total_len = 1, that is, aggr_buf allocates 1 byte of heap memory, and then copies aggr_buf, which can eventually overflow N bytes
An attacker could construct GATT payload to cause crash or remote code execution
The GATT payload as follows:

FIX SUGGESTIONG
if(blufi_env.offset + hdr->data_len  - 2 <= blufi_env.total_len){
    memcpy(blufi_env.aggr_buf + blufi_env.offset, hdr->data + 2, hdr->data_len  - 2)
}

any progress？

any progress?

Hi @pokerfacett ,
Thank you for your suggestion, we will fix it as soon as possible.

Hi @WCCWCC ,
Thank you for relay ,if you fix it please put commit link here and i will close this issue

Hi @pokerfacett ，
I have submitted the MR and it will be modified as follows.

@WCCWCC hi，First of all, thank you for submitting the repair. Secondly, considering that ESPIDF has thousands of users and is mounted on tens of millions of devices, the vulnerability will remotely cause code execution and denial of service attacks on the device.In order to maintain the security of the entire ecology for development, we recommend that ESPIDF notify developers through github's security announcement to fix the problem on their own version to achieve risk avoidance and reduction.


Add libglib2

As requested by Synthesis team.

added in https://github.com/neuro-inc/neuro-base-environment/pull/422


Make 'retry' argument of InfluxDB::Client configurable

Currently after #77 retry argument passed to InfluxDB::Client.new is 10. Sure, it is not possible for any influxdb installation. Please, do this argument configurable by resource properties.

Hi @vills, thanks for reporting this.  Should be an easy enhancement to make.

#161 has been merged.


Implement "pause debugger now" API

The target can call duk_debugger_pause() to initiate an immediate debugger pause, enabling implementation of a "break into debugger now" hotkey or similar mechanism.
The intent is that duk_debugger_pause() will pause execution at the exact point where it was called.  I'm not sure if this is true in practice due to the way the interrupt counter works.

If this is called from outside of the debugger message loop, it could reset the interrupt counter so that the pause gets noticed right away.
If this can be called from inside the debugger message loop (hopefully not!) then that should hopefully not be necessary at all, but care would have to be taken that we don't change the interrupt counter if it has been already primed for the next execution segment. I don't think this is the case; the final interrupt counter value is selected after the debugger message loop exits.

I don't envision this being callable from the debugger message loop, no.  The intent is that a running target would call this when a certain key is pressed to allow the user to break out of e.g. infinite loops.
Concretely, what would have to be done to implement the interrupt counter reset?

If we're outside the debugger code, it's basically this with ctr == 1:
https://github.com/svaarala/duktape/blob/master/src/duk_js_executor.c#L1764-L1766

So, like this then:
DUK_EXTERNAL void duk_debugger_pause(duk_context *ctx) {
	duk_hthread *thr = (duk_hthread *) ctx;

	if (DUK_HEAP_IS_DEBUGGER_ATTACHED(thr->heap)) {
		DUK_HEAP_SET_PAUSED(thr->heap);
		thr->interrupt_init = 1;
		thr->interrupt_counter = 0;
	}
}

Incidentally, is it safe to call DUK_HEAP_SET_PAUSED() while already paused, or should I check for that first?

Right now it is: https://github.com/svaarala/duktape/blob/master/src/duk_heap.h#L277-L281.

I implemented that interrupt counter thing, and added a note to the top of the function that it can't safely be called from inside the debugger.

That could maybe be just an assert, e.g. DUK_ASSERT(thr->heap->dbg_processing == 0). This won't be the strongest possible assert but enough to catch most abuses.

Does this need any documentation other than website changes?

I took a stab at adding a website page, let me know if I did it right. :)

Usually I try to have all of the following in pulls if possible:

Testcases
Website changes e.g. guide mentions / links for high level feature changes
If the feature needs or benefits from Wiki changes, ensure there's a Wiki issue/pull (doesn't have to be completed right away or in sync with the pull)
Website API documentation changes
Releases entry

Looking at the diff quickly looked good to me, I'll take a closer look tomorrow.

I didn't add a releases entry, I'll do that tonight.

I'll fill in whatever's missing but much appreciated if you go through the motions :)

Okay, I added the feature to RELEASES.rst.

Alright, I added a note on the new function to the Debug API section of debugger.rst.  This is merge-ready pending nitpicks. :)

It might be worth emphasizing in the documentation that no calls into Duktape API are thread safe, so that it's not safe to call duk_debugger_pause() unless: (a) there is no call into Duktape in progress, or (b) the duk_debugger_pause() call comes from the same thread that is currently running.
Although this is the case for all API calls, it may be especially tempting with this API call I think.

I agree.  The obvious use case is for debugging infinite loops, and it'd be very tempting to try to initiate the pause from another thread.  Where do you want me to add the note?

I guess the API document (website/api/duk_debugger_pause.yaml) would be the logical place.
You can use a
...
 (see https://github.com/svaarala/duktape/blob/master/website/api/duk_to_pointer.yaml#L23-L29) if that feels appropriate, it'll look like this: http://duktape.org/api.html#duk_to_pointer. The NOTE: text is added automatically.

As such this calls would in 99.99% cases work correctly from another thread, but it wouldn't be 100% without some form of locking.
It'd actually be quite easy to make this call thread sage: duk_debugger_pause() would be changed to just write a volatile "please pause" flag to 1, and let the executor interrupt read the flag and do the rest when it notices the flag is != 0 (and then reset the flag to 0). The downside is that the interrupt wouldn't be immediate, as updating the interrupt counters concurrently is definitely not safe because it's not atomic (and they're also not declared volatile).
But it'd be a bit odd for one API call to have threading support while others didn't.

Another API call which would be useful is to force throw an uncatchable error like the script timeout. It'd be useful to be able to call that from another thread too. I'll make a note of these two calls on the "relax threading" work list :)

For future work it would actually be useful if Duktape had some way for the user to plug in their own locking primitives, through preprocessor magic perhaps, which Duktape could then use to ensure synchronization.  Then it wouldn't matter if the underlying threading framework were pthreads, Win32, or whatever.

Yeah, adding those macros would be easy but figuring out all the places where they're needed (each individual API call?) is not so easy :) An explicit duk_lock() and duk_unlock() would be an easy solution but awkward to use.

Okay, note added to both the API entry as well as debugger.rst.

I don't mind so much the core API calls being non-thread-safe (since JS itself is single-threaded), but the debugger is one case where running it on a separate native thread would be useful.  Anyway we can figure that out later. :)

Hmm, one sequence what needs a bit of checking is where:

The user requests a pause
The debugger gets detached before the pause can take effect (either user requests the detach or it's caused by another trigger)

I think this should work out fine, because we don't check for paused state or debugger handling in general, if the debugger is not attached. So, any changes to dbg_paused wouldn't matter.
Also a corner case: what happens when this API call happens when detach1() has executed by we haven't detached yet fully. This case might happen e.g. when the user calls duk_debugger_notify() which causes debugger protocol writes which trigger detach1() but we're still outside the debugger message loop. Then, before we get to finish the detach in detach2(), the user calls duk_debugger_pause().

You're right - the user might request a pause while an duk_debugger_cooperate() event loop is running and would expect it to pause the next time some JS code is run.  So the current behavior is okay.
As for the detach handling, the call should be safe even in that case - The only state changes as far as I know are heap->dbg_paused = 1 and setting the interrupt counter to trigger on the next opcode.  That shouldn't interfere with detach handling, should it?

Logically no, but the only way to be sure is to read the related code sections. I can do that before merging.

Changes made in the last push:

Remove the "immediate" references and clarify what happens if a native call is in progress
Move the thread safety note into the main text of the API entry
Add a debug print when duk_debugger_pause() is called
Fix spacing in documentation (use 2 spaces after a period)

Travis must be having difficulties, the test builds never seem to start...

Okay, tests passed.  This looks good from my end, just waiting on a final review.

So I was getting mixed signals - do you want me to remove the assert?

I guess the only thing that could possibly go wrong with pausing from a debugger loop is the interrupt counter manipulation.  I think interrupt handling is already fully completed by the time the debugger message loop is entered.
Come to think of it, would a debugger pause even work during Eval?  Message processing can't be entered into recursively (barring bugs), so assume everything is working properly the call should have no effect.

I'm going to go out for a walk, I'll do some in-house testing on this when I get back to try various scenarios.  That might help us decide what to do about that assert. :-)

Actually the interrupt counter is reset when we come out of the debugger message loop. So here's what I think would happen if duk_debugger_pause() was called from the message loop -- regardless of whether that message loop is executing because we're paused or because we peeked and noticed an inbound Eval while running:

The executor enters the debugger message loop which (for argument's sake) reads and handles an Eval.
The Eval code triggers a call to duk_debugger_pause(). This sets dbg_paused=1 and dbg_state_dirty=1.
We then exit the "handle one message" call and resend Status based on the dirty flag: https://github.com/svaarala/duktape/blob/master/src/duk_debugger.c#L2097
On the next message loop we'd notice we're paused and want to read in blocking mode, regardless of why we originally entered the loop: https://github.com/svaarala/duktape/blob/master/src/duk_debugger.c#L2091-L2093
We can only exit the loop via a Resume or a step command.
We end up returning to the executor interrupt and initializing the interrupt counter for the next segment of opcode execution: https://github.com/svaarala/duktape/blob/master/src/duk_js_executor.c#L1754-L1767
The "immediate" flag controls what value is ultimately used; immediate is set by the debugger integration helper e.g. when we're stepping.

So based on this I think it should work correctly within the current handling structure.

So I think that scenario should be tested in practice, and I think the duk_debugger_pause() code block which writes the interrupt counter etc should have a comment that this works also in the message loop because the interrupt counter is reinitialized (overwritten) before returning to the executor.

Just to make sure, it won't actually pause DURING the Eval, right?  As we've seen in the past, bad things happen when an Eval is halted. :)

No, it will just flag to we want to remain paused and this affects the current message loop as it needs to.

One solid test for this would be:

Implement an AppRequest "CallPause" which just unconditionally causes duk_debugger_pause().
Execute an infinite loop in Ecmascript with debugger attached and running.
While Duktape is in running state, execute the "CallPause" command.
Duktape's periodic transport peek would cause that call to execute with the effect that we should remain paused and responsive in the message loop, just as if we paused via an explicit request.
From this state, the critical tests are to see that (1) Resume works, and (2) e.g. StepInto works. These two exercise the two different counter reinitialization alternatives.

A little bit better test would be to do the same as above but:

Implement a native callPause() Duktape/C binding which calls duk_debugger_pause().
In the same setup as above (Duktape running, infinite loop) send an Eval for "callPause()".
See that the Eval runs and exits, and then stops in the message loop for further commands.

Btw, I can do one or both tests before the merge (with the assert removed), but I wouldn't want to merge the change until it's tested in practice.
But assuming it works the way I described above, the changes needed for this pull would be:

Remove the assert.
Add a comment to duk_debugger_pause() that it's safe to call both from outside and inside the debugger message loop. The message loop will correctly react to getting paused e.g. during Eval or AppRequest, and the executor interrupt will rewrite the counter when we've exited the message loop and are about to resume execution.
Ensure documentation is in line with this. I think both debugger.rst and the API document are fine as is, because there are no surprising limitations with the assert removed.

I set up a test harness using minisphere and SSJ to test this.  It was actually more difficult than originally anticipated because SSJ is designed to lock the user out of the command line while the target is running.  I had to refactor it a bit to show the command prompt while not paused.
Here's the test setup:
https://github.com/fatcerberus/minisphere/blob/duktape-pause-harness/src/ssj/inferior.c#L248-L259
https://github.com/fatcerberus/minisphere/blob/duktape-pause-harness/src/minisphere/debugger.c#L299-L301
https://github.com/fatcerberus/minisphere/blob/duktape-pause-harness/src/minisphere/api.c#L844-L849
Test 1: While target is running, enter pause at SSJ command line.  This triggers an AppRequest which ultimately calls duk_debugger_pause() on the target.
Result: Target pauses.  Stepping commands work correctly, and continue causes execution to resume as normal.
Test 2: While target is running, enter e DebugPause().  This calls a Duktape/C function which calls duk_debugger_pause().
Result: Target pauses. Stepping commands work correctly.  continue resumes normal execution.
So yes, it looks like the tests checked out and the assert can be removed. :)

Ok, sounds good :)

I removed that assert, wrote a comment explaining why the interrupt counter manipulation is safe, and made some edits to the documentation to clarify some things.  In particular: I made the thread safety notice a footnote again (as it's simply a reminder of the contract already in place), and added mentions that, if a native call is in progress, execution will halt once -EITHER- the native call returns -OR- it calls back into Duktape.
I also gave the new code a quick review and fixed a few nits I noticed.
Let me know if you want me to do anything else.  If you want to run those two tests yourself that's okay by me, otherwise this is merge-ready.

Looks good to me, except clarifying when we'll become paused: the current text was a bit misleading.
I'll take your word for the tests, thanks for going through them :)

How's this?  If that's good I'll squash the fixup commit before merge.

Looks good to me!

Okay, squashed.  Thanks for the help with this, it's not always obvious what the various corner cases are.
It seems like the simplest changes in theory end up being the most difficult to get right.

Yeah, anything to do with the execution state needs careful thought because there are natural dependencies between a lot of different mechanisms and states. It's not always obvious what's easy and what's hard.
For example, adding a debugger command to "resume until expression is true", where an expression would be Eval'd before every opcode executed, would have less corner cases than this API call, and would be pretty easy to implement :)

I should be done with pulls for 1.5.x - this makes everything I need for minisphere 3.0.  I'm angling for a March 28 release, by the way. ;-)

That sounds possible for Duktape 1.5.0 release too.


Netperf version 1.5 TODO: Automatic Provisioning of Azure VMs

We need to figure out a solution to automatically deploy/provision the Azure VMs on demand. We might be able to do this easily enough by having a 'setup job/step' that uses the Azure CLI to create the VMs and then run the provisioning script on them. This script would start add the VM to the desired pool (for a single job). Then all subsequent jobs could leverage the pool.

Windows work reliably well.
Linux is WIP.


Add BlockFace#fromDirection

Add a convenience method to BlockFace that simply helps retrieving a BlockFace from the corresponding Direction (Direction.UP -> BlockFace.TOP, Direction.EAST -> BlockFace.EAST, etc)

Cherry picked in https://github.com/Minestom/Minestom/commit/82725b5e325088a67c75810b4bb5ced76a67de9d


Small improvements to Readme

Thank you for your kindly PR!
I will need some time to reorganize the TODO list in a better way - probably as a issues...
Feel free to use the open a new PR directly if you have some other tip like this, and don't forget the discussions too for any question about the project.
Luca


chore(ui): Fix errors and turn on no-explicit-any lint rule

Description
We disabled some rules to limit the number of changes in #8629

To expedite turning on this rule, I added disable comments where I am not sure about types:

handler argument in DuplicatePolicyForm
targetProps?.datum?.xName in ViolationsByPolicyCategoryChart

TypeScript errors related to ClientPolicy and Policy were confusing.
Although I think some of the type declarations are incorrect, I added as unknown idiom which we can find in the future to sort it out.

yarn build displays only the first error.
yarn start displays some errors in its overlay.
Visual Studio Code displays errors only for open files. The delayed cause and effect between errors felt like dominos falling in slow motion.

Checklist

[ ] Investigated and inspected CI test results
[x] ~Unit test and regression tests added~

Testing Performed

yarn lint in ui/apps/platform
yarn build in ui
yarn start in ui

Because same clusters test failed for all 3 ui-e2e-tests, I rewrote newCluster conditional that I apparently broke.
That was the only change that affected compiled code, so confirms decision to change only types elsewhere.

ocp-4-11-ui-e2e-tests failure not related to changes:

Timed out retrying after 10000ms: cy.wait() timed out waiting 10000ms for the 1st request to the route: mypermissions. No request ever occurred.

/test ocp-4-11-ui-e2e-tests


Linker Error in Project

clang: error: linker command failed with exit code 1 (use -v to see invocation)

What are you running to get this error?

ping

ping

Closing for lack of activity.
If you can give me more info about how to get this error, I'll be happy to reopen it! :smile:

hey this is not an error.... i got the solution for that...... when we add libraries we don't open it directly....  we want to open the workspace file... Then it works like charm... :) :+1:

Please reopen, I have the same problem, but I don't know where


Add exception type that represents an AMQP "exception".

AMQP indicates "exceptions" via the closure of the channel, we need an exception that represents this condition.
DeclareException, ResourceLockedException and ResourceNotFoundException are all specific cases of this exception that are likely to be useful to the library user, and as such should be subclasses of this new exception.

As discussed, this can be AmqpException, it'll have a create() method that takes the code and text from a ChannelCloseFrame or ConnectionCloseFrame.
I've changed my mind about the other 3 classes being subclasses, because it depends on which public API method you're calling as to the exact meaning of some of the codes.

BTW, if you do a search for new RuntimeException in the code, most of those places will need to instead throw one of these exceptions.


Feature/parallel ndt build

Added multi-threaded NDT voxel grid build

@SakodaShintaro There are several issue with this branch so I close this PR temporarily.


fbctf hintUrl generation

Added option for hintUrl generation with fbctf.
Hint will be added to hint the hint description if choosen.
Hint and hintUrl costs are added

Coverage increased (+0.06%) to 97.115% when pulling b0d2c7959ed5893c4adf0f495fc04af20f205f86 on fbctf-enhancements into 7accf81b251953f6404ec62689346f2d4b1ae18c on develop.

That is a brilliant solution! :+1:


Node.js v18 image lacks ESM bundles in @aws-sdk

I was hoping that the Node.js v18 image would allow me to skip packaging @aws-sdk, but it seems the packages provided in the image are stripped and only contain the dist-cjs bundles but not the dist-es bundles. I went all out on ES modules and this is kind of a bummer.
> docker run --rm -it --entrypoint /bin/ls amazon/aws-lambda-nodejs:18 /var/runtime/node_modules/@aws-sdk/client-s3 -l
total 132
-rw-r--r--   1 root root 100375 Feb  6 09:55 CHANGELOG.md
drwxr-xr-x   7 root root   4096 Feb 28 14:21 dist-cjs
-rw-r--r--   1 root root  11390 Feb  6 09:55 LICENSE
-rw-r--r--   1 root root   4295 Feb  6 09:55 package.json
-rw-r--r--   1 root root   5780 Feb  6 09:55 README.md

> npm pack<EMAIL_ADDRESS>--dry-run 2>&1 | sed -n "/Tarball Contents/,/Tarball Details/{p}" | grep -v Tarball | cut -c20- | cut -d/ -f1 | sort | uniq
dist-cjs
dist-es
dist-types
CHANGELOG.md
LICENSE
package.json
README.md

@elieux Slightly relevant question, but where you able to get the entrypoint to work correctly? Say your handler is index.mjs, how do you set the Dockerfile CMD?
If I use CMD ["index.handler"], it is still looking up index.js and not index.mjs, even if the package type is set to module.

AWS SDK v3 is designed to be used from ES Modules, independent of its internal implementation. In other words, dist-cjs folder is sufficient for it to be imported as an ESM:
Dockerfile:
FROM public.ecr.aws/lambda/nodejs:18

COPY index.mjs /var/task/

CMD ["index.handler"]

index.mjs:
import { S3Client } from "@aws-sdk/client-s3";

const s3 = new S3Client({ region: process.env.AWS_REGION });

export const handler = async (event) => {
  const response = {
    statusCode: 200,
    body: JSON.stringify({
      "typeof S3Client": typeof S3Client,
      "config.serviceId": s3.config.serviceId,
    }),
  };
  return response;
};

Result:
{
  "statusCode": 200,
  "body": "{\"typeof S3Client\":\"function\",\"config.serviceId\":\"S3\"}"
}

@paambaati, I haven't tried in Docker, but in Lambda it just works. The runtime code looks for different files based on package.json and a predefined order. I hope it's okay to paste the code here.
    async function _tryRequire(appRoot, moduleRoot, module2) {
      verbose("Try loading as commonjs: ", module2, " with paths: ,", appRoot, moduleRoot);
      const lambdaStylePath = path.resolve(appRoot, moduleRoot, module2);
      const extensionless = _tryRequireFile(lambdaStylePath);
      if (extensionless) {
        return extensionless;
      }
      const pjHasModule = _hasPackageJsonTypeModule(lambdaStylePath);
      if (!pjHasModule) {
        const loaded2 = _tryRequireFile(lambdaStylePath, ".js");
        if (loaded2) {
          return loaded2;
        }
      }
      const loaded = pjHasModule && await _tryAwaitImport(lambdaStylePath, ".js") || await _tryAwaitImport(lambdaStylePath, ".mjs") || _tryRequireFile(lambdaStylePath, ".cjs");
      if (loaded) {
        return loaded;
      }
      verbose("Try loading as commonjs: ", module2, " with path(s): ", appRoot, moduleRoot);
      const nodeStylePath = __require.resolve(module2, {
        paths: [appRoot, moduleRoot]
      });
      return __require(nodeStylePath);
    }

@krk, thanks. I'd tried it and it didn't work. I'll check again and report back.

This is an issue depending on what you are importing, regardless of whether you're using CommonJS or ESModule. The issue is that, for some reason, not all the exported types are resolving. Here's a quick example to illustrate...
import { S3Client } from "@aws-sdk/client-s3";

import pkg from '@aws-sdk/client-s3'

const s3 = new S3Client({});

export const handler = async(event) => {
    console.log(`MetadataDirective = ${pkg.MetadataDirective}`);
    if (pkg) console.log(`MetadataDirective.REPLACE = ${pkg.MetadataDirective.REPLACE}`);
};

In this example, S3Client has no problem resolving as a named export. If I try to include MetadataDirective as a named export, I get an error stating Named export 'MetadataDirective' not found and recommending that I use the default export. The above code then uses the default export, but MetadataDirective is undefined. This even though this is exported in models_0.* in the client-s3 package.
Fallback is simply to include the SDKs as dependencies in my Lambda which inflates the size, but works. Would be helpful to understand which exports we can count on being present so we can make informed decisions on devDependencies.

Hmm. It seems to work now, which is nice.
For future reference, this is the error I was getting previously:
{
    "errorType": "Error",
    "errorMessage": "Cannot find package '@aws-sdk/client-lambda' imported from /var/task/XXX.mjs",
    "code": "ERR_MODULE_NOT_FOUND",
    "stack": [
        "Error [ERR_MODULE_NOT_FOUND]: Cannot find package '@aws-sdk/client-lambda' imported from /var/task/XXX.mjs",
        "    at new NodeError (node:internal/errors:400:5)",
        "    at packageResolve (node:internal/modules/esm/resolve:894:9)",
        "    at moduleResolve (node:internal/modules/esm/resolve:987:20)",
        "    at moduleResolveWithNodePath (node:internal/modules/esm/resolve:938:12)",
        "    at defaultResolve (node:internal/modules/esm/resolve:1202:79)",
        "    at nextResolve (node:internal/modules/esm/loader:163:28)",
        "    at ESMLoader.resolve (node:internal/modules/esm/loader:842:30)",
        "    at ESMLoader.getModuleJob (node:internal/modules/esm/loader:424:18)",
        "    at ModuleWrap.<anonymous> (node:internal/modules/esm/module_job:77:40)",
        "    at link (node:internal/modules/esm/module_job:76:36)"
    ]
}

The bundles SDK is an older version which doesn't export (all) types, I ran into the same issue. The "solution" is to pack the relevant AWS SDKs yourself.


Degree-based orderings, take 2

This PR implements three additional orders from the ColPack paper: SmallestLast, IncidenceDegree, DynamicLargestFirst (new version of #18).
It builds upon #123 for clean graph structures, so #123 needs to be reviewed first.

[x] Implement degree-based orders for AdjacencyGraph and BipartiteGraph.
[x] Test that they return plausible results.
[ ] Test that they return correct results (not sure how to do that).
[ ] Optimize performance (maybe best left for a future PR).

The big question is where to find small examples to check that we return the correct orders. This is especially tricky because ties are broken in an arbitrary manner.

You also implemented DynamicDegreeBasedOrder, right?

You also implemented DynamicDegreeBasedOrder, right?

Yeah, basically these three new orders are variants of the same underlying paradigm, which is DynamicDegreeBasedOrder. So we don't need three different implems, just a single one with a few toggles here and there

gdalle, what is SaturationDegree in #15?
Is it the missing combination forward + up?

No, it's a very different one. I just re-read the section of the ColPack paper and I think we can skip it?

Saturation Degree ordering, due to Brélaz [1979], is another known ordering technique for greedy coloring. It can be best viewed in contrast to the ID ordering technique: in a coloring algorithm that uses an ID ordering, the ith vertex is a vertex with the largest number of already colored adjacent vertices, whereas in an algorithm that uses an SD ordering it is a vertex that has the largest number of distinctly colored adjacent vertices. Clearly, the characterization of an SD ordering cannot be separated from the rest of the coloring algorithm, and the ordering is nonamenable for a linear-time implementation in the size of the graph. For these reasons we chose not to include it in ColPack. Turner [1988] has shown that using careful choice of data structures, a coloring algorithm that in effect employs an SD ordering can be implemented such that its complexity is O(|E|log|V|). We have implemented such an algorithm and we will report results on it in the experiments discussed in Section 8.

@amontoison friendly bump, do you want us to try and merge this one?

Following our chat with @amontoison, I'm merging this with clear warnings that the new orders are experimental and need further tests. That way we can start experimenting.