unable to connect to index.crates.io

Problem
When attempting to download crates, I get a warning: spurious network error (3 tries remaining): [35] SSL connect error (Recv failure: Connection was reset). After the next tries it causes the install to fail with:
Caused by: [35] SSL connect error (Recv failure: Connection was reset)
This started happening on my new windows 11 machine.
Steps
Clean installation on a new windows 11 system, all updates installed. Attempting to install any crate fails.
Possible Solution(s)
No response
Notes
The toolchain used makes no difference. Already tried setting these options in the global config.toml, based on some previous issues:
[source.crates-io]
index = "https://github.com/rust-lang/crates.io-index" # I set the source to the official github index
[http]
proxy = "" # leaving this as a blank string resets proxy to default
http.check-revoke = false
#check-revoke = false
None of which worked. I am not using Windows defender. I am using Eset antivirus as I was on my previous computer, where I didn't have this problem.
Version
cargo 1.78.0-nightly (fc1d58fd0 2024-02-09)
release: 1.78.0-nightly
commit-hash: fc1d58fd0531a57a6b942a14cdcdbcb82ece16f3
commit-date: 2024-02-09
host: x86_64-pc-windows-msvc
libgit2: 1.7.2 (sys:0.18.2 vendored)
libcurl: 8.6.0-DEV (sys:0.4.71+curl-8.6.0 vendored ssl:Schannel)
os: Windows 10.0.22635 (Windows 11 Core) [64-bit]

Got the same issue but when I build an image with docker.
The update of index failed due to the same error.
Cannot build Rust App Image anymore ... :(
Any help welcome!
Rgds,

seems to be an similar issue to this one #7096 i guess

Not at all!
I am on Linux and it is a DNS issue when building an image
with a Rust App. On my host, everything works fine!
It seems that in the Dockerfile, when cargo build is triggered, cargo update
execute first to update the index with crates.io, and it is here I got the issue.
Even more, the initial image is downloaded successfully!
Something is wrong with index.crates.io and their DNS ...

For me it turns out that this is caused by curl and not by cargo. Don't know how, but sure.

For me on Linux
$> curl https://crates.io
=> WORKS!
But:
$> curl https://index.crates.io
<Error>NoSuchKey<Message>The specified key does not exist.</Message><Key>index.html</Key><RequestId>EFEJM9PM6CVJRN9V</RequestId><HostId>iXnil6CJiEMRL4+scgXj AgNJyvrloC3rdfJNc7PU1nzBtflcfKlRSGB5/oDisRBT8N1T7BnNPEM=</HostId></Error>%
is NOT working...

For the folks with the SSL error, can you run curl with verbose output? That might reveal more details about the kind of SSL error:
curl -Iv https://index.crates.io/config.json

@jdno  You are right about the curl issue; indeed, my DNS issue is still open
and unfortunately not fixed. Something is wrong with the Dockerfile and cargo
build...

I located this to the issue being the curl connection, however this only happens on crates.io. It works for every other website.

Sure, here is the output. I'll preface it with saying that using -H"Pragma: no-cache" doesn't really help. The operation succeeds like 1 in 10 or 20.
The output from a failure:

Trying [2600:9000:25e8:a800:1f:a9f5:69c0:93a1]:443...
Connected to index.crates.io (2600:9000:25e8:a800:1f:a9f5:69c0:93a1) port 443
schannel: disabled automatic use of client certificate
ALPN: curl offers http/1.1
Recv failure: Connection was reset
schannel: failed to receive handshake, SSL/TLS connection failed
Closing connection
schannel: shutting down SSL/TLS connection with index.crates.io port 443
Send failure: Connection was reset
schannel: failed to send close msg: Failed sending data to the peer (bytes written: -1)
curl: (35) Recv failure: Connection was reset

The output from a success:

Trying [2600:9000:25e8:a800:1f:a9f5:69c0:93a1]:443...
Connected to index.crates.io (2600:9000:25e8:a800:1f:a9f5:69c0:93a1) port 443
schannel: disabled automatic use of client certificate
ALPN: curl offers http/1.1
Recv failure: Connection was reset
schannel: failed to receive handshake, SSL/TLS connection failed
Closing connection
schannel: shutting down SSL/TLS connection with index.crates.io port 443
Send failure: Connection was reset
schannel: failed to send close msg: Failed sending data to the peer (bytes written: -1)
curl: (35) Recv failure: Connection was reset

C:\Users\rysza>curl -Iv https://index.crates.io/config.json

Trying [2600:9000:25e8:a800:1f:a9f5:69c0:93a1]:443...
Trying <IP_ADDRESS>:443...
Connected to index.crates.io (<IP_ADDRESS>) port 443
schannel: disabled automatic use of client certificate
ALPN: curl offers http/1.1
ALPN: server accepted http/1.1
using HTTP/1.1

HEAD /config.json HTTP/1.1
Host: index.crates.io
User-Agent: curl/8.4.0
Accept: /

schannel: remote party requests renegotiation
schannel: renegotiating SSL/TLS connection
schannel: SSL/TLS connection renegotiated
< HTTP/1.1 200 OK
HTTP/1.1 200 OK
< Content-Type: application/json
Content-Type: application/json
< Content-Length: 76
Content-Length: 76
< Connection: keep-alive
Connection: keep-alive
< Last-Modified: Mon, 23 May 2022 19:23:53 GMT
Last-Modified: Mon, 23 May 2022 19:23:53 GMT
< x-amz-version-id: OLJqOQ3bTJ8rGepGZwmcEkt2gR_Fm1bY
x-amz-version-id: OLJqOQ3bTJ8rGepGZwmcEkt2gR_Fm1bY
< Accept-Ranges: bytes
Accept-Ranges: bytes
< Server: AmazonS3
Server: AmazonS3
< Date: Wed, 21 Feb 2024 19:38:52 GMT
Date: Wed, 21 Feb 2024 19:38:52 GMT
< ETag: "7a39851bdb021fcef53172317860f970"
ETag: "7a39851bdb021fcef53172317860f970"
< Vary: Accept-Encoding
Vary: Accept-Encoding
< X-Cache: Hit from cloudfront
X-Cache: Hit from cloudfront
< Via: 1.1 cf82d48fdf484813132bbd9c90904672.cloudfront.net (CloudFront)
Via: 1.1 cf82d48fdf484813132bbd9c90904672.cloudfront.net (CloudFront)
< X-Amz-Cf-Pop: CDG52-P2
X-Amz-Cf-Pop: CDG52-P2
< X-Amz-Cf-Id: 2VaixxGo5GWVsJaReNE98OmQGBgMVzxMxHuhwUmjeXXkArZejCXOsw==
X-Amz-Cf-Id: 2VaixxGo5GWVsJaReNE98OmQGBgMVzxMxHuhwUmjeXXkArZejCXOsw==
< Age: 2222
Age: 2222

<

Connection #0 to host index.crates.io left intact

I can confirm R-Gdoc's output (https://github.com/rust-lang/cargo/issues/13457#issuecomment-1957834320)
Running on Ubuntu 22.04, curl 7.81, openssl 3.x
Oddly, if I use the -Iv keys it nearly always works. If I drop the -I and just use -v it nearly  fails about 50% of the time.
Success
$ curl -v https://index.crates.io/config.json
*   Trying <IP_ADDRESS>:443...
* Connected to index.crates.io (<IP_ADDRESS>) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.0 (OUT), TLS header, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS header, Finished (20):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.2 (OUT), TLS header, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=crates.io
*  start date: Dec 26 00:00:00 2023 GMT
*  expire date: Jan 23 23:59:59 2025 GMT
*  subjectAltName: host "index.crates.io" matched cert's "index.crates.io"
*  issuer: C=US; O=Amazon; CN=Amazon RSA 2048 M02
*  SSL certificate verify ok.
* Using HTTP2, server supports multiplexing
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* Using Stream ID: 1 (easy handle 0x561d774f1a60)
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
> GET /config.json HTTP/2
> Host: index.crates.io
> user-agent: curl/7.81.0
> accept: */*
>
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* Connection state changed (MAX_CONCURRENT_STREAMS == 128)!

* Connected to index.crates.io (<IP_ADDRESS>) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.0 (OUT), TLS header, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* OpenSSL SSL_connect: Connection reset by peer in connection to index.crates.io:443
* Closing connection 0
* TLSv1.0 (OUT), TLS header, Unknown (21):
* TLSv1.3 (OUT), TLS alert, decode error (562):
curl: (35) OpenSSL SSL_connect: Connection reset by peer in connection to index.crates.io:443

Failure for me is happening on a simple cargo build from the command line of a mostly empty (new) repo - just two dependencies.

A possible workaround to this problem is disabling the sparse checkout of the registry by adding
[registries.crates-io]
protocol = "git"

to your .cargo/config.toml. The massive downside is needing to fetch the Repo containing the whole registry. As a side note opening https://index.crates.io/config.json in a browser isn't failing at all.

Im using Windows. For me curl 8.4 on Win seems to work if I specify -4. Specifying -6 makes it fail.
`*   Trying [2600:9000:2670:a600:1f:a9f5:69c0:93a1]:443...

Connected to index.crates.io (2600:9000:2670:a600:1f:a9f5:69c0:93a1) port 443
schannel: disabled automatic use of client certificate
ALPN: curl offers http/1.0
Send failure: Connection was reset
schannel: failed to send initial handshake data: sent -1 of 201 bytes
Closing connection
schannel: shutting down SSL/TLS connection with index.crates.io port 443
Send failure: Connection was reset
schannel: failed to send close msg: Failed sending data to the peer (bytes written: -1)`
Looking at it with wireshark, the handshake completes, but then the server immediately sends a RST Frame. The Client doesn't get the opportunity to send the tls client hello.

WSL curl 7.68.0 -4 works. v6 isn't configured for v6 so cargo defaults to ipv4 so it works. For me the workaround is to disable ipv6 in my network adapter, at least when I need cargo to run. Is there an option to force cargo to v4?
At least for @R-Goc the failure is v6 and the success v4 as well. I don't think the other errors are of the same cause as they show a different error. @kdopen is able to send a Client Hello as far as I can tell.

If someone is still going through this issue on Windows, please try by adding registries.crates-io.protocol = "git" to /.cargo/config.toml. It worked for me.

If someone is still going through this issue on Windows, please try by adding registries.crates-io.protocol = "git" to /.cargo/config.toml. It worked for me.

I tried that before I found the workaround for me with ipv4 only. Getting the index worked. Downloading the crates didn't work for me.

How would you set cargo to use ipv4?

a (clearly suboptimal) workaround is to make cargo use git for the protocol, and git-cli for its git work .. the CLI can be whatever you have installed and it usually uses ipv4 instead of ipv6 .. so ..
cargo-net.git-fetch-with-cli = true
registries.crates-io.protocol = "git"

in cargo.toml should be enough for that ..
now if whatever git-cli you have is still doing git via ipv6, you could configure it with something like :
git config --global url."https://github.com/".insteadOf git://github.com/
git config --global url."https://".insteadOf git://

there's also the more intrusive way of just going into windows network adapter advanced setting, finding the ipv4 and ipv6 specific advanced properties and configuring the metric manually to give ipv4 lower (i.e higher preference) metric compared to ipv6 instead of the typically selected 'Automatic metric' checkbox for both of them .. in theory that should make windows prefer the lower-metric option
and for those who might only occasionally run into the issue, you could prob simply get by increasing the number or retries cargo makes before it gives up .. by setting up CARGO_NET_RETRY=10 etc in the env (instead of the default 3)
(and for those who know what they're doing and find that it is indeed a ssl cert revocation issue for them, you could add CARGO_HTTP_CHECK_REVOKE=false to your env too)

I can confirm that this is a bug with Windows - cargo is doing nothing wrong. Turning Windows Defender Firewall off has fixed the issue for me (however this may not be advisable). The firewall has been breaking other apps for me as well (mainly the Helldivers 2 updater), so I'm not too upset about turning this feature off.
If anyone has the guts to brave Microsoft's bug reporting process, just know that you have my support.

I now tried again and everything seems to be working perfectly. I don't know what fixed it sadly. I didn't change any network settings. My windows version is newer now, but I also updated cargo. Current version:
cargo 1.80.0-nightly (4de0094ac 2024-05-09)
release: 1.80.0-nightly
commit-hash: 4de0094ac78743d2c8ff682489e35c8a7cafe8e4
commit-date: 2024-05-09
host: x86_64-pc-windows-msvc
libgit2: 1.7.2 (sys:0.18.3 vendored)
libcurl: 8.6.0-DEV (sys:0.4.72+curl-8.6.0 vendored ssl:Schannel)
os: Windows 10.0.22635 (Windows 11 Core) [64-bit]


stake-pool: Use delegation.stake for accounting

Problem
Stake pool accounting has an issue because it always uses a stake account's full lamports() to decide how much is under management.  There's a possible attack where a person can send additional lamports to stake accounts to artificially inflate the value of pool tokens.  Someone could do this to screw over another user from getting their lamports out.
Solution
Whenever looking at validator stake accounts, always take into account Stake.delegation.stake instead of the account's lamports.  For the reserve, we can still use lamports() because all of that is accessible.
While fixing this, an issue came up regarding stake merges on deposit.  When merging two active stakes, the source account's lamports (outside of the delegation.stake) get added to the destination account, but not as active stake.  To properly reward users for the stake account provided, we still give them pool tokens for the full lamports() amount, BUT we also withdraw the extra amount to the reserve, to be restaked by the stake pool staker.  The rent exempt amount was being lost as the lamports() in the validator stake account.
There's still a weird "attack" where a person someone can deposit a stake account that has a lot of lamports, but a small amount of stake, get the pool tokens, and then immediately withdraw into a stake account.  I'm not sure if this is really a valid attack though since the person is already maintaining a stake account.
@ruuda could you take a look at this? This should address the issue we discussed during our last meeting

There's a possible attack where a person can send additional lamports to stake accounts to artificially inflate the value of pool tokens. Someone could do this to screw over another user from getting their lamports out.

Can you explain how this works? What I thought would happen in this case, is that the additional lamports act effectively like a reward; pool token holders now own a share of a larger pool. Is it not possible to withdraw the additional lamports from the stake account?
It does crease this weird state, where this “donation” to the pool is in a stake account, but not staked. I think you never want such accounts, and you want to stake the donation as soon as possible, but automating that might keep the transient stake account busy ... How do you want to handle such accounts? Just leave the additional lamports sitting there? And would they be considered part of the pool?

Can you explain how this works? What I thought would happen in this case, is that the additional lamports act effectively like a reward; pool token holders now own a share of a larger pool. Is it not possible to withdraw the additional lamports from the stake account?

They unfortunately don't act as rewards, since they aren't staked -- they're just additional lamports in the account.  It is possible withdraw those lamports directly to the reserve as unstaked SOL, so perhaps we can also add a step in the update_validator_list_balance instruction to check for any additional lamports in the stake accounts and withdraw them into the reserve if they're found.  After that, the stake pool staker can stake those additional lamports as they like.  Thankfully, we don't need to involve the transient stake accounts for this.

How do you want to handle such accounts? Just leave the additional lamports sitting there? And would they be considered part of the pool?

with this PR, they won't be considered if someone just transfers SOL to a stake account, If they're withdrawn into the reserve though, then they would be considered part of the pool.  It sounds like I should add that functionality here.  What do you think?

they're just additional lamports in the account. It is possible withdraw those lamports directly to the reserve as unstaked SOL, so perhaps we can also add a step in the update_validator_list_balance instruction to check for any additional lamports in the stake accounts and withdraw them into the reserve if they're found

Ah, yes, that makes sense.

with this PR, they won't be considered if someone just transfers SOL to a stake account, If they're withdrawn into the reserve though, then they would be considered part of the pool. It sounds like I should add that functionality here. What do you think?

I’m ambivalent about this, on the one hand it feels like a lot of complexity for a weird situation that nobody has an incentive to create, apart from messing with the pool. If the stake pool consistently only uses the stake balance and not the account balance, and there is no way for the additional lamports to ever “leak back” into the pool, then I think it would be best to simply ignore them. It does feel like a very fragile situation, because you always need to be extremely careful to use the right balance. It’s always nicer to have something that can’t go wrong by construction. So on the other hand, given that these weird states could arise, having something that can move a weird state back to a good state is nice to have. :thinking:

@ruuda sorry for the lateness here, I'm still out today, but I'll get you a full answer tomorrow

Thanks for the great review! You brought up the difference between the stake delegation and account lamports during one of our calls, so I assumed you had already noticed this issue :laughing: regarding your points about invariants...

These are the accounts managed by the stake pool.

The accounts managed by the stake pool are all of the validator stake accounts + transient stake accounts + reserve

The stake balance of the pool is defined to be «some sum involving stake accounts and other accounts».

The stake balance of the pool is a bit tricky.  The concept is that it should represent all of the SOL deposited by users into the pool, and NOT anything provided by others.  To start, all stake accounts in the pool (reserve + validator stake accounts) have a rent-exempt amount, which is not considered, since it can't be accessed by the depositors (they can't add or remove validators and reclaim the rent exempt lamports).  Next, there's an additional "exempt" amount in the stake accounts, required to keep them in the correct state.  The validator stake accounts require 1 SOL so that new stakes can merge in easily, and the reserve requires just 1 lamport to stay a stake account.  Transient accounts are totally fair game, since they are merged into other accounts all the time.
So, the formula then for the stake pool balance is:
reserve_lamports - rent_exemption - 1 lamport + sum(account.lamports for account in transient_accounts) + sum(account.lamports - 1 SOL - rent_exemption for account in validator_stake_accounts)

There is also sum(account.lamports for account in accounts_controlled_by_pool). If this sum is not equal to the stake balance, what is the meaning of the difference?

Good question, that would usually mean that someone contributed outside lamports

Stake pool tokens represent a share of ... the stake balance?

Exactly, yes.
With regard to your other point, since a few of your comments lead to the answer of "we need to withdraw additional SOL from the validator stake accounts into the reserve during update", it sounds like we should definitely do it.  This PR already seems pretty big, so I'd prefer to do it in a follow-up.  What do you think?

With regard to your other point, since a few of your comments lead to the answer of "we need to withdraw additional SOL from the validator stake accounts into the reserve during update", it sounds like we should definitely do it.

I’ve been thinking about this for Lido, and there I think we’ll go with something simpler. I see three ways to go about these extra lamports that somebody might “donate” to a stake account:

Say “thanks for the donation” and count them as part of the pool. They are not gaining validation rewards, but hey, we got this donation so we shouldn’t complain, we are not worse off than before the donation. The additional lamports can be withdrawn, so they are really part of the pool.
Say “there are unstaked lamports, we’re missing out on rewards”. In this case we do need to withdraw the additional SOL and then stake it properly.
Say “anything that we didn’t put in ourselves (or got rewarded for our own stake), we don’t consider part of the pool”. Then we only look at delegation.stake to compute the balance. I haven’t thought deeply about this, but I think it can become a challenge to do the accounting properly, and if we make a mistake, there can be places where some operation that wasn’t supposed to change the balance actually does.

For Lido we went with option 1 for now, because it’s the simplest one. I think option 2 is entirely an optimization on top of option 1 that can get a bit more rewards in the weird case of a donation, so if you care deeply about maximizing validation rewards, you should go with option 2. I don’t think it is required for correctness.

This PR already seems pretty big, so I'd prefer to do it in a follow-up. What do you think?

Yes, definitely :)

Sounds good to me! It should be a good approach for most situations.


uint -> uint256

Fixes

Fixes # https://github.com/ethereum-optimism/roadmap/issues/284

Contributing Agreement

[x] I have read and understood the Optimism Contributing Guide and Code of Conduct and am following those guidelines in this pull request.

LGTM! Let's hold off on merging this one just yet, need to confirm that changing the L2ToL1 event hash is not gonna break some other things.

Just tested in remix, event hash doesn't change when switching from uint to uint256.

This looks good to go, based on what @kfichter found, should be good to merge to master.


Importing yft's not responding till i close it with task manager

when i try importing a yft it just doesnt respond until i kill it with task manager

when i try importing a yft it just doesnt respond until i kill it with task manager

Can you send the original yfts

Possibly same issue as #862. It is fixed but not released yet so you can try installing the latest commit.

when i try importing a yft it just doesnt respond until i kill it with task manager

Can you send the original yfts

it was ripples valor i needed models so i could texture in substance

when i try importing a yft it just doesnt respond until i kill it with task manager

Can you send the original yfts

it was ripples valor i needed models so i could texture in substance

not really sure what 'ripples valor' is, can you link it?

Can't fix anything if we can't reproduce it. Anyway, as I mentioned, this is probably a duplicate of #862. Try installing the latest commit (not release) that includes the fix and import again.


feat: avatar decorations

Ref: https://github.com/discord/discord-api-docs/pull/5723
This also is based on new seasonal avatar decorations being prefixed with v2_ to denote using an avatar decoration preset.

Hey there! :wave:
If this PR is still alive, then it might be worth refactoring it to the latest iteration: https://github.com/discord/discord-api-docs/pull/6464
That one is going to stay for a while, I think. Whereas the ones shown before seemed to be seasonal testing rounds…

Will refactor with the latest iteration later this weekend.

Sidenote: I removed dynamicAvatarDecorationURL from the User class since it seems that avatar decoration presets don't use size query strings


About the code

Gread job! When will the code be released?

Gread job! When will the code be released?

Gread job! When will the code be released?


ERROR: No matching distribution found for mxnet-mkl

I cloned the repo locally, then ran pip install -e . and pip install mxnet-mkl but I get the error:
ERROR: Could not find a version that satisfies the requirement mxnet-mkl (from versions: none)
ERROR: No matching distribution found for mxnet-mkl

How can I fix it?

The program is not occurred by min-scoring.
I think it is "pip install mxnet."

@Tanesan Now I get the error:
    from mlm.scorers import MLMScorer, LMScorer
  File ".mlm-scoring/src/mlm/scorers.py", line 8, in <module>
    import gluonnlp as nlp
  File "./venv_3.8/lib/python3.8/site-packages/gluonnlp/__init__.py", line 23, in <module>
    from . import loss
  File "./venv_3.8/lib/python3.8/site-packages/gluonnlp/loss/__init__.py", line 23, in <module>
    from .activation_regularizer import *
  File "./venv_3.8/lib/python3.8/site-packages/gluonnlp/loss/activation_regularizer.py", line 25, in <module>
    from mxnet.gluon.loss import Loss
  File "./venv_3.8/lib/python3.8/site-packages/mxnet/__init__.py", line 23, in <module>
    from .context import Context, current_context, cpu, gpu, cpu_pinned
  File "./venv_3.8/lib/python3.8/site-packages/mxnet/context.py", line 23, in <module>
    from .base import classproperty, with_metaclass, _MXClassPropertyMetaClass
  File "./venv_3.8/lib/python3.8/site-packages/mxnet/base.py", line 351, in <module>
    _LIB = _load_lib()
  File "./venv_3.8/lib/python3.8/site-packages/mxnet/base.py", line 342, in _load_lib
    lib = ctypes.CDLL(lib_path[0], ctypes.RTLD_LOCAL)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/ctypes/__init__.py", line 373, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: dlopen(./venv_3.8/lib/python3.8/site-packages/mxnet/libmxnet.dylib, 6): Symbol not found: ____chkstk_darwin
  Referenced from: ./venv_3.8/lib/python3.8/site-packages/mxnet/libmxnet.dylib
  Expected in: /usr/lib/libSystem.B.dylib
 in ./venv_3.8/lib/python3.8/site-packages/mxnet/libmxnet.dylib
python-BaseException

Any idea? I am using MAC and Python3.8

I don't know much about it, but this URL may be helpful.
https://github.com/apache/incubator-mxnet/issues/5487

@orko19 I was unable to reproduce your bug on a Mac.
Can you try again by creating a new virtual environment and installing MXNet and PyTorch using the following command?
pip install torch mxnet


Handle etcd compacted revisions

vip-manager currently doesn't seem to handle etcd compacted revisions gracefully:
Feb 22 15:17:03 host vip-manager[3872968]: 2024/02/22 15:17:03 IP address 10.x.y.z/16 state is true, desired true
Feb 22 15:17:13 host vip-manager[3872968]: 2024/02/22 15:17:13 IP address 10.x.y.z/16 state is true, desired true
Feb 22 15:17:21 host vip-manager[3872968]: 2024/02/22 15:17:21 etcd watcher returned error: etcdserver: mvcc: required revision has been compacted
Feb 22 15:17:21 host vip-manager[3872968]: 2024/02/22 15:17:21 IP address 10.x.y.z/16 state is true, desired false
Feb 22 15:17:21 host vip-manager[3872968]: 2024/02/22 15:17:21 Removing address 10.x.y.z/16 on ens192

Restarting vip-manager fixes this, but of course the database is not accessible until then.

I noticed this too when I was testing failover. The new leader requires a restart of vip-manager. Im trying to figure out a way to auto restart vip-manager daemon using monit on failover.

Any chance you guys can throw me a link to get idea of compacted revisions without chatting with AI or Google? :)
Thanks in advance!

I guess I am less aware of "impacted revisions" and more referring to auto-recovery.
I am using vip-manager as part of https://github.com/vitabaks/postgresql_cluster. If I hard stop (power off the VM, simulating a hardware failure) the leader of the cluster, vip-manager never recovers until I restart vip-manager on the new leader. Sorry for the confusion.

@pashagolub please help.

etcdserver: mvcc: required revision has been compacted

maybe it will be enough to simply retry here to re-read the value of the latest version of the key?

I'd rather fix this error. But I cannot find how I can reproduce it in a simple way

I cannot find how I can reproduce it in a simple way

Add ETCD_AUTO_COMPACTION_RETENTION="1" option to etcd.conf
Example:
ETCD_NAME="pgnode01"
ETCD_LISTEN_CLIENT_URLS="http://<IP_ADDRESS>:2379,http://<IP_ADDRESS>:2379"
ETCD_ADVERTISE_CLIENT_URLS="http://<IP_ADDRESS>:2379"
ETCD_LISTEN_PEER_URLS="http://<IP_ADDRESS>:2380"
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://<IP_ADDRESS>:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-postgres-cluster"
ETCD_INITIAL_CLUSTER="pgnode01=http://<IP_ADDRESS>:2380,pgnode02=http://<IP_ADDRESS>:2380,pgnode03=http://<IP_ADDRESS>:2380"
ETCD_INITIAL_CLUSTER_STATE="new"
ETCD_DATA_DIR="/var/lib/etcd"
ETCD_ELECTION_TIMEOUT="5000"
ETCD_HEARTBEAT_INTERVAL="1000"
ETCD_INITIAL_ELECTION_TICK_ADVANCE="false"
ETCD_AUTO_COMPACTION_RETENTION="1"

@pashagolub, Hi, there is new information.
In connection with the detected problem, for the time being until the new version and fix of the problem, I turned off maintenance for ETCD on our DB clusters today and that's what an interesting moment I discovered.
On the cluster where vip-manager 2.3.0 was, when restarting ETCD, vip manager gave the error described above and vip disappeared before restarting vip-manager, and on the second cluster, where vip-manager version 2.1.0 is used, I did not find any problems with vip-manager, perhaps this is due to hardware, because on in the second cluster, the hardware is better, then the ETCD restart time was faster, but perhaps this will give a hint in finding a solution.
I will try to reproduce this scenario in my test zone with different versions as soon as possible and I will unsubscribe with an additional comment.

Hi people. Would you please try #217
Thanks in advance!

@pashagolub Hi!
I've done several iterations of testing.
In the first case, I did apt-get --purge --autoremove remove vip-manager for the old version, after installing the version from the 208-handle-etcd-compressed-revisions branch, and in this case, it was extremely difficult to repeat the error, but in one of the many attempts to cause this error, I managed to get it to call.
Next, I deployed the cluster from 0 and installed vip-manager immediately from the 208-handle-etcd-compressed-revisions branch, and this time, no matter how many different attempts I made, the error could not be reproduced. I tried to stop etcd on one of the 3 nodes, restart etc also on each of the nodes, but the error did not reproduce.


293_post_interest_job

@AvinashKumarG : I know it's a bit late but would it be possible for you to resolve the conflict, test it once and update your PR?

PR ping.  Is this Pull Request still alive?

ping
Is this pull request still "alive"? Will Nazeer and Avik's question receive an answer? If there's no response in a week this pull request will be closed.

@AvinashKumarG thank you for your interest in Mifos.  I'm an (old) volunteer who got reminded of Mifos this week-end at the FOSDEM conference, and I'm taking a moment to help the project clean up some of it's old issues and pull requests etc.
Given that there was no reaction to above in over 1 year, I took the liberty to close this PR, to avoid confusion for future new contributors looking at this repository - hope you understand and that's fair.


.NET Core: Exception in WebApiToSwaggerGenerator when action has more than one parameter

LoadParameters in WebApiToSwaggerGenerator.cs throws exception for controller actions which have multiple parameters (which is valid in AspNetCore).
Consider this Action-Method:
public Task Get([FromQuery] Model model, CancellationToken cancellationToken)
there could be more arguments which are decorated with [FromServices]
see here: https://docs.asp.net/en/latest/mvc/models/model-binding.html

Improved support https://github.com/NSwag/NSwag/commit/ab46164c8c4dfb418da0f71882dec65642998650

Do you think this is enough for now?

Works, thanks.


actor/task .call are missing waitSecs option

The type ActorStartOptions has waitForFinish but actor('actor').call (and task) where it is used should have waitSecs instead. I would do a PR but not sure how you want to structure the types

Out of curiosity, why is that? The api expects waitForFinish, not waitSecs. Is this just a rename in the client for clarity sake or is it something else?

@vladfrangu I think this was designed by @mnmkng but as I understand it: The .start() is basically the normal client run so it passes all the params to the API (waitForFinish is API param). But .call() is a wrapper of .start() that waits for the run to finish fully. This is feature of the client, not supported but the API so the waitSecs is option only to the client, not API.
I agree that it is a bit confusing since vast majority of the client just reflects the API but we generally agreed that the client should have utility logic on top of it.

Oh crap, I think I see what you're saying. .call re-uses the type for .start but in the case of waitSecs vs waitForFinish, .call expects the former, and .start the latter. I'll fix this, thx


How to read the passcode(pincode) (CON-1389)

Using esp-matter-mfg-tool to generate files and download them onto the chip, only the discriminator can be read, but the passcode cannot be read. I tried using '-- enable dynamic passcode', but it also failed
code:
   ESP32FactoryDataProvider dataProvider;
   if(dataProvider.GetVendorId(device_info.VendorId) == CHIP_NO_ERROR)
{
   USER_LOGI(TAG,"VendorId :%d",device_info.VendorId);
}
if(dataProvider.GetProductId(device_info.ProductID) == CHIP_NO_ERROR)
{
   USER_LOGI(TAG,"ProductID :%d",device_info.ProductID);
}
if(dataProvider.GetSetupDiscriminator(device_info.Discriminator) == CHIP_NO_ERROR)
{
   USER_LOGI(TAG,"device_info.Discriminator :%d",device_info.Discriminator);
}
if(dataProvider.GetSetupPasscode(device_info.PassCode) == CHIP_NO_ERROR)
{
   USER_LOGI(TAG,"device_info.PassCode :%ld",device_info.PassCode);
}

log:

esp-matter-mfg-tool --discriminator 2508 --passcode 10230953
--product-name 81MtAc01 --product-id 0x2024
--vendor-name XzElec --vendor-id 0x1575
--hw-ver 1003 --hw-ver-str <IP_ADDRESS>
--dac-in-secure-cert --target esp32s3
--dac-cert  /home/alen/esp-matter/connectedhomeip/connectedhomeip/credentials/user/TrustAsia_cert/certificate.pem
--dac-key  /home/alen/esp-matter/connectedhomeip/connectedhomeip/credentials/user/TrustAsia_cert/privateKey.pem
--pai -c      /home/alen/esp-matter/connectedhomeip/connectedhomeip/credentials/user/TrustAsia_cert/Changsha_Xizhen_Electronic_Matter_Test_PAI.pem
-cd             /home/alen/esp-matter/connectedhomeip/connectedhomeip/credentials/user/TrustAsia_cert/esp_dac_1575_2024.der
partition.csv：

Partition.csv  does not have a pass code, but the pass code can be seen in another file

Is there any way for me to read the passcode from the factory partition? I need to generate QR codes and ManualPairingCode

As per (2) in https://github.com/espressif/esp-matter/issues/1126#issuecomment-2430989163, we are not recommending people to store the Passcode along with verifier. So, it won't be in the factory partition generated by esp-matter-mfg-tool.
For -enable-dynamic-passcode, please check the help content of --enable-dynamic-passcode
g_commissioning.add_argument('--enable-dynamic-passcode', action="store_true", required=False,
    help='Enable dynamic passcode. If enabling this option, the generated binaries will \
    not include the spake2p verifier. so this option should work with a custom \
    CommissionableDataProvider which can generate random passcode and \
    corresponding verifier')

That said, if you use that option you will not have the passcode in the factory partition binary generated by esp-matter-mfg-tool and your app/custom-provider must be capable to generate one (as demonstrated in light_switch).
PS: Can you please describe the problem that you are solving rather than wondering why a solution (may not be intended for the problem you are solving) is not working for you. Can you please share (If you are willing/allowed to) what kind of device you are developing. Because, the dynamic passcode makes sense if your device really has a display or some sort of output capability.  For device such as light, won't be able to showcase the onboarding payload by itself and hence will be a part of packaging.
PPS: I would suggest you to please go through section 5.1.7. Generation of the Passcode in Core Matter Specification for better understanding about Passcode/verifier storage.

As per (2) in #1126 (comment), we are not recommending people to store the Passcode along with verifier. So, it won't be in the factory partition generated by .esp-matter-mfg-tool
For , please check the help content of --enable-dynamic-passcode-enable-dynamic-passcode
g_commissioning.add_argument('--enable-dynamic-passcode', action="store_true", required=False,
    help='Enable dynamic passcode. If enabling this option, the generated binaries will \
    not include the spake2p verifier. so this option should work with a custom \
    CommissionableDataProvider which can generate random passcode and \
    corresponding verifier')

That said, if you use that option you will not have the passcode in the factory partition binary generated by and your app/custom-provider must be capable to generate one (as demonstrated in light_switch).esp-matter-mfg-tool
PS: Can you please describe the problem that you are solving rather than wondering why a solution (may not be intended for the problem you are solving) is not working for you. Can you please share (If you are willing/allowed to) what kind of device you are developing. Because, the dynamic passcode makes sense if your device really has a display or some sort of output capability. For device such as light, won't be able to showcase the onboarding payload by itself and hence will be a part of packaging.
PPS: I would suggest you to please go through section 5.1.7. Generation of the Passcode in Core Matter Specification for better understanding about Passcode/verifier storage.

We are developing a temperature controller device with a screen. Due to production reasons, it is necessary to display the QR code and manual pairing code. Of course, it does not need to be dynamic. I saw that there is a "Get Setup Passcode()", so I thought it could read the information like "Get Setup Discretionar()", making it more convenient and efficient to generate QR code information in the code and display it on the screen. If I cannot read it, I can only manually enter the Passcode to write it into NVS and extract it from NVS. This is very inconvenient when producing products

Thanks for the information.
If it has screen to display onboarding code, I'd suggest a using dynamic passcode. Passcode is the proof of possession and initial PAKE happens using this as a password.
Yes, there are APIs like GetSetupPasscode or SetSetupPasscode in CommissionableDataProvider interface. But, in ESP32FactoryDataProvider or dynamic_commissionable_data_provider we have implemented the recommendation from the specification.

If I cannot read it, I can only manually enter the Passcode to write it into NVS and extract it from NVS.

You can definitely do that. But make sure you comply with those SHALL statements from 5.1.7. Generation of the Passcode (please check attached screenshot)

This is very inconvenient when producing products

I know right? But, when you think of security, it makes more sense!

Thank you for your answer. I will store the passcode in another way


Support vi mode ; and , motions

The f, F, t, and T left-right motions can be repeated with the ; motion, or reversed with the , motion:

;
Repeat latest f, t, F or T [count] times. See cpo-;
,
Repeat latest f, t, F or T in opposite direction [count] times. See also cpo-;

I named these "to-till" motions as the first word of the description for the f and T motions are "To" and "Till" respectively.
Add storage for the last To or Till motion on the Vi struct, and pass it through to parse_command() so ; or , can re-use it.
When a to-till motion is fully parsed the new to-till motion is stored in the Vi struct.
I'm not the happiest with finding a new to-till motion in the parsed command. I can't think of a better way to match it due to the Vec, but am open to suggestions.
It's probably reasonable to make parse_command() a method of Vi if my implementation is acceptable.

I need to fix , behavior. For fabcdefabcdef with the cursor on the first f issuing ff;,, should return you to the first f. In my implementation , swaps between the second and third fs


Coursera - Course 3 Final Project Updates

These are updates to Coursera - Course 3 Final Project Updates

Pushing final project updates


Split build component by service

Separate build component to:

build-service
jvm-build-service
pipeline-service
shared-resources
build-templates

LGTM. A minor comment: Should be renamed from build-service to pipeline-service here?

Updated, moved under tekton-pipelines since pipeline-service namespace does not exist. But Anyway I am thinking to move these .tekton folders under one place and into one namespace in next PRs

LGTM from me.

:+1:


fix(3156115079): align the header's tabs to Figma design

Description of the Changes

Adding new color
change Divider width, height, and color
change the tab's font size

Solves #3156115079

Checklist

[X] Manually tests of the main Application flows are done and passed.
[ ] New unit / functional tests have been added (whenever applicable).
[ ] Docs have been updated (whenever relevant).
[X] PR title is follow the Conventional Commits convention: <type>[optional scope]: <description>, e.g: fix: prevent racing of requests

This change is

:tada: This PR is included in version 1.9.0-dev.12 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:

:tada: This PR is included in version 1.9.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:

:tada: This PR is included in version 1.9.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


geo_point insert error

Given table
geocity (
geo geo_point
)
http Statement
{"stmt":"INSERT INTO "geocity" ("geo") VALUES (?) ",
"bulk_args":[[[1,1]]]}
Results in
"SQLActionException[ArrayStoreException in java.util.Arrays.copyOf(Arrays.java:3213)]"
Yet the following works fine
{"stmt":"INSERT INTO "geocity" ("geo") VALUES (?) ",
"bulk_args":[["POINT(1 1)"]]}
On an aside it would be nice if you also supported the mapped method of inserting the geo-point..
thanks

---
Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/29467486-geo_point-insert-error?utm_campaign=plugin&utm_content=tracker%2F505659&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F505659&utm_medium=issues&utm_source=github).

Seems like it fails because the points should be double's and not integers.
See:
http -v localhost:4200/_sql stmt="insert into geocity (geo) values (?)" bulk_args:='[[ [1.0, 1.0] ]]'
POST /_sql HTTP/1.1
{
    "bulk_args": [
        [
            [
                1.0,
                1.0
            ]
        ]
    ],
    "stmt": "insert into geocity (geo) values (?)"
}

HTTP/1.1 200 OK
Content-Length: 51
Content-Type: application/json; charset=UTF-8

{
    "cols": [],
    "duration": 2,
    "results": [
        {
            "rowcount": 1
        }
    ]
}

I'm not sure if it makes sense to be more lenient in this case. But we should at least improve the error message.
What exactly do you mean with

On an aside it would be nice if you also supported the mapped method of inserting the geo-point..

Well d'oh, it appears the encoder in go tends to use integers when possible ..
Re second point see also Basically allow for a use of a mapped value for lat and lon
{
    "bulk_args": [
        [
            {
                "lat":1.0,
                "lon":1.0
            }
        ]
    ],
    "stmt": "insert into geocity (geo) values (?)"

}

hi @notzippy
we probably won't support the ES specific object notation for GEO_POINT type. since we use GeoJSON for GEO_SHAPE type, we could think about supporting that for GEO_POINT, too. will put that into our backlog and we'll discuss that internally.

@notzippy The issue you reported has popped up again so've changed the behaviour and made it more lenient so that conversion from arrays of integers works too.
See https://github.com/crate/crate/pull/3294

Just for the record, I bumped into this issue today, and had to workaround it by using is string 'POINT(lon lat)' representation as @notzippy pointed out above. We're using javascript for this task, and it's pretty cumbersome to enforce "float type" in a typeless language like javascript (basically the JSON just encode the field as a number and does not add '.0' and the end).
Supporting "integers" (a.k.a. a subset of float numbers) directly from crate will ease many things to the developer. :+1:

Ran into a rather bigger annoyance with this issue version .55
Inserting a geo_point into an array of geo_points - the only form you can do so is using the "point (1 1)" method. An error is thrown if you nest an array within the array. So in some cases you need to define one way others another..


Setup JavaScript testing framework

Vision
Implement unit testing framework for JavaScript widgets.
Current State
No unit testing currently in nightshade-core.
Scope

[ ] Implement testing framework (ideally Ava, or Mocha)

Anti-Vision
Integration/system testing not required at this time.

Not going to run it via Gulp because NYC is not available for gulp. The plan is to run tests via NPM script.


remove head

remove head , There is a default df getting shown after every checkbox. So remove head to avoid printing twice

closed in another pr


Docker plugin

TL;DR
Add a plugin to build imagespec using a docker buildx driver/builder
Type

[ ] Bug Fix
[ ] Feature
[x] Plugin

Are all requirements met?

[x] Code completed
[ ] Smoke tested
[x] Unit tests added
[x] Code documentation added
[ ] Any pending items have an associated Issue

Complete description
This is a quick pass at a docker / buildkit builder for flytekit. Basically, this plugin provides an alternative interface to ImageSpec which will allow the usage of preexisting Dockerfiles that may be too complex to recreate with envd.
ImageSpec itself seems very tailored to envd, so I'm not sure if this is the best way to go about adding this functionality.
Notable changes

flytekit/remote/remote.py

alter task-id (version) based on ImageSpec

Because of how images are built/checked for existence i have to resort to some hacks that I don't think should make it into the final PR. See horrible_horrible_hack in plugins/flytekit-docker/flytekitplugins/docker/image_builder.py

Because of this, It will likely require a deeper discussion with the team before merging

[ ] Need to create requirements.txt file correctly (getting weird errors)
[ ] Test the buildkit_build_extra_output option
[ ] Check if it Is it possible to test this further with the current infrastructure
[x] Check that it works

Yea I just wanted to get something that works by touching the least amount possible.  I have an idea of what needs to happen, but I wanted to get some feedback before I dumped a bunch more time into it or pushed the scope wider.
I'll probably keep plugging away at this till I'm happy with it, but any direction / thoughts would be greatly appreciated so I don't spend too much time in the wrong places.
Briefly:

Should ImageSpec be abstracted to ImageSpecBase + ImageSpecEnvD + ImageSpecDockerfile?

I'm not a huge fan of this solution, I think it would be nice to not have the plugins installed on the node
The current way is a bit too kitchen-drawer for me.

if non-python files are going to be considered during hashing - that has to be managed delicately as it is not what flyte currently does without copy-all

horrible_horrible_hack is because the Dockerfile isn't packaged with the script and sent to the remote. So when the remote calls .build_image and tries to determine a hash, it either crashes (folder doesn't exist) or it determines a new one and thus returns a new image_name.  This causes the builder to try and build the image again and then crash because docker doesn't exist.  To prove to myself it could be done I just used this hack but as previously mentioned I don't want it to be merged.


Add option to set knife ssh timeout

This adds a config to 'knife ssh' to allow a connection timeout to be specified.
This will prevent knife ssh from hanging if one of the hosts is down or not reachable.
@chef/client-core @tyler-ball

There should be a corresponding DOC_CHANGES modification but otherwise :+1:

Yep, will add DOC_CHANGES


Replace cake wallet with Monero.com

Replacing with Monero only wallet. Closes #2219

Title
downloads: monero.com instead of cake wallet


Expand github action pinning guidance to include update approach

SHA pinning is a sensible approach to mitigate potential supply chain attacks. See some great blog posts here on the approach:
https://blog.rafaelgss.dev/why-you-should-pin-actions-by-commit-hash also
https://michaelheap.com/improve-your-github-actions-security/#using-pin-github-action
However one down side is that SHA's are not very human readable. It can be difficult to tell from the SHA if the version we've pinned has an update, or if that update is a security or important fix.
Best practice therefore is to place a comment after the pinned version listing the semantic version for a third party github action.
This gets you best of both worlds, maintainability plus certainty.
It might look at bit like this:
jobs:
  check-pull-request:
    runs-on: ubuntu-latest
    steps:
      - name: Check out repository code
	uses: actions/checkout@ee066bloop # pin @v2
      - name: Install Ruby uses:
	ruby/setup-ruby@22acsewblah # pin@v1

Consistency here also helps us manage this code in line with the GDS Way requirement to Update dependencies frequently when managing third party dependencies:
https://gds-way.digital.cabinet-office.gov.uk/standards/tracking-dependencies.html#update-dependencies-frequently
Since October 2022 DependaBot will now look for comments on SHA pinning and automatically suggest updates. Similar approaches may be possible for other dependency management tools.
Dependabot currently supports a range of different comment syntaxses which can be viewed here:
https://github.com/dependabot/dependabot-core/pull/5951#issuecomment-1298147084
I've tried to keep the guidance general and open, leaving detail to this commit history, given the range of different tools on use across GDS.
The principles are:

Pin your actions using SHAs
Ensure human readability by commenting the semver on the line with the action
Explore if your usual dependency management process, especially automated ones like DependaBot can help flag and raise visibility on new versions.

This seems like a good change to me and is inline with what we're already doing in some places. I'm happy to approve when the comments are resolved.

Rightio folks,
Finally got this done, ready for a re-review if one of @andyloughran, @stephengrier, @willp-bl wouldn't mind another look?


Duplicate entries created by sync_service

I restarted several times the project on my local environment with Docker and it seems that the sync_service is pushing all the same entries every time it restarts (or gets them from the nodefetch container).
Can you check that the deduplication is working correctly ?

I have not yet implemented an duplication(checksum) check in the sync-service. Every time I receive an request I simply merge it with an existing Request from the stayhome api or create a new one. The service also re-calculates the checksum and sends sends that one as well to the stayhome api.
I though that I need to send all the requests coming  in to the sync-servie to the stay-home api (eg. not filtering them out based on the checksum) - so that the stayhome-api can then handle that (e.g decide to update a request, refresh the ttl etc).

Here is how I see the relation between the sync-service and the web-api :

At startup, the sync-service fetch all existing requests from the web-api in order to have an initial state.
When the the sync-service receives a "new" requests, it has to compute the checksum of this request and see if it already exists in its state.
** If the request already exists, the sync-state must perform an update of the request in the web-api in order to update the TTL. The checksum field must not change, the TTL field must be updated. (PUT on /requests/<request_uuid>/)
** If the request does not exists, the sync-state must create the request on the web-api. (POST on /requests/)
** If the request changed, the sync-state must update the request in the web-api. The new checksum must be given and the TTL field must be updated. (PUT on /requests/<request_uuid>/)
If the sync-service does not receive the external request for an existing entry in its state after the duration of the TTL, the entry can be deleted in its state. No need to send a DELETE to the web-api, the deletion will be automatic.

I guess I found at least on issue (the sync-service send a field called: checkSum instead of checksum

Yeap, is will not help :-)
Just to be 100% clear : the checksum field is not used by the web-api to check the uniqueness of the request. It is only used to detect a change in an existing request. The uniqueness is warranted by the uuid field of the request that is generated by the web-api at creation.

Also remember that there's a field called "source_uuid" that you can use to store your own unique key for a request, for example if the external source provides an ID.
Of course if you don't get an ID from the external source you have to find another way ;-)

with regards to https://github.com/stayhomech/stayhomech/pull/54
Example seqence:
First run:

we receive a request (source_id: test-001) from an integrator
we fetch an potential existing request from the web-api (find by source_uuid= test-001)
we build the checksum: 7777 and TTL: 1234
since we have not yet found a request in the web-api we create -> POST it

Some time passes:

I'd assume the stayhome-backend starts to decrease the TTL ?

Next run (no change in fields):

we receive a request (source_id: test-001) from an integrator
we fetch an potential existing request from the web-api (find by source_uuid= test-001)
we build the checksum: 7777 and TTL: 1234
since we have  found an existing request in the web-api we update-> PUT it

Some time passes:

I'd assume the stayhome-backend starts to decrease the TTL ?

Next run (change in fields):

we receive a request (source_id: test-001) from an integrator
we fetch an potential existing request from the web-api (find by source_uuid= test-001)
we build the checksum: 6666 and TTL: 1234
since we have  found an existing request in the web-api we update-> PUT it

1234 times (seconds) passes:

I'd assume the stayhome-backend deletes that request

Next run (we do not receive that entry from the datasource anymore):

nothing happens in the sync-service

Sounds good to me !

@mageo so shall I make the sync-service statefull and send you a delete request for an request that we haven't fetched since XXX TTL times?

We decided that the web-api (backend) is going to handle the TTL/expiry check


Update of font-awesome

Version 4.7 is old and unmaintained and Fedora Linux is in the process of updating to version 6, see: https://fedoraproject.org/wiki/Changes/FontAwesome6
https://github.com/jupyter/nbclassic/blob/8313c3d32f382c712a6c6eae2953ca9157410215/bower.json#L11
Would it make sense to update the version here as well?

We try to maintain status-quo and not change if it is not needed. What would font-awesome 6 bring?


Help with an error message when processing huge VTU file

Hi,
I have a program written in Python which post-processes an output VTU file from a fluid dynamics framework. The program is massively parallel in the sense that it repeats the exact same calculations at different locations in space, distributed on a grid. I have used Charm4Py to parallelise the workload on a large, university-like cluster, especially the pool functionality as it seemed to be the most appropriate to me. Everything is working properly on "regular" VTU files, and I am obtaining results which compare really well to multiprocessing on a single node and Ray on a multi-node environment. However, I am encountering an issue when I provide a "huge" VTU. What I mean by huge is 3 GB, whereas other files are on the order of a few 100 MB. I am pasting below the output generated by Charm4Py and the traceback for the first PE; the computer I have run this on has 64 GB of RAM. I would be really grateful if anyone could help me with this error and explain to me what it means so that I can attempt to fix it. I am more than happy to provide additional information about the program in itself.
Thank you for any help.
Charmrun> scalable start enabled.
Charmrun> started all node programs in 1.311 seconds.
Charm++> Running in non-SMP mode: 17 processes (PEs)
Converse/Charm++ Commit ID: v6.10.0-beta1-17-ga5b6b3259
Warning> Randomization of virtual memory (ASLR) is turned on in the kernel, thread migration may not work! Run 'echo 0 > /proc/sys/kernel/randomize_va_space' as root to disable it, or try running with '+isomalloc_sync'.
Charm++> scheduler running in netpoll mode.
CharmLB> Load balancer assumes all CPUs are same.
Charm4py> Running Charm4py version 1.0 on Python 3.8.0 (CPython). Using 'cython' interface to access Charm++
Charm++> Running on 1 hosts (1 sockets x 10 cores x 2 PUs = 20-way SMP)
Charm++> cpu topology info is gathered in 0.001 seconds.
Initializing charm.pool with 16 worker PEs. Warning: charm.pool is experimental (API and performance is subject to change)
----------------- Python Stack Traceback PE 2 -----------------
  File "charm4py/charmlib/charmlib_cython.pyx", line 863, in charm4py.charmlib.charmlib_cython.recvGroupMsg
  File "/home/thomas/.local/lib/python3.8/site-packages/charm4py/charm.py", line 253, in recvGroupMsg
    header, args = self.unpackMsg(msg, dcopy_start, obj)
  File "charm4py/charmlib/charmlib_cython.pyx", line 739, in charm4py.charmlib.charmlib_cython.CharmLib.unpackMsg
------------- Processor 2 Exiting: Called CmiAbort ------------
Reason: UnpicklingError: pickle data was truncated
[2] Stack Traceback:
  [2:0] _Z14CmiAbortHelperPKcS0_S0_ii+0x60  [0x7f78509ea920]
  [2:1] +0x260a51  [0x7f78509eaa51]
  [2:2] +0x18e0e  [0x7f7850d0ae0e]
  [2:3] _PyObject_MakeTpCall+0x28f  [0x5fff6f]
  [2:4]   [0x4ffbbf]
  [2:5] _PyEval_EvalFrameDefault+0x53f0  [0x57dbb0]
  [2:6] _PyFunction_Vectorcall+0x19c  [0x602b2c]
  [2:7] _PyEval_EvalFrameDefault+0x88d  [0x57904d]
  [2:8] +0x11cd0  [0x7f7850d03cd0]
  [2:9] +0x13474  [0x7f7850d05474]
  [2:10] +0x16857  [0x7f7850d08857]
  [2:11] +0x40601  [0x7f7850d32601]
  [2:12] _ZN8GroupExt13__entryMethodEPvS0_+0xb5  [0x7f7850895445]
  [2:13] CkDeliverMessageFree+0x21  [0x7f7850898761]
  [2:14] _Z15_processHandlerPvP11CkCoreState+0x88a  [0x7f78508a18ea]
  [2:15] CsdScheduleForever+0x77  [0x7f78509aa717]
  [2:16] CsdScheduler+0x2d  [0x7f78509aa97d]
  [2:17] ConverseInit+0x64a  [0x7f78509f046a]
  [2:18] StartCharmExt+0x255  [0x7f785089edd5]
  [2:19] +0x3d7be  [0x7f7850d2f7be]
  [2:20] _PyObject_MakeTpCall+0x28f  [0x5fff6f]
  [2:21]   [0x4ffbbf]
  [2:22] _PyEval_EvalFrameDefault+0x53f0  [0x57dbb0]
  [2:23] _PyEval_EvalCodeWithName+0x25c  [0x5765ec]
  [2:24] _PyFunction_Vectorcall+0x442  [0x602dd2]
  [2:25] _PyEval_EvalFrameDefault+0x88d  [0x57904d]
  [2:26] _PyEval_EvalCodeWithName+0x25c  [0x5765ec]
  [2:27]   [0x662c2e]
  [2:28] PyRun_FileExFlags+0x97  [0x662d07]
  [2:29] PyRun_SimpleFileExFlags+0x17f  [0x663a1f]
  [2:30] Py_RunMain+0x20e  [0x687dbe]
  [2:31] Py_BytesMain+0x29  [0x688149]
  [2:32] __libc_start_main+0xe7  [0x7f78efe25bf7]
  [2:33] _start+0x2a  [0x607daa]

Hello,
Thank you for reaching out. Did you install Charm4Py from pip or did you build Charm++ to use with Charm4Py manually? Also, are you distributing the entire 3GB data to each chare?

Thanks for having a look @ZwFink.
The error message provided results from a pip installation of Charm4Py. I have just tried with an MPI build of Charm++ (./build charm4py mpi-linux-x86_64 -j8 --with-production) and it yields a very similar error message:
Running on 17 processors:  /usr/bin/python3.8 PyDRex.py Navid_3d_transform_HUGE.vtu --charm
charmrun>  /usr/bin/setarch x86_64 -R  mpirun -np 17  /usr/bin/python3.8 PyDRex.py Navid_3d_transform_HUGE.vtu --charm
Charm++> Running on MPI version: 3.1
Charm++> level of thread support used: -1 (desired: 0)
Charm++> Running in non-SMP mode: 17 processes (PEs)
Converse/Charm++ Commit ID: v6.11.0-beta1-29-gd35885331
Isomalloc> Synchronized global address space.
CharmLB> Load balancer assumes all CPUs are same.
Charm4py> Running Charm4py version 1.0 on Python 3.8.0 (CPython). Using 'cython' interface to access Charm++
Charm++> Running on 1 hosts (1 sockets x 10 cores x 2 PUs = 20-way SMP)
Charm++> cpu topology info is gathered in 0.009 seconds.
Initializing charm.pool with 16 worker PEs.  Data from the VTU is read in the main function (the one given to charm.start()). It is then used to create 3-D Scipy Interpolator objects using, for example, NearestNDInterpolator. These objects are then passed to each function execution through charm.pool.
I am using the multi_future argument of map_async, making sure 60,000 futures at most are created, and I provide a function foo with a partial construct to pass a dictionary that holds variables which will be accessed by each process. These variables are never modified, they are only read or, in the case of the Interpolator objects mentioned above, called. I paste below the relevant snippet.
    if inputArgs.charm:  # Charm4Py
        nBatch = np.ceil(np.sum(varDict['indArr'] == 0) / 6e4).astype(int)
        for batch in range(nBatch):
            nodes2do = np.asarray(varDict['indArr'] == 0).nonzero()
            futures = charm.pool.map_async(
                partial(foo, dictGlobals=dictGlobals),
                list(zip(*[nodes[:60_000] for nodes in nodes2do])),
                multi_future=True)
            for future in charm.iwait(futures):
                output = future.get()
                for i, var in enumerate(outVar):
                    varDict[var][output[0]] = output[i + 1]
                varDict['indArr'][output[0][::-1]] = 1
                varDict['nodesComplete'] += 1


Add bitwise_xor_agg Presto aggregate function

Fixes #6690
Fuzzer results:
 ./velox/exec/tests/velox_aggregation_fuzzer_test --logtostderr --only "bitwise_xor_agg" --duration_sec 3600
I20230924 20:15:29.420713 3407272 AggregationFuzzer.cpp:780] ==============================> Done with iteration 5310
I20230924 20:15:29.420739 3407272 AggregationFuzzer.cpp:1503] Total functions tested: 1
I20230924 20:15:29.420745 3407272 AggregationFuzzer.cpp:1504] Total masked aggregations: 812 (15.29%)
I20230924 20:15:29.421257 3407272 AggregationFuzzer.cpp:1506] Total global aggregations: 410 (7.72%)
I20230924 20:15:29.421263 3407272 AggregationFuzzer.cpp:1508] Total group-by aggregations: 3880 (73.06%)
I20230924 20:15:29.421272 3407272 AggregationFuzzer.cpp:1510] Total distinct aggregations: 544 (10.24%)
I20230924 20:15:29.421276 3407272 AggregationFuzzer.cpp:1512] Total window expressions: 477 (8.98%)
I20230924 20:15:29.421281 3407272 AggregationFuzzer.cpp:1514] Total aggregations verified against DuckDB: 487 (9.17%)
I20230924 20:15:29.421286 3407272 AggregationFuzzer.cpp:1516] Total failed aggregations: 0 (0.00%)
[==========] Running 0 tests from 0 test suites.
[==========] 0 tests from 0 test suites ran. (0 ms total)
[  PASSED  ] 0 tests.

@mbasmanova can you help review this simple PR?


sigproc writer block needs some love

The current sigproc reader block a) doesn't treat angles correctly, and b) will crash in most cases! My fix so far has been to use astropy, but that adds a dependency, so starting an issue to discuss merits.
Angles
The sigproc definition for angles is actually a float, HHMMSS.ss. For example, 17h45m00.12s needs to be written as a float 174500.12. This code is ugly but parses hms or dms:
from astropy.coordinates import Angle

def to_sigproc_angle(angle_val):
    """ Convert an astropy.Angle to the ridiculous sigproc angle format string. """
    x = str(angle_val)

    if '.' in x:
        if 'h' in x:
                d, m, s, ss = int(x[0:x.index('h')]), int(x[x.index('h')+1:x.index('m')]), \
                int(x[x.index('m')+1:x.index('.')]), float(x[x.index('.'):x.index('s')])
        if 'd' in x:
            d, m, s, ss = int(x[0:x.index('d')]), int(x[x.index('d')+1:x.index('m')]), \
            int(x[x.index('m')+1:x.index('.')]), float(x[x.index('.'):x.index('s')])
    else:
        if 'h' in x:
            d, m, s = int(x[0:x.index('h')]), int(x[x.index('h')+1:x.index('m')]), \
            int(x[x.index('m')+1:x.index('s')])
        if 'd' in x:
            d, m, s = int(x[0:x.index('d')]), int(x[x.index('d')+1:x.index('m')]), \
            int(x[x.index('m')+1:x.index('s')])
        ss = 0
    num = str(d).zfill(2) + str(m).zfill(2) + str(s).zfill(2)+ '.' + str(ss).split(".")[-1]
    return np.float64(num)

...

# Get RA and DEC and convert to sigproc format
ra = Angle(ihdr['raj'], unit='hourangle')
dec = Angle(ihdr['dej'], unit='degree')
sigproc_hdr['src_raj'] =  to_sigproc_angle(ra)
sigproc_hdr['src_dej'] =  to_sigproc_angle(dec)

Crash bug
The code fails  on  L204 and L291 as axnames is of type list, while the RHS a tuple.
Easiest fix is:
 if ndim >= 3 and axnames[-3:] == ('time', 'pol', 'freq'):
becomes
 if ndim >= 3 and tuple(axnames[-3:]) == ('time', 'pol', 'freq'):

Thanks for finding these bugs.
Couldn't you use http://docs.astropy.org/en/stable/api/astropy.coordinates.Angle.html#astropy.coordinates.Angle.dms
Astropy does seem like a heavyweight solution for this. How/where do you need to interact with the angle value in the headers? E.g., are you setting them in a source block, loading them from a Sigproc read block, something else? I guess we need to decide on a standard representation of them in headers.

Yes, generally read from guppi raw files, which are often strings (unsure what the spec is, or if people actually follow it).
An yes, that code is particularly horrible (not written by me in my defense!), and came from a non-astropy-using codebase.
Astropy angles and units (http://docs.astropy.org/en/v0.2.1/units/index.html) are both pretty useful. I think we either use them extensively (instead of pint?), or not at all...

@telegraphic is this still a thing?  It looks like the crash has been fixed but nothing was done about the angle representation.


iOS - onPartialResult null string value

Hello! We have this implemented for both Android and iOS - Android works like a charm, but on iOS we get a null value passed in for the onPartialResult callback function.
Example logs output from both the onPartialResult and the onResult on iOS:
 LOG  partial result:  null
 LOG  partial result:  null
 LOG  partial result:  null
 LOG  partial result:  null
 LOG  partial result:  null
 LOG  final result:  check check one to one two

So we can see that the event is firing but no value for res in
const partialResultEvent = vosk.onPartialResult(async (res) => {
           console.log('partial result: ', res);
});
const resultEvent = vosk.onResult(async (res: string) => {
           console.log('final result: ', res);
});

Any guidance or workaround would be greatly appreciated!

What are our thoughts on the timing of this - should we give them till the weekend to update the release, or do we want to temporarily check-in a locally built library until then?

I'll Check on that wednesday, sorry for the delay 😅
Le lun. 25 mars 2024, 23:04, Jon Campbell @.***> a
écrit :

What are our thoughts on the timing of this - should we give them till the
weekend to update the release, or do we want to temporarily check-in a
locally built library until then?
—
Reply to this email directly, view it on GitHub
https://github.com/riderodd/react-native-vosk/issues/47#issuecomment-2018999667,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAT7RRRG47ETFSBYYNO3DN3Y2CNQLAVCNFSM6AAAAABFB5PXMWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMJYHE4TSNRWG4
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

Thanks @riderodd! Btw it looks like #48 fixes it - tested on my devices and partials are coming through

Fixed on 0.2.2


Extending DocList with itself causes unexpected recursion

Description
When extending a DocList with itself, the code seems to add the list recursively, causing a huge increase in the length of the list. For example, the following code produces a DocList of length 9,052,000 (I had to interrupt the execution):
from docarray import DocVec, BaseDoc, DocList
from docarray.documents.image import ImageDoc
docs = DocList[ImageDoc]([ImageDoc(url=f'http://url.com/{i}.png') for i in range(10)])
docs.extend(docs)

However, a similar operation works fine with Python lists:
l = list(range(10))
l.extend(l)
print(l)  # [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

Questions

Why does DocList behave differently than Python lists in this situation?
Are there plans to support the extend method for DocVec in the future?

Steps to reproduce

Run the following code snippet:

from docarray import DocVec, BaseDoc, DocList
from docarray.documents.image import ImageDoc
docs = DocList[ImageDoc]([ImageDoc(url=f'http://url.com/{i}.png') for i in range(10)])
docs.extend(docs)

Interrupt the execution because it hangs.

Observe that the resulting DocList has an enourmous length, rather than the expected 20.

Hey @movchan74 ,
Thanks for the very helpful reports. We recently merged a PR #1457, which I suspect may fix the issue. Maybe u can try on the main branch? otherwise I will try tomorrow. In case it does solve it, this will be released very soon.
As per plans for supporting extend of DocVec, there is no explicit plan for it, but if community asks we would be happy to consider adding it.

I've tried it on the main branch but the issue is still present.
Regarding the extend for DocVec, it would be really nice to have the same behavior for DocVec and DocList.

Comment on my side:
This is the error we are making in the code, with an equivalent code and result:
class MyList(list):
    def extend(self, __iterable) -> None:
        def generator(aux):
            for a in aux:
                yield a

        return super().extend(generator(__iterable))

    def append(self, __object) -> None:
        return super().append(__object)

l = MyList([1, 2, 3, 4])

l.extend(l)

Hey @movchan74 ,
Let me try to answer the 2 questions:

Why does it behave differently? Because we do some validations and we iterate over the Documents in the container as a generator, which makes it infinite since it keeps adding to the list. This will be fixed soon.

As if we want to add extend to DocVec. We are not so sure about this, DocVec is much more planned to be static in terms of size, and will be complex to provide a coherent list-like behavior. In past versions we tried to fit every pattern in a single type of object, and we now decided to clarify the interfaces. We believe it is better to do DocVec.to_doc_list, do the extension and then convert back. However, we welcome you to join our discord channel to discuss this or any other design choices.

Are there plans to support the extend method for DocVec in the future?

Here we want to be cautious about how we integrate an extended method on DocVec because extending on a DocVec will be slower that on a DocList (because of underlying implementation)
The way I see it we could either have:
1- extend as a method of DocVec that add the new DocVec to self
2- cat function like in torch/numpy that will fuse to DocVec into one
1 is more coherent with DocList. 2 is more intuitive and gives more insight into what is happening under the hood.
@movchan74 any opinion on what you would prefer ?

I think extend makes more sense to keep it consistent with DocList.

@movchan74 ,
Our concern is that we wanted in purpose not to have consistency with DocList so that it is clear that they are different things and behave differently.
We have the impression that adding appends and extends support people will expect list-like behavior which we cannot ensure

BTW; I created this discussion https://github.com/docarray/docarray/discussions/1495
So that this valuable discussion is not hidden in a not-so-related bug report


Feature extraction

Dear @zhaoyue-zephyrus ,
I want to extract features (for a custom dataset) generated by TSN. I reckon that the tensor containing features is present here (https://github.com/open-mmlab/mmaction/blob/eb7ace3458df6d551b99a1026abee2b209376a6b/mmaction/models/recognizers/TSN2D.py#L123). While running my dataset on TSN2D, the dimension of the tensor that i get after the above referenced line is (32768 x 1). And there are 22 such tensors getting generated for each epoch of training. I dont understand what these numbers (22 & 32768)mean. Kindly help me.
TESTING:
I reckon that the features getting generated for videos in test split will be present here(https://github.com/open-mmlab/mmaction/blob/eb7ace3458df6d551b99a1026abee2b209376a6b/mmaction/models/recognizers/TSN2D.py#L155). By printing this, I'm getting tensors of dimension (62 x 1). And there are 384 such tensors getting generated.
And the values in test set tensors were mostly negative... Do they mean anything?

Plz help me understand these tensors & their dimensions.
P.S:
My dataset has 61 classes. In the training split, each class has number of videos in the range of (5-200). Number of vids in test split of approximately 1/3rd of those.

Hi, @sparsh-b did you find solutions to your question? I am also trying to figure out the same, please let me know.

Hello @akashsharma29 !
I'm sorry. I wasn't able to figure it out either. I regret being unable to help you.


TIKA-3403 Create example for Transcription

This issue addresses https://issues.apache.org/jira/browse/TIKA-3403
In addition to implementing the example file, it proposes the following improvements

minor upgrade of aws libraries to 1.11.1018
adds a new configuration option for the AWS transcriber allowing client to write to a specific region cf. transcribe.REGION
makes use of SelectObjectContentRequest which filters the contents of an Amazon S3 object (transcription) based on a simple Structured Query Language (SQL) statement. In the request, along with the SQL expression, we specify JSON as the data serialization format of the object. Amazon S3 uses this to parse object data into records, and returns only records that match the specified SQL expression. In our case this means we ONLY return the transcription text. This dramatically (orders of magnitude) reduces the amount of data we egress from s3 to client.
the implementation will now automatically create the bucket (to store the transcription) if one does not already exist. This is a merely a utility feature.
introduces a LOT of exception handling and checks which will assist the client in debugging errors/anomalies.
Reformatted GoogleTranslator.java with 4-space indents.

Thanks about it.
CC @rohan2810 FYI

@rohan2810 can you please tag the rest of the HackIllinois crew? Thanks

Sure @lewismc
@phantuanminh @abehara2 @nprate2


Rails 3.2: can't write unknown attribute `gmaps' when using attr_accessor

I might be mistaken, but I don't think the following will work in rails 3.2.0 when using an attr_accessor for :checker. (attr_accessor :gmaps for instance):
if Gmaps4rails.condition_eval(self, gmaps4rails_options[:check_process])
    self[gmaps4rails_options[:checker]] = true
end

https://github.com/rails/rails/issues/4583

I see but didn't test myself, do you have some suggestion to fix this?

I'm not using attr_accessor but I'm seeing a similar exception:
NoMethodError (undefined method `gmaps' for #<User:0x007fd06c47c8b0>)
activemodel (3.2.1) lib/active_model/attribute_methods.rb:407:in `method_missing'
activerecord (3.2.1) lib/active_record/attribute_methods.rb:126:in `method_missing'
gmaps4rails (1.4.3) lib/gmaps4rails/acts_as_gmappable.rb:15:in `process_geocoding'
...

I guess that in your case, the part failing is:
self.send(gmaps4rails_options[:checker]) == true

But honestly, I can't see why if you have a gmaps column in your User model...

I'll install Rails 3.2 to have a look.

It may have been related to the same problem, I'll give it a try and see if it's fixed now.


Remove comment blocks from module - Backlog Item 83064

Overview/Summary
Remove multi-line comments from the top of each module that contains module info as duplicated info (version, author etc.)
Replace this with a brief description of what this Pull Request fixes, changes, etc.
This PR fixes/adds/changes/removes

Removed comment blocks from ALZ Bicep modules
Updated Contributing guide to remove the comment block from Bicep authoring best practices

Breaking Changes

Comment block removal from modules
Contributing guide updated to reflect removal of comment blocks

Testing Evidence

As part of this Pull Request I have

[X] Read the Contribution Guide and ensured this PR is compliant with the guide
[X] Checked for duplicate Pull Requests
[N/A] Associated it with relevant GitHub Issues
[X] (ALZ Bicep Core Team Only) Associated it with relevant ADO Items
[X] Ensured my code/branch is up-to-date with the latest changes in the main branch
[X] Performed testing and provided evidence.


Improve performance of compareTo

This PR improves performance of compareTo. This is accomplished by saving the expensive call to compareUnsignedLongs for longs that are know to be equal.
This saves 0 to 40 percent of time, depending on the joint portion of the slices.

Can you add a JMH benchmark?

@martint Provided the class for benchmark and the results. I have also added a commit with direct use of address with unsafe.
Original Version
Benchmark                 Mode  Cnt   Score   Error  Units
BenchmarkSlice.compareTo  avgt   20  32,132 ± 0,044  ns/op

Reorded compare
Benchmark                 Mode  Cnt   Score   Error  Units
BenchmarkSlice.compareTo  avgt   20  28,729 ± 0,323  ns/op

Using directly address with unsafe
Benchmark                 Mode  Cnt   Score   Error  Units
BenchmarkSlice.compareTo  avgt   20  26,221 ± 0,240  ns/op

I made a few changes and merged it. Thanks!
BTW, here's a more comprehensive benchmark result for different sizes:
Before:

Benchmark                 (size)  Mode  Cnt     Score     Error  Units
BenchmarkSlice.compareTo       1  avgt   20     6.248 ±   0.535  ns/op
BenchmarkSlice.compareTo       7  avgt   20    14.806 ±   0.233  ns/op
BenchmarkSlice.compareTo       8  avgt   20     5.967 ±   0.098  ns/op
BenchmarkSlice.compareTo      16  avgt   20    10.031 ±   0.171  ns/op
BenchmarkSlice.compareTo      32  avgt   20    15.668 ±   0.295  ns/op
BenchmarkSlice.compareTo      64  avgt   20    24.333 ±   0.675  ns/op
BenchmarkSlice.compareTo     127  avgt   20    50.837 ±   0.741  ns/op
BenchmarkSlice.compareTo   32779  avgt   20  8359.595 ± 122.491  ns/op

After:

Benchmark                 (size)  Mode  Cnt     Score    Error  Units
BenchmarkSlice.compareTo       1  avgt   20     6.036 ±  0.097  ns/op
BenchmarkSlice.compareTo       7  avgt   20    11.913 ±  0.195  ns/op
BenchmarkSlice.compareTo       8  avgt   20     6.176 ±  0.133  ns/op
BenchmarkSlice.compareTo      16  avgt   20     7.561 ±  0.128  ns/op
BenchmarkSlice.compareTo      32  avgt   20     9.654 ±  0.132  ns/op
BenchmarkSlice.compareTo      64  avgt   20    14.359 ±  0.361  ns/op
BenchmarkSlice.compareTo     127  avgt   20    27.731 ±  0.361  ns/op
BenchmarkSlice.compareTo   32779  avgt   20  3666.218 ± 70.419  ns/op


Fix merging multi-value longhand into single-value shorthands

Here's an example from background property:
.block {
  background:url(one.png);
  background-repeat:no-repeat,repeat-x;
}

which when optimized ends up as:
.block {
 background:url(one.png) no-repeat, url(one.png) repeat-x;
}

which should rather be:
.block {
 background:url(one.png) no-repeat, repeat-x;
}

~or maybe second value can be dropped altogether since the shorthand doesn't declare a 2nd value?~
EDIT: in this particular case it may be OK to drop it but not in general, e.g. imagine having a two-value background-image declaration afterwards. So I believe the solution above is the correct one.
.block {
 background:url(one.png) no-repeat;
}

Here's the opposite case:
.block {
  background:repeat-x,repeat-y;
  background-image:url(one.png);
}

should be
.block {
  background:url(one.png) repeat-x,repeat-y;
}

However background-url stands out as the only property which behaves differently when pulling into a shorthand, e.g background-repeat:
.block {
  background: url(one.png) repeat-x, url(two.png) repeat-y;
  background-repeat: no-repeat;
}

equals to
.block {
  background: url(one.png) no-repeat, url(two.png) no-repeat;
}

Here are all background-* property examples - https://jsfiddle.net/e8vkxyy4/2/


pyarrow does not know how to serialize objects of type

sql_lab and also visualization doesn't work because of pyarrow can't serialize objects
Expected results
return query result
Actual results
pyarrow does not know how to serialize objects of type <class 'pandas.core.arrays.integer.IntegerArray'>.

Screenshots

How to reproduce the bug

Go to sql_lab
Run a query like SELECT *
See error

Environment

superset version: branch master on 2019-10-12 18:38
python version: 3.7
node.js version: 10.16.3
npm version: 6.9.0

Checklist
Make sure these boxes are checked before submitting your issue - thank you!

[x] I have checked the superset logs for python stacktraces and included it here as text if there are any.
[x] I have reproduced the issue with at least the latest released version of superset.
[x] I have checked the issue tracker for the same issue and I haven't found one similar.

Please share database type and column types of the table causing the error.

It doesn't matter! this error happens on this code:
data = (
                pa.default_serialization_context()
                .serialize(cdf.raw_df)
                .to_buffer()
                .to_pybytes()
            )

here is the refrenc uri on sql_lab:
https://github.com/apache/incubator-superset/blob/master/superset/sql_lab.py#L275
Also, you can test this issue with trying this code:
import pandas as pd
import pyarrow as pa
from  pandas.core.arrays.integer import IntegerArray

int_array = pd.array([1, None, 3], dtype=pd.Int32Dtype())
pa.default_serialization_context().serialize(int_array).to_buffer().to_pybytes()

Here is the arrow issue tracker Uri, that I reported this bug:
https://issues.apache.org/jira/browse/ARROW-6900?page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel&focusedCommentId=16952842#comment-16952842

Thanks for reporting this issue. This may be related to https://github.com/apache/incubator-superset/issues/8225. Are you using Presto by chance?
You can disable msgpack and pyarrow serialization by setting RESULTS_BACKEND_USE_MSGPACK = False in superset_config.py until this issue is resolved.

Yes. I'm using Presto. But I tried the sqlite too.

@blcksrx This should now be resolved via https://github.com/apache/incubator-superset/pull/8733


CSE Machine: enable visualization when running programs with runtime errors

Description

Type of change

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[X] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update
[ ] Code quality improvements

How to test

Checklist

[X] I have tested this code
[ ] I have updated the documentation

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

2 of 11 (18.18%)  changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.003%) to 36.305%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

src/commons/sagas/WorkspaceSaga.ts
1
4
25.0%

src/commons/sideContent/content/SideContentCseMachine.tsx
1
7
14.29%

Totals

Change from base Build<PHONE_NUMBER>:
0.003%

Covered Lines:
4965

Relevant Lines:
12810

💛  - Coveralls


[Event Request] Codeunit 5836 , 1 new Event in Function „CalcProdOrderLineExpCost“

We need new some "forgotten Events from the old Issue-Repo" in Codeunit 5836 to handle own ISV Code (https://github.com/Microsoft/AL/issues/3351)

Function CalcProdOrderLineExpCost

The Event needs the following parameters:

Function CalcActTimeAndQtyBase

The Event needs the following parameters:

Function CalcActOperOutputAndScrap

The event needs the following parameters:

Thanks

Request 1 is now available but Requests 2 and 3 for filtering are still missing in BC14.3!
Could you please give us any Infos why??
Thanks

Last 2 events have been implemented at July 19, so probably they will be available in next update only.

Would it be possible to add some more info like "planned in release 14.3"?? This would help us partners very much!


Do versions prior to react native 0.68 support installation?

Do versions prior to react native 0.68 support installation?

Please have a try, out SDK don't require RN version.


Authentication required

Hi all!
When a composer need an secure repo to install, it doesn't seems to work. I have this error:
[Composer\Downloader\TransportException]
The 'https://repo.magento.com/packages.json' URL required authentication.
You must be using the interactive console to authenticate
I checked and I created the auth.json before. If I run on CLI, it work just fine. I looked on google and every solution seems to point to an absent auth.json but it work on CLI for me. So it must be something else. Composer doesn't like puppet for authentication?
Anyone have a solution for this?
Thanks!

I fixed the problem by explicit set the "user" in the class{ composer }. I don't know why but the user variable was empty and was keeping using root and a mixed with my user. It wasn't finding the auth.json for authenticating to the repo.
I'm not sure what lesson we can get from that. Is that the default behavior to run as root with include composer? And then use the user in composer::project ?
This is what I need to install composer:
class { 'composer':
user  => "$owner",
composer_home => "/home/$owner/.composer",
suhosin_enabled => false,
}
then my project:
composer::project { 'magento2':
project_name   => 'magento/project-community-edition',  # REQUIRED
target_dir     => "$docroot", # REQUIRED
version        => '2.0.7', # Some valid version string
prefer_source  => true,
stability      => 'stable', # Minimum stability setting
keep_vcs       => false, # Keep the VCS information
dev            => false, # Install dev dependencies
repository_url => "https://repo.magento.com/", # Custom repository URL
user           => "$owner", # Set the user to run as
}


Make UI More Responsive

refactor: Improve responsiveness and layout in multiple components
In this commit, I made the following improvements:
1. FooterContent:

Enhanced responsiveness by adjusting links for better display on various screen sizes.

2. TabAccount:

Improved button and item responsiveness to enhance the user experience on mobile devices.

3. TabInfo, TabSecurity, and TabAccount:

Replaced the use of marginRight with a flex-based layout to provide consistent spacing between elements.
Added gap between elements to maintain proper spacing when items wrap.

4. StatisticsCard:

Modified the layout to display items in a 2/row grid instead of 1/row.
This change reduces the card's height and enhances its overall appearance.

These changes contribute to better responsiveness and a more polished user interface in the mentioned components.

Hi @sourabhkumhar,
Thanks for showing an interest in contributing.
As we have updated the template with the App Router of Next.js, we have considered responsiveness for all the scenarios. Also, images are draggable by default and most users keep draggable itself.
Thus, we are not considering merging this PR.


Current CUDA Device does not support bfloat16. Please switch dtype to float16

Here's where you set the dtype:


Is this library still maintained

Hi,
Is this library still maintained or will it not be updated anymore?
Thanks,

I recently submitted a pull request for migration to  Androidx and support for onscreen fingerprint, but it has not been approved. I'm going to wait till the month-end. If they don't merge it by then, I will move the library to a separate repository and continue managing it.

Hello @devmike01,
Thanks for your answer. For my own need, I just switched to the new androidx biometric library. It is what I need and it's an official library.
So I think, I can just close this issue.
P.S: you should not wait and directly manage the library on your own repo.

Good one.


WX style Base Reflectivity palette

;NEXRAD Level-3 palette for product code 56 - NxS Spectum Width
Product: BR
Units: DBZ
Step: 5
Color: -32 0 0 0
Color: -20 64 64 80
Color:  -6 100 100 130
Color:  -5 31 42 104
Color:   0 53 77 164
Color:  11 79 118 236
Color:  12 3 101 31
Color:  20 41 172 49
Color:  31 80 245 69
Color:  32 255 247 39
Color:  40 255 176 28
Color:  50 255 84 13
Color:  63 188 0 0
Color:  64 231 7 203
Color:  75 253 132 250
Color:  76 190 158 245
Color:  80 0 235 230
Color:  81 255 255 255
Color:  95 255 255 255

Thanks! I'm currently working on a page to host user-submitted color tables.
For the color tables that are found elsewhere (e.g., #173, #174), for now, I would prefer to link to those sites to not drive traffic away from them.
I currently link to:

https://grlevelxusers.com/grlevelx-goodies/categories/color-tables/
https://www.wxtools.org/

Am I missing any sources?
I currently link to them here (I'll probably promote these links to the User Guide to make them easier to find):

https://supercell-wx.readthedocs.io/en/stable/getting-started/initial-setup.html#customization


Use API client libraries in place of lower level code for API integrations.

We are currently using "raw"/request based integrations for accessing GitHub and presumably other APIs. It would be preferable to use API providers' client libraries to reduce low-level code, and simplify implementation.  GitHub would be the obvious place to start.  There are several candidates listed here: https://developer.github.com/libraries/
#payforpullcandidate.

@BK01 will be fleshing this issue out as a possible candidate for P4P


Fix prepare email none

 In the button Action for performing action (cancellation, acceptation, send an email) by the recommenders -> reviewer, remove the possibility to send email when there is a "None" in place of the email of the reviewer


EAS Testing for OnePlus3T - error running generic.py

I am working on implementing EAS for the OP3T which uses the same SD821 SoC. So with that I have been able to setup the testing environment but I am running into an issue when running:
lisa-test tests/eas/generic.py
It will get through the testing and give me some information.  But this error, I know nothing about. I tried to research it but I am apparently lacking in python and panda  data frames. if you have time, i would love any assistance. Here it is:
Traceback (most recent call last):
File "/home/renderbroken/lisa/libs/utils/test.py", line 219, in experiment_test
wrapped_test(experiment, tasks, *args, **kwargs)
File "/home/renderbroken/lisa/tests/eas/generic.py", line 513, in test_task_placement
self._test_task_placement(experiment, tasks)
File "/home/renderbroken/lisa/tests/eas/generic.py", line 291, in _test_task_placement
exp_power = self.get_expected_power_df(experiment)
File "/home/renderbroken/lisa/tests/eas/generic.py", line 256, in get_expected_power_df
task_utils_df.apply(exp_power, axis=1))
File "/usr/local/lib/python2.7/dist-packages/pandas/core/frame.py", line 4152, in apply
return self._apply_standard(f, axis, reduce=reduce)
File "/usr/local/lib/python2.7/dist-packages/pandas/core/frame.py", line 4248, in _apply_standard
results[i] = func(v)
File "/home/renderbroken/lisa/tests/eas/generic.py", line 251, in exp_power
expected_utils = nrg_model.get_optimal_placements(task_utils)
File "/home/renderbroken/lisa/libs/utils/energy_model.py", line 678, in get_optimal_placements
power = self.estimate_from_cpu_util(util, freqs=freqs)
File "/home/renderbroken/lisa/libs/utils/energy_model.py", line 613, in estimate_from_cpu_util
freqs, idle_states, combine=True)
File "/home/renderbroken/lisa/libs/utils/energy_model.py", line 553, in _estimate_from_active_time
_idle_power = max(node.idle_states[idle_states[c]] for c in cpus)
File "/home/renderbroken/lisa/libs/utils/energy_model.py", line 553, in
_idle_power = max(node.idle_states[idle_states[c]] for c in cpus)
KeyError: ('C2', u'occurred at index 0.448395')
Kind Regards,
Zachariah

Hi Zachariah,
Cool to hear that you're trying LISA out on a new device! I guess you used the EnergyModel that LISA provides for Pixel?
I suspect this error is due to a difference between the Pixel and OnePlus kernels in how CPU IDs are reported in ftrace. I'll try to find time to stare at the code and figure it out, then get back to you with more questions :)

Actually, please disregard. I found it was a typo error on my side. I forgot to rename a certain line from pixel to op3:
-pixel_energy = EnergyModel(
+op3_energy = EnergyModel(
Here is a commit that shows more info:
https://github.com/RenderBroken/lisa_toolkit/commit/564c88691d1276b4f1d8ee23e8f16e84f169c169
Thanks again though for the amazing fast response! I am hoping to contribute what little I can to this project. I am weak at python and really new to all of this but I am sure I could possibly help with documentation. Getting Lisa cloned to working like it is today was a journey. Maybe I can make that easier for others to be able to give it a shot and hopefully contribute as well.
Kind Regards,
Zachariah


Support -data in kube config

Currently, it only supports certificate-authority, client-certificate-... referencing filenames. It would be nice if it supported certificate-authority-data and the like. The current behavior is that it runs unverified requests.

Could you provide more information on what exactly fails for you? *-data should be supported. See BytesOrFile in pykube.config.

When I use a -data cert I see Python warning messages about SSL not being
validated so I assumed it was because the cert wasn't being loaded.
On Tue, Feb 21, 2017, 5:16 AM Brian Rosner<EMAIL_ADDRESS>wrote:

Could you provide more information on what exactly fails for you? *-data
should be supported. See BytesOrFile in pykube.config.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/kelproject/pykube/issues/114#issuecomment-281328886,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAeFZ3VLq7F-sS5sqlpPbQDoUuj0PvLqks5retWxgaJpZM4Lx_le
.


Back button needs 2 presses to go back to another page (or quit)

Testing this after your fix, I'm still seeing something odd: say I'm on page A (not The Lounge), then type the URL of The Lounge (localhost when testing), then I have to hit "Back" twice to go back to A.

For anyone who tries to fix this, a good starting point to understand the issue would be to change these lines to trigger click via
.trigger("click", {
  pushState: false,
})

to avoid pushing a state entry to the history when loading the page, and observe the effects.


Reset Timer When Receiving new Until Props

When passing in a new time based on a state change it doesn't update the timer to reflect state change.

@etcetera8 added beb2190a872a9630a05f3cc74ae42054873bc157


Change unittests for Device

More mocking is used to be able to test device without dependencies

Codecov Report
Base: 55.87% // Head: 52.18% // Decreases project coverage by -3.69% :warning:

Coverage data is based on head (5292573) compared to base (ae248a9).
Patch has no changes to coverable lines.

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #16      +/-   ##
==========================================
- Coverage   55.87%   52.18%   -3.70%
==========================================
  Files           9        9
  Lines         596      596
  Branches       79       79
==========================================
- Hits          333      311      -22
- Misses        263      285      +22

Impacted Files
Coverage Δ

PyStageLinQ/Network.py
13.67% <0.00%> (-18.81%)
:arrow_down:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

/fast-forward

/fast-forward


Tweak WithVolume method for usability

Fixes #2738
Fixes #2739
Microsoft Reviewers: Open in CodeFlow

I originally allowed a null name, but feedback was anonymous volumes weren't supported by DCP. I never tried. Have you tested DCP is happy with a null name?
cc @karolz-ms

I originally allowed a null name, but feedback was anonymous volumes weren't supported by DCP. I never tried. Have you tested DCP is happy with a null name?

@JamesNK yes I tested it locally and it works

FYI you might need to resolve conflicts from https://github.com/dotnet/aspire/pull/2809
(or I will, depending on which merges first)


Provide seccomp profile for the operator

What would you like to be added:
I think a default profile for the operator itself could be a good starting point for a demo. It should be highly restrictive and we could create some documentation around how to build such profiles as well as providing additional application profiles out of the box.
Why is this needed:
We should aim for the maximum amount of security for the operator deployment, a seccomp profile will help with that.

/kind feature

/priority important-soon

/assign