Curves syscall: EdwardPoint and RistrettoPoint compression

Problem
It would be nice to be able to decompress pubkeys to curve points within the Solana SVM runtime.
The Edward curve syscalls already provide the computation required to decompress() a standard public key to an EdwardsPoint. However, this computation is discarded and only a boolean value is returned to the caller.
Context
With feature gate 7rcw5UtqgDTBBv2EcynNfYckgdAaH1MAsCjKgXMkN7Ri, the curve25519 group ops and validation syscalls were rolled out. These syscalls offer a ton of potential but are not usable with standard pubkeys as specified by RFC 8032.
Proposed Solution
It would be nice to offer a different syscall to compress/decompress pubkey slices or to change the implementation so that the validate function also returns the EdwardsPoint itself.
cc @samkim-crypto @cavemanloverboy

Having had some time to dive deeper on this, it seems that the crux of the issue is in defining what compress and decompress would look like at the syscall level. Specifically, the uncompressed ed25519_dalek EdwardPoint/RistrettoPoint struct's don't have a direct serialization, instead, they use compress() internally. This was probably done to ensure FieldElement could be of variable type internally.
Perhaps we could assume FieldElement51 and serialize to [u32; 5] or [u64; 5]. Also, it is not clear to me if returning a ed25519_dalek type is preferred over a generic POD type.
I'd love feedback on what the preferred approach would be.

What you pointed out is correct. There are multiple representation of decompressed curve points and conversions between the representations are generally done on-the-fly when different ops are applied to them.
The curve25519 syscalls were designed to work with the compressed (i.e. POD) types. The syscalls for add, subtract, multiply, etc. all take in pod types, decompresses them, applies the relevant ops, and then compresses them again. Except for advanced zk applications, these ops should be sufficient for most standard cryptographic applications (sig verification, vrf, etc.). It is unclear to me what the additional decompress() operation could provide.

@samkim-crypto
Sounds like Code are working on an advanced Zero-Knowledge (ZK) feature in their new protocol - as you pointed out, some protocols might require working with uncompressed point representations for computations or proofs.
Other benefits for the general population - custom cryptographic protocols, better interoperability, audit/debugging.
Perhaps a solution is to:

Define compress() to return a fixed-size byte array ([u8; 32] for compressed form).
Define two versions of decompress():
a. One that returns the appropriate ed25519_dalek type (EdwardPoint or RistrettoPoint).
b. Another that returns a generic POD type (e.g. a struct containing the coordinates).

NB it's easier to get from u8 to u32 or u64 and you'd worry less about endianness.

@samkim-crypto Thanks for the context, I appreciate it. I agree that it makes sense to use the compressed representation as the default.
It might be helpful to walk through an example use-case. As you've mentioned, one potential use case is that of sig verification. So lets take a look at what that might look like.
Lets assume I have a dynamically generated message that I want to verify inside an instruction. In this case, the use of the native ed25519 pre-compute instruction is out of the question (it requires the message, pubkey, and signature to be hard-coded).
Here is what a dynamic signature verify would look like:
    // Lets assume these are not hard-coded
    let message: &[u8]   = b"hello world";
    let pubkey: [u8; 32] = [
        73, 73, 170, 112, 75, 235, 154, 81, 203, 8, 44, 245, 233, 18, 204, 136,
        162, 9, 233, 49, 154, 201, 171, 175, 47, 6, 223, 101, 105, 80, 95, 166
    ];
    let sig: [u8; 64] = [
        164, 121, 89, 242, 88, 29, 80, 177, 104, 20, 102, 176, 48, 133, 68, 8,
        105, 33, 58, 86, 28, 108, 198, 140, 160, 219, 62, 184, 154, 181, 140, 33,
        35, 102, 183, 203, 111, 33, 55, 170, 180, 138, 92, 196, 185, 201, 122, 167,
        15, 112, 9, 228, 226, 112, 111, 10, 142, 73, 85, 43, 81, 152, 204, 13
    ];

    // Normally, we could verify the signature using the Solana SDK
    // (This is not SVM compatible; expensive internally)
    let sdk_sig = solana_sdk::signature::Signature::from(sig);
    assert!(sdk_sig.verify(&pubkey, &message));

    // Instead, lets verify the signature "by hand" using the new edward curve syscalls
    // (could be SVM compatible, except for the expensive decompress/compress functions)
    let pubkey_point: EdwardsPoint = CompressedEdwardsY(pubkey)
        .decompress() // <-- Expensive decompress()
        .unwrap();

    let mut sig_lower: [u8; 32] = [0u8; 32];
    let mut sig_upper: [u8; 32] = [0u8; 32];
    sig_lower.copy_from_slice(&sig[..32]);
    sig_upper.copy_from_slice(&sig[32..]);

    let sig_R = CompressedEdwardsY(sig_lower);
    let sig_s: Scalar;
    if sig[31] & 240 == 0 {
        sig_s = Scalar::from_bits(sig_upper)
    } else {
        sig_s = Scalar::from_canonical_bytes(sig_upper).unwrap();
    }

    let minus_A: EdwardsPoint = -pubkey_point;
    let signature_R : EdwardsPoint = sig_R.decompress().unwrap(); // <-- Expensive decompress()
    if signature_R.is_small_order() || pubkey_point.is_small_order() {
        panic!("Signature verification failed: small order point");
    }

    let mut h: Sha512 = Sha512::new();
    h.update(sig_R.as_bytes());
    h.update(&pubkey);
    h.update(&message);

    let f = h.finalize();
    let k = Scalar::from_bytes_mod_order_wide(f.as_ref());

    let k_bytes = k.to_bytes();
    let minus_A_bytes = minus_A.compress().to_bytes(); // <-- Expensive compress()
    let sig_s_bytes = sig_s.to_bytes();
    let g_bytes = G.compress().to_bytes(); // basepoint, could technically be precomputed

    let a = PodScalar(k_bytes);
    let A = PodEdwardsPoint(minus_A_bytes);
    let b = PodScalar(sig_s_bytes);
    let B = PodEdwardsPoint(g_bytes);

    // R = aA + bB
    // R = (k * minus_A) + (sig_s * G)

    let aA = multiply_edwards(&a, &A).unwrap();
    let bB = multiply_edwards(&b, &B).unwrap();
    let R = add_edwards(&aA, &bB).unwrap();

    let lhs = R.0;
    let rhs = *(signature_R.compress().as_bytes());

    println!("{:?}", lhs);
    println!("{:?}", rhs);

    assert!(lhs == rhs);

Is there a world where this type of thing could be enabled within the SVM?

Okay, it's a bit clunky but works.
        let message: &[u8]   = b"hello world";
        let pubkey: [u8; 32] = [
            73, 73, 170, 112, 75, 235, 154, 81, 203, 8, 44, 245, 233, 18, 204, 136,
            162, 9, 233, 49, 154, 201, 171, 175, 47, 6, 223, 101, 105, 80, 95, 166
        ];
        let sig: [u8; 64] = [
            164, 121, 89, 242, 88, 29, 80, 177, 104, 20, 102, 176, 48, 133, 68, 8,
            105, 33, 58, 86, 28, 108, 198, 140, 160, 219, 62, 184, 154, 181, 140, 33,
            35, 102, 183, 203, 111, 33, 55, 170, 180, 138, 92, 196, 185, 201, 122, 167,
            15, 112, 9, 228, 226, 112, 111, 10, 142, 73, 85, 43, 81, 152, 204, 13
        ];

        // Normally, we could verify the signature using the Solana SDK
        // (This is not SVM compatible; expensive internally)
        let sdk_sig = solana_sdk::signature::Signature::from(sig);
        assert!(sdk_sig.verify(&pubkey, &message));

        // Instead, lets verify the signature "by hand" using the new edward curve syscalls
        let pubkey_point = PodEdwardsPoint(pubkey);
        let neg_one = PodScalar((-Scalar::one()).to_bytes());

        let mut sig_lower: [u8; 32] = [0u8; 32];
        let mut sig_upper: [u8; 32] = [0u8; 32];
        sig_lower.copy_from_slice(&sig[..32]);
        sig_upper.copy_from_slice(&sig[32..]);

        let sig_R = PodEdwardsPoint(sig_lower);
        let sig_s: Scalar;
        if sig[31] & 240 == 0 {
            sig_s = Scalar::from_bits(sig_upper)
        } else {
            sig_s = Scalar::from_canonical_bytes(sig_upper).unwrap();
        }

        /*
        // todo...
        if signature_R.is_small_order() || pubkey_point.is_small_order() {
            panic!("Signature verification failed: small order point");
        }
        */

        let mut h: Sha512 = Sha512::new();
        h.update(sig_R.0);
        h.update(&pubkey);
        h.update(&message);

        let f = h.finalize();
        let k = Scalar::from_bytes_mod_order_wide(f.as_ref());
        let minus_A = multiply_edwards(&neg_one, &pubkey_point).unwrap();

        let k_bytes = k.to_bytes();
        let minus_A_bytes = minus_A.0;
        let sig_s_bytes = sig_s.to_bytes();
        let g_bytes = G.compress().to_bytes(); // basepoint, could technically be precomputed

        let a = PodScalar(k_bytes);
        let A = PodEdwardsPoint(minus_A_bytes);
        let b = PodScalar(sig_s_bytes);
        let B = PodEdwardsPoint(g_bytes);

        // R = aA + bB
        // R = (k * minus_A) + (sig_s * G)

        let aA = multiply_edwards(&a, &A).unwrap();
        let bB = multiply_edwards(&b, &B).unwrap();
        let R = add_edwards(&aA, &bB).unwrap();

        let lhs = R.0;
        let rhs = sig_R.0;

        println!("{:?}", lhs);
        println!("{:?}", rhs);

        assert!(lhs == rhs);

@samkim-crypto given that I don't see a path forward with serializing uncompressed points, I'm going to close this (unless anyone has objections).
Both you and @cavemanloverboy cleared a lot up for me; much appreciated!


Add timeout to rpc client and fn wait_for_event

adds a timeout to the WsRpcClient. By default it is 300 secs ( = 5 minutes). Finalization may take quite some time, so I think that's a reasonable amount of time.
the wait_for_event functions now take an additional optional timeout parameter. Because the event may be expected to happen somewhen far in the future, I did not want to introduce a default timeout just like that.

closes #134

Closig this PR - doesn't make sense currently because connect() from ws is blocking. So recv_timeout does not help. As the ws server will be exchanged anyway I think, I'll close for now.


test: add apply-mapping test condition for custom fields in data file

/test release-pipelines-e2e-suite

/test release-pipelines-e2e-suite


[QUESTION] Background task not executed on high load

I have a simple function that inserts in the database (RethinkDB) a generated user.
The functions looks like this:
`    @router.get('/api/v1/create_load_testing_user')
async def create_load_testing_user(request: Request, background_tasks: BackgroundTasks):
    api_user_1 = copy.deepcopy(default_user_model)
    api_user_1["uid"] = get_uuid_number()
    api_user_1["profile"]["name"] = "API USER 1"
    api_user_1["login"]["client_id"] = get_uuid_number()
    api_user_1["login"]["client_secret"] = get_uuid_number()
    api_user_1["is_performance_test_user"] = True

background_tasks.add_task(
    login_test_performance_insert,
    api_user_1,
    request
)

return api_user_1

async def login_test_performance_insert(api_user_1, request):
    await r.table("users").insert(api_user_1).run(await get_rethink_conn(request))
    return True

`
Now, i am doing a performance test with AB:
ab -n 8000 -c 1000 https://link
And i am getting:
Complete requests:      8000 Failed requests:        6294    (Connect: 0, Receive: 0, Length: 6294, Exceptions: 0) Non-2xx responses:      1706
This means that i just lost 1706 user entries in the db
What am i doing wrong?
I have a http middleware that acquires a db connection, from a pool, before executing the route and releases it after the function is completed

Any reason why you insert in db using a background task?
If not I'd try without , background tasks run in a seperate thread, you may exhaust them?

Because i want to return an answer faster.
Without the background tasks i would get around 250 req/sec and with backgroundtasks i get 550 req/sec (with 8000 users in db ).

Interesting, will try tomorrow to look at it, in the meantime what is ab? I
just know wrk, but I guess it's similar?
And how do you serve your app, gunicorn behind uvicorn or just uvicorn,
well could you share a little bit more about those settings :)
Le mar. 24 sept. 2019 à 9:15 PM, razvanavram<EMAIL_ADDRESS>a
écrit :

Because i want to return an answer faster.
Without the background tasks i would get around 250 req/sec and with
backgroundtasks i get 550 req/sec (with 8000 users in db ).
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/tiangolo/fastapi/issues/565?email_source=notifications&email_token=AAINSPSQFSEIBEFXPK736ETQLJRMBA5CNFSM4I2DSRUKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7PP3QI#issuecomment-534707649,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAINSPV7XTPQJEFNDGUZH43QLJRMBANCNFSM4I2DSRUA
.

ab (apache benchmarking tool) is just like wrk.
I've just switched from flask (from tiangolo) to FastApi and now i have this problems so please bear with me :).
I am serving my app with gunicorn -k uvicorn.workers.UvicornWorker -c "$GUNICORN_CONF" "$APP_MODULE"
And my gunicorn_conf looks like this:
web_concurrency = 4
loglevel = use_loglevel
workers = web_concurrency
bind = use_bind
keepalive = 120
timeout = 1200
backlog = 2048
limit_request_field_size = 0
errorlog = "-"
Also i use use_bind = "unix:/tmp/uvicorn.sock"

ok, bear in mind I never ever used rethinkdb, here's what I did and how it went (good from what I can tell from ab results), I used only uvicorn to serve the app as you'll see below

I ran rethinkdb with docker run --name some-rethink -p 28015:28015 -d rethinkdb
I used this snippet, adapted from yours, there are discrepancies, maybe that explains why all requests end up fine. In particular I used a simple await to the connection in the background task and didn't use the request
run the script, it will only use uvircorn, I wanted to keep things simple and seperate eventual issues
run ab ab -n 8000 -c 1000 http://localhost:8000/create_load_testing_user

(rapidFastAPItest) ➜  rapidFastAPItest git:(master) ✗ ab -n 8000 -c 1000 http://localhost:8000/create_load_testing_user
This is ApacheBench, Version 2.3 <$Revision: 1843412 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 800 requests
Completed 1600 requests
Completed 2400 requests
Completed 3200 requests
Completed 4000 requests
Completed 4800 requests
Completed 5600 requests
Completed 6400 requests
Completed 7200 requests
Completed 8000 requests
Finished 8000 requests

Server Software:        uvicorn
Server Hostname:        localhost
Server Port:            8000

Document Path:          /create_load_testing_user
Document Length:        226 bytes

Concurrency Level:      1000
Time taken for tests:   21.235 seconds
Complete requests:      8000
Failed requests:        0
Total transferred:      2816000 bytes
HTML transferred:       1808000 bytes
Requests per second:    376.73 [#/sec] (mean)
Time per request:       2654.421 [ms] (mean)
Time per request:       2.654 [ms] (mean, across all concurrent requests)
Transfer rate:          129.50 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0  865 1640.7     10    7145
Processing:    44  977 615.7    898    4716
Waiting:        3  819 603.0    735    4542
Total:         45 1842 1869.0   1037    9023

Percentage of the requests served within a certain time (ms)
  50%   1037
  66%   1734
  75%   1988
  80%   2864
  90%   4173
  95%   5521
  98%   8362
  99%   8368
 100%   9023 (longest request)

So the issue you face may come from how you handle the connection, would be interesting to see how you deal with it, again I'm a total rethinkdb newbie so I may be totally off

using gunicorn -k uvicorn.workers.UvicornWorker -c 565_gunicorn.conf "565_background_high_load:app" I get 1400 req/s without one failing
however I used bind = "<IP_ADDRESS>:8000" as you can see in conf used
what's the purpose of using a socket in your conf ?

Pfff. I will definitely try without a connection pool and spawn a new connection every time.
I will try your example (since it's a huge difference in the req/sec and no errors) and come back with the results.
The use of socket is for nginx communication. I use nginx because i need to redirect http to https and i didn't knew how to do it with FastApi. The traffic comes from a google load balancer that has the ssl cert.
Sidenote:
I use the http middleware to acquire a connection from the pool, execute the route function and after the function, i release the connection back to the pool. The db connection is then assigned to the request.state (eg. request.state.rethinkdb_connection) and accessed where i need it.
If i use the same approach with a new connection every time (each request) the background thread will throw an error saying that the connection is closed. And it's correct since after the route function i close the connection and the background thread runs after the request completed.

i would like to link my issue about production configuration of gunicorn here - https://github.com/tiangolo/fastapi/issues/551
I'm very keen to figure out this tuning. Is it recommended to run  gunicorn with threads ?
also - how do you create the connection pool in the middleware ? we are using sqlalchemy with postgresql, but a lot of the same things will apply.
The first place i learnt this from was the official sqlalchemy/postgresql starter for fastapi - https://github.com/tiangolo/full-stack-fastapi-postgresql/blob/b96d78db99d8c4f37f0df64df587ffebda4bfc56/{{cookiecutter.project_slug}}/backend/app/app/main.py#L33

@sandys 'Is it recommended to run gunicorn with threads ?' - > i don't know what to say
'how do you create the connection pool in the middleware ? ' i am not creating the pool in the middleware. On the application startup i create the pool. In the middleware i just acquire a connection from the pool.

@euri10
I ran your test and i get (approx.) the same results.
I made some changes on my side and the problems with the dropped requests came from nginx with 2 timeouts.
Fixed that (dropped requests) but now the problem is that i am getting around 500 req/seq instead of 1300-1400 req/sec.
The only reason i use nginx is to redirect http to https.
Is there a solution to upgrade http to https using starlette/fastapi?

Is there a solution to upgrade http to https using starlette/fastapi?
not that I'm aware of, and nginx is supposed to be fastest than traefik which I currently use , but maybe others have better insights on that part

@euri10 well, thanks for the help

Have you guys tested haproxy ? Generally speaking it's faster (and simpler
config than nginx)
On Wed, 25 Sep, 2019, 19:14 razvanavram<EMAIL_ADDRESS>wrote:

@euri10 https://github.com/euri10 well, thanks for the help
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/tiangolo/fastapi/issues/565?email_source=notifications&email_token=AAASYU7P35YZTWX4MYCVR5TQLNTNFA5CNFSM4I2DSRUKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7R6TOY#issuecomment-535030203,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAASYU3NX3G56SYBROZSDLDQLNTNFANCNFSM4I2DSRUA
.

Thanks for the help here @euri10 ! :clap: :bow:
Thanks for reporting back and closing the issue @razvanavram  :+1:

@euri10
I ran your test and i get (approx.) the same results.
I made some changes on my side and the problems with the dropped requests came from nginx with 2 timeouts.
Fixed that (dropped requests) but now the problem is that i am getting around 500 req/seq instead of 1300-1400 req/sec.
The only reason i use nginx is to redirect http to https. (running behind a google cloud loadbalancer)
Is there a solution to upgrade http to https using starlette/fastapi?

hi @razvanavram its been a year. May I know how you fixed nginx part to avoid dropped requests?
I'm facing this issue on PROD.
Currently Im running almost the same your gunicorn config with uvicorn workers and nginx is being used as proxy (running behind Cloudflare loadbalancer.
But when I have 600+ requests / sec, the app start not working, gunicorn threw worker timeouts, at the same time it made high cpu load and memory run out on rethinkdb.
Can anyone please help?

@thaodt I have abandoned the whole nginx, uvicorn & gunicorn server idea and moved to AWS Lambda with mangum


URL Not Found when using action camel case and params

### What steps will reproduce the problem?
Controller :  ReportsController
Action : actionTecnicalReport($id)
Try Access url :  reports/tecnical-report/1
### What is the expected result?
Accessing the page
### What do you get instead?
Not Found 404
### Additional info
if i remove param ($id) the url works
Controller :  ReportsController
Action : actionTecnicalReport
url :  reports/tecnical-report

'urlManager' => [
'class' => 'yii\web\UrlManager',
'showScriptName' => false,
'enablePrettyUrl' => true,
'rules' => array(
'<controller:\w+>/<id:\d+>' => '<controller>/view',
'<controller:\w+>/<action:\w+>/id:\d+' => '<controller>/<action>',
'<controller:\w+>/<action:\w+>' => '<controller>/<action>',
),
],

Q
A

Yii version
2.0.5 July 11, 2015

PHP version
5.5.9

Operating system
ubuntu 4.13

'<controller:\w+>/<action:\w+>/id:d+' => '<controller>/<action>',

Did you try:
 '<controller:\w+>/<action:\w+>/<id:\d+>' => '<controller>/<action>',
?

[\w-]+

https://regex101.com/r/nN6uI5/1

you do as https://regex101.com/r/nN6uI5/2

Tks !!   It works !
'urlManager' => [
'class' => 'yii\web\UrlManager',
'showScriptName' => false,
'enablePrettyUrl' => true,
'rules' => array(
'<controller:\w+>/<id:\d+>' => '<controller>/view',
'<controller:\w+>/<action**:[\w-]+**>/<id:d+>' => '<controller>/<action>',
'<controller:\w+>/<action:\w+>' => '<controller>/<action>',
),
],
I think that if yii2 developers have developed friendly url with traces  (controller/action-with-trace) have also developed the default  urlmanager  configuration

I think that if yii2 developers have developed friendly url with traces (controller/action-with-trace) have also developed the default urlmanager configuration

where did you find the default configuration without -? it is correct in the docs here: http://www.yiiframework.com/doc-2.0/yii-web-urlmanager.html#$rules-detail

Possibly an old documentation when the Yii version was 2.0.5
(Now its up-to-date and works fine ! )
Tks


NPR Music Tiny Desk Sporadically Broken

Checklist

[X] I'm reporting a broken site
[X] I've verified that I'm running yt-dlp version 2021.12.01. (update instructions)
[X] I've checked that all provided URLs are alive and playable in a browser
[X] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[X] I've searched the bugtracker for similar issues including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue
[X] I've read about sharing account credentials and I'm willing to share it if required

Region
United States
Description
I've found an instance of a NPR Tiny Desk Concert that does not work.
Most concerts — such as this one — work no sweat.
Oddly, this one does not. It plays just fine in Safari.
Verbose log
[debug] Command-line config: ['-Uv', 'https://www.npr.org/2021/10/26/1048372074/ed-sheeran-tiny-desk-home-concert']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8 (No ANSI), err utf-8, pref UTF-8
[debug] yt-dlp version 2021.12.01 [91f071af6]
[debug] Python version 3.10.0 (CPython 64bit) - macOS-12.0.1-arm64-arm-64bit
[debug] exe versions: ffmpeg 4.4.1 (setts), ffprobe 4.4.1, rtmpdump 2.4
[debug] Optional libraries: Cryptodome, mutagen, sqlite, websockets
[debug] Proxy map: {}
[debug] [Npr] Extracting URL: https://www.npr.org/2021/10/26/1048372074/ed-sheeran-tiny-desk-home-concert
WARNING: [Npr] Failed to download m3u8 information: HTTP Error 404: Not Found
ERROR: [Npr]<PHONE_NUMBER>: Unable to download SMIL file: HTTP Error 404: Not Found (caused by <HTTPError 404: 'Not Found'>); please report this issue on  https://github.com/yt-dlp/yt-dlp . Make sure you are using the latest version; see  https://github.com/yt-dlp/yt-dlp  on how to update. Be sure to call yt-dlp with the --verbose flag and include its complete output. (caused by <HTTPError 404: 'Not Found'>); please report this issue on  https://github.com/yt-dlp/yt-dlp . Make sure you are using the latest version; see  https://github.com/yt-dlp/yt-dlp  on how to update. Be sure to call yt-dlp with the --verbose flag and include its complete output.
  File "/opt/homebrew/Cellar/yt-dlp/2021.12.1/libexec/lib/python3.10/site-packages/yt_dlp/extractor/common.py", line 715, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "/opt/homebrew/Cellar/yt-dlp/2021.12.1/libexec/lib/python3.10/site-packages/yt_dlp/YoutubeDL.py", line 3385, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File<EMAIL_ADDRESS>line 525, in open
    response = meth(req, response)
  File<EMAIL_ADDRESS>line 634, in http_response
    response = self.parent.error(
  File<EMAIL_ADDRESS>line 563, in error
    return self._call_chain(*args)
  File<EMAIL_ADDRESS>line 496, in _call_chain
    result = func(*args)
  File<EMAIL_ADDRESS>line 643, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)

i checked and it fails because the website returns a cookie consent webpage instead of the page the extractor expects...

Looks like most (or perhaps all) SMIL links returned by api.npr.org
are 404s.
Currently, the _extract_smil_formats call in NprIE does not
pass fatal=False. Consequently, the call fails and makes the
download fail.
If fatal=False is passed to _extract_smil_formats, the download
works:
$ python -m yt_dlp https://www.npr.org/2021/10/26/1048372074/ed-sheeran-tiny-desk-home-concert
[Npr]<PHONE_NUMBER>: Downloading JSON metadata
[Npr]<PHONE_NUMBER>: Downloading m3u8 information
WARNING: [Npr] Failed to download m3u8 information: HTTP Error 404: Not Found
[Npr]<PHONE_NUMBER>: Downloading SMIL file
WARNING: [Npr] Unable to download SMIL file: HTTP Error 404: Not Found
[Npr] hlsUrl: Downloading m3u8 information
[download] Downloading playlist: Ed Sheeran: Tiny Desk (Home) Concert
[Npr] playlist Ed Sheeran: Tiny Desk (Home) Concert: Collected 1 videos; downloading 1 of them
[download] Downloading video 1 of 1
[info]<PHONE_NUMBER>: Downloading 1 format(s): hls-3740
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 338
[download] Destination: Ed Sheeran - Tiny Desk (Home) Concert [1048379421].mp4
[download] 100% of 618.08MiB in 10:16
[FixupM3u8] Fixing MPEG-TS in MP4 container of "Ed Sheeran - Tiny Desk (Home) Concert [1048379421].mp4"
[download] Finished downloading playlist: Ed Sheeran: Tiny Desk (Home) Concert

Here's the patch:
diff --git a/yt_dlp/extractor/common.py b/yt_dlp/extractor/common.py
index e15763953..6cd2551bf 100644
--- a/yt_dlp/extractor/common.py
+++ b/yt_dlp/extractor/common.py
@@ -2332,7 +2332,7 @@ class InfoExtractor(object):

         if smil is False:
             assert not fatal
-            return []
+            return [], {}

         namespace = self._parse_smil_namespace(smil)

diff --git a/yt_dlp/extractor/npr.py b/yt_dlp/extractor/npr.py
index 9d1122f0c..49f062d7a 100644
--- a/yt_dlp/extractor/npr.py
+++ b/yt_dlp/extractor/npr.py
@@ -91,7 +91,8 @@ class NprIE(InfoExtractor):
                     elif format_id == 'smil':
                         smil_formats = self._extract_smil_formats(
                             format_url, media_id, transform_source=lambda s: s.replace(
-                                'rtmp://flash.npr.org/ondemand/', 'https://ondemand.npr.org/'))
+                                'rtmp://flash.npr.org/ondemand/', 'https://ondemand.npr.org/'),
+                            fatal=False)
                         self._check_formats(smil_formats, media_id)
                         formats.extend(smil_formats)
                     else:

Does this look good? Should I go ahead and open a PR?

Should I go ahead and open a PR?

please do

Should I go ahead and open a PR?

please do

Done.

Just tested this on the new release. Works perfectly. Thank you, @r5d!

Seems this has cropped up again, unfortunately. Currently seeing a failure with this video.
Log Here
[debug] Command-line config: ['-Uv', 'https://www.npr.org/2022/03/15/1084896560/bonobo-tiny-desk-home-concert']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8 (No ANSI), err utf-8, pref UTF-8
[debug] yt-dlp version 2022.04.08 [7884ade65]
[debug] Python version 3.10.2 (CPython 64bit) - macOS-12.3.1-arm64-arm-64bit
[debug] Checking exe version: ffmpeg -bsfs
[debug] Checking exe version: ffprobe -bsfs
[debug] exe versions: ffmpeg 5.0.1 (setts), ffprobe 5.0.1, rtmpdump 2.4
[debug] Optional libraries: brotli, certifi, Cryptodome, mutagen, sqlite, websockets
[debug] Proxy map: {}
[debug] [Npr] Extracting URL: https://www.npr.org/2022/03/15/1084896560/bonobo-tiny-desk-home-concert
ERROR: [Npr]<PHONE_NUMBER>: No video formats found!; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/yt-dlp/2022.4.8/libexec/lib/python3.10/site-packages/yt_dlp/YoutubeDL.py", line 1408, in wrapper
    return func(self, *args, **kwargs)
  File "/opt/homebrew/Cellar/yt-dlp/2022.4.8/libexec/lib/python3.10/site-packages/yt_dlp/YoutubeDL.py", line 1834, in __process_iterable_entry
    return self.process_ie_result(
  File "/opt/homebrew/Cellar/yt-dlp/2022.4.8/libexec/lib/python3.10/site-packages/yt_dlp/YoutubeDL.py", line 1548, in process_ie_result
    ie_result = self.process_video_result(ie_result, download=download)
  File "/opt/homebrew/Cellar/yt-dlp/2022.4.8/libexec/lib/python3.10/site-packages/yt_dlp/YoutubeDL.py", line 2487, in process_video_result
    self.raise_no_formats(info_dict)
  File "/opt/homebrew/Cellar/yt-dlp/2022.4.8/libexec/lib/python3.10/site-packages/yt_dlp/YoutubeDL.py", line 981, in raise_no_formats
    raise ExtractorError(msg, video_id=info['id'], ie=info['extractor'],
yt_dlp.utils.ExtractorError: [Npr]<PHONE_NUMBER>: No video formats found!; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U

For the Bonobo link, the format and stream fields, under the
multimedia object returned by the API, are empty:

curl dump

curl -s "http://api.npr.org/query?id=1084896560&fields=audio,multimedia,title&format=json&apiKey=MDAzMzQ2MjAyMDEyMzk4MTU1MDg3ZmM3MQ010" | python -m json.tool
{
    "version": "0.94",
    "list": {
        "title": {
            "$text": "Stories from NPR"
        },
        "teaser": {
            "$text": "Assorted stories from NPR"
        },
        "miniTeaser": {
            "$text": "Custom NPR News Feed API.  Visit https://www.npr.org/templates/apidoc/index.php for more information."
        },
        "link": [
            {
                "type": "api",
                "$text": "http://api.npr.org/query?id=1084896560&apiKey=MDAzMzQ2MjAyMDEyMzk4MTU1MDg3ZmM3MQ010"
            },
            {
                "type": "html",
                "$text": "https://www.npr.org/2022/03/15/1084896560/bonobo-tiny-desk-home-concert?ft=nprml&f=1084896560"
            }
        ],
        "story": [
            {
                "id": "1084896560",
                "link": [
                    {
                        "type": "html",
                        "$text": "https://www.npr.org/2022/03/15/1084896560/bonobo-tiny-desk-home-concert?ft=nprml&f=1084896560"
                    },
                    {
                        "type": "api",
                        "$text": "http://api.npr.org/query?id=1084896560&apiKey=MDAzMzQ2MjAyMDEyMzk4MTU1MDg3ZmM3MQ010"
                    }
                ],
                "title": {
                    "$text": "Bonobo: Tiny Desk (Home) Concert"
                },
                "multimedia": [
                    {
                        "id": "1086468851",
                        "title": {},
                        "duration": {
                            "$text": "1061"
                        },
                        "format": {},
                        "dataUrl": {},
                        "rightsHolder": {},
                        "caption": {},
                        "credit": {
                            "$text": "Courtesy of the artist"
                        },
                        "width": {
                            "$text": "624"
                        },
                        "height": {
                            "$text": "351"
                        },
                        "altImageUrl": {
                            "$text": "https://media.npr.org/assets/img/2022/03/14/20220310_tdhc-bonobo_thumbnail_wide-d5df4d272116d5fef9b89fb46d56bf71820ee300.jpg"
                        },
                        "permissions": {
                            "download": {
                                "allow": "false"
                            },
                            "stream": {
                                "allow": "true"
                            },
                            "embed": {
                                "allow": "true"
                            }
                        },
                        "stream": {
                            "active": "false"
                        }
                    }
                ]
            }
        ]
    }
}

The NPR extractor depends on the link in the format and stream
fields for downloading the video. Consequently, the download fails.
Forunately, the NPR web page contains a JSON-LD with the "stream" link
in the JSON-LD VideoObject's "embedUrl" field.
I've opened PR #3455 that updates the NPR Extractor to use the "embedUrl"
as a stream url if the format and stream fields returned by the
API is empty.
The Bonobo link downloads successfully with the patch in PR #3455:
$ python -m yt_dlp https://www.npr.org/2022/03/15/1084896560/bonobo-tiny-desk-home-concert
[Npr]<PHONE_NUMBER>: Downloading JSON metadata
[Npr]<PHONE_NUMBER>: Downloading webpage
[Npr] hlsUrl: Downloading m3u8 information
[download] Downloading playlist: Bonobo: Tiny Desk (Home) Concert
[Npr] playlist Bonobo: Tiny Desk (Home) Concert: Collected 1 videos; downloading 1 of them
[download] Downloading video 1 of 1
[info]<PHONE_NUMBER>: Downloading 1 format(s): hls-3620
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 266
[download] Destination: Bonobo - Tiny Desk (Home) Concert [1086468851].mp4
[download] 100% of 470.85MiB in 08:38
[FixupM3u8] Fixing MPEG-TS in MP4 container of "Bonobo - Tiny Desk (Home) Concert [1086468851].mp4"
[download] Finished downloading playlist: Bonobo: Tiny Desk (Home) Concert

@cliss, you could use the NPR Music youtube feed to download tiny
concerts -- https://www.youtube.com/feeds/videos.xml?channel_id=UC4eYXhJI4-7wSWc8UNRwD4A
i've (written and) been using fern to download tiny desk
concerts. fern parses youtube feeds and uses yt-dlp to download the
videos.
Something like this:
   {
      "id": "tiny-desk",
      "source": "https://www.youtube.com/feeds/videos.xml?channel_id=UC4eYXhJI4-7wSWc8UNRwD4A",
      "schema": "youtube",
      "title-contains": "tiny desk",
      "last": 10
   }

in the fern config will make fern download just tiny desk videos from
the NPR Music feed.
See https://pkg.go.dev/ricketyspace.net/fern if you want to try fern.

@r5d Thanks for the fix! Excited to try it.
Thanks for the tip wrt fern; I’ll keep that in mind. Generally speaking I prefer to download Tiny Desks piecemeal, rather than en masse, but I’m happy to have another tool to keep in mind. :)

@cliss, You may test the fix with:
git clone --branch npr-1934-jsonld-fallback https://github.com/r5d/yt-dlp.git
cd yt-dlp
python3 -m yt_dlp https://www.npr.org/2022/03/15/1084896560/bonobo-tiny-desk-home-concert

Confirmed, @r5d: worked no sweat. 😗👌🏻


Using gitbook for docs

As discussed in Gitter, we agree that GitBook would be a good choice to merge the masstransit.github.io and the readthedocs.
For now the most realistic scenario is:

Keep the docs in the masstransit.github.io repo
Use GH Pages to host the front page (index.html)
Set up the MassTransit organisation on gitbook.com
Create a book for MT docs and make it build from the masstransit.github.io GH repo
Add custom domain name to the gitbook book, may be use existing docs.masstransit-project.com
Change index.html links to point to gitbook

I have created a sample set up for this on http://mt.digilica.com/
At the bottom, the Installation link points to the gitbook with custom domain
If @phatboyg and @drusellers agree, I will continue with this approach.
We could reconsider keeping the docs source in the main repo should we feel the need for it, later on.

Okay, the reason for moving to GitHub from RTD, was to be hosted entirely on GitHub. I don't want to move from RTD to GitBook, and host on something other than GitHub. Otherwise, I'd be fine staying with RTD.

If you want GH Pages to host the docs, then it needs to be built.

I would be okay with that, which I'm okay just making a parallel folder in the main repository (we have doc, maybe it's docs) and have the CI server build it perhaps?

And then it publishes to masstransit.github.io (which is already supporting masstransit-project.com).

Yes, this was one of the options I proposed yesterday. I will make a prototype for this, will try using FAKE script and Appveyor.

Yeah, and we could do a separate repository with a separate build, that would be fine too, I can set that up on AppVeyor as part of the projects. That keeps the PR process separate I suppose.

I like moving issues from in progress to "document" though in the ZenHub board, so that documentation eventually gets written.

The issue with pushing to masstransit.github.io is that user/org pages can only be published from the master branch. This is the GH Pages limitation. So if we can only host the index.html there and the docs will be built from the main repo folder and published to the gh-pages branch, the docs site will appear at masstransit.github.io/masstransit. We can also have a custom domain name for it. Will this be okay?

Separate docs project would be okay as well.

Does MT have a ZenHub board?

@waynebrantley yes

@alexeyzimarev this repository: https://github.com/MassTransit/masstransit.github.io
I think Dru said we're fine if the entire site is built with GitBook. I know he worked on the layout of the index page, but I don't think it's a big deal if it's restructured around the format of GitBook.

Okay, we can try to work out the index page from readme later on.
On Tue, 10 Jan 2017 at 16:53, Chris Patterson<EMAIL_ADDRESS>wrote:

Assigned #759 https://github.com/MassTransit/MassTransit/issues/759 to
@alexeyzimarev https://github.com/alexeyzimarev.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/MassTransit/MassTransit/issues/759#event-917713647,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACsMVR79mwM5Mbs9B5alfV_ehNON3GKpks5rQ6ldgaJpZM4LfQWV
.

@phatboyg One more thing - it we use masstransit.github.io, this means the MT docs will be on the org page. Do you have any plans for docs for Automatonymous and GreenPipes, for example?

That was the point of a separate repository for documentation, to create the documentation for everything on a single output site - Automatonymous, Green Pipes, MT, etc. Thoughts on how best to manage that?

A gitbook for every project and then build/output them to masstransit.github.io as part of that single site? I'm up for whatever makes the most sense.

Well, yes. Docs folder and gh-pages branch in each repo and attic content
on the org site
On Tue, 10 Jan 2017 at 17:49, Chris Patterson<EMAIL_ADDRESS>wrote:

A gitbook for every project and then build/output them to
masstransit.github.io as part of that single site? I'm up for whatever
makes the most sense.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/MassTransit/MassTransit/issues/759#issuecomment-271629568,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACsMVWsmLNHY8UcVavzgP7CBTpRaDiUHks5rQ7a3gaJpZM4LfQWV
.

Ok, so what if...in each project there was a docs folder. Now each project can have its own documentation that lives with the project and can evolve with the project.
I like the docs following the mdBook style approach which is easy to automate using AppVeyor, etc. This gives us a clean structure for each project.
Ideally, changes to docs would not trigger a CI build for NuGet type activities.
We could then have another site masstransit.githbub.io that has the landing page that keeps it pretty and for the most part just provides simple short cuts to the GitBook/MdBook site. The purpose of the mt.github.io site is to aggregate links and provide a dashboard of links to NuGets etc. It gets update as a part of a common release process as well.

Yeah, this was the original prototype but I am not sure how to exclude the
docs folder from the CI build trigger. Not sure if Appveyor allows that.
On Tue, 10 Jan 2017 at 19:06, Dru Sellers<EMAIL_ADDRESS>wrote:

Ok, so what if...in each project there was a docs folder. Now each
project can have its own documentation that lives with the project and can
evolve with the project.
I like the docs following the mdBook style approach which is easy to
automate using AppVeyor, etc. This gives us a clean structure for each
project.
Ideally, changes to docs would not trigger a CI build for NuGet type
activities.
We could then have another site masstransit.githbub.io that has the
landing page that keeps it pretty and for the most part just provides
simple short cuts to the GitBook/MdBook site. The purpose of the
mt.github.io site is to aggregate links and provide a dashboard of links
to NuGets etc. It gets update as a part of a common release process as well.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/MassTransit/MassTransit/issues/759#issuecomment-271651042,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACsMVbsC0RyuHoRa_CVsgF8H3HgMiZPsks5rQ8izgaJpZM4LfQWV
.

@phatboyg Is there a url for the zenhub for MT?

https://www.appveyor.com/docs/how-to/filtering-commits/#commit-files-github-and-bitbucket-only

Yeah, I pasted to gitter a couple of minutes ago...

So basically the build that already exists this folder and some files need to be excluded. It seems like @phatboyg has it configured in the GUI? Can't find the yml file. It would be nice if we can switch to yml file to I can submit a PR to change it for ignoring files/folders that belong to docs.
Then I can add a new yml file for the docs build. It will just run npm and it should work just fine.

@waynebrantley As I can see it, zenboard is just an extension to your browser so when you have it, connected to github, logged in and opened the right repo - you will see the board.

@waynebrantley no idea, it just works for me once I enabled integration with GitHub. There is a "boards" icon for me using Chrome.

Ah, I got it now..thats pretty cool.  Thx.

Lots of pages added https://alexeyzimarev.github.io/MassTransit/
Still a lot to go... Doing a little restructuring along the way.

Looks awesome so far!

"Using MT" is done. Comments welcome. Retries needs comments related to the latest change (3.5.4)

Closing as done


Bug: page numbers don't always show on hover

Describe the bug
After a knowledge graph is generated, sometimes the page numbers aren't showing on hover and instead it says "no page in metadata". However, as it is a document type that should have page numbers, this should not be the case

To Reproduce
Steps to reproduce the behavior:

Upload of this document has so far consistently resulted in failure.

AMOS B01 - Team and Tools.pptx
Expected behavior
For document types of .docx, .pdf, .pptx where there are page numbers available, the page numbers should always be included in the metadata and thus show on hover over an entity.

@kuoirene I tried for .docx files to find a way to read the page numbers but it seems there is not straightforward and reliable way. The package does not support it. Also from a quick search it seems that this is on purpose since like I mentioned there is not way to do that for these docs since it depends on the render engine and most of the time pages are found based on the line breaks.
For .pptx files the package allowed it and it is fixed in the PR.


Publish Microsoft Authored templates to azure samples browser

Publish cookiecutter-relecloud template to azure samples browser.
@pamelafox    According to the video introduction in the Readme, when we tried to generate templates locally through cookiecutter-relecloud, we encountered an FailedHookException error. Therefore, we cannot verify the accuracy of this PR. Do you have any ideas on how to generate these templates?
@hemarina for notification.

@jianingwang123 - can you include the exact error and the operating system and python version that you're using?

@kjaymiller
operating system : Windows 11
Python version : Python 3.11.5
The exact error :
FileNotFoundError: [WinError 2] The system cannot find the file specified.  Stopping generation because pre_gen_project hook script didn't exit successfully
FailedHookException: Hook script failed (exit status: 1)

The purpose of this PR to add metadata to publish Microsoft Authored templates has been achieved by https://github.com/Azure-Samples/Azure-Python-Standardization-Template-Generator/pull/9, I will close this PR.


[v5 only] Backport of my most recent fixes to the KS files

I saw that the master branch is still receiving changes for maintenance, so I thought I'd backport my most recent fixes that I already posted some days ago and that have been included in v6 so that @nolimits4web  can evaluate if that's the case for also include them in v5.

Thanks!


-add callback for order/bank/sale

@marcghelas Looks good but let's wait for the final decision before merging


Recommended workflow for storing results returned as numpy.ndarray?

Several functions that operate on AnalogSignal data return simple numpy.ndarrays, e.g.

the spectral analysis methods
correlation measures

Most users will probably want to store the results in some way. For the AnalogSignal outputs, Neo provides easy-to-use saving to disk. For the spectral and correlation measures however, Neo does not offer this (yet).
Are there any best practices what to do with these "pure" results, or plans towards implementing a Spectrum data model including I/O?

Hi Joscha,
thanks for your input. Indeed the returns of analysis functions is a hot topic on our agenda. For some analysis functions, we used Neo objects as return types for precisely the reasons you mentioned. However, this situation quickly comes to its limits. For example, a time histogram could be interpreted as an analog signal, but then again, in a way its more than just that -- it has a concept of bin width, for example.
Therefore, we are in the planning to move to an alternative representation building something like Neo, not for input data but for analysis results. The idea here is that a minimal number of objects are able to represent  the analysis results, certain key metadata and additional info like Neo annotations, and of course a serialization to disk (maybe even the option to temporarily dump objects to disk, similar to Neo's lazy loading, to deal with large analysis results). These objects would not become part of Neo since structurally this would not fit, however, it is possible to draw links between the tools nevertheless. An early prototype of how this could look for a TimeHistogram object you can find here:
https://github.com/INM-6/elephant/blob/feature/basic_provenance/elephant/buffalo/objects/histogram.py
Implementing such objects would further simplify the interoperability with a companion project, alpaca (first release pending within the next weeks, https://alpaca-prov.readthedocs.io/en/latest/), to capture the provenance of an analysis workflow. This work on provenance we had prioritized over the data objects, however, we are confident that the data objects will be on the agenda this year (together with a new object to represent an experimental trial, which is what we are currently working on.)
I hope this goes in the direction of what you are thinking. Of course, we are very open for any ideas, suggestions and contributions to this topic!

Hi @mdenker, great to hear that there are some plans for this. I agree that this is probably out-of-scope for Neo.
In general I like the direction that the AnalysisObject is taking, and alpaca looks very interesting!
As data analysis in electrophysiology is often exploratory and constantly changing, it may already be great to offer something a little more rigid than saving a complete workspace in MATLAB, but not too much. Often an analysis result is not much more than a couple of numpy arrays plus meta-data, which could be stored in simple, future-proof formats such as JSON and (flat) HDF or NPY. If I understood it correctly, alpaca is basically already almost doing that, except serializing the arrays. Correct?
From an architectural point of view, I'm not sure if each analysis method needs to implement its own result class inheriting from the AnalysisObject. This may be useful to achieve forward-compatibility of the stored analysis results, but I'm not sure sure that's achievable or necessary. I think, AnalysisObject could be treated as a flexible container that stores as much metadata as possible (auto-magically) together with the arrays, and serializes results using simple, flat data formats.

Thinking about it, maybe a dataclass, which tells the user and developer what attributes should be there, in combination with a metadata-enhanced serializer could be robust enough.

Hi, and thanks for all your great comments ideas and suggestions. I agree that your idea of having a generic AnalysisObject type container that "always works" is very interesting concept that could already help a lot. At the same time it may still be beneficial to have more specialized (e.g., subclassed) objects to describe certain recurring types of analysis results that define the structure of the analysis at greater depth and help -- in the long run -- with interoperability and clarity of code. I think both concepts could work well together.
(Regarding alpaca, it's aimed at merely tracking provenance and data flow of inputs and outputs during a script execution, but does not get involved with the structure of serialization of data as such. However, such approaches could be seen synergistic in this discussion.)


Support for different subnets in fargate profile configurations

I have issues
I want to create an eks cluster with both fargate and node group support. I also want to have many fargate profiles with different subnet configurations.
I'm submitting a...

[ ] bug report
[x] feature request
[ ] support request - read the FAQ first!
[ ] kudos, thank you, warm fuzzy

What is the current behavior?
The current behavior is that all fargate profiles shares the same private subnets as the EKS cluster. Specifically the ones passed by the variable subnets.
What's the expected behavior?
The expected behavior is to be able to customize the fargate profiles subnets input variable for each profile
Are you able to fix this problem and submit a PR? Link here if you have already.
I have already contributed to a fix, this is the PR link.
Environment details

Affected module version: latest
OS: all
Terraform version: 0.13

Any other relevant info

Running into same issue.
Created clusters with both private and public subnets, although the worker groups are restricted to private subnets. Can't really remember the reason. Maybe it was the initial requirement of managed nodes to be in public subnets, maybe I goofed up.

Running into same issue.
Created clusters with both private and public subnets, although the worker groups are restricted to private subnets. Can't really remember the reason. Maybe it was the initial requirement of managed nodes to be in public subnets, maybe I goofed up.

The problem seems to be that fargate only supports private subnets but the module attaches all subnets with no regard to private public (thus attaching public) here

@myoung34 actually the patch provided gives you the ability to pass any subnet to the fargate profile. If you don't pass a specific subnet it's going to use the var.subnets. In that case yes, if var.subnets contains public subnet it may be a problem (did not verify it) but is not really related with this patch, it keeps the same default behaviour as before. I can actually close this since the patch has been merged.

That's part of the problem @aittam
there's only one subnets variable. Per doc: A list of subnets to place the EKS cluster and workers within.
This makes it so that you cant have a public control plane and private fargate workers, no?

of course you can, you can pass whatever subnet you want to the fargate profile regardless of var.subnet. Look at the example. You can specify the subnets inside the fargate profile.

Ah, looks like the patch fixed the issue I had. I'll try again when I get back to this one

closing since we now have subnets argument for fargate profile.


[Feature Request] [Windows] Add the snap layout feature introduced in Windows 11

There's much hope to get the snap layout released within next week. The implementation is almost done now and I'm currently finishing the details and testing it locally.

@JulienMaille  The snap layout is now fully supported by FramelessHelper, can you test the code from the main branch? If no serious issues are found, I'll release it as the 2.1 version.

I'll do my best but I'm quite busy right now. Have you implemented it in the Mainwindow example?

No need to hurry, bro! I've tested the code on my PC locally already and everything seems to work fine.

Have you implemented it in the Mainwindow example?

Snap layout is supported by all the examples, we should show as many features as possible in the demo applications anyway :)

Done in 2.1


Address getting extension working with updated jupyter lab

This commit gets the extension working with latest Jupyterlab
Changes include:
package updates in package.json,
syntax error fixes in index.ts and voyagerpanel.js,
filetype fixes for opening .csv and other supported filetypes correctly in index.ts,
changes for prop and react error fixes caused by updating of packages in tsconfig.json
known bugs so far:
export/save doesn't work,
some .csv may not open saying invalid format
columns with quotes (Ex: "date") cannot be parsed correctly

Ooops looks like i need to clean up the commented code, but before can you please review and let me know if that line is needed or not

Hey @saulshanabrook having a bit of trouble getting the cypress test to work...not too sure what I'm missing tbh. this is my first time working with cypress, but it looks pretty straightforward, it seems as if the mousemove is command is not triggering a visible menu when hovering over 'Open With'...seems odd as it fires the even and adds the new .p-Menu div onto the DOM.
I'm not sure if this is an actual issue with cypress, but it works fine when i manually hove -- I posted details about the issue on cypress... any ideas on what could be happening here, kind of stumped? :/

@playermanny2 Thanks for looking into the testing.
I will take a look at this locally.

@saulshanabrook appears to an issue with our environment specifically... was able to do some testing in https://github.com/cypress-io/cypress/issues/3945 with one of the contributors...Not sure if i messed something up internally within jupyterlab so now it can't properly receive those programmatic hover commands.
Down the rabbit hole...haha

@saulshanabrook @jredding
Dug into a bit more, was able to isolate the cypress testing issue to Jupyterlab version ^0.35.
Open With -> Voyager appears correctly on 0.34.12 -- only thing i haven't been able to get working right is that after opening a test file in voyager and reloading, the voyager menu is not there (via cypress), however it works normally if i click without cypress and reload. I suspect this could be another weird bug with this jupyterlab version.
Overall, I think we have two options moving forward, let me know what you think is best

Revert core @jupyterlab/ dependencies to be on ^18.0, and change travis jupyterlab version to 0.34.12 for testing
Only change travis jupyterlab version to 0.34.12 for testing purposes

Not sure why this would be an issue with the new Jupyterlab ^.35.0 but we can create a post in the repository to start conversations about how to get that fixed.
@jredding  not 100% sure how to revert only the whitespace changes, would it be okay to leave for now? in the future definitely can focus on creating cleaner diffs

I'm going to leave the whitespace decision up to @saulshanabrook - personally I think it would be good to revert those. I know it's a pain, but it does make it much cleaner to understand what changes were made so when this needs to be updated again we have a good historical reference.
In regards to the tests.. did the HTML structure change between .34 and .35?
The Cypress tests is stating:

CypressError: Timed out retrying: Expected to find element: '.p-Menu-item div:contains("Open With")', but never found it.

In .35.4 I'm see this within
`

Open With

`
So could the test be written to look for  'div.p-Menu-itemLabel:contains("Open With")', ?
or do you think this is a loading issue (i.e. looking for it prior to it being rendered?)

@jredding  the DOM didn't change for those menu items between versions. You can either reference using a more general .p-Menu-item with cy.contains to search the children, or you can access it using .p-Menu-itemLabel specifically.
This issue appears to be a bug rather than a change in DOM structure. Happy to set-up some time in order to show the issue in more detail

@playermanny2 I can replicate your issues locally.
Here is the changelog between 34 and 35, I will look for relevant changes. Thanks for helping track this down.

This looks like it could be the relevant change: https://github.com/jupyterlab/jupyterlab/commit/ec0a5737722b59118edb3dea36a4c877a959b9e0#diff-457faa59a5ec6fc44e00a122e66d5abc

@playermanny2 I couldn't figure out how to get the tests to pass.
I am fine merging this in as is, I tried it locally and it seemed to work.

Only change travis jupyterlab version to 0.34.12 for testing purposes. This means extension can work with only the latest juyterlabs (^.35.x)

I would just say, forget the tests for now. If you can do this easily, that's fine, but the version are usually pinned and so it might be more trouble than its worth.
I am happy with the state it's in, feel free to merge it in as you see fit.

@saulshanabrook Awesome, yea I'll go ahead and merge this in as is for now so we can have a working repo...once merged, will you be able to take care of updating the npm package?
As far as the updates that caused the test to not work, do you think that it's a bug within cypress or jupyter? or is it something that just requires updating to use the right API and we should address in another ticket?

@saulshanabrook Awesome, yea I'll go ahead and merge this in as is for now so we can have a working repo...once merged, will you be able to take care of updating the npm package?

Sure, I will publish a new version.

As far as the updates that caused the test to not work, do you think that it's a bug within cypress or jupyter? or is it something that just requires updating to use the right API and we should address in another ticket?

I really don't know... I think it is probably a weirdness with how JuptyerLab is doing even handling. You could open up an issue on the main repo, with steps to reproduce, if you are motivated.
Thanks again for pushing through on this PR! It's great to have this working again :D

@saulshanabrook I'll open up an issue on the main repo with these details.
My pleasure on the PR, glad to be helping! Let's see what we can tackle next xD


Add content folder to tailwind paths by default

Is your feature request related to a problem? Please describe.
When using @nuxt/content, the content/ folder will not be checked by Tailwind's JIT engine by default.
Describe the solution you'd like
That it will be checked and classes will be generated
Describe alternatives you've considered
Adding it manually which is error-prone 🙈

Actually it should be added by default, you can test with running yarn dev @manniL
See https://github.com/nuxt/content/blob/005fdfd19aa9658f821cb64a868d7f4d09582e1f/src/module.ts#L594-L597

Interesting! I'll have a deeper look once again and report back ☺

Yes, my bad! The project still had tailwindcss as buildModule causing behavior.
Maybe a warning for Nuxt3 users who accidentially use buildModules would make sense here?

Don't you have the TS warning?


Added more euler rotation orders support.

Added more euler rotation orders support.
Fixed floating point issue on the old one.
Fixed the equation on the get_euler_yxz function.
Added unit tests.

After the compilation, you can start the unit test using ./bin/godot.linuxbsd.tools.64 --test basis.
This work has been kindly sponsored by IMVU.

Out of curiosity, what are some possible use cases for this?

In general, when you deal with Euler rotations, to avoid dead locks, you need to use a different rotation order.
@RevoluPowered is using these into its FBX importer; For example, in the past, during the integration of an input 3D sensor into Godot, the guy that was integrating it was using the set_euler_xyz function; since the device was returning Euler rotations.
I've integrated these for the FBX importer; I thought that someone may need it sooner or later.

We're now using this directly in our PR for FBX, merged
#39418 Required to fix regressions with problem rotations / intrinsic vs non-intrinsic rotations.
Fixes complicated rigs not working too with some Euler angles.
Has unit tests too :)

@Calinou can you tag this for discussion for merge with 3.2 rather than 4.0?

@RevoluPowered The milestone refers to the branch this PR will initially be merged in. It can be cherry-picked to the 3.2 branch later if needed.

Thanks! That's good to hear, I will be so happy to merge FBX once approved :)

Thanks!

Cherry-picked for 3.2.2.


Q447 root_keywords SELECT and FROM are not right aligned

Hi!
I have this query:
CREATE OR REPLACE VIEW registration_list AS
         SELECT col1, col2, col3, col4, col5, col6, col7,
                col8, col9, SUBSTRING(col1 FROM 1 FOR 8) AS col10
              FROM table
           WHERE col5 IS NULL WITH CHECK OPTION

and the flake-8 kept giving me the error:
Q447 root_keywords SELECT and FROM are not right aligned
I tried many possibilities but i'm not able to remove the error, i would appreciate if you can help me in pointing the issue.

@pgjones can you help me out in this?

I think it is confused by the FROM in the SUBSTRING (although in the example the FROM isn't right aligned with the SELECT). I'm unsure how to fix this right now, so I'd add a # noqa Q447 to the line it notes as having the error.

so that's an issue right now? And yes this is what i thought as well because upon removing the SUBSTRING it was working fine. So we will let this issue remain open?

This should be fixed by d1a1000e870db0c3fd88cf5e5c7a62c33aad51d8 and released in 0.4.0


Fix some typos in the api docstrings

What is the purpose of the pull request
This PR fixes some typos in the API docstrings

Merged into main, thanks for your contribution @hussein-awala !


Current electricity tariff 'unknown'

Yep once again zonneplan changed their API that breaks this plugin. This time they changed the current electricity tariff.
In the attributes of the current_electricity_tariff sensor the name of the electricity price is changed from 'price' to 'electricity_price'. My (could be wrong) assumption is that this also breaks the plugin.

See https://github.com/fsaris/home-assistant-zonneplan-one/issues/63


Training cost

Could you please tell me the training cost for this project, i.e. how many GPUs, how long to finish the training process?

And what version of python is the code running on?

Hi, thanks for your attention to our paper.
Each experiment in this project cost 1 GPU (2080Ti) for 2 days. The code is tested on python 3.6.


Precondition does not provide any efficiency in cg

My issue is about precondition in the sparse solvers (cg, gmers). I tried an incomplete LU factorization as preconditioner. However, this does not provide any decrease in consuming time even greatly lags the program more than 10 times!
I am confused by this situation, my original problem is a Poisson equation defined on a square domain, and I discrete this problem with Q1 finite element method. The mesh size reaches 512*512.
Reproducing code example:

def _get_preconditioner(op: csr_matrix):
    M = spilu(op.tocsc())
    def Mx(x): return M.solve(x)
    return LinearOperator(op.shape, Mx)

Scipy/Numpy/Python version information:
1.3.1 1.17.2 sys.version_info(major=3, minor=7, micro=5, releaselevel='final', serial=0)

Two things to check:

Does the number of CG iterations go down with preconditioning? (In other words, check that the preconditioner is correct.)
Are you recomputing the factorization with each iteration? You will have to trace the code to see if that is the case.

We did not hear back for this elsewhere and seems like we don't have much to do with the issue hence closing.


feat: improve design

This PR improves the design of the popup. It uses a dark-ish shade of orange to look more sleek. It uses a gray shade instead of black. And it uses either sans-serif or a Noto font depending on which is available.

An issue I'm yet to fix, is the fact on sites with light-mode enabled. The popup font looks, distorted.

@Chiroyce1 I fixed the conflicts!

why the description change??

It seemed more descriptive and formal.

In any case, it's a lot less formal.

Sorry, not merging any PRs right now.


[FEATURE] Update REST client used in OpenSearchRestTemplate

Is your feature request related to a problem?
The client used in org.opensearch.data.client.orhlc.OpenSearchRestTemplate, org.opensearch.client.RestHighLevelClient, has been deprecated in Elasticsearch since v7.15. The OpenSearch documentation also considers it deprecated.
What solution would you like?
The OpenSearchRestTemplate should use the Java client org.opensearch.client.opensearch.OpenSearchClient instead.
What alternatives have you considered?
Continuing to use the deprecated client runs the risk of breakage through its removal.
Do you have any additional context?
No

Note to maintainers:
Consider adding an issue type that's neither a bug nor an enhancement --  E.g., Technical debt, wherein this particular issue would be more accurately categorized.

And thanks for this fabulous Spring Data implementation!

@curena-contrast Want to take this on?

@curena-contrast this is a duplicate of https://github.com/opensearch-project/spring-data-opensearch/issues/19


Value<PHONE_NUMBER>776 cannot be encoded in 40 bits.

Hey, I'm running the tool on a contract that uses uint40 variables as input params and return values of functions, gives the following error:
2021-07-01 15:04:33,104 - Analysis - INFO - Generation number 0          Code coverage: 29.95% (2016/6731)         Branch coverage: 31.76% (108/340)       Transactions: 57 (56 unique)            Time: 0.3796710968017578
2021-07-01 15:04:33,558 - Analysis - INFO - Generation number 1          Code coverage: 29.95% (2016/6731)         Branch coverage: 31.76% (108/340)       Transactions: 102 (81 unique)           Time: 0.8341164588928223
2021-07-01 15:04:34,472 - Individual - ERROR - Value<PHONE_NUMBER>776 cannot be encoded in 40 bits.  Must be bounded between [0,<PHONE_NUMBER>775]
2021-07-01 15:04:34,473 - Individual - ERROR - 0xf5faf0ce: ['uint40'] -> [1099511627776]
2021-07-01 15:04:34,474 - Analysis - INFO - -----------------------------------------------------
2021-07-01 15:04:34,474 - Analysis - INFO - Number of generations:       2
2021-07-01 15:04:34,474 - Analysis - INFO - Number of transactions:      102 (81 unique)
2021-07-01 15:04:34,474 - Analysis - INFO - Transactions per second:     58
2021-07-01 15:04:34,474 - Analysis - INFO - Total code coverage:         29.95% (2016/6731)
2021-07-01 15:04:34,474 - Analysis - INFO - Total branch coverage:       31.76% (108/340)
2021-07-01 15:04:34,475 - Analysis - INFO - Total execution time:        1.75 seconds
2021-07-01 15:04:34,475 - Analysis - INFO - Total memory consumption:    110.65 MB

Any suggestions on how to get past this error?

Hi @banescusebi, thanks for reporting! We analyzed this bug and fixed it with our latest commit. Please check it out and let us know if the error still occurs.


Generate clusters dynamically

https://issues.redhat.com/browse/DPTP-2635
/cc @openshift/test-platform
/hold
require https://github.com/openshift/release/pull/24421

/hold cancel


[async_slam_toolbox_node-1] laserrangescan contains 456 range readings, expected 451

[async_slam_toolbox_node-1] laserrangescan contains 456 range readings, expected 451
getting this error in slam_toolbox can someone help me out?
thanks and regards

Hi @0jeetparmar0 , I am a LD19 user also encountering this issue.
You can refer to issues #4 and #11 of this repo. There also seems to be an opened pull request that fixes this issue, but have not been merged by the (rather inactive) owners.
Discussions about the root cause of this issue can also be found on slam_toolbox's repo (an issue raised back in 2020) and also similar repos utilising LDROBOT LiDARs


⚠ Spryr has degraded performance

In fbba978, Spryr (https://spryr.com) experienced degraded performance:

HTTP code: 200
Response time: 3862 ms

Resolved: Spryr performance has improved in 24e27da after 17 minutes.


Maven plugin should not need true

I don't see the maven plugin doing anything that would require it to be an extension.
The main use case for declaring a plugin as an extension is:

plug-ins which provide lifecycle enhancements.

https://maven.apache.org/guides/mini/guide-using-extensions.html
On the other hand extension plugins can be very powerful -- which means they could be doing anything. And this makes the build significantly harder to understand.
If I remove the extension attribute the plugin no longer executes it's goals.
Using the extension mechanism just to register some executions (=save typing) it's wrong.
The documentation should explain why is the extension needed for.
If you want to keep using the extension to save typing, the documentation should also show how to configure the plugin manually without extensions.

In my case I changed the <extension>true</extension>
to
                <executions>
                    <execution>
                        <id>generate-tests</id>
                        <goals>
                            <goal>generateTests</goal>
                        </goals>
                    </execution>
                </executions>

Not only I can now understand the build just by reading the POM file. As a bonus I got rid of the convert and generateStubs executions that I had no use for.

Hey, thank you for the feedback.
Nobody requires you to use the extensions. It's not mandatory to set them. If you don't want to set them just attach your executions to whatever phase you want.
Are you willing to file a PR to patch the documentation?


Introductory Slide Not open

In Pre-requisite Introductory Slide Not open.

@gauravrattan I have fixed the link. Please check and confirm.

@ajeetraina yes issue is fixed.


Build fails at Step 9 for nginx

running Docker 1.7.0 I run into this error.
Step 8 : RUN rm /etc/php5/fpm/pool.d/www.conf
---> Using cache
---> 1815c3b0671f
Step 9 : ADD http://storage.accesstomemory.org/releases/atom-2.1.2.tar.gz /
Service 'nginx' failed to build: Get https://storage.accesstomemory.org/releases/atom-2.1.2.tar.gz: x509: certificate signed by unknown authority

Huh! Trying to do a wget of atom-2.1.2.tar.gz results in the following error:
Connecting to storage.accesstomemory.org (storage.accesstomemory.org)|<IP_ADDRESS>|:443... connected.
ERROR: cannot verify storage.accesstomemory.org's certificate, issued by ‘CN=COMODO RSA Domain Validation Secure Server CA,O=COMODO CA Limited,L=Salford,ST=Greater Manchester,C=GB’:
  Unable to locally verify the issuer's authority.
To connect to storage.accesstomemory.org insecurely, use `--no-check-certificate'.

Looks like Artefactual needs to fix their SSL cert. I'll see if there's a way I can disable checks for Docker ADD command in the meantime and see if I can't let Artefactual know about their cert as well. Thanks for letting me know!

Aaaaand SSL is fixed! Or whatever, it's working again. OK!


Python 3.11 needed to build the example

typing.Self was introduced in python 3.11 - the readme mentions 3.9 and may want bumping.
I was using rustup and had to create a file rust-toolchain with the content nightly in the example directory to build (though; I'm very new to rust so not entirely sure what I'm doing).
After that I could build and run the example in dosbox (though on first runthrough I lost concentration so had run make not make debug or `make release).
$ make
sh build-dos-lib.sh --release
Building release target...
    Finished release [optimized] target(s) in 0.04s
Converting ELF objects to COFF-GO32...
Traceback (most recent call last):
  File "/home/stu/.virtualenvs/elf2djgpp/bin/elf2djgpp", line 8, in <module>
    sys.exit(main())
  File "/home/stu/projects/mine/dos-rust-projects/elf2djgpp/elf2djgpp/main.py", line 55, in main
    coff_obj.write(f)
  File "/home/stu/projects/mine/dos-rust-projects/elf2djgpp/elf2djgpp/coff_go32.py", line 194, in write
    from . import binfmt
  File "/home/stu/projects/mine/dos-rust-projects/elf2djgpp/elf2djgpp/binfmt.py", line 26, in <module>
    class COFFFileHeader:
  File "/home/stu/projects/mine/dos-rust-projects/elf2djgpp/elf2djgpp/binfmt.py", line 36, in COFFFileHeader
    def from_coff(cls, coff: coff_go32.COFF) -> typing.Self:
AttributeError: module 'typing' has no attribute 'Self'
make: *** [Makefile:32: build/release/librust_lib.a] Error 1

typing.Self was introduced in python 3.11 - the readme mentions 3.9 and may want bumping.

Doh! Fixed in 6f7db413a6e68913dc6df2e0ac75064bff87a0f2.

I was using rustup and had to create a file rust-toolchain with the content nightly in the example directory to build (though; I'm very new to rust so not entirely sure what I'm doing).

That makes two of us :smile:
Since the example requires nightly, I added a rust-toolchain.toml file to the example project in 42e5646c47b1f4c3438c5944cae46ebfbb51c81d.

That makes two of us :smile:
That's reassuring, I think being able to make stuff for DOS might be what gets me into rust :)

Re: needing rust nightly - I don't know why it's the place that you get -Z
error: the `-Z` flag is only accepted on the nightly channel of Cargo, but this is the `stable` channel

if that's an experimental feature, or just something special about nightly that only it will ever have to allow you to add targets.

TLDR: Probably worth mentioning in the readme as well; maybe not everyone uses rustup ?
Closing this ticket as the 3.11 thing is sorted.


Fix issue #73 of cpp-netlib.

Hi:
This is the change I mentioned in the comment of issue #73.
Regards,
diego.

Sorry to be a pain, but can you ask to merge this to the 0.9-devel branch? Thanks again.

Mmm... I can see only 1.0-devel and 0.8-devel, but not 0.9-devel...

OK, found, sorry. Trying again. Not used to github...


Does not work on Windows

Describe the bug
When using on Windows10 with the command rst2myst convert *.rst
it produces:
Usage: rst2myst convert [OPTIONS] [PATHS]...
Try 'rst2myst convert -h' for help.

Error: Invalid value for '[PATHS]...': Path '*.rst' does not exist.

I triple-checked the if I am in the right directory, where my .rst files are stored.
Also tried quoting the path.
Does anybody experienced that issue? On Linux it worked as expected.
Reproduce the bug

in a conda environment install with pip install rst-to-myst[sphinx]
in terminal navigate to the folder where *.rst files are stored
execute the command rst2myst convert *.rst

List your environment
No response

@fmfreeze You can get it working when upgrading your click package to 8.1.3, this works for me on python 3.8 and 3.10 on windows.

v0.3.3 (just released) now allows click v8


New website design

Hi, I offered to help @robertlacok with this website, so here are a bunch of changes! Disclaimer: I work at Deepnote.
We've already talked about a bunch of this stuff offline.
Notable changes:

the website is now implemented in Next.js, which is more mainstream than Svelte so easier for contributors
there are wayyy more details for each notebook tool, which allows for richer comparisons and details
there's a snazzy new design

Hey, awesome work @Pinpickle, thanks a lot!
We'll still need to keep the old URLs working, so I'm going to add redirects. I can't seem to push edits to this PR, so I'll merge it, add an extra commit, and deploy.


Support for unregistered accounts

[x] Import balances
[ ] Everywhere else

Users must import to a registered account.


Aligning login buttons

Hi,
From looking through the issues, I have found out that
<meteor-include src="loginButtons"></meteor-include>

has been depreciated for :
<blaze-template
   name="loginButtons">
</blaze-template>

This works, however, the default dropdown is to the right. I wish to make it dropdown to the left.
I attempted using :
align="left"

Didn't work so I thought I'd ask here if there is any specific way to make it dropdown to the left.
Thanks

Good question, you can read how here:
http://angular-meteor.com/api/blaze-template#passingargumentstoblazetemplate


VaultList Fix

Now VaultList component calls GetVaults async function.
GetVaults should be called on page level and pass it to VaultList component as props, not load on component level.

https://github.com/vultisig/vultisig-windows/pull/58


Other Devices interface, Remove button is too big

Other Devices interface, Remove button is too big

Fixed!


Features/cron

Make time event scheduling for sending google form based on the data on database
The function is send on 19:30 everyday

TODO: Delete date after finish reserve
(Check for success status after sending form too?)


Feature request - Playback + Recording change at the same time

Thanks for this very handy little utility.
One thing I'd like to see added is some type of convention to change the playback and recording devices at the same time. Either through a hotkey or favorite selection.
Use case is that I've got two different device "profiles" (playback device + recording device) that I'd like to quickly switch between. I can almost get there now, it takes a couple clicks to do it, but it would be great if a combination could be created to change both at the same time.

This is already planned for the Major 2.0 release.
It includes a scripting engine (JavaScript) in which you can write your own "profiles" and switch between them.
I originally thought of doing profiles as you suggested, but fleshed it out into scripts. This allows you to not only change more than one device at the same time, but you can also reset the volume levels of the devices, etc...
It's kind of a "Rolls Royce" solution, so everyone can do exactly what they want.
These scripts will be able to be "started" by macro events, which include an application starting, hotkeys, and even (ctrl/shift/alt) clicking on the icon in the tray.
This has already been built as in in 2.0, so will close this issue.
Cheers.


Update JMH to the latest version

OK, I've already triggered the following builds for you:

PullRequestFeedback build

@bot-gradle test this

OK, I've already triggered the following builds for you:

PullRequestFeedback build

@bot-gradle test and merge

Your PR is queued. See the queue page for details.

OK, I've already triggered a build for you.


ConfigureUtil.configureSelf doesn't seem to have a replacement

Context
We're trying to replace a usage of configureSelf here: https://github.com/gradle-nexus/publish-plugin/issues/152
The use case is implementing this DSL (assuming Groovy consumer here):
nexusPublishing { // NexusPublishExtension
    repositories { // NexusRepositoryContainer
        sonatype() // typed method on NexusRepositoryContainer
        sonatype { ... } // typed method on NexusRepositoryContainer
        somethingElse { ... } // maybeCreate magic by `project.container`!
    }
}

Current implementation (click to expand)
Here's the actual API that exists in Kotlin:
class NexusPublishExtension {
    val repositories: NexusRepositoryContainer = project.objects.newInstance(...)
    fun repositories(action: Action<in NexusRepositoryContainer>) { action(repositories) }
}

interface NexusRepositoryContainer : NamedDomainObjectContainer<NexusRepository> {
    fun sonatype(): NexusRepository // similar to RepositoryHandler.mavenCentral() DSL
    fun sonatype(action: Action<in NexusRepository>): NexusRepository
}

Now, the problem is the part that I conveniently omitted (...). We need to create:

an instance of a NamedDomainObjectCollection<NexusRepository>
with a specific factory (project.container)
+ some mixin methods (i.e. we need a specific strongly-typed interface).

All this while supporting Kotlin and Groovy consumers.
The current solution (using configureSelf) is to implement the container like this:
class NexusPublishExtension {
    val repositories: NexusRepositoryContainer = project.objects.newInstance(
        /*type = */ DefaultNexusRepositoryContainer::class,
        /*delegate = */ project.container(NexusRepository::class) { name ->
            project.objects.newInstance(NexusRepository::class, name, project)
        }
    )
}

internal open class DefaultNexusRepositoryContainer @Inject constructor(
    private val delegate: NamedDomainObjectContainer<NexusRepository>
) : NexusRepositoryContainer, NamedDomainObjectContainer<NexusRepository> by delegate {

    // Implement NexusRepositoryContainer
    override fun sonatype(): NexusRepository = sonatype {}
    override fun sonatype(action: Action<in NexusRepository>): NexusRepository = create("sonatype") {
        // defaults
        action.execute(this) // override possibility
    }

    // Amazing hack, ensuring `repositories { target: NexusRepositoryContainer }`.
    override fun configure(configureClosure: Closure<*>): NamedDomainObjectContainer<NexusRepository> =
        // Heavy use of internal API, copied from AbstractNamedDomainObjectContainer
        ConfigureUtil.configureSelf( // Deprecated.
            configureClosure,
            this,
            org.gradle.api.internal.NamedDomainObjectContainerConfigureDelegate(configureClosure, this)
        )
}

Related: https://github.com/gradle/gradle/issues/17446 (but not asking for the same, just guidance on a use case for a removed API)
Expected Behavior
Either there should be no need for the configure hack, when using by delegate, or another method exists.
Current Behavior
Have to use internal APIs to achieve this, I tried looking at how these are implemented:

RepositoryHandler (Project.repositories),
ArtifactHandler (Project.artifacts),
ConfigurationContainer (Project.configurations),

but no luck mimicking that behavior, because they all just extend internal API and use internal API to instantiate.
The biggest problem is that to remove the configure hack, we would need to remove the Kotlin delegation as well (by delegate), but then we would need to change the signature to:
internal open class DefaultNexusRepositoryContainer @Inject constructor(
) : NexusRepositoryContainer, org.gradle.api.internal.DefaultNamedDomainObjectSet<NexusRepository>()

so we would still be using internal APIs.
I tried a few variations, but all failed in some way.

If anyone knows a plugin that does something similar, please mention it here.

It looks like the above suggestion with adding an extra method to Configurable won't work.
For instance:
open class DefaultNexusRepositoryContainer @Inject constructor(
    private val delegate: NamedDomainObjectContainer<NexusRepository>
) : NexusRepositoryContainer, NamedDomainObjectContainer<NexusRepository> by delegate {

    override fun getConfigureTarget() = this
}

val container: DefaultNexusRepositoryContainer

When the user calls container.configure(..) it would delegate to, well, delegate, and it would execute AbstractNamedDomainObjectContainer.configure, and it will never upcall to DefaultNexusRepositoryContainer.getConfigureTarget.

Thank you for your interest in Gradle!
This feature request is in the backlog of the relevant team but this area of Gradle is currently not a focus one so it might take a while before it gets implemented.

(It might be just a documentation update, if it's already possible.)

if it's already possible

It is not possible as seen in NamedDomainObjectContainer code

Implementing a custom NamedDomainObjectContainer is not something we intend the users to do. We recognize the missing capability of having user-defined type-safe factory methods in the containers, though.
@gradle/bt-declarative-dsl, this use case might be of interest to you too.

If the configurations / repositories / sourceSets (and probably other) blocks can do it coming from built-in plugins, then why don't you want to allow third party plugins do the same? KGP (KMP targets), Publishing (target repos), AGP (build types, flavors, Gradle Managed Devices and groups) are all valid uses cases for this, aren't they?


Unable to resolve features with complex (but valid) semantic versions

The devcontainer CLI is able to resolve features with simple versions like this:
devcontainer features info manifest myociregistry.com/devcontainers/features/myfeature:1.0.2

However, it is not able to resolve features with more complex versions like this:
devcontainer features info manifest myociregistry.com/devcontainers/features/myfeature:1.0.2-ABC

or
devcontainer features info manifest myociregistry.com/devcontainers/features/myfeature:1.0.2+ABC

Both 1.0.2-ABC and 1.0.2+ABC are valid semantic versions. You can check with this regex checker linked from semver.org. However, the CLI will fail to resolve features with those versions.
1.0.2-ABC results in a No manifest found! error, and 1.0.2+ABC results in:
'1.0.2+ABC failed validation. Expected digest to match regex '/^[a-zA-Z0-9_][a-zA-Z0-9._-]{0,127}$/''

Looks like that regex doesn't even allow + in the version.
Is this the expected behaviour? Are we limited to simple versions?

That's correct that today only a simple x.y.z version will be recognized by the implementation.  Our sub-command that publishes Features will also expect to see the x.y.z pattern, and will republish x, x.y, and x.y.z for each new x.y.z version.


[stable/stolon]Data persistance is not working in postgres with chart version v0.13.0

Bug Description
Version of Helm and Kubernetes: v0.13.0
Which chart: stolon
What happened:
We expected data to be retained every time in the keepers.
What you expected to happen:
All the databases and tables created are gone when we restart the pods(helm delete)
How to reproduce it (as minimally and precisely as possible):
1. Install the postgres cluster using helm charts
2. Add some data to the postgres pointing to stolon-proxy service.
3. Restart the pods using helm delete command
4. Check if the data exists.

Anything else we need to know:
We are using proxy pods to connect to postgres. Let us know what is the issue.
We have downloaded charts from the below URL: https://github.com/sorintlab/stolon/releases
Version: v0.13.0
Im using dockerfile of postgres v10.8 as postgres image

Not sure if it helps, but Persistent Volumes (PV) and Persistent Volume Claims (PVC) need to be created for persistence.
An example of how to creat PV and PVC is in https://github.com/helm/charts/issues/14390

Hi @docktermj ,
I don't think its a PV/PVC creation problem, but if you still believe its PV/PVC problem then let me know after looking at below details.
Yesterday i deployed this chart with persistence enabled and it has created PVC/PV.
[root@server]# kubectl get pvc | grep stolon
data-si-postgres-stolon-keeper-0       Bound    pvc-14ffb823-8cfd-11e9-81a5-fa163eda9ef5   10Gi       RWO            cinder-az-nova           19h
data-si-postgres-stolon-keeper-1       Bound    pvc-25321774-8cfd-11e9-81a5-fa163eda9ef5   10Gi       RWO            cinder-az-nova           19h
And when pods are getting created it's using same volume which has been created earlier but even then the data is missing.
[root@server]# kubectl get pod | grep stolon
si-postgres-stolon-create-cluster-5pb97         0/1     Completed      0          5m43s
si-postgres-stolon-keeper-0                     1/1     Running        0          5m43s
si-postgres-stolon-keeper-1                     1/1     Running        0          5m33s
si-postgres-stolon-proxy-8496794555-bt6sx       1/1     Running        0          5m43s
si-postgres-stolon-proxy-8496794555-rtwmw       1/1     Running        0          5m43s
si-postgres-stolon-sentinel-5bcc8b65db-c4d4r    1/1     Running        0          5m43s
si-postgres-stolon-sentinel-5bcc8b65db-gfjjc    1/1     Running        0          5m43s
[root@server]# kubectl describe pod si-postgres-stolon-keeper-0 |grep pvc-14ffb823-8cfd-11e9-81a5-fa163eda9ef5
Normal  SuccessfulAttachVolume  5m55s  attachdetach-controller      AttachVolume.Attach succeeded for volume "pvc-14ffb823-8cfd-11e9-81a5-fa163eda9ef5"
[root@server]#

I see the below form one of the job which runs every time i deploy the chart:
[root@server]# k logs -f si-postgres-stolon-create-cluster-5pb97
WARNING: The current cluster data will be removed
WARNING: The databases managed by the keepers will be overwritten depending on the provided cluster spec.
[root@server]#
Does it makes sense to anyone why data persistence might not be happening for me?

Are you setting job.autoCreateCluster: false? If not - your current cluster will be removed.


No referer checking

It doesn't look like we have any referer checking in our middleware right now.
Look at porting over Django's checks.

Actually, this might not be needed (we have no cookie). Need to think about this more.

OK, from what I now understand referer checking is only required when storing a CSRF token in a cookie.
There's still the session cookie to consider, but that's out of scope IMO (HSTS can prevent session theft by making the client only use HTTPS, for example)
The Django docs could be a little clearer on this stuff, but this is the paper they keep referring to


Widen fixnum range

I'm hitting a problem getting pub to solve with Dart 2.13.4 on our codebase due to not being able to use fixnum 1.0.0. I've tracked it back to this repo that limits fixnum to ^0.10.9 but specifies a SDK range of >=2.7.0 <3.0.0.
It seems like if you wanted to use this library with null safety, you'd want to be able to solve to fixnum 1.0.0 which is null-safe.
Indeed when running the migration tool it reports that the version of fixnum isn't nullsafe.
pub outdated --mode=null-safety shows

So, it seems like a good change in order support both older and newer SDK version.
If this ends up merging, I would need this released as a patch version on the 1x line.

To get eyes, pinging @kevmoo and @sigurdm.

@robbecker-wf – take a look at the failures here?

Failing on dart analyze in the api_benchmark dir from something not being generated
error - lib/benchmark.dart:7:8 - Target of URI hasn't been generated: 'generated/benchmark.pb.dart'. Try running the generator that will generate the file referenced by the URI. - uri_has_not_been_generated

The other failures are formatting code under protoc_plugin.
I wouldn't expect this range widening to have changed either of these other subdirs. Is it possible this was failing before this PR or newer SDK or dart_style means code is being formatted differently?

Failing on dart analyze in the api_benchmark dir from something not being generated
error - lib/benchmark.dart:7:8 - Target of URI hasn't been generated: 'generated/benchmark.pb.dart'. Try running the generator that will generate the file referenced by the URI. - uri_has_not_been_generated

The other failures are formatting code under protoc_plugin.
I wouldn't expect this range widening to have changed either of these other subdirs. Is it possible this was failing before this PR or newer SDK or dart_style means code is being formatted differently?

Indeed! Hrm...

Aaah I just discovered we can avoid needing to update this repo to widen the fixnum range if we remove dependencies on build_vm_compilers. So if you want to close this PR and forget it ever happened, I understand. On the otherhand, I may have uncovered something legit that other people haven't reported yet (null safety & failing CI).

Closing this for now since there's no urgency on either side.

Hey @robbecker-wf – FYI. There is a null-safe branch that needs to get merged in.
This has the v2 of things.


UniswapV3 contracts are not downloaded

Any uniswapv3 contract failed witn an error 'Invalid address'
https://etherscan.io/address/0x0002e63328169d7feea121f1e32e4f620abf0352
https://etherscan.io/address/0x0001fcbba8eb491c3ccfeddc5a5caba1a98c4c28

Hmm I think that this may be an etherscan API issue. Here is the full output:
  {
    "status": "1",
    "message": "OK",
    "result": [
      {
        "SourceCode": "",
        "ABI": "Contract source code not verified",
        "ContractName": "",
        "CompilerVersion": "",
        "OptimizationUsed": "",
        "Runs": "",
        "ConstructorArguments": "",
        "EVMVersion": "Default",
        "Library": "",
        "LicenseType": "Unknown",
        "Proxy": "0",
        "Implementation": "",
        "SwarmSource": ""
      }
    ]
  }

But looking at the contract section,
Note: This contract matches the deployed ByteCode of the Source Code for Contract 0x8f8ef111b67c04eb1641f5ff19ee54cda062f163
I tested here and it load the contract using this address?
Using 0x8f8EF111B67C04Eb1641f5ff19EE54Cda062f163 address works for you?


Not breaking when git not available

@giovanni-guidini @dana-yaish is this still relevant or can we close it?


Move Conan stuff to custom path

Hi!
This PR solves the issue #114
After to have Conan Package Tools 0.18.0, we are able to use test_package in a custom path.
Close #114
Regards!

Coverage remained the same at 100.0% when pulling 3d3c389221ff1990023c2425ede87d33b019003f on feature/move-conan into 641bccc3e21ec5135994deb7ca1dc4219368bcd7 on master.

I now see both TravisCI and AppVeyor running twice!? This delays feedback and waste resources.
Can we do something about this? (I have to go to work now, if you know how to fix it please go ahead, otherwise I'll have a look an the weekend)

I now see both TravisCI and AppVeyor running twice!? This delays feedback and waste resources.

It's because one job is related for the feature branch, and the second is for the PR.
On my last commit, I used a fork, so the second job didn't appear on PR list.


Notify is available in 0.6.0 ?

Hi,
I was wondering if the new feature Notify is available. I don't know how to check manually. Maybe you can teach me how for the next feature :)

@miljan-aleksic pushed the notify component later to the develop branch then the 0.6.0 commit. So no.

@Nayir, all new features in dev branch will be available as 0.7.0 once considered stable enough. Meantime you can build manually and install Vuikit from your local repo. Something like this:
// in your vuikit local repo folder
npm install
npm run build

// in your vue project folder
npm install /local/path/to/vuikit/repo

Great. Thanks


next export cannot run properly

I am getting an error when I try to execute next export. Although it says "DONE Compiled successfully in...", it is not compiling everything.
I get an error that says this.
DONE Compiled successfully in 4711ms

> Ready on http://localhost:3000
> No `pages` directory found. Did you mean to run `next` in the parent (`../`) directory?

Any idea?

See my comment in #70. I don't think there is such a command.

Close according to this comment. I had a misunderstood. Sorry for the inconvenience.


When sending the outputs to email, it lost all the formats, as shown in the pic.
![QQ20240409-170533@2x.png](https://files.boardx.us/beBT5B83vbXnsw7R5/P7LsMHdYhqJp8AABQ/TzEiN9aSSW

When sending the outputs to email, it lost all the formats, as shown in the pic.

userAgent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/<IP_ADDRESS> Safari/537.36
email<EMAIL_ADDRESS>username: xiaodanzhang

See how it looks like in the email as above