Tracing regression in 1.5.x - nested traces are intermittently incomplete

Bug description
We are having a problem where traces aren't coming through to Jaeger correctly, where the spans are complaining that the parentSpanId is missing.
This is only an issue on 1.5.1, using a 1.4.6 sidecar (with 1.5.1 control plane) this traces are always complete.  Therefore I believe this to be a 1.5 regression.
It'd be great if someone who more deeply understands istio/envoy/zipkin and the istio debug logs could help me wade through the information here to see what is up.  If you're struggling to reproduce i'm more than happy to remote share with you to debug this.
Example failure
As an example, we have a simple app setup of:
nginx -> istio-test-app-1 -> istio-test-app-2 ->istio-test-app-3
Which has been around since istio 1.0 days, to help us test these sorts of things, nothing has changed in them with regards to header forwarding logic.
As you can see in Jaeger the trace for 38e310d5619723f6 is messed up:

The istio-test-app-2 span is missing.
Another example here on trace 64f13d7f39b4a5f2:

In this case, the span for istio-test-app-1 is missing.
I have captured istio-proxy-debug logs for each of the test apps, which contain the requests mentioned above (as well as some other noise, sorry).
istio-test-app-1.log
istio-test-app-2.log
istio-test-app-3.log
Expected behavior
Traces to be complete
Steps to reproduce the bug
For us, it's using a 1.5.1 sidecar
Version (include the output of istioctl version --remote and kubectl version and helm version if you used Helm)
1.5.1
How was Istio installed?
Helm
Environment where bug was observed (cloud vendor, OS, etc)

This needs to be fixed for 1.5.x releases as well.
@fpesce is it better to make this a 1.5 milestone or leave it as 1.6 ?

@douglas-reid if this is considered a regression, I think it would be better to fix it in 1.5.x.

@bianpengyuan can you investigate ?

@objectiser if you have any insights here, that'd also be appreciated.

I was able to reproduce this in my test environment. Looks like the drop is random. I need to dig more.

@douglas-reid Jaeger has various queues in its architecture where spans may be dropped if the component gets flooded by spans being reported - the collector being one of them. However, as @Stono pointed out, this only occurs when using a specific version of the proxy.
The other interesting point is that, rather than one span randomly going missing, which might happen occasionally - in both the example failures shown in the description, the 2 spans from the same service are missing.
So in the first failure - two spans (server and client) are missing for istio-test-app-2, whereas the second failure has two spans (server and client) for istio-test-app-1 missing.
Trace context is still being propagated successfully, so doesn't appear to be a problem with whether the trace is being sampled or not.

Hi,
Just to clarify a couple of things.  Istio test app 1,2 and 3 are the same code base and we’ve used them to test istio for almost a year.
In terms of load, this is on a testing cluster under no load, to an in memory jaeger collector.  The same behaviour manifests on all other clusters on 1.5.1 which back onto elasticsearch.
In terms of the correlation that it was the second span, that’s just coincidental I believe, as on other tests it’s spans at various levels that disappear.  I will share some other examples just to confirm when I’m back at my desk

@objectiser @bianpengyuan @douglas-reid i've tried to extract from the logs everything for trace 38e310d5619723f6, which from the information span 9fa88920d6b8778a was reporting a parent span of 70818c6c00118cea, which was missing.
However from the l
istio-test-app-1 (sidecar receiving request):
 - 'x-b3-spanid', 'ac7b27f7f9160c5e'
 - 'x-b3-parentspanid', 'e7008aa18fd4c5f6'

istio-test-app-1 (sidecar)-> istio-test-app-1 (app):
 - 'x-b3-spanid', '45749a1e06c2512d'
 - 'x-b3-parentspanid', 'ac7b27f7f9160c5e'

istio-test-app-1 (app) -> istio-test-app-1 (sidecar):
 - 'x-b3-spanid', '45749a1e06c2512d'
 - 'x-b3-parentspanid', 'ac7b27f7f9160c5e'

istio-test-app-1 (sidecar) -> istio-test-app-2 (sidecar):
 - 'x-b3-spanid', '9862709f56a411ae'
 - 'x-b3-parentspanid', '45749a1e06c2512d'

istio-test-app-2 (sidecar receiving request):
 - 'x-b3-spanid', '9862709f56a411ae'
 - 'x-b3-parentspanid', '45749a1e06c2512d'

istio-test-app-2 (sidecar) -> istio-test-app-2 (app):
 - 'x-b3-spanid', '6275ede08ea50c4a'
 - 'x-b3-parentspanid', '9862709f56a411ae'

istio-test-app-2 (app) -> istio-test-app2 (sidecar):
 - 'x-b3-spanid', '6275ede08ea50c4a'
 - 'x-b3-parentspanid', '9862709f56a411ae'

istio-test-app-2 (sidecar) -> istio-test-app-3 (sidecar):  <---- MISSING SPAN
 - 'x-b3-spanid', '70818c6c00118cea'
 - 'x-b3-parentspanid', '6275ede08ea50c4a'

istio-test-app-3 (sidecar receiving request):
 - 'x-b3-spanid', '70818c6c00118cea'
 - 'x-b3-parentspanid', '6275ede08ea50c4a'

istio-test-app-3 (sidecar) -> istio-test-app-3 (app):
 - 'x-b3-spanid', '9fa88920d6b8778a'
 - 'x-b3-parentspanid', '70818c6c00118cea'

So looking at the logs for istio-test-app-2, you can see the request from istio-test-app-2 -> istio-test-app-3 on line 178, at 18:50:25.302.   So I decided to follow down to where the Jaeger POST to report the spans happened and I saw this:
[Envoy (Epoch 0)] [2020-04-15 18:50:27.428][28][debug][router] [external/envoy/source/common/router/router.cc:614] [C0][S17259990231804564098] router decoding headers:
':method', 'POST'
':path', '/api/v2/spans'
':authority', 'zipkin'
':scheme', 'http'
'content-type', 'application/json'
'x-envoy-internal', 'true'
'x-forwarded-for', '<IP_ADDRESS>'
'x-envoy-expected-rq-timeout-ms', '5000'

[Envoy (Epoch 0)] [2020-04-15 18:50:27.428][28][debug][pool] [external/envoy/source/common/http/http1/conn_pool.cc:104] [C15] using existing connection
[Envoy (Epoch 0)] [2020-04-15 18:50:27.428][28][debug][router] [external/envoy/source/common/router/router.cc:1711] [C0][S17259990231804564098] pool ready
[Envoy (Epoch 0)] [2020-04-15 18:50:27.429][28][debug][router] [external/envoy/source/common/router/router.cc:1115] [C0][S17259990231804564098] upstream headers complete: end_stream=false
[Envoy (Epoch 0)] [2020-04-15 18:50:27.429][28][debug][http] [external/envoy/source/common/http/async_client_impl.cc:95] async http request response headers (end_stream=false):
':status', '400'
'content-type', 'text/plain; charset=utf-8'
'vary', 'Origin'
'x-content-type-options', 'nosniff'
'date', 'Wed, 15 Apr 2020 18:50:27 GMT'
'content-length', '133'
'x-envoy-upstream-service-time', '0'

[Envoy (Epoch 0)] [2020-04-15 18:50:27.429][28][debug][client] [external/envoy/source/common/http/codec_client.cc:104] [C15] response complete
[Envoy (Epoch 0)] [2020-04-15 18:50:27.429][28][debug][pool] [external/envoy/source/common/http/http1/conn_pool.cc:201] [C15] response complete
[Envoy (Epoch 0)] [2020-04-15 18:50:27.429][28][debug][pool] [external/envoy/source/common/http/http1/conn_pool.cc:239] [C15] moving to ready

Note the 400 BAD REQUEST with a upstream-service-time of 0?  Feels like for whatever reason envoy rejected this request with a 400.
Looking through the logs further, we occasionally see these 400's after a POST to jaeger and it marries up with traces that are missing spans.

So i then followed this through to the jaeger collector and TCP dumped out the incoming requests and was able to see the 400s:
09:42:04.259148 IP <IP_ADDRESS>.39940 > istio-tracing-5845b6fbd4-7br5j.9411: Flags [P.], seq<PHONE_NUMBER>:4163820027, ack 504468475, win 333, options [nop,nop,TS val<PHONE_NUMBER> ecr 175011169], length 4719
E.....@.?.(=
...
..u..$............M9%.....
z.+.
nuaPOST /api/v2/spans HTTP/1.1
host: zipkin
content-type: application/json
x-envoy-internal: true
x-forwarded-for: <IP_ADDRESS>
x-envoy-expected-rq-timeout-ms: 5000
transfer-encoding: chunked

119c
[{"traceId":"c71df96d08ec50ef","name":"app.istio-test-app-2.svc.cluster.local:80/*","timestamp":1.58703012360987e+15,"duration":7912,"kind":"CLIENT","localEndpoint":{"port":0,"serviceName":"istio-test-app-1","ipv4":"<IP_ADDRESS>"},"id":"aa29007755b8cbac","tags":{"upstream_cluster":"outbound|80||app.istio-test-app-2.svc.cluster.local","guid:x-request-id":"6ce4516f-0da3-48b4-8b24-17e2a54da720","http.protocol":"HTTP/1.1","component":"proxy","http.url":"http://app.istio-test-app-2/downstream?servers=app.istio-test-app-3/instant","user_agent":"-","downstream_cluster":"-","request_size":"0","response_flags":"-","response_size":"428","node_id":"sidecar~<IP_ADDRESS>~istio-test-app-1-cd9cd559d-m9hql.istio-test-app-1~istio-test-app-1.svc.cluster.local","http.status_code":"200","http.method":"GET"},"parentId":"6c2b3d8c42f24f05"},{"id":"6c2b3d8c42f24f05","tags":{"response_size":"633","node_id":"sidecar~<IP_ADDRESS>~istio-test-app-1-cd9cd559d-m9hql.istio-test-app-1~istio-test-app-1.svc.cluster.local","http.status_code":"200","http.method":"GET","upstream_cluster":"inbound|80|http-web|app.istio-test-app-1.svc.cluster.local","guid:x-request-id":"6ce4516f-0da3-48b4-8b24-17e2a54da720","http.protocol":"HTTP/2","component":"proxy","http.url":"https://app.istio-test-app-1.testing.k8.atcloud.io/downstream?servers=app.istio-test-app-2/downstream?servers=app.istio-test-app-3/instant","user_agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.163 Safari/537.36","downstream_cluster":"-","response_flags":"-","request_size":"0"},"parentId":"0b45ec791cb2b9d9","traceId":"c71df96d08ec50ef","name":"app.istio-test-app-1.svc.cluster.local:80/*","timestamp":1587030123608213,"duration":11523,"kind":"CLIENT","localEndpoint":{"port":0,"serviceName":"istio-test-app-1","ipv4":"<IP_ADDRESS>"}},{"duration":8036,"kind":"CLIENT","localEndpoint":{"port":0,"serviceName":"istio-test-app-1","ipv4":"<IP_ADDRESS>"},"id":"2271a862646ef79b","tags":{"downstream_cluster":"-","user_agent":"-","response_flags":"-","request_size":"0","response_size":"428","node_id":"sidecar~<IP_ADDRESS>~istio-test-app-1-cd9cd559d-m9hql.istio-test-app-1~istio-test-app-1.svc.cluster.local","http.status_code":"200","http.method":"GET","upstream_cluster":"outbound|80||app.istio-test-app-2.svc.cluster.local","guid:x-request-id":"9c0c4bbc-777b-4414-a36b-e2ff6a0aa30b","http.protocol":"HTTP/1.1","component":"proxy","http.url":"http://app.istio-test-app-2/downstream?servers=app.istio-test-app-3/instant"},"parentId":"514b42930dfd5f51","traceId":"b10d0bc09ddd3574","name":"app.istio-test-app-2.svc.cluster.local:80/*","timestamp":1587030124065573},{"traceId":"b10d0bc09ddd3574","name":"app.istio-test-app-1.svc.cluster.local:80/*","timestamp":1587030124063938,"duration":11654,"kind":"CLIENT","localEndpoint":{"port":0,"serviceName":"istio-test-app-1","ipv4":"<IP_ADDRESS>"},"id":"514b42930dfd5f51","tags":{"http.url":"https://app.istio-test-app-1.testing.k8.atcloud.io/downstream?servers=app.istio-test-app-2/downstream?servers=app.istio-test-app-3/instant","downstream_cluster":"-","user_agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.163 Safari/537.36","response_flags":"-","request_size":"0","response_size":"633","node_id":"sidecar~<IP_ADDRESS>~istio-test-app-1-cd9cd559d-m9hql.istio-test-app-1~istio-test-app-1.svc.cluster.local","http.status_code":"200","http.method":"GET","upstream_cluster":"inbound|80|http-web|app.istio-test-app-1.svc.cluster.local","guid:x-request-id":"9c0c4bbc-777b-4414-a36b-e2ff6a0aa30b","http.protocol":"HTTP/2","component":"proxy"},"parentId":"a663b2da260c5333"},{"localEndpoint":{"port":0,"serviceName":"istio-test-app-1","ipv4":"<IP_ADDRESS>"},"id":"ea393e07f90061e6","tags":{"http.method":"GET","upstream_cluster":"outbound|80||app.istio-test-app-2.svc.cluster.local","guid:x-request-id":"45501d11-f9cc-442f-b7b7-8e2822de8469","http.protocol":"HTTP/1.1","component":"proxy","http.url":"http://app.istio-test-app-2/downstream?servers=app.istio-test-app-3/instant","downstream_cluster":"-","user_agent":"-","request_size":"0","response_flags":"-","response_size":"428","node_id":"sidecar~<IP_ADDRESS>~istio-test-app-1-cd9cd559d-m9hql.istio-test-app-1~istio-test-app-1.svc.cluster.local","http.status_code":"200"},"parentId":"334d87d8578e0e3e","traceId":"dac5d6d050bbf254","name":"app.istio-test-app-2.svc.cluster.local:80/*","timestamp":1587030124251379,"duration":7252,"kind":"CLIENT"}]
0

09:42:04.259428 IP istio-tracing-5845b6fbd4-7br5j.9411 > <IP_ADDRESS>.39940: Flags [P.], seq 1:308, ack 4719, win 331, options [nop,nop,TS val 175039204 ecr<PHONE_NUMBER>], length 307
E..g..@.@.).
..u
...$..............K'......

n..z.+.HTTP/1.1 400 Bad Request
Content-Type: text/plain; charset=utf-8
Vary: Origin
X-Content-Type-Options: nosniff
Date: Thu, 16 Apr 2020 09:42:04 GMT
Content-Length: 133

Unable to process request body: json: cannot unmarshal number 1.58703012360987e+15 into Go struct field Span.timestamp of type int64

I think this is the relevant issue. It relates to a couple of changes made to the zipkin v2 timestamp field, and it refers to a comment about the format: https://github.com/openzipkin/zipkin-go/pull/161#issuecomment-598558072
So potentially need to upgrade to latest envoy commit.

great sleuthing @Stono .
Does this mean every request should fail ? With HTTP_JSON endpoint,  every timestamp should be scientific notation, or does it change from number to scientific dynamically ?

Summarizing offline conversations, etc.: In the linked issue, it looks like others were reporting the it sometimes uses scientific notation. A patch was just merged and we need to verify and then cherry-pick to the impacted releases.

1.5 cherry-pick PR: https://github.com/istio/envoy/pull/194

Summarizing offline conversations, etc.: In the linked issue, it looks like others were reporting that it sometimes uses scientific notation. A patch was just merged and we need to verify and then cherry-pick to the impacted releases.

@douglas-reid I'm confused as to why this will cause intermittent failures?

@bianpengyuan the cherry-pick is only on master, not 1.5

@mandarjog not sure I follow. This needs to be fix on 1.5 right? the linked pr is to cherry-pick fix into istio/envoy release-1.5.
@nrjpoddar It is random behavior from protobuf struct -> json serialization with large integer.

Summarizing offline conversations, etc.: In the linked issue, it looks like others were reporting that it sometimes uses scientific notation. A patch was just merged and we need to verify and then cherry-pick to the impacted releases.

@douglas-reid I'm confused as to why this will cause intermittent failures?

@nrjpoddar i haven't looked closely at the patch so I don't know the underlying cause, but https://github.com/envoyproxy/envoy/issues/9341#issuecomment-577051483 shows that others have observed intermittent failures.

Sorry to be that naggy person but this is causing us quite a few ops issues.  When do you anticipate being able to cut a 1.5.2 with the fix?
I ask because if it’s far away I’m going to have to look at rolling back everywhere to 1.4.6
Cheers for your help!

This has been cherry-picked into 1.5 and will be released with 1.5.2. I've verified the fix in my test env. @Stono Please try out gcr.io/istio-testing/proxyv2:1.5-alpha.287e52ea3c53f7a15853db550edef4dbe243b4ca in your env when you have a chance, thanks!

@fpesce ^^ about 1.5.2 eta.

ETA for 1.5.2 is realistically Wednesday, we are fixing https://github.com/istio/istio/issues/23038 right now, I'll probably trigger the build when https://github.com/istio/istio/pull/23037 is tested and merged

Thank you for the update both.  I’ll test that image shortly

@fpesce @bianpengyuan I've tested the above image on the deployment setup I used to replicate this issue and can confirm the issue is resolved.
This deployment has constant load at it and is also patch restarted every 5-10 minutes to generate churn so i'll leave it there for a good few days and let you know if I spot anything untoward.
Ta

@bianpengyuan and just to be clear, that envoy patch is definitely included in 1.6 moving forward ?

@douglas-reid with https://github.com/istio/envoy/pull/196 it should be.

This should be fixed in 1.6. I will use this issue to track how to make the test more strict in order to catch a random bug like this.

Close in favor of https://github.com/istio/istio/issues/23260

Bad news, looks like this is still broken.
Please can we reopen it @bianpengyuan @douglas-reid @fpesce and i'll try and debug more tomorrow

Is it still broken with 1.5.2?

Seems so yes :-(

Ok thanks! I will also try out 1.6 installation and see if I can reproduce.

How often does it happen in your env? I cannot find any occurrence of this bug in my cluster installed head. If I remembered correctly it happened quite often before the fix. So not sure if it is just a transient error that you caught.

No unfortunately I have a lot of examples.  It's quite late here so I'll collect more information for you tomorrow.

08:11:16.180772 IP <IP_ADDRESS>.47740 > istio-tracing-54f89c8554-ldvkv.9411: Flags [P.], seq 38021791:38026659, ack 73850609, win 4416, options [nop,nop,TS val<PHONE_NUMBER> ecr 244643951], length 4868
<EMAIL_ADDRESS>...
<EMAIL_ADDRESS>L......oPOST /api/v2/spans HTTP/1.1
host: zipkin
content-type: application/json
x-envoy-internal: true
x-forwarded-for: <IP_ADDRESS>
x-envoy-expected-rq-timeout-ms: 5000
transfer-encoding: chunked

1231
[{"localEndpoint":{"port":0,"serviceName":"ingress-nginx-external","ipv4":"<IP_ADDRESS>"},"id":"90baa147bf5ed88a","tags":{"http.status_code":"200","http.method":"GET","upstream_cluster":"outbound|80||app.consumer-gateway.svc.cluster.local","guid:x-request-id":"58c807e0dbabe597-MAN","http.protocol":"HTTP/1.1","component":"proxy","http.url":"https://www.autotrader.co.uk/json/finance-calculator?useDefault=true&advertId=202004309122948&selectedDid=21342&includePartEx=true&channel=cars","user_agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.122 Safari/537.36","downstream_cluster":"-","request_size":"0","response_flags":"-","response_size":"1351","node_id":"sidecar~<IP_ADDRESS>~ingress-nginx-external-controller-6758c68b6f-ccjzm.ingress-nginx~ingress-nginx.svc.cluster.local"},"parentId":"4a9ad30d09410d00","traceId":"0d645df36d72d419","name":"app.consumer-gateway.svc.cluster.local:80/*","timestamp":1.58832067599851e+15,"duration":143176,"kind":"CLIENT"},{"localEndpoint":{"port":0,"serviceName":"ingress-nginx-external","ipv4":"<IP_ADDRESS>"},"id":"91a4b9d86579e01c","tags":{"component":"proxy","http.url":"https://www.autotrader.co.uk/","downstream_cluster":"-","user_agent":"Mozilla/5.0 (iPhone; CPU iPhone OS 13_3_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.0.5 Mobile/15E148 Safari/604.1","response_flags":"-","request_size":"0","response_size":"47157","node_id":"sidecar~<IP_ADDRESS>~ingress-nginx-external-controller-6758c68b6f-ccjzm.ingress-nginx~ingress-nginx.svc.cluster.local","http.status_code":"200","http.method":"GET","upstream_cluster":"outbound|80||app.consumer-gateway.svc.cluster.local","guid:x-request-id":"58c807e17f61ce1f-LHR","http.protocol":"HTTP/1.1"},"parentId":"cfefa6fc30ac0959","traceId":"14d19010d8cf9c61","name":"app.consumer-gateway.svc.cluster.local:80/*","timestamp":1588320676096608,"duration":45625,"kind":"CLIENT"},{"id":"462cb9066426f1c7","tags":{"http.status_code":"204","http.method":"POST","upstream_cluster":"outbound|80||app.consumer-gateway.svc.cluster.local","guid:x-request-id":"58c807e1d9e60686-LHR","http.protocol":"HTTP/1.1","component":"proxy","http.url":"https://www.autotrader.co.uk/json/js-log/error","downstream_cluster":"-","user_agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36 Edge/18.18362","response_flags":"-","request_size":"35","response_size":"0","node_id":"sidecar~<IP_ADDRESS>~ingress-nginx-external-controller-6758c68b6f-ccjzm.ingress-nginx~ingress-nginx.svc.cluster.local"},"parentId":"8d2b9c59742c31c6","traceId":"b769794e36f7a05c","name":"app.consumer-gateway.svc.cluster.local:80/*","timestamp":1588320676152718,"duration":5986,"kind":"CLIENT","localEndpoint":{"port":0,"serviceName":"ingress-nginx-external","ipv4":"<IP_ADDRESS>"}},{"tags":{"http.status_code":"204","http.method":"POST","upstream_cluster":"outbound|80||app.consumer-gateway.svc.cluster.local","guid:x-request-id":"58c807e1ed67d210-MAN","http.protocol":"HTTP/1.1","component":"proxy","http.url":"https://www.autotrader.co.uk/json/js-log/error","downstream_cluster":"-","user_agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36 Edge/18.18363","response_flags":"-","request_size":"35","response_size":"0","node_id":"sidecar~<IP_ADDRESS>~ingress-nginx-external-controller-6758c68b6f-ccjzm.ingress-nginx~ingress-nginx.svc.cluster.local"},"parentId":"20a6a93d1f52ec6c","traceId":"6d16137acf04708a","name":"app.consumer-gateway.svc.cluster.local:80/*","timestamp":1588320676165394,"duration":4196,"kind":"CLIENT","localEndpoint":{"port":0,"serviceName":"ingress-nginx-external","ipv4":"<IP_ADDRESS>"},"id":"e6a586c45cef35f7"},{"tags":{"downstream_cluster":"-","user_agent":"swcd (unknown version) CFNetwork/1125.2 Darwin/19.4.0","response_flags":"-","request_size":"0","response_size":"0","node_id":"sidecar~<IP_ADDRESS>~ingress-nginx-external-controller-6758c68b6f-ccjzm.ingress-nginx~ingress-nginx.svc.cluster.local","http.status_code":"301","http.method":"GET","upstream_cluster":"outbound|80||app.consumer-gateway.svc.cluster.local","guid:x-request-id":"58c807e1ea780070-LHR","http.protocol":"HTTP/1.1","component":"proxy","http.url":"https://autotrader.co.uk/.well-known/apple-app-site-association"},"parentId":"4caa0bdd26697db3","traceId":"d174821f86615120","name":"app.consumer-gateway.svc.cluster.local:80/*","timestamp":1588320676162625,"duration":17704,"kind":"CLIENT","localEndpoint":{"port":0,"serviceName":"ingress-nginx-external","ipv4":"<IP_ADDRESS>"},"id":"580389b9404e3ee7"}]
0

08:11:16.181063 IP istio-tracing-54f89c8554-ldvkv.9411 > <IP_ADDRESS>.47740: Flags [P.], seq 1:308, ack 4868, win 331, options [nop,nop,TS val 244644002 ecr<PHONE_NUMBER>], length 307
E..g<EMAIL_ADDRESS>.'.
...$..|.f...D=....Kl......
....L...HTTP/1.1 400 Bad Request
Content-Type: text/plain; charset=utf-8
Vary: Origin
X-Content-Type-Options: nosniff
Date: Fri, 01 May 2020 08:11:16 GMT
Content-Length: 133

The source pod:
❯ k get pods --all-namespaces -o wide| grep <IP_ADDRESS>
ingress-nginx                              ingress-nginx-external-controller-6758c68b6f-ccjzm                                            2/2     Running             0          22d     <IP_ADDRESS>    gke-delivery-platform-core-20191114-af4e6139-61hg     <none>           1/1

Which is actually on 1.5.1!  Some how, it's been missed during our sidecar rollout...
2020-04-08T17:58:10.931836Z	info	Version 1.5.1-9d07e185b0dd50e6fb1418caa4b4d879788807e3-Clean
2020-04-08T17:58:10.931956Z	info	Obtained private IP [<IP_ADDRESS>]
2020-04-08T17:58:10.931989Z	info	Proxy role: &model.Proxy{ClusterID:"", Type:"sidecar", IPAddresses:[]string{"<IP_ADDRESS>", "<IP_ADDRESS>"}, ID:"ingress-nginx-external-controller-6758c68b6f-ccjzm.ingress-nginx", Locality:(*envoy_api_v2_core.Locality)(nil), DNSDomain:"ingress-nginx.svc.cluster.local", ConfigNamespace:"", Metadata:(*model.NodeMetadata)(nil), SidecarScope:(*model.SidecarScope)(nil), MergedGateway:(*model.MergedGateway)(nil), ServiceInstances:[]*model.ServiceInstance(nil), WorkloadLabels:labels.Collection(nil), IstioVersion:(*model.IstioVersion)(nil)}

Sorry for the false positive guys!

I'm still facing this issue on istio 1.5.2, using opentelemetry library 0.7.0, and nodejs 8.9.4

@romil-punetha Please check and make sure all of your proxies are updated to 1.5.2.

I checked. Both the services and the gateway from the above image are using envoy:1.5.2

I've tried to reproduce this with 1.5.2 but did not see this anymore. Could you use @Stono's TCP dump command https://github.com/istio/istio/issues/22968#issuecomment-614537948 and see if you can capture any 400s? Then it would be nice to check that the proxy sending the trace is also 1.5.2.

@bianpengyuan This is currently marked as a release blocker on the spreadsheet. If it's no longer a release blocker please either change the priority or mark it as closed.

@brian-avery I believe this is a release blocker, and it is currently marked closed. However, I think we are trying to determine if it should in fact be considered closed, or if more work is needed.

I was not able to reproduce this with either 1.6.0-beta.0 or 1.5.2. I tend to treat this as fixed and not blocking, unless there are evidences show otherwise.

@bianpengyuan I was able to fix this by configuring the Jaeger propagator in Opentelemetry library to use jaeger-agent.istio-system:6832 as the backend.


Enum typename considering name in enumeratedValues

svd2rust has a good mechanism to parse enumeratedValues into Rust enums. However, current code would pick name of the first field the enum resides, and use pub type to re-export it into other names.
SVD files provide name field in enumeratedValues, this commit make use of them to name the Rust enums it generated. After this commit, more information of SVD file is considered in output pac crate. Users may also discover a significant drop in amount of types, which will speed up rustdoc generation and crate compilation.
Before this pull request:

After this pull request:

SVD file for this example:

            <field>
              <name>fifo_error_enable</name>
              <description>Transmit or receive FIFO error interrupt enable</description>
              <lsb>29</lsb>
              <msb>29</msb>
              <writeConstraint>
                <useEnumeratedValues>true</useEnumeratedValues>
              </writeConstraint>
              <enumeratedValues>
                <name>InterruptEnable</name>
                <enumeratedValue>
                  <name>enable</name>
                  <description>Enable interrupt</description>
                  <value>1</value>
                </enumeratedValue>
                <enumeratedValue>
                  <name>disable</name>
                  <description>Disable interrupt</description>
                  <value>0</value>
                </enumeratedValue>
              </enumeratedValues>
            </field>
            <field>
              <name>arbitrate_lost_enable</name>
              <description>Arbitration lost interrupt enable</description>
              <lsb>28</lsb>
              <msb>28</msb>
              <writeConstraint>
                <useEnumeratedValues>true</useEnumeratedValues>
              </writeConstraint>
              <enumeratedValues derivedFrom="InterruptEnable" />
            </field>
            <field>
              <name>not_acknowledged_enable</name>
              <description>Not-acknowledged response interrupt enable</description>
              <lsb>27</lsb>
              <msb>27</msb>
              <writeConstraint>
                <useEnumeratedValues>true</useEnumeratedValues>
              </writeConstraint>
              <enumeratedValues derivedFrom="InterruptEnable" />
            </field>
<!-- and all other fields in this register -->

r? @Emilgardis
(rust-highfive has picked a reviewer for you, use r? to override)

Some of enums can be exported from other registers/peripherals (read modules)

Looks like MSRV 1.51.0 does not support new style format macro: format!("{variable}")

Looks like MSRV 1.51.0 does not support new style format macro: format!("{variable}")

it was added in 1.58.0

im not sure I like this change as it currently stands, it is imo less clear what is happening in the docs

There are register spec type parameters in BitWriters, resulting in that write proxies cannot be re-exported properly. Any good opinion?

cannot be re-exported properly

and should not be
I like idea to use enumaratedValues name for enums, but not for readers and writers.

Could you move ConstantCase, docs and other code style changes in separate PR?
Possibly it makes sense to collect all case related traits in 1 ToCase.

I'll try to split this pull request, thanks! ;)


Wrap S3 in storage adapter interface

Codecov Report
Base: 43.96% // Head: 38.97% // Decreases project coverage by -4.98% :warning:

Coverage data is based on head (4cc9619) compared to base (3ff0864).
Patch coverage: 0.00% of modified lines in pull request are covered.

Additional details and impacted files
@@            Coverage Diff             @@
##           master      #16      +/-   ##
==========================================
- Coverage   43.96%   38.97%   -4.99%
==========================================
  Files           5        7       +2
  Lines         414      467      +53
  Branches       26       28       +2
==========================================
  Hits          182      182
- Misses        229      280      +51
- Partials        3        5       +2

Impacted Files
Coverage Δ

src/api.ts
0.00% <0.00%> (ø)

src/index.ts
0.00% <0.00%> (ø)

src/storage/S3StorageAdapter.ts
0.00% <0.00%> (ø)

src/storage/StorageAdapter.ts
0.00% <0.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Update whats-new-apps-19h1.md

Change in wordings # Snip & Sketch

@lizap - Can you please assist to check, approve and merge the content of this PR based on issue #181?
Thank you.

@msbemba - please make the necessary typo corrections to this PR as suggested by @ItzLevvie. Thank you.

@lizap - Can you please assist to check, approve and merge the content of this PR based on issue #181?
Thank you.


on Dledger  community recommendation

In Dledger mode, the official recommended version is 4.? .
Is there an official stress test report under Dledger?
在Dledger 模式下的，官方推荐版本是4.几。
官方有  Dledger 下的压测报告吗。

推荐最新的版本，暂时没有压测报告。

The report has not opened now. IMO, there are many others who do the same thing in the community, let's call for help if anyone sees this issue.


New Machine requirement: Windows dockerBuild containers

I need to request a new machine:

New machine operating system (e.g. linux/windows/macos/solaris/aix): Windows
New machine architecture (e.g. x64/aarch32/arm32/ppc64/ppc64le/sparc): x64
Provider (leave blank if it does not matter): Docker :-)
Desired usage: Build containers, similar to what we have for Linux
Any unusual specification/setup required:
How many of them are required: n/a - they should be created dynamically

Please explain what this machine is needed for:
Running builds in an isolated way where we can achieve SLSA build level 3 compliance on Windows along with the other primary platforms. Ideally we'll be able to create windows-on-windows container images which we share and then download and run the builds in.
As background info:

https://learn.microsoft.com/en-us/virtualization/windowscontainers/quick-start/set-up-environment?tabs=dockerce#windows-server-1 looks to be a good resource on the software options for containers on Windows
Eclipse Temurin does provide container images for Windows at https://hub.docker.com/_/eclipse-temurin/ - these are created from the dockerfiles in https://github.com/adoptium/containers/tree/main/21/jdk/windows which may give a good starting point for this work on investigating windows containers.

So the tasks required would be:

[x] Identify the appropriate software for running containers and ensure no licensing concerns (Likely something from the microsoft site linked above)
[x] See if we can verify that a "basic" dockerfile works in that environment and whether we can map directories into it (same as -v on linux) which are read+write in the container
[ ] Determine whether we can create a container from the playbooks using a dockerfile equivalent to the Linux ones
[ ] Once we create the container, map a directoryi from the host into it with -v and use that to build Temurin in the container on the mapped volume so that the output is visible on the host system.
[ ] Understand whether we can reasonably push the resulting container images with the compiler up to dockerhub
[ ] Integrate this into the build pipelines
[ ] Implement processes to regenerate the images when playbook updates are made, - likely an addition to what we do for Linux in https://github.com/adoptium/infrastructure/blob/master/FAQ.md#what-about-the-builds-that-use-the-dockerbuild-tag
[ ] Declare SLSA Build level 3 on Windows :-)

Once this level of analysis and expertise is gained it will likely make windows installer testing, or any other such activities simpler and give us more options moving forward.

Please, assign this task to me. Thank you.

Of the three options listed on the Microsoft website:

The first (Docker CE / Moby) seems to work well out of the box
The second (Mirantis) appears to be a commercial offering
The third (Containerd+nerdctl) appers functional although networking doesn't work out of the box and it seems to fail to be able to start the eclipse-temurin container's default jshell process.

OK First phase done ...

docker run -p 5986:5986 -v c:\Users\sxa:c:\cabbage mcr.microsoft.com/windows/servercore:ltsc2022
Run ConfigureRemotingForAnsible.ps1 with the usual parameters with the netsh commands disabled (They require windows defender which isn't in the image)
Create a user to connect with for the playbooks:

net user ansible MyPassword /ADD
net localgroup "Administrators" ansible /ADD
net localgroup "Remote Management Users" ansible /ADD

This allows the machine to be accessible via ansible running on a remote machine :-)

Playbook execution notes:

VS2013 requires the archive under /Vendor_Files/windows, otherwise MSVS_2013 needs to be skipped
NTP_TIME needs to be skipped as that has issues that are presumably related to running in a container: FAILED! => {"changed": false, "msg": "Unhandled exception while executing module: Service 'Windows Time (W32Time)' cannot be started due to the following error: Cannot start service W32Time on computer '.'."}
In the absence of the fixed layout files for VS2019 and VS2022, adoptopenjdk needs to be skipped to allow them to complete successfully

ansible can be run on the host to point at the container if you install cygwin which has ansible as one of its installable options (You probably want to include git too if it's a clean install on the host system). Noting that if you use localhost/<IP_ADDRESS> in your hosts file you should specify -e git_sha=12345 or something appropriate otherwise the execution will trip up over https://github.com/adoptium/infrastructure/blob/4aa7788325c224484f99aa1ae000f117e9b081d7/ansible/playbooks/AdoptOpenJDK_Windows_Playbook/roles/logs/tasks/main.yml#L14

Latest attempt is with:
--skip-tags adoptopenjdk,reboot,MSVS_2017,NTP_TIME
Playbook changes to make it complete:

Set ansible_connection/ansible_winrm_transport  in ansible.cfg
Set ansible_user/ansible_password in group_vars/all/adoptopenjdk_variables.yml
Remove win_reboot: from Common/roles/main.yml Line 60
Remove win_reboot: from MSVS_2013 role line 50
Remove win_reboot: from MSVS_2017 role line 37
Remove checksum parameters MSVS_2022 role line 103 as it's been updated
Remove win_reboot from WMF_5.1 role line 29
Remove win_reboot from cygwin role line 45 (Although it's already covered with th reboot tag)

docker commit didn't work on my image:
Error response from daemon: re-exec error: exit status 1: output: mkdir \\?\C:\Windows\SystemTemp\hcs376450290\Files: Access is denied
This is specific to the new image which has had the playbook run on it  and does not occur when attempting to commit a image with only basic changes applied.

This is unfortuante. The builds aren't working because it looks like the automatic shortname generation  (fsutil behavior set disable8.3 0) does not appear to be working within the container but is mandatory for the openjdk build process. Directories can have a shortname created manually with fsutil file setshortname "Long name" shortname but that is not ideal to do for each possible path.

Manually created a few of the shortnames that the configure step was objecting to and I have a JDK21u build complete in a container, so this seems feasible 👍🏻

Noting that we should look at doing this with the MS build tools installer which is suitable for use by Open Source projects. The jdk21u builds currently use:
10:04:20  * C Compiler:     Version 19.37.32822 (at /cygdrive/c/progra~1/micros~3/2022/commun~1/vc/tools/msvc/1437~1.328/bin/hostx64/x64/cl.exe)
10:04:20  * C++ Compiler:   Version 19.37.32822 (at /cygdrive/c/progra~1/micros~3/2022/commun~1/vc/tools/msvc/1437~1.328/bin/hostx64/x64/cl.exe)

Other references (this numbering is more confiusing that I realised - I thought we only had the '2022' vs '19.xx' versioning differences to worry about before today...)

https://devblogs.microsoft.com/oldnewthing/20221219-00/?p=107601 (Would suggest, extrapolating from the table, that _MSC_VER 1937 maps to VS 17.7)
https://learn.microsoft.com/en-us/cpp/overview/compiler-versions?view=msvc-170#service-releases-starting-with-visual-studio-2017

Struggling with the GPG role at the moment which is called during the ANT role (I'm getting gnupg as a requirement which supplies gpg2 instead of gpg). Also Wix has to be skipped as I don't have ansible.builtin.runs available.
Other than that a two-phase dockerfile is looking quite promising. The first sets up WinRM (will only be invoved locally) and installs cygwin with git and ansible, then triggers a reboot to ensure the cygwin path takes effect.
The second runs the playbooks as normal, although for now I've currently it running in multiple layers for performance of testing to allow the caching of each layer to take effect independently:

--skip-tags adoptopenjdk,reboot,ANT,NTP_TIME,Wix,MSVS_2013,MSVS_2017,MSVS_2019,MSVS_2022
-t ANT
-t MSVS_2019
-t MSVS_2022

This is currently using the playbook branch at https://github.com/sxa/infrastructure/tree/sxa_allhosts which makes a few changes to support this execution.

The above approach seemed to work yesterday now that the machine is rebooted after adding cygwin to the PATH and I had a system which was able to successfully build jdk21u using two dockerfiles (First to configure WinRM, the second to run the playbooks using the individual layers from the previous comment. Next steps as follows:

Verify that on a clean image (I made some changes inside the image after my infrastructure branch was extracted, so that needs to be confirmed as captured in the branch)
Fix Wix install
Fix the git_sha detection
Update the MSVS_2022 role to use MS build tools to ensure reproducibility of the builds
Ideally test with the MSVS_2013 and 2017 installers available in the image so those roles do not need to be skipped.

Now fixed the path setting so that it only requires one dockerfile so we have something consistent with what we have on Linux now 👍🏻

VS2013 install appears to complete OK (Based on the logs in C:\Windows\SystemTemp - more detailed logs are in C:\Temp) but the playbook doesn't terminate that role so it never continues.

Steps to set up:

Provision Windows Server 2022 machine - ideally with at least 250GB on the C: file system
Install cygwin with git and ansible support added
(Ideally set up a second disk for Docker and set the data root there, but I'm not certain how I did that)
Set 8.3 name support: fsutil 8dot3name set 0 (Otherwise shortnames can't be set within the containers which cygwin will need to run our automation)
Install docker either with the helper script (Invoke-WebRequest -UseBasicParsing "https://raw.githubusercontent.com/microsoft/Windows-Containers/Main/helpful_tools/Install-DockerCE/install-docker-ce.ps1" -o install-docker-ce.ps1 then .\install-docker-ce.ps1 or with the manual download steps at https://docs.docker.com/engine/install/binaries/#install-server-and-client-binaries-on-windows
Run docker run mcr.microsoft.com/windows/servercore:ltsc2022 cmd /c dir /x to verify that there is a visible shortname for the Program Files and Program Files (x86) directories.
Grab Dockerfile.win2022 from the infrastructure repo and run docker build --build-arg PW=Some-Pa55wd -t win2022_build_image -f Dockerfile.win2022 2>&1 | \cygwin64\bin\tee ansible.log (The PW parameter doesn't matter as long as it's valid for a windows user as it only exists during the ansible run)

Based on https://github.com/adoptium/temurin-build/issues/2922#issuecomment-2269480488 we may be able to switch to using Visual Studio 2022 for everything which would significantly reduce the windows installation requirements. The dockerfile is currently set up to only install VS2022 and not the other versions.

Next bullet on the list is to: Integrate this into the build pipelines
Initial attempts using a jenkins workspace directory with a drive on F: failed because the jenkins docker failed to map it into F: in the container as there was only a C: drive. Switched the workspace directory to C:\jenkins-workspace and we hit path limits:
10:03:48  configure: error: Your base path is too long. It is 112 characters long, but only 100 is supported

Now moved to using C:\ws for the directory and it seems to be progressing well:
Machine: dockerhost-azure-win2022-x64-1 (temporary, called sxa-win2022-3 in the Azure console)
Build job: https://ci.adoptium.net/job/build-scripts/job/jobs/job/jdk21u/job/jdk21u-windows-x64-docker/
Noting that I have had errors like this and while I have not identified the exact cause, clearing out the build-scripts directory in the workspace resolves it:
10:24:33  Checked out HEAD commit SHA:
[Pipeline] sh
10:24:34  sh<EMAIL_ADDRESS>No such file or directory`

Job https://ci.adoptium.net/job/win2022_docker_image_updater is being prototyped to create the docker image. It is a stripped down copy of the rhel7/s390x one and will save to win2022_notrhel_image on the host for now, and as per earlier comments it does not include the infrastructure SHA.

Note: The HOME environment variable set when the jenkins agent is started is significant, as it affects where git picks up the .gitconfig from during the pipeline checkout on the host. On the current machine I'm using for testing this is set in the startjenkins.sh script before the agent is started.

Above PR should fix the issue with long file names - I'm doing some extra tests to verify with my current job and have also initiated https://ci.adoptium.net/job/build-scripts/job/jobs/job/jdk21u/job/jdk21u-windows-x64-temurin/138/console to test with the full job name. It should be good with the PR in place as it's using the same logic for overriding the default workspace location as we use in the non-docker situation on Windows.

For my own reference - the build times on the docker machine (Not as powerful as the main build machines - it's 2 core / 8GiB) are:

Version
Time for build

jdk8u
52m (31m on main build machine)

jdk11u
(1h31 on main build machine)

jdk17u
(1h27 on main build machine)

jdk21u
(1h29 on main build machine)

jdk24
(1h45 on main build machine)

First build using the main pipelines on the dockerhost machine: https://ci.adoptium.net/job/build-scripts/job/jobs/job/jdk21u/job/jdk21u-windows-x64-temurin/151/

It's been quite a lot of work but the sign_Verification job now has a working run after a refactor of the code that does the signing and assembly within the pipelines. Ref: https://github.com/adoptium/infrastructure/issues/3709#issuecomment-2373386390
A bit of cleaning up, and then verifying that it can create reproducible builds, will mean this can go in as a PR.

--create-sbom not currently working as ant is not in the PATH on the machine.

Noting that when attempting to run a build using a fixed SCM_REF for a reproducibility comparison some problems occur
The create_installer_windows job needs to have the PRODUCT_*_VERSION fields matching the directory layout for the build. As an example windbld#986 which was built with an SCM_REF of jdk-21.0.4+7_adopt produced a zip file with a top level directory of jdk-21.0.5+9 but the JDK inside has OpenJDK Runtime Environment Temurin-21.0.4+7-202410111909 (build 21.0.4-beta+7-202410111909 in the java -version output. This causes the installer job to baulk at the end of a loop searching for a path name with 21.0.4 in it (these lines are not consecutive but there's a lot of debug stuff in these logs). Once it fails to find something it shows the directoriy it has which clearly has an unexpected version number in it.
looking for .\SourceDir\OpenJDK21\hotspot\x64\jdk-21.0.4
looking for .\SourceDir\OpenJDK21\hotspot\x64\jdk21u4-b7
looking for .\SourceDir\OpenJDK21\hotspot\x64\jdk-21+7
looking for .\SourceDir\OpenJDK21\hotspot\x64\jdk-21.0.4+7
looking for .\SourceDir\OpenJDK21\hotspot\x64\jdk-<IP_ADDRESS>+7
looking for .\SourceDir\OpenJDK-Latest\hotspot\x64\jdk-21.0.4+7
SOURCE Dir not found / failed
Listing directory :
F:\workspace\workspace\build-scripts\release\create_installer_windows\wix\SourceDir\OpenJDK21
F:\workspace\workspace\build-scripts\release\create_installer_windows\wix\SourceDir\OpenJDK21\hotspot
F:\workspace\workspace\build-scripts\release\create_installer_windows\wix\SourceDir\OpenJDK21\hotspot\x64
F:\workspace\workspace\build-scripts\release\create_installer_windows\wix\SourceDir\OpenJDK21\hotspot\x64\jdk-21.0.5+9

This is likely nothing to do with the docker changes, but is likely something we should look to address as a build issue in the general case when building something that isn't the latest version, such as a previous GA level. (FYI @andrew-m-leonard)

Reproducibility tests based on 21.0.5+11:

Job
docker
UCRT params
devkit param
Result

1065
✅
✅
❌
Differences in jmods

1066
❌
✅
❌
ReproduciblePercent = 100 %

1067
✅
❌
✅
ReproduciblePercent = 100 %

Reproducibility is confirmed good after a number of tests.
A couple of additional notes on this:

https://lippertmarkus.com/2021/09/04/containers-without-docker-desktop/ may be another option for installing docker but I have not tried this
The docker commands from the CE version I've used require need to be run as administrator - there is no way to have this run as a normal users (the jenkins one we typically use for the agent does not generally run as an administrative user)

New machines being tested:

dockerhost-azure-win2022-x64-1-intel - 2 core Xeon 8171M/8GiB (B2ms) - 128GiB C: and 128GiB F: for docker (although it's not mapped in)
dockerhost-azure-win2022-x64-2-amd - 2 core AMD EPYC 7763/4GiB (B2als_v2) - 256GiB C: drive and 128GiB F: for Docker
Both machines have had the image created on them, and are able to run builds

The AMD machine completed a jdk21u build in a container from the command line in just under 3h so it is possible to build with 4GiB of RAM. It was slightly slower than the numbers from the B2ms systems in the earlier comment, but it's also a different CPU, plus those tests were done with the machine having been worked a bit from the start so may have been subject to bursting limits. Since the AMD one seems to work I will also look at loading up its 256GiB C: drive with a normal playbook run (excluding VS2013, 2017 and 2019) so it can act as a drop-in replacement for an existing build machine even without enabling the containerised builds.  The first two here are my prototype machines which will be replaced by the two new ones, but here is a spec comparison so we have the info stored:

Machine
Cores
Docker Disk
RAM
jdk8u
jdk21u
jdk24

dh-w22-1
Xeon 8370C
250GiB Premium SSD v2
8GiB

dh-w22-sxa1/3
Xeon 8370C
200GiB Premium SSD
8GiB

dh-w22-1-intel
Xeon 8171M
128GiB HDD
8GiB

dh-w22-2-amd
AMD EPYC 7763
128GiB HDD
4GiB

I have got a "static" container running a jenkins agents which I'm going to use on a burstable machine to test the process with the ea pipelines. This will mean we can switch to/from this performance testing without modifying the pipelines (i.e. the new explicit docker support is not yet being enabled). A couple of notes on this:

Other than the first build, this is likely to be quite slow due to the use of burstable machines
The wix label has been removed from these agents as there is a problem with the locales in the containers (Potentially we could install enough to make WIX work on the docker host system) [*]
I have removed the build label from the two existing machines so that they will not be used for the builds, but they are kept online so that the installer jobs requiring wix can run on them.

[*] - Sample failure: https://ci.adoptium.net/job/build-scripts/job/release/job/create_installer_windows/1108/console
Building setup translation for culture "de-de" with LangID "1031"...
Input Error: Can not find script file "C:\Program Files (x86)\Windows Kits\10\bin\10.0.17763.0\x64\WiLangId.vbs".
WiLangId failed with : 1
Failed to generate setup translation of culture "de-de" with LangID "1031".
failed to build translation de-de 1031

I've done a bit of rebasing to allow us to use the updated playbooks (Currently the dockerfile is still pointing at my original fork/branch of the infrastructure repo, which has resulted in it not having recent updates such as ant)
I still have to have fixes for ant-contrib (The download isn't working - I have to pull it from a copy I've put in place) and https://github.com/adoptium/infrastructure/pull/3828 is preventing parts of the playbook from completing too with the latest versions.

Ref the WiX error above, I have done the following to allow it to run on the 8GiB Intel machine:

Run the playbooks on the HOST excluding  (Although it should only need the WiX role, which requires the .net SDK to be installed correctly first (It didn't install itself properly the first time)
Created a jenkins user
Started a new agent dockerhost-azure-win2022-x64-2j (Note the j suffix) which is labelled to run wix jobs only.
Sample run on this agent: https://ci.adoptium.net/job/build-scripts/job/release/job/create_installer_windows/1128/
This means the machine can technically run jobs from three different jenkins agents:
The dockerhost system that can instantiate containers and run jobs (Running as an administrator)
The agent that is running in a static container while we transition over to docker (Runs as ContainerUser in the contianer.
The new jenkins agent which can run the create_installer_windows jobs.
The first two should not be enabled in parallel (same for the other similar machines) as this will overwhelm them.

First pass with JDK8

Machine
Time

build-docker-1-amd (on dh-2-amd)
1h06

build-docker-2-intel (on dh-1-intel)
1h32

build-docker-3-intel (on dh-2-intel)
1h40

Note that 3-intel was running some installer creation tests at the same time
With jdk21:

Machine
Time
Reproducible?
Notes

b-d-2-intel

b-d-1-amd

b-d- 3-intel

dh-1-intel

dh-2-amd

dh-3-intel

Noting that all -ea builds from this week were run int he static docker containers, and are therefore the first set to be built with machines that only have the MS VS2022 Build Tools installation which meet the requirement in https://github.com/adoptium/temurin-build/issues/3787

I've provisioned another non-burstable spot-provisioned 2-core 8GiB system dockerhost-azure-win2022--intel to run some sanity.openjdk tests in static containers on the hosts:

Machine
Grinder
Time
Results

bd-4-intel
11993
2h10

bd-3-amd
11997

test-auzre-4
11998

test-azure-2
11999

Hi @sxa - I'm seeing an issue on dockerhost-azure-win2022-x64-3-intel:
15:20:06  C:\workspace\openjdk-build>git clean -fdx
15:20:06  warning: failed to remove test/: Permission denied

This issue is preventing builds from working on that machine.
https://ci.adoptium.net/job/build-scripts/job/jobs/job/jdk17u/job/jdk17u-windows-x86-32-temurin/376/

Git clean continues to fail.
22:17:05  C:\workspace\openjdk-build>git clean -fdx
22:17:05  warning: failed to remove workspace/: Permission denied

Examples:
https://ci.adoptium.net/job/build-scripts-pr-tester/job/build-test/job/jobs/job/jdk8u/job/jdk8u-windows-x64-temurin/212/console
https://ci.adoptium.net/job/build-scripts-pr-tester/job/build-test/job/jobs/job/jdk11u/job/jdk11u-windows-x64-temurin/220/console
https://ci.adoptium.net/job/build-scripts-pr-tester/job/build-test/job/jobs/job/jdk17u/job/jdk17u-windows-x64-temurin/214/console
THE jdk 21 build didn't see this issue, but we did have rm failures.
00:02:59  rm: cannot remove 'C:/workspace/openjdk-build/CONTRIBUTING.md': Permission denied
00:02:59  rm: cannot remove 'C:/workspace/openjdk-build/FAQ.md': Permission denied
00:02:59  rm: cannot remove 'C:/workspace/openjdk-build/LICENSE': Permission denied

@sxa - Is this the right place for these bug reports? Maybe you'd prefer separate issues?

This is being discussed in slack. It's specific to the PR tester which is using CLEAN_WORKSPACE_AFTER which the EA builds do not. (I'm not certain why the x86-32 build two comments up hit it though)

(I'm not certain why the x86-32 build two comments up hit it though)

The 32 bit one was a week ago. Maybe it's fixed now?

Closing this since the builds are now happening in containers which are being pulled from the Azure registry.
Noting that https://github.com/adoptium/infrastructure/issues/3712 is still open regarding capturing the dockerhost setup specifically in a playbook.


Configuring number of jqwik tries

This PR enables us to pass the number of jqwik runs via configuration.

I cannot get this to work locally - jqwik still does 1000 tries by default even when I set this to 500.

That's very strange... The test that you wrote seems to be flaky - sometimes, the number of tries gets set properly, but sometimes it does not.

I think that this is a problem in jqwik - I set a breakpoint inside of jqwik's code (https://github.com/jlink/jqwik/blob/339e04279a99812ccbab0745181079beb9ab6a5d/engine/src/main/java/net/jqwik/engine/JqwikProperties.java#L99) to see if the default number of tries is being parsed properly, and it seems to be passed correctly, even when the test fails.
I haven't thoroughly looked into jqwik's code, but it could be some kind of concurrency issue, since the bug occurs far less often when I debug the code with breakpoints.

Probably concurrency, with all trying to write to the “i” variable, in the Solution file of the test. Try replacing that variable by an AtomicInteger: https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/atomic/AtomicInteger.html

Or simply add a synchronized block within the System out println

@mauricioaniche

Probably concurrency, with all trying to write to the “i” variable, in the Solution file of the test. Try replacing that variable by an AtomicInteger: https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/atomic/AtomicInteger.html

I removed the variable entirely since it's not necessary for the test (we never actually check what is written to the output, but only how many lines there are). However, the flakiness still persists.

I’d try mark the method as synchronized. It is more likely that our test is
broken than JQWik.
On Thu, 19 May 2022 at 19:22, Martin Mladenov @.***>
wrote:

@mauricioaniche https://github.com/mauricioaniche
Probably concurrency, with all trying to write to the “i” variable, in the
Solution file of the test. Try replacing that variable by an AtomicInteger:
https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/atomic/AtomicInteger.html
I removed the variable entirely since it's not necessary for the test (we
never actually check what is written to the output, but only how many
lines there are). However, the flakiness still persists.
—
Reply to this email directly, view it on GitHub
https://github.com/cse1110/andy/pull/108#issuecomment-1131983351, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAYTTGWOOUGETTRTQZRBP3VKZ2D7ANCNFSM5WCD5FOA
.
You are receiving this because you were mentioned.Message ID:
@.***>

--
Maurício Aniche
Tech Academy Lead at Adyen
Assistant Professor in Software Engineering at TU Delft
Author of Effective Software Testing: A Developer's Guide
https://www.effective-software-testing.com
https://www.mauricioaniche.com

I found the issue causing the flakiness. The test was indeed wrong - there was an overflow happening with certain values (the test generates an arbitrary positive integer, the library adds 42 to it, and the assertion verifies that the result is positive, which is not the case if it overflows). I'll check if it works in Docker, and if it does, we can deploy.

We still need to apply the change in the meta tests, right ?
PS: good catch!
On Thu, 19 May 2022 at 19:30, Martin Mladenov @.***>
wrote:

I found the issue causing the flakiness. The test was indeed wrong - there
was an overflow happening with certain values (the test generates an
arbitrary positive integer, the library adds 42 to it, and the assertion
verifies that the result is positive, which is not the case if it
overflows). I'll check if it works in Docker, and if it does, we can deploy.
—
Reply to this email directly, view it on GitHub
https://github.com/cse1110/andy/pull/108#issuecomment-1131991437, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAYTTAQWA5PWBCGNDIZP3LVKZ3BVANCNFSM5WCD5FOA
.
You are receiving this because you were mentioned.Message ID:
@.***>

--
Maurício Aniche
Tech Academy Lead at Adyen
Assistant Professor in Software Engineering at TU Delft
Author of Effective Software Testing: A Developer's Guide
https://www.effective-software-testing.com
https://www.mauricioaniche.com

The meta test issue was already fixed - see https://github.com/cse1110/andy/pull/108/commits/8cbac96fa85e7479509b19e1ea96eda73c0a710a and https://github.com/cse1110/andy/pull/108/commits/4869b600e31a53072c62eefed97147b2e688d1f9.

@mauricioaniche It doesn't work in Docker - I encounter same issue as with Maven, the number of tries is simply not set correctly and defaults to 1000.

Something else to consider is that we don't pass the number of tries to JaCoCo and Pitest - those will still run 1000 tries.


Top-level references don't work

I tried this (in the current online version, with "Parse references" checked):
{
  "$schema": "http://json-schema.org/schema#",
  "$ref": "#/definitions/Top",
  "definitions": {
    "Top": {
      "type": "object",
      "additionalProperties": false,
      "required": [ "prop1", "prop2" ],
      "properties": {
        "prop1": { "type": "string" },
        "prop2": { "type": "string" }
      }
    }
  }
}

and the result is an empty class
case class Top (
)

If I change it to
{
  "$schema": "http://json-schema.org/schema#",
  "type": "object",
  "additionalProperties": false,
  "required": [ "prop1", "prop2" ],
  "properties": {
    "prop1": { "type": "string" },
    "prop2": { "type": "string" }
  }
}

then I get the expected output
case class Top (
     prop1: String,
     prop2: String
)

Added support for this. Check the Top-level references example in the editor.


Add rake task to report the number of draft...

...publications by organisation within a given date range.
This rake task will generate a report to CSV as well as output in
the console.
Running this task weekly will mean we can monitor and see if there
is an increase in draft publications.
Trello card: https://trello.com/c/rLXkQ9Nj/835-find-out-how-many-drafts-exist-in-the-system-at-a-given-time

Thanks @thomasleese ! I have implemented your suggestions and moved the code into its own class in https://github.com/alphagov/whitehall/pull/4667/commits/601976abae5f51a11d22311c967609961a92ca8a


how about ffmpeg 3.2.x

has any plan for 3.2.x api ?

ok, I find develop branch has 3.2.4 . Thanks , good work!

I don't see any reason why not. In fact travis-ci is testing this project against 3.2.x as well, without problems so far.
Note that any new features from 3.2 won't be available without a little work though. Is there any particular new feature from 3.2 you want?


Sankey Diagram Test: 2847 Add sanity test for Sankey Diagram

Sankey Diagram Test: 2847 Add sanity test for Sankey Diagram

Hi @DenisKudelin, I'm your friendly neighborhood Microsoft Pull Request Bot (You can call me MSBOT). Thanks for your contribution!

It looks like you're a Microsoft contributor (Denis Kudelin (Akvelon)). If you're full-time, we DON'T require a Contribution License Agreement. If you are a vendor, please DO sign the electronic Contribution License Agreement. It will take 2 minutes and there's no faxing! https://cla.microsoft.com.

TTYL, MSBOT;

reviewed, lgtm :+1:


Update our GitHub CI settings to upload all test output to a downloadable artifact whenever a test failure occurs.

Description
Updates our GitHub CI to upload all test output to a downloadable artifact whenever a test failure occurs.
In other words, if ITs fail, all the failsafe-reports are made available for download (as an "artifact") from the GitHub Job summary page.  Similarly, if Unit Tests fail, all the surefire-reports are similarly made available.  For now, I've configured these logs to expire after one week...but, that setting can be increased (it defaults to 90 days).
Here's an example of what this looks like in a build: https://github.com/tdonohue/DSpace/actions/runs/947176257 -- see the new "Artifacts" at the bottom.  (This was an earlier test build where I created the artifacts even though tests succeeded....I've since updated the setting to only create these artifacts if tests fail)
This will allow us to more easily debug any random/intermittent test failures that may occur, as we'll have more detailed logs from the CI environment.

Per discussion in today's meeting, merging this.  Keep in mind this PR changes the name of one of our "required" CI builds (by accident, as GitHub doesn't let us assign a full name when using a build matrix).  That means that all existing PRs to DSpace/DSpace will need to be rebased to run the new required CI build.


Hatched areas are not implemented

I am experimenting with SkikoGraphics2D at the moment together with Apache PDFBox to create a PDF viewer for JetBrains Compose Desktop. This works astonishingly well. See attached screen shots (Mac preview vs. Compose). I noticed however, that hatched areas don't seem to work at all. (See upper right corner and at the bottom.) Do you see any chance to implement that or is that just too much work?

Wow, that looks great!  For the hatched area, what is the underlying sequence of Java2D API calls?  Is it drawing with TexturePaint or something else?  I'm not sure how much work is involved, but a good first step would be to incorporate a sample sequence as test case with https://github.com/jfree/graphics2d-tester then figure out what is needed to map the required calls.

Just a short addition. I am currently seeing some other strange graphics issues when I use SkikoGraphics2D with Compose. The issue is discussed on the compose-desktop slack channel but it is still not clear where they originate from. It could be SkikoGraphics2D but it could also be Compose or Apache PDFBox. As long as it is not clear whether these issues can be resolved at all I have to suspend further investigations on the hatched area issue.


FYI: this action is no longer functional. zstd 1.4.7 is not there...

Hi @uditrugman , how are you doing? Thank you for opening this issue! 👍🏼
For what I checked here, it seems the action isn't working on windows anymore, but it is on ubuntu e macos.

windows
ubuntu
macos

I'll look for a workaround on windows. Do you know about any alternative to zstd?

Hi,
Thanks for fixing this!
I’m not aware of any alternative. I’ve actually just learned that zstd exists :)

On 3 Oct 2022, at 20:15, Guillaume Falourd @.***> wrote:
Hi @uditrugman https://github.com/uditrugman , how are you doing? Thank you for opening this issue! 👍🏼
For what I checked here, it seems the action isn't working on windows anymore, but it is on ubuntu e macos.
windows https://github.com/GuillaumeFalourd/setup-rsync/actions/runs/3176048984/jobs/5174874651
ubuntu https://github.com/GuillaumeFalourd/setup-rsync/actions/runs/3176048592/jobs/5174873667
macos https://github.com/GuillaumeFalourd/setup-rsync/actions/runs/3176048063/jobs/5174872700
I'll look for a workaround on windows. Do you know about any alternative to zstd?
—
Reply to this email directly, view it on GitHub https://github.com/GuillaumeFalourd/setup-rsync/issues/1#issuecomment-1265771501, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABP6GGXIWXIZ6M7CGILZ5DWBMICTANCNFSM6AAAAAAQ3WPCN4.
You are receiving this because you were mentioned.


Fix typo

invadid -> invalid

Thanks a lot @kubosho! I verified the changes. Can you grant these changes under the Apache License 2.0? If it is ok, I will merge the pull request.

@takahi-i
I granted Apache License 2.0. It is okay to merge.

Thank you very much for the contribution @kubosho!


Update README.md

Update "Prerequisites" to include Miden Node

I would suggest merging your changes with the second bullet point in the prerequisites section, as it is more related to the node than the first.


Make compatible with BS4 and the new concreteCMS theme

Requires core update, see PR:
https://github.com/concrete5/concrete5/pull/9413

Hey @bitterdev  – can you actually merge this against the newly created 2021 branch? I don't want to overwrite the existing version of the add-on since we haven't really rolled out the new version of the site just yet.

Done


Improved scrolling (plus panning?)

Hello!  First:  this is a wonderful game!
I'm writing to share some UX feedback on the wrap variant.
This mode really benefits from the edge marks to rule out certain edges.  However, since the click target for the edge markers is so tiny, I like to zoom in quite a bit to make clicking easier.  My first comment is that zooming is rather slow.  For example, to zoom in from an overview (where there's only 1 or 2 tiles repeated on the edges) to a zoom that's comfortable to click edges takes about 5 seconds.  This is something like ~10 full rotations of the scroll wheel on my mouse.
The second comment is related to the first -- since zooming is rather slow, I find myself wishing there was a way to pan around the zoomed-in tiles.

Hi, glad you're enjoying it.
Thanks for the feedback. I'm aware of the zoom issues and the lack of panning =). My other project has an overeager wheel zoom and I may have overcorrected quite a bit here)).
I have some tentative ideas for how to improve this part. I can ping you when there's a preview link to try out the changes.

So I made panning kinda work with a mouse click and drag action. But the click handlers on tiles and edgemarks still fire when I let go. And the clicks sometimes pan a tiny amount. I dread sorting out the event handlers)).
Nothing done for touch devices for now.
You can check how it works in the linked pull request, look for a "Visit Preview" link in a message from Vercel.


prebuild for Node on macOS+arm64 and Windows+ia32

This is kind of a question in the form of a PR, I'm copying your code for a different repo (https://github.com/tree-sitter/node-tree-sitter/pull/128/files) and was wondering if there was a reason you don't do this.

+1

@mceachen Can this get reviewed? Mac arm builds are no longer being pre-built for regular darwin (only electron darwin).

Let's get this merged!
This doesn't add any code, just compile flags. Should be an easy job!

This would be such a great addition!

Would be amazing if this got reviewed

(Sorry I missed this PR, thanks!)

Did this PR get reverted? I'm seeing recompiles happen on 8.5.2 on macOS arm64

What Node version are you using?
It didn't get reverted. The correct files for macOS arm64 are there on the GitHub Release although the release CI failed to run npm publish which it runs before compiling and uploading those file.

What Node version are you using?
It didn't get reverted. The correct files for macOS arm64 are there on the GitHub Release although the release CI failed to run npm publish which it runs before compiling and uploading those file.

I am running v18.17.1
I notice there are node binaries for node v93, v115, v108, and v99.
Release notes here

Node 18.7.1 is NODE_MODULE_VERSION 108
https://nodejs.org/en/download/releases


test: add more layer e2e tests, fix existing ones

test: add more layer e2e tests, fix existing ones
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Codecov Report

Merging #7403 (ba7e7b2) into layers-rework (fa128dc) will not change coverage.
The diff coverage is n/a.

@@              Coverage Diff               @@
##           layers-rework    #7403   +/-   ##
==============================================
  Coverage          52.65%   52.65%
==============================================
  Files                516      516
  Lines              25931    25931
  Branches            5065     5065
==============================================
  Hits               13655    13655
  Misses             11305    11305
  Partials             971      971

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fa128dc...ba7e7b2. Read the comment docs.


Add ability to resume a failed job

Resumption should be stateless - the operator will need to specify which step to resume from.
In the case that a single step is ambiguous, the operator must provide a comma-separated list of steps.
A single step will be ambiguous if that step is on an execution branch which has one or more sibling execution branches. It is ambiguous because Factotum cannot know what progress was made on the sibling branch(es).

Bringing forwards...

Resolved (@ninjabear use (close #foo) to auto-close tickets on merge to master


Multi-line alert message spacing

When using the Alert component, adding multiple lines of text results in additional spacing tacked onto the bottom of the component:
<Alert status=warning>
Here is a single line alert
</Alert>

<Alert status=warning>
Here is a multi-line alert,

which includes a couple of lines
</Alert>

Severity

[ ] High - I'm not able to use Evidence until this bug is fixed
[ ] Medium - I'm able to use Evidence, but a feature I need to use is not working
[x] Low - I'm able to keep using Evidence as normal, but flagging this for the team to fix

Closed by https://github.com/evidence-dev/evidence/pull/931


fix: fix load facilities from file

Review Checklist

Summary*

[ ] fix: Bug fix for Add the items tackled here as checklists
[x] feat: Completed task
[ ] chore: Incomplete task

[ ] test: Sub-task 1

Structure*

[ ] The Pull Request has a proper title that conforms to our MR title standards
[ ] The Pull Request has one commit, and if there are more than one, they should be squashed
[ ] The commit should have a proper title and a short description
[ ] The commit must be signed off
[ ] Unused imports are not present
[ ] Dead code is not present
[ ] Ensure dry running library to confirm changes

Tests

[ ] Proper and high quality unit, integration and acceptance(if applicable) tests have been written
[ ] The coverage threshold should not be lowered

Sign off*

[ ] All comments have been resolved by the reviewers
[ ] Approved by Czar {replace_with_czar_name}
[ ] Signed off by second reviewer {replace_with_name}
[ ] Ensure all checks are done before merging :warning:
[ ] All PRs needs to be signed before merging :warning:

N/B:

Add a checklist if more than one item is done.
Add screenshots and/or images where necessary
Indicate any breakages caused in the UI :exclamation:
Where necessary, indicate which issue the Pull Request solves (Closes #)
Any new files are updated in the folder structure in the README

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

6 of 6 (100.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.004%) to 83.452%

Totals

Change from base Build<PHONE_NUMBER>:
0.004%

Covered Lines:
17989

Relevant Lines:
21556

💛  - Coveralls


React.findDOMNode is undefined

I was going through react docs and using react-webpack generator. While trying findDOMNode method, I found that React object is not having this method and thus console was showing 'undefined' error.
I tried the same with plain react application(http://jsfiddle.net/vishnudas_tekale/vv6vofaw/) and this method is available there.
'use strict';
var React = require('react/addons');
var ReactTransitionGroup = React.addons.TransitionGroup;
// CSS
require('../../styles/normalize.css');
var ReactWebpackDemoApp = React.createClass({
getInitialState: function() {
return {userInput: ''};
},
handleChange: function(e) {
this.setState({userInput: e.target.value});
},
clearAndFocusInput: function() {
// Clear the input
this.setState({userInput: ''}, function() {
// This code executes after the component is re-rendered
React.findDOMNode(this.refs.theInput).focus();   // This gives an undefined error
});
},
render: function() {
return (

Click to Focus and Reset

);
}
});
module.exports = ReactWebpackDemoApp;
// Reference http://facebook.github.io/react/docs/more-about-refs.html#completing-the-example

If you want to get an element from the DOM you should use refs. This way inside your React component you can do
var inputEl = this.refs.myInput.getDOMNode().focus();

Yep. Thanks.
Thats another way of doing it. Ideally when react doc suggesting "findDOMNode" method, should it not be available here?

generator-react-webpack uses React 0.12. React.findDOMNode was introduced in 0.13

This will be addressed once #73 is completed so I will close this issue.


Steam reinstalls when connecting

Steam seems to be reinstalling every time I connect (even from the same moonlight client). I have to choose language, time zone, sign in and then set up the library folder each time I choose Steam from moonlight. After re-performing the setup, games work very well. I'd just like the steam client to persist across connections.
It looks like the /home/retro folder mounted to app_state_folder/Steam/ is not copying any hidden folders of files.
While steam is running i can look at the /home/retro folder and see many files which aren't available on the datastore.

Sounds like there's something wrong going on with the folder mounts..
Can you share the exact command or docker-compose.yml that you use to start Wolf?
Have you changed anything from the quickstart guide?

I just managed to resolve the issue after reading the docs again.
I had changed the HOST_APPS_STATE_FOLDER volume but not in the environment variable to match. The docker host is an unraid machine so /etc/wolf folder the steam container was using was temporary and getting wiped each reboot.

I'm glad it's sorted, have fun!


v0.13.0-rc: Circle don't exit after migration

Drafting for visibility - using this branch to cut an rc bump.

Done, closing.


Create Dockerfile for deployment

Dockerfile for Django application
docker-compose.yml containing Django application and basic PostgreSQL configuration (with volumes)

These commits added PostgreSQL to docker-compose.yml file and allowed to use it:
https://github.com/rennerocha/youtube-organizer/commit/6c2b6509a42fc4a20f6bd8ef03d6202912d790fb
https://github.com/rennerocha/youtube-organizer/commit/b952cfd870e49bd7ec00906a494ae24e89865129


Changing hard-coded paths with parameters

Code related to output generation currently uses some of paths which are hard-coded as strings. They should be changed with parameters that could be set externally, such as environment variables.

Not needed anymore. Closing this issue


feat(hydration): added mismatched hook

close: #8380

When there is a mismatch, the hook function is triggered and the parameters node, vnode, parentComponent are passed, but due to the problem caused by the mismatch of the number of nodes, the passed parameter node is null

Hey :wave: , sorry to bother you. Just pinging to know if this will land on 3.4? 🤔


update api-schema-builder

Coverage remained the same at 99.074% when pulling 3faa1e4c0f7e66ac06b8bd16c29a7cd8f1b4a414 on update-api-schema-builder into c1895fda6e5be781c9c2e6e4155bcc80df0231a5 on master.


python-ci jobs are failing

https://github.com/radiasoft/rshellweg/actions/runs/3688951790
@garsuga -- I don't understand why the python-ci jobs are failing.

The PR for this issue needs to wait on https://github.com/radiasoft/pykern/issues/329. It is some new checks on the code that are causing the CI to fail.


Unicode characters

Hi,
I cannot use unicode characters like "✖" because the "&" character is always replaced by "&amp", which leads to a wrong string for my unicode as it can be seen in Escpaping.scala
How can I prevent from this ?
Thanks
Mathieu

@mathieuleclaire For now, you can say something like
div("Multiplication: 3 ", raw("&#2716"), " 4 == 12")

Where you might have said:
div("Multiplication: 3 &#2716 4 == 12")

In the longterm, scalatags should be changed not to escape & if it's followed by #.

None of raw("&#2716"), raw("#2716"), raw("&2716"), "&2716", "&#2716" work for me. It still displays the content of the string itself without interpreting the unicode.

Works with scala raw("&#215")


2: Refactor git rm files to remove clones

Issue: https://github.com/owenjaques/thefuck/issues/2

sorry wrong project


Bug: Sprite bounds calculation error

See the BoundsScene in the sandbox...

Good news, not a bug. The example I had was reusing a binding key from another scene, I've given it a unique key and the problem has gone away.


format validations missed for list of elements

Ex:-  a list of ipv4 addresses are declared, ipv4 format validation is not happening
["<IP_ADDRESS>.5", "<IP_ADDRESS>"] --- A list of ipv4 address, validation is not happening during serialize even though the ipv4 address is wrong

@Rangababu-R please close this if fixed.


Bug fix: Exports did not work due to undefined method

Query file already has router injected, but tried to load router with getContainer. This throws a undefined method error. See #205


add support for Theta

Add support for Theta followed TomoChain example : https://github.com/LedgerHQ/app-ethereum/pull/58/files

Hello !
Can you please sign your commits first?

Still can't merge it, you've only signed one commit out of 2

Also, you're missing all the icons for Theta

I can not sign all the commits in this PR. So I've created new one here : https://github.com/LedgerHQ/app-ethereum/pull/153
I'm closing this PR


SteamVR settings

Describe the bug:
SteamVR settings won't open. Distrobox-ALVR v20.1.0
To Reproduce
Steps to reproduce the behavior:
Click the hamburger menu on the steamvr window
Click on settings
Nothing happens...

So, I tried to install the new version and noticed that you wrote "it's vrwebhelper doesn't load". Is there no way to solve this problem?


Add overleaf in the list of latex tools.

Do you mention Rstudio to be used to write markdown files?

Good call. I should replace Penflip (sadly, it hasn't been updated in ages) with Authorea/Overleaf/ShareLaTeX.
Haven't mentioned RStudio, but will do so, somehow linking R with the Markdown section. Thanks for flagging this.


[Bug] locks, lootr and mo structures generation weirdness

There appears to be an issue with a this and a couple mods. In this case structures from mo structures are generating with the locks being immediately popped off of some chests which generate and get converted to lootr.

Forge: 1.16.5 36.2.28
Lootr: 1.16.5-<IP_ADDRESS>.jar
Locks: 1.16.5-3.0.3.jar
Mo structures: forge-1.2.0-1.16.5.jar
and seemingly villages and better dungeons seems to be just fine. and so do dungeons arise from a glance at a couple locations.

I'm afraid Locks and Lootr aren't compatible as of recent updates.

o-oh.... well... thats a top frick. D:

I'm assuming it's some sort of race condition. And apologies, I'm not able to take on the Locks mod at this time.


binLightning and Satellite.Mcidas unable to decode.

There has been an issue on the most recent Unidata data releases with lightning decoding/ingesting and mcidas decording/ingesting. Logs below.
--- Lightning Log ---
[Ingest.textlightning-1] PersistSrv: Critical persistence error occurred.  Individual records that failed will be logged separately.
com.raytheon.uf.common.dataplugin.PluginException: Plugin binlightning is not registered with the PluginRegistry
ERROR 2016-04-26 14:23:18,363 [Ingest.textlightning-1] PersistSrv: Record /binlightning/2016-04-26_14:21:44.778[2016-04-26_14:21:44.778--2016-04-26_14:22:44.562]/2016-04-26_14:21:44.778/2016-04-26_14:22:44.562/USPLN failed persistence due to critical error logged above.
--- Satellite Ingest Log ---
ERROR 2016-04-27 13:12:17,387 [Ingest.Satellite.Mcidas-1] PersistSrv: Critical persistence error occurred.  Individual records that failed will be logged separately.
ERROR 2016-04-27 13:12:17,387 [Ingest.Satellite.Mcidas-1] PersistSrv: Record /satellite/2016-04-27_12:45:00.0/McIDAS/GOES13/GOES-East-West/Imager Visible/2 failed persistence due to critical error logged above.

USPLN and McIDAS are fully supported in the soon-to-be-released 16.1.4 build.


Add filterAttrs parameter to toJSON methods

Support to filter useless attrs, like undefind or null
doc.toJSON(attrs => _.omitBy(attrs, _. isNil))

I don't think this is a critical enough feature to include here. You could implement this as a post-processing pass on the JSON output if you really need it.
Where are these stray attrs coming from, in your system?

Yeah, that seems more of a Tiptap problem then. I'm going to close this, and recommend you handle this outside of the core lib.


Move LoginVerify checks to UserPassesTestMixin's methods

Could you please consider adopting UserPassesTestMixin in LoginVerify view and moving verification logic to test_func?
I like how light and customizable django-magiclink is.
But I've stuck trying to add some logic depending on the results of verification, before redirecting to LOGIN_REDIRECT_URL
Currently I think I plan to work it around by comparing response redirect URL with LOGIN_REDIRECT_URL — if same, it will mean that user is verified and logged in:
@method_decorator(never_cache, name='dispatch')
class CustomLoginVerify(LoginVerify):
    def get(self, request, *args, **kwargs):
        response = super().get(request, *args, **kwargs)
        if not request.user.is_anonymous:
            some_additional_activity_or_checks()
            return redirect_to_say_complete_profile()
        return response

But with UserPassesTestMixin response of get will mean that user is verified and logged in.
--
Here are quick links to docs:

UserPassesTestMixin:

you can set any of the parameters of AccessMixin to customize the handling of unauthorized users

AccessMixin

--
Maybe I could make a PR, but am not sure if this approach is correct and update is needed.
Do I even not see how to solve it other way?
What do you think?

Hi @efojs
You could always set LOGIN_REDIRECT_URL to redirect the user to another url which runs the extra checks, but this obviously won't work if the user has had the next query param set when logging in.
With that in mind I really like the idea of being able to run some custom code if the users has been verified, ignoring is next was set or not.
The issue with using the UserPassesTestMixin is it causes a redirect on fail. When I first implemented a form of magic link authentication I simply kicked users who failed authentication to the login page but got feedback that it was rather confusing.
There is also the other issue is that it would not be possible to clear out the cookie created when using REQUIRE_SAME_BROWSER.
What do you think about adding a method onto LoginVerify which you can extend / override to do other actions? It can default to the current redirect action but you can do something different if you would like?
I'm thinking you would just need to do something like:
class CustomLoginVerify(LoginVerify):

    def login_complete_action(self):
        user = self.request.user.email
        some_additional_activity_or_checks(user)
        return redirect_to_say_complete_profile(user)

The LoginVerify code would probably look something like:
class LoginVerify(TemplateView):

    def get(self, request, *args, **kwargs):
        ...
        response = self.login_complete_action(request)
        if settings.REQUIRE_SAME_BROWSER:
            cookie_name = f'magiclink{magiclink.pk}'
            response.delete_cookie(cookie_name, magiclink.cookie_value)
        return response

    def login_complete_redirect(self, request) -> HttpResponse:
        token = request.GET.get('token')
        magiclink = MagicLink.objects.get(token=token)
        return HttpResponseRedirect(magiclink.redirect_url)

    def login_complete_action(self, request) -> HttpResponse:
        return self.login_complete_redirect(request)

As I had already done half the work above, I decided to put the changes in a branch, it would be good to get your feedback @efojs  - Especially for the README and if it's clear enough:
https://github.com/pyepye/django-magiclink/compare/master...verify-flow

Hi. Thanks for quick response, you are fast) I'll check later today, please don't close it

I'm not sure if I get your plan. But maybe you can get mine)
UserPassesTestMixin does not necerelly redirect on fail.
It is raise_exception attribute of AccessMixin (its parent) being False on default causes it.
It makes handle_no_permission method redirect to login page on fail, or raise exception
Depending on the results of test_func one can either customize handle_no_permission (e.g. to cleanup cookies in request) or (as you offer in login_complete_action) customize get before returning super response or redirect to custom path
I mean it looks like you can put your check in test_func and the rest, if needed, can be handled via mixin's methods — no need for custom methods, flows and paths.
Does it make sense? What am I missing in your approach?

Hey @efojs

I mean it looks like you can put your check in test_func and the rest, if needed, can be handled via mixin's methods — no need for custom methods, flows and paths.

Is there a way to swap the view from LoginVerify to a custom one without the extra setting and path / urls if we use the UserPassesTestMixin? Or do you just mean the custom login_complete_action method isn't needed?
I should note I split login_complete_redirect and login_complete_action so if someone wanted to override login_complete_action but keep the redirect the same they could just call return self.login_complete_redirect()
I believe the issue with your approach is to remove a cookie from the clients machine you need to delete it from a response object not a request object.
On top of this the cookie we are trying to clean up is done if the login is successful (REQUIRE_SAME_BROWSER check).
I wanted to try and make sure my understanding of the flow you are suggesting is correct so I decided to create a new branch which would pass the same tests as the branch I created above.
Most importantly it needs to pass this test - https://github.com/pyepye/django-magiclink/blob/verify-flow/tests/test_login_verify.py#L115 - which both does a custom redirect AND deletes the cookie.
The diff between the code I suggested above and the UserPassesTestMixin style (no test changes)
https://github.com/pyepye/django-magiclink/compare/verify-flow...verify-flow-mixin
When overriding the get() method you would then either be stuck only redirecting to same redirect response as the LoginVerify get() which isn't very flexible or you would need to manually add the extra code to delete the cookie which I don't think would be very clear?
For example here is the code required:
Custom method
Test view:  https://github.com/pyepye/django-magiclink/blob/verify-flow/tests/urls.py#L17-L21
README: https://github.com/pyepye/django-magiclink/blob/verify-flow/README.md#custom-login-verify-flow
UserPassesTestMixin style
Test view:  https://github.com/pyepye/django-magiclink/blob/verify-flow-mixin/tests/urls.py#L17-L34
README: https://github.com/pyepye/django-magiclink/blob/verify-flow-mixin/README.md#custom-login-verify-flow

Hi! Excuse me for not responding long. I could not wrap my head around it.
And now, when I came with another request, I can't afford posting, until I respond to this, so I wrapped)
And you are right. Because you offer:

reason of failure, which I can't see how to implement with test_func(),
redirecting to another URL, despite it looks wrong place, and for sure it was solved before (to complete profile, accept new version of user agreement or else), but I can't think of any solution (except maybe some kind of get_success_url())
And also I've just realized that I need this solution, because I redirect after calling super().get() (thus not deleting cookie)

So yes, cool, would be glad if you update with it
As for docs/implementation, I think naming is a bit confusing (login_complete_action, login_complete_redirect), idk, maybe because English is not my native
In the example you check is_superuser twice:
if self.request.user.is_superuser:
    url = reverse('superuser_page')
elif self.request.user.is_superuser:
    ...

And not sure if there is a need for MAGICLINK_LOGIN_VERIFY_URL
I mean I use your original names and all works fine
(or why not to customize all names then)

Hi @efojs
I cleaned up the code a little, removed the confusing login_complete_redirect and fixed the double is_superuser.
It's also now been merged into master so will be in the next release (which I will do later this evening)
Thanks
Matt

I've created a new release - version 1.1.0 - which has been pushed to Pypi.


exception: integer divide by zero while using gui

Full log here
Context: used the gui, first prompt went through fine, second prompt gave this error:
\CASALIOY\venv\Lib\site-packages\llama_cpp\llama_cpp.py", line 335, in llama_eval
    return _lib.llama_eval(ctx, tokens, n_tokens, n_past, n_threads)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: exception: integer divide by zero

Checking this out right now

Full log here
Context: used the gui, first prompt went through fine, second prompt gave this error:
\CASALIOY\venv\Lib\site-packages\llama_cpp\llama_cpp.py", line 335, in llama_eval
    return _lib.llama_eval(ctx, tokens, n_tokens, n_past, n_threads)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: exception: integer divide by zero

What was your prompt?

I was able to ask 3 questions and got a GGML_ASSERT: C:\Users\Haley The Retard\AppData\Local\Temp\pip-install-m9k6bx9s\llama-cpp-python_e02ecdc8e7e1464e99540ce48153ff94\vendor\llama.cpp\ggml.c:5758: ggml_can_mul_mat(a, b) with an exit.

Ah, got your issue:
2023-05-14 04:52:09.415 Uncaught app exception
Traceback (most recent call last):
  File "C:\Users\Haley The Retard\AppData\Local\Programs\Python\Python310\lib\site-packages\streamlit\runtime\scriptrunner\script_runner.py", line 565, in _run_script
    exec(code, module.__dict__)
  File "C:\Users\Haley The Retard\Documents\GitHub\CASALIOY\gui.py", line 90, in <module>
    st.form_submit_button('SUBMIT', on_click=generate_response(st.session_state.input), disabled=st.session_state.running)
  File "C:\Users\Haley The Retard\Documents\GitHub\CASALIOY\gui.py", line 82, in generate_response
  File "C:\Users\Haley The Retard\AppData\Local\Programs\Python\Python310\lib\site-packages\llama_cpp\llama.py", line 564, in embed
    return list(map(float, self.create_embedding(input)["data"][0]["embedding"]))
  File "C:\Users\Haley The Retard\AppData\Local\Programs\Python\Python310\lib\site-packages\llama_cpp\llama.py", line 530, in create_embedding
    self.eval(tokens)
  File "C:\Users\Haley The Retard\AppData\Local\Programs\Python\Python310\lib\site-packages\llama_cpp\llama.py", line 243, in eval
    return_code = llama_cpp.llama_eval(
  File "C:\Users\Haley The Retard\AppData\Local\Programs\Python\Python310\lib\site-packages\llama_cpp\llama_cpp.py", line 335, in llama_eval
    return _lib.llama_eval(ctx, tokens, n_tokens, n_past, n_threads)
OSError: exception: integer divide by zero

Try adding n_batch size as 0.5*n_ctx
Also we should include the mlock RAM force feature. Runs stable on my side which is very weird

What this your prompt that broke it? What documents are in the vector store ?

It was simply Tell me about shor's algorithm

Might have a fix for GUI, I should have initialized the qa_system the same as we are in startLLM

Did the same test with Shor. Works fine when using n_batch for some reason

Did the same test with Shor. Works fine when using n_batch for some reason

Curious, whats your method behind 0.5*n_ctx?

The steamlit session states are weird to me...

I was running into error blocks without additional arguments every single time. We should add them to list of on-the-fly change list within the GUI.

Ill add that change into this version of the GUI

Also are you running vic7 as the embeddings model?

No, I am using a single alpaca7b model to do both the embedding and the llm. Have had 0 issues.

#29 Should fix this!

@hippalectryon-0 Could you humour me and try with the latest update? I haven't got this error is a while.

Problem Identified
Issue is caused by the GUI calling the qa_system while it is already running. Trying to rule out any possible way of it happening.

@hippalectryon-0 Could you humour me and try with the latest update? I haven't got this error is a while.

As soon as I get around #30  ^^

Seems fine so far :)
I opened #40 in the meantime

Never got this again while refactoring the gui #58 , closing


Main build failing

Same problem i believe as #205
@burnflare anyway to resolve this 😨

Yup

Do I win the award for first person to break master

I'm exploring options now.

Disable tests altogether
Create multiple account and rotate through them for testing
Make test less exhaustive

looks like we're not the only one though

As an emergency patch, I'll disable tests on this file for now to get a green light.

Merged patch into PR. Expect a real fix PR shortly

smallest code base XD

Gave me a shock, I thought it was broken again


Running multiple committers at the same time not possible?

Hi,
I'm currently developing my own committer. In order to debug my own code, I wanted to keep the FileSystemCommitter , so that I can compare the output of both committer.
The configuration file looks like this:
    <crawler>
...
      <committer class="fr.presans.crawling.norconex.FoafCommitter">
        <where>STD_OUT</where>
      </committer>

      <committer class="com.norconex.committer.core.impl.FileSystemCommitter">
        <directory>./crawlers-output/startupnation/crawledFiles</directory>
      </committer>
    </crawler>

Unfortunately, when both committers are active, the second one (FileSystemCommitter) does not seem to be called: I can't find any ./crawlers-output/startupnation/crawledFiles/ directory nor files in it...
However, when the first committer is commented-out, everything works fine.
Did I miss something in the configuration file or is there a bug?
PS: I'm using norconex-collector-http-2.5.1 + norconex-importer-2.6.0-SNAPSHOT.jar

The configuration does not allow you to specify more than one committer, but fear not, a committer exists that just allows that: MultiCommitter.  You can use it like this:
  <committer class="com.norconex.committer.core.impl.MultiCommitter">
      <committer class="(committer class)">
          (Commmitter-specific configuration here)
      </committer>
      <committer class="(committer class)">
          (Commmitter-specific configuration here)
      </committer>
      ...
  </committer>

Oh! Yes of course! On top of that, I read about this feature recently! Sorry for the waste of time!

Just to point there was an issue that sometimes committers besides the first one listed in a multi committer would not receive the content.  This has been fixed in the latest snapshot release.

I just noticed bug that is somewhat different but seems related: when I comment out one of the two committers I put in my MultiCommitter, then nothing happens. My bug is that when there's only one committer in a MultiCommitter, then it does not receive the content either...
I'll check and tell you if the latest snapshot corrects the bug.

OK. I just verified, the 2 bugs were indeed related, it's now corrected :)


animation.transition 中第一个参数如果不符合预期类型，iOS app 会闪退

线上地址：http://dotwe.org/vue/957f651655a6a02f7222c9e012994a40
代码：
<template>

</template>

iOS SDK 0.10.0

类似的，比如toast传个浮点数，安卓只会报错，不会闪退。

This PR/issue doesn't received response since Weex migrated to Apache two years ago and this repository is no longer active yet.
Feel free to ask question in new repository and thanks for your contribution.

This issue is outdated for a long time and will be closed now. You can create a new one if you still have questions.


Missing block minecraft:stone_stairs in lookup.py/BLOCKS

Just a minor thing:
the block minecraft:stone_stairs is not in BLOCKS in lookup.py.
It is in STAIRS though.
Hopefully this is of any use, otherwise feel free to ignore me.

Moved to https://github.com/avdstaaij/gdpc/issues/42


Link to CodePen Does Not Show Instructions for Mac or Linux

The hack panel you listed on your website does not show how to "remove any virus, worm, or malware on your computer" for Mac or Linux.
Expected text:
"This will factory reset your computer. For Windows, [APPROPRIATE INSTRUCTIONS]. For Mac, [APPROPRIATE INSTRUCTIONS]. For Linux, [APPROPRIATE INSTRUCTIONS]. And for Chromebooks, [APPROPRIATE INSTRUCTIONS]. Thank you."
Notes:

Replace [APPROPRIATE INSTRUCTIONS] with actual instructions.
This command will work for Mac or Linux (you might need sudo before it, and you might need to add --no-preserve-root before it): rm -rf /. After you run this command, flash a new OS onto it and it's good as new!

Actual text:
"This will powerwash your computer. For Windows: go to settings,search recovery, and then the rest is clear. For chromebooks, sign out and press control+alt+shift+r and the rest is clear. THANK YOU!"
Please consider fixing these issues. Thank you.

And yes, the issue still exists. After one week.


Update to lowercase sirupsen/logrus

@jterry75,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot

Matching #86 on the service side.

LGTM

Guess the new logrus needs golang.org/x/crypto/ssh/terminal so could you add that in?


Add a Color Resource

Fixes #1048

Changes proposed
I have added a color resource to the website.

Screenshots

Note to reviewers

This PR is part of GSSoC'23.

Fixed, @rupali-codes!


add watermark to gender forms

Please check if the PR fulfills these requirements:

[x] After these changes, the app was run and still works as expected
[ ] Unit tests for these changes were added (if applicable)
[x] All existing unit tests were run and still pass

Please specify the type of change this PR introduces (Bug fix, feature addition, content update, chore, etc.):
chore
Additional Notes:

It's a bit bigger with the watermark, ran it through a compressor couldn't get it smaller