Envoy Proxy disconnects when http header contains "chunked, chunked" or equivalent

Envoy sidecar proxy is dropping connection with application container and causing this error with 503 status and UC response flag.  upstream connect error or disconnect/reset before headers. reset reason: connection termination,
[X ] Networking
[X] Performance and Scalability
Expected behavior
Get 200 response for all requests.
Steps to reproduce the bug
Not sure,  it happens when request gets large data set from backend.
Version (include the output of istioctl version --remote and kubectl version and helm version if you used Helm)
istioctl version --remote
client version: 1.6.2
control plane version: 1.6.2
data plane version: 1.6.2 (4 proxies)
How was Istio installed?
Istio Operator (default profile)
Environment where bug was observed (cloud vendor, OS, etc)
EKS, 1.15
We are seeing an issue with 1.5 istio version, where sidecar proxy is dropping connection with application container and causing this error with 503 status and UC response flag.  upstream connect error or disconnect/reset before headers. reset reason: connection termination,   Debug logs shows following error messages.  It's only happen when we try to get large data set from backend which is outside k8s cluster.  Response from backend is always 200, but the connection btw sidecar and application container drops with multiple retries.  We tried 1.6.2 version today, seeing same issue.  This is not happening in 1.2.2 version.   We also tried making the same request from with in pod from side proxy using localhost and it works fine,  issue is only when routed through envoy proxy.  Appreciate any pointers or help to resolve this issue.
[/http/codec_client.cc:127] [C61] protocol error: http/1.1 protocol error: unsupported transfer encoding
[/network/connection_impl.cc:101] [C61] closing data_to_write=0 type=1
[/network/connection_impl.cc:192] [C61] closing socket: 1
[/http/codec_client.cc:91] [C61] disconnect. resetting 1 pending requests
[/http/codec_client.cc:114] [C61] request reset
[/router/router.cc:990] [C59][S8598556017247194609] upstream reset: reset reason connection termination
More debug logs in envoy proxy:
2020-06-17T16:54:53.025416Z	debug	envoy client	[external/envoy/source/common/http/codec_client.cc:104] [C297] response complete
2020-06-17T16:54:53.026436Z	debug	envoy pool	[external/envoy/source/common/http/http1/conn_pool.cc:48] [C297] response complete
2020-06-17T16:54:53.026454Z	debug	envoy pool	[external/envoy/source/common/http/conn_pool_base.cc:93] [C297] destroying stream: 0 remaining
2020-06-17T16:54:53.026592Z	debug	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1100] wasm log stats_outbound: [extensions/stats/plugin.cc:629]::report() metricKey cache miss _istio_requests_total , stat=12
2020-06-17T16:54:53.026610Z	debug	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1100] wasm log stats_outbound: [extensions/stats/plugin.cc:629]::report() metricKey cache miss _istio_response_bytes , stat=6
2020-06-17T16:54:53.026770Z	debug	envoy client	[external/envoy/source/common/http/codec_client.cc:127] [C295] protocol error: http/1.1 protocol error: unsupported transfer encoding
2020-06-17T16:54:53.026783Z	debug	envoy connection	[external/envoy/source/common/network/connection_impl.cc:109] [C295] closing data_to_write=0 type=1
2020-06-17T16:54:53.026786Z	debug	envoy connection	[external/envoy/source/common/network/connection_impl.cc:200] [C295] closing socket: 1
2020-06-17T16:54:53.026834Z	debug	envoy client	[external/envoy/source/common/http/codec_client.cc:91] [C295] disconnect. resetting 1 pending requests
2020-06-17T16:54:53.026845Z	debug	envoy client	[external/envoy/source/common/http/codec_client.cc:114] [C295] request reset
2020-06-17T16:54:53.026856Z	debug	envoy router	[external/envoy/source/common/router/router.cc:1018] [C294][S17503853286743059635] upstream reset: reset reason connection termination
2020-06-17T16:54:53.026887Z	debug	envoy http	[external/envoy/source/common/http/conn_manager_impl.cc:1475] [C294][S17503853286743059635] Sending local reply with details upstream_reset_before_response_started{connection termination}
2020-06-17T16:54:53.026927Z	debug	envoy http	[external/envoy/source/common/http/conn_manager_impl.cc:1706] [C294][S17503853286743059635] encoding headers via codec (end_stream=false):
':status', '503'
'content-length', '95'
'content-type', 'text/plain'
'x-envoy-peer-metadata', 'ChoKCkNMVVNURVJfSUQSDBoKS3ViZXJuZXRlcwofCgxJTlNUQU5DRV9JUFMSDxoNMTAwLjY0LjEwLjIyMwqJBAoGTEFCRUxTEv4DKvsDChMKCmFkcC5jb20vZGMSBRoDYXdzChQKC2FkcC5jb20vZW52EgUaA2RpdAoZChBhZHAuY29tL2Vudi10eXBlEgUaA2RpdAohCg9hZHAuY29tL3Byb2plY3QSDhoMcmVsZWFzZS1uYW1lCiwKGmFwcC5rdWJlcm5ldGVzLmlvL2luc3RhbmNlEg4aDHJlbGVhc2UtbmFtZQooChxhcHAua3ViZXJuZXRlcy5pby9tYW5hZ2VkLWJ5EggaBlRpbGxlcgo2ChZhcHAua3ViZXJuZXRlcy5pby9uYW1lEhwaGm1pY3JvYXBpLXdvcmtlci1vcGVyYXRpb25zCioKDWhlbG0uc2gvY2hhcnQSGRoXd2ViLXNlcnZpY2UtaXN0aW8tMS4xLjAKGQoMaXN0aW8uaW8vcmV2EgkaB2RlZmF1bHQKIQoRcG9kLXRlbXBsYXRlLWhhc2gSDBoKNTZmODY0NGM1YwokChlzZWN1cml0eS5pc3Rpby5pby90bHNNb2RlEgcaBWlzdGlvCj8KH3NlcnZpY2UuaXN0aW8uaW8vY2Fub25pY2FsLW5hbWUSHBoabWljcm9hcGktd29ya2VyLW9wZXJhdGlvbnMKLwojc2VydmljZS5pc3Rpby5pby9jYW5vbmljYWwtcmV2aXNpb24SCBoGbGF0ZXN0ChoKB01FU0hfSUQSDxoNY2x1c3Rlci5sb2NhbAo1CgROQU1FEi0aK21pY3JvYXBpLXdvcmtlci1vcGVyYXRpb25zLTU2Zjg2NDRjNWMtczZwaHYKFgoJTkFNRVNQQUNFEgkaB3Zhbi1kaXQKXgoFT1dORVISVRpTa3ViZXJuZXRlczovL2FwaXMvYXBwcy92MS9uYW1lc3BhY2VzL3Zhbi1kaXQvZGVwbG95bWVudHMvbWljcm9hcGktd29ya2VyLW9wZXJhdGlvbnMKHAoPU0VSVklDRV9BQ0NPVU5UEgkaB2RlZmF1bHQKLQoNV09SS0xPQURfTkFNRRIcGhptaWNyb2FwaS13b3JrZXItb3BlcmF0aW9ucw=='
'x-envoy-peer-metadata-id', 'sidecar~<IP_ADDRESS>~microapi-worker-operations-56f8644c5c-s6phv.van-dit~van-dit.svc.cluster.local'
'date', 'Wed, 17 Jun 2020 16:54:52 GMT'
'server', 'istio-envoy'
2020-06-17T16:54:53.026948Z	debug	envoy filter	[src/envoy/http/authn/http_filter.cc:49] Called AuthenticationFilter : onDestroy
2020-06-17T16:54:53.026965Z	debug	envoy pool	[external/envoy/source/common/http/conn_pool_base.cc:265] [C295] client disconnected, failure reason:
2020-06-17T16:54:53.027048Z	debug	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1100] wasm log stats_inbound: [extensions/stats/plugin.cc:612]::report() metricKey cache hit , stat=12
2020-06-17T16:54:53.027058Z	debug	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1100] wasm log stats_inbound: [extensions/stats/plugin.cc:612]::report() metricKey cache hit , stat=6
2020-06-17T16:54:53.027062Z	debug	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1100] wasm log stats_inbound: [extensions/stats/plugin.cc:612]::report() metricKey cache hit , stat=10
2020-06-17T16:54:53.027065Z	debug	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1100] wasm log stats_inbound: [extensions/stats/plugin.cc:612]::report() metricKey cache hit , stat=14
2020-06-17T16:54:53.027103Z	debug	envoy pool	[external/envoy/source/common/http/conn_pool_base.cc:93] [C295] destroying stream: 0 remaining
2020-06-17T16:54:53.027322Z	debug	envoy connection	[external/envoy/source/common/network/connection_impl.cc:558] [C294] remote close
2020-06-17T16:54:53.027333Z	debug	envoy connection	[external/envoy/source/common/network/connection_impl.cc:200] [C294] closing socket: 0
2020-06-17T16:54:53.027370Z	debug	envoy connection	[external/envoy/source/extensions/transport_sockets/tls/ssl_socket.cc:298] [C294] SSL shutdown: rc=1
2020-06-17T16:54:53.027375Z	debug	envoy connection	[external/envoy/source/extensions/transport_sockets/tls/ssl_socket.cc:226] [C294]
2020-06-17T16:54:53.027389Z	debug	envoy conn_handler	[external/envoy/source/server/connection_handler_impl.cc:111] [C294] adding to cleanup list
2020-06-17T16:54:53.029577Z	debug	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1100] wasm log stats_outbound: [extensions/stats/plugin.cc:629]::report() metricKey cache miss istio_request_duration_milliseconds , stat=10
2020-06-17T16:54:53.031995Z	debug	envoy filter	[external/envoy/source/extensions/filters/listener/original_dst/original_dst.cc:18] original_dst: New connection accepted
2020-06-17T16:54:53.032017Z	debug	envoy filter	[external/envoy/source/extensions/filters/listener/tls_inspector/tls_inspector.cc:78] tls inspector: new connection accepted
2020-06-17T16:54:53.032072Z	debug	envoy filter	[external/envoy/source/extensions/filters/listener/tls_inspector/tls_inspector.cc:148] tls:onServerName(), requestedServerName: outbound.8080_._.microapi-worker-operations.van-dit.svc.cluster.local
2020-06-17T16:54:53.032085Z	debug	envoy filter	[external/envoy/source/extensions/filters/listener/http_inspector/http_inspector.cc:37] http inspector: new connection accepted
2020-06-17T16:54:53.032133Z	debug	envoy conn_handler	[external/envoy/source/server/connection_handler_impl.cc:403] [C298] new connection
2020-06-17T16:54:53.032286Z	debug	envoy connection	[external/envoy/source/extensions/transport_sockets/tls/ssl_socket.cc:191] [C298] handshake expecting read
2020-06-17T16:54:53.032297Z	debug	envoy connection	[external/envoy/source/extensions/transport_sockets/tls/ssl_socket.cc:191] [C298] handshake expecting read
2020-06-17T16:54:53.032410Z	debug	envoy connection	[external/envoy/source/extensions/transport_sockets/tls/ssl_socket.cc:176] [C298] handshake complete
2020-06-17T16:54:53.032499Z	debug	envoy http	[external/envoy/source/common/http/conn_manager_impl.cc:268] [C298] new stream
2020-06-17T16:54:53.032564Z	debug	envoy http	[external/envoy/source/common/http/conn_manager_impl.cc:781] [C298][S5686435881520010723] request headers complete (end_stream=true):
':authority', 'xxxxxxxxxxxxxxxxxxxxxxxxxxxx'
':path', '/hr/v2/workers?$top=47'
':method', 'GET'
'user-agent', 'curl/7.50.3'
'accept', 'application/json'
'acsgroups', '%LDAP%'
'sm_transactionid', '1234567'
'x-forwarded-for', '<IP_ADDRESS>'
'x-forwarded-proto', 'http'
'x-envoy-internal', 'true'
'x-request-id', '53189ebd-b390-96bb-9bbb-96c8d7dcbd0e'
'x-envoy-decorator-operation', 'microapi-worker-operations.van-dit.svc.cluster.local:8080/microapi-worker-operations/1/xyz/*'
'x-envoy-peer-metadata', 'ChoKCkNMVVNURVJfSUQSDBoKS3ViZXJuZXRlcwoeCgxJTlNUQU5DRV9JUFMSDhoMMTAwLjY0LjEyLjI1CpYCCgZMQUJFTFMSiwIqiAIKHQoDYXBwEhYaFGlzdGlvLWluZ3Jlc3NnYXRld2F5ChMKBWNoYXJ0EgoaCGdhdGV3YXlzChQKCGhlcml0YWdlEggaBlRpbGxlcgoZCgVpc3RpbxIQGg5pbmdyZXNzZ2F0ZXdheQohChFwb2QtdGVtcGxhdGUtaGFzaBIMGgo2YzliY2RmYjlkChIKB3JlbGVhc2USBxoFaXN0aW8KOQofc2VydmljZS5pc3Rpby5pby9jYW5vbmljYWwtbmFtZRIWGhRpc3Rpby1pbmdyZXNzZ2F0ZXdheQovCiNzZXJ2aWNlLmlzdGlvLmlvL2Nhbm9uaWNhbC1yZXZpc2lvbhIIGgZsYXRlc3QKGgoHTUVTSF9JRBIPGg1jbHVzdGVyLmxvY2FsCi8KBE5BTUUSJxolaXN0aW8taW5ncmVzc2dhdGV3YXktNmM5YmNkZmI5ZC1yNmw2agobCglOQU1FU1BBQ0USDhoMaXN0aW8tc3lzdGVtCl0KBU9XTkVSElQaUmt1YmVybmV0ZXM6Ly9hcGlzL2FwcHMvdjEvbmFtZXNwYWNlcy9pc3Rpby1zeXN0ZW0vZGVwbG95bWVudHMvaXN0aW8taW5ncmVzc2dhdGV3YXkKpwEKEVBMQVRGT1JNX01FVEFEQVRBEpEBKo4BCiAKDmF3c19hY2NvdW50X2lkEg4aDDgzODAxMDkwNDQ5MwolChVhd3NfYXZhaWxhYmlsaXR5X3pvbmUSDBoKdXMtZWFzdC0xYQooCg9hd3NfaW5zdGFuY2VfaWQSFRoTaS0wNzhmYTYwYzBjNjc3ODBlNQoZCgphd3NfcmVnaW9uEgsaCXVzLWVhc3QtMQo5Cg9TRVJWSUNFX0FDQ09VTlQSJhokaXN0aW8taW5ncmVzc2dhdGV3YXktc2VydmljZS1hY2NvdW50CicKDVdPUktMT0FEX05BTUUSFhoUaXN0aW8taW5ncmVzc2dhdGV3YXk='
'x-envoy-peer-metadata-id', 'router~<IP_ADDRESS>~istio-ingressgateway-6c9bcdfb9d-r6l6j.istio-system~istio-system.svc.cluster.local'
'x-envoy-expected-rq-timeout-ms', '50000'
'x-b3-traceid', 'e8d7653e211045ebf7cad02b32970102'
'x-b3-spanid', 'f7cad02b32970102'
'x-b3-sampled', '1'
'x-envoy-original-path', '/microapi-worker-operations/1/abcd/hr/v2/workers?$top=47'
'content-length', '0'

Similar to https://github.com/istio/istio/issues/23020 but should be fixed in this version...

@howardjohn I tested our service with 1.5.6 and also 1.6.2 version,  and still seeing this issue.  I can confirm that its not happening with 1.4.6 version.

@kyaparla Could you enable the trace log on http topic?
kubectl exec -it pod/<POD> -- curl -X POST localhost:15000/logging?http=trace
I have something below. the completed header: key=transfer-encoding should be a little bit above the http1.1 protocol error.
Looks like envoy support only "chunked". No other values are allowed. It shouldn't be a issue with default istio config.
 It's petty clear that there are duplicated transfer-encoding since http headers are case-insensitive.
2020-06-23T21:40:36.631195Z trace envoy http [external/envoy/source/common/http/http1/codec_impl.cc:433] [C528] completed header: key=transfer-encoding value=chunked
...
2020-06-23T21:40:36.631207Z trace envoy http [external/envoy/source/common/http/http1/codec_impl.cc:433] [C528] completed header: key=Transfer-Encoding value=chunked

IIRC envoy would do concatenation on duplicate headers: so the value is "chunked, chunked".
Confirmed on 1.6.2

Can you confirm the next hop which directly respond duplicated Transfer-Encoding? Is it another envoy? If it's envoy, is it 1.6.2 or 1.5.6?
I could not reproduce: If the upstream of envoy responds either "Transfer-Encoding" or "transfer-encoding", envoy doesn't generate duplicated these headers as in above logs.

@lambdai
This was collected from 1.6.2 version.  Are there any changes to envoy proxy around headers between 1.4 and 1.5 that could impact this?  We are thinking of using 1.4.6 version, as it supports service entries, and with out this particular issue.
Request flow is as follows.
inbound request through envoy proxy to the application,  application make outbound request to outside k8s cluster ( tried through service entry & direct ).  When payload size is big inbound request fails with 503 response,  but outbound request is always 200.
Here is the flow.
Route53->elb->ingress gateway->envoy proxy-> application -> egress->outbound->external service
Please let me know, for additional information.

@lambdai
Tested enabling http trace on Istio 1.4.6, with same service and I dont see Transfer encoding: chunked text.  Is this something new with Istio 1.5?
Is this an issue with Istio or the application doing some thing not supported by Istio?

There is no intended behavior change regarding to handle Transfer-Encoding: chunked.
As we can see in http trace, it means envoy received duplicated Transfer-Encoding. Can you figure out the first hop producing the duplicated Transfer-Encoding?

@lambdai
I see ingressgateway http trace shows only 1 header,  with small case.  Next hop from here is side car proxy right?  Does side car introduces another header?
Do we have to find out where below header is coming from?
http/http1/codec_impl.cc:433] [C905872] completed header: key=server value=istio-envoy
http/http1/codec_impl.cc:573] [C905872] onHeadersCompleteBase
http/http1/codec_impl.cc:433] [C905872] completed header: key=transfer-encoding value=chunked
http/http1/codec_impl.cc:986] [C905872] Client: onHeadersComplete size=11

"server: istio-envoy" says so. Unless you have a manipulated process pretending to be envoy.
However, when I setup an istio-envoy inside istio 1.6.2 and 1.5.6, that envoy doesn't produce 2 "transfer-encoding"
curl <-> envoy(1.5.6 or 1.6.2) <-> "nc -l"

nc is introduced so that I can feed http 200 with my own "transfer-encoding", lower case or upper case. That envoy could either abort when I produce double transfer-encoding from nc(which is what saw in your istio-ingressgateway), or that envoy produces single `Transfer-Encoding" header.

@lambdai
We enabled further debugging,  and found that the backend,  our service is interacting with is returning this header and our application is setting these headers and returning the response.  We modified our code to delete transfer-encoding header, and see the issue is not happening.
Wanted to understand more,  on why this is working in versions < 1.5, but breaks from versions > 1.5.  And also if the headers are case insensitive, shouldnt it do dedup and consider only 1 header instead of 2?

@kyaparla Great to see you resolve it! Thank you for the update.
Back to topic of istio's behavior. Istio dataplane is all in envoyproxy. Envoy was improving the performance of http header operations and I believe some regressions are accidentally introduced. #23020 is one of them. I believe you are reporting anther one.
I agree with you that duplicate chunked is not ideal but should not fail the http request. I see curl doesn't report error. Earlier version envoy didn't report either.
The follow up: I will fix the upstream envoy.

@kyaparla Discussed in https://github.com/envoyproxy/envoy/pull/11916
And envoy won't add the tolerance back. Because

two transfer-encoding: chunked is equivalent to transfer-encoding: chunked, chunked as per RFC
transfer-encoding: chunked, chunked doesn't have the same semantic as transfer-encoding: chunked
Before your fix, the body in your http response is actually should be decoded by transfer-encoding: chunked but you are sending  transfer-encoding: chunked, chunked

We know

tl;dr
chunked, chunked should not be considered "chunked" although it happens to resolve the failure.
Close if for now.
We can raise another discussion if there overwhelming cases

tl;dr
chunked, chunked should not be considered "chunked" although it happens to resolve the failure.
Close if for now.
We can raise another discussion if there overwhelming cases

@lambdai
Just looking for more details on what changed between 1.4 and 1.5 versions,  that introduced this?  Was transfer-encoding being interpreted differently before 1.5?

Hi guys,
I've tested istio from versions 1.6.0 to 1.6.8 and happens the same issue when header has Transfer-Encoding:chunked.
I've set reject_unsupported_transfer_encodings = false on proxy as workaround and the issue is temporarily solved.
## Set envoy feature ##
kubectl exec istio-ingressgateway-XXXXXXX-- \
curl -X POST http://localhost:15000/runtime_modify?envoy.reloadable_features.reject_unsupported_transfer_encodings=false

Is that behaviour expected when connection is HTTP 1.1 ?
Is that a envoy bug ?
Is there any solution I could apply on istio to have this solved ?

Thank you for the workaround! Please fix the upstream http app which populate double chucked at high priority. Envoy is removing the reject_unsupported_transfer_encodings soon
AFAIK some user report Tomcat cors plugin is a potential culprit but I didn't verify.
Your pasted log shows this is double "chunked". See the below explanation. It's the double which is illegal.
A single "Transfer-Encoding:chunked" is fine.
2020-08-14T11:35:02.404669Z trace envoy http [external/envoy/source/common/http/http1/codec_impl.cc:672] [C862] message begin
...
2020-08-14T11:35:02.404697Z trace envoy http [external/envoy/source/common/http/http1/codec_impl.cc:433] [C862] completed header: key=Transfer-Encoding value=chunked
...
2020-08-14T11:35:02.404765Z trace envoy http [external/envoy/source/common/http/http1/codec_impl.cc:573] [C862] onHeadersCompleteBase
## The log line right below onHeadersCompleteBase is the last header
2020-08-14T11:35:02.404767Z trace envoy http [external/envoy/source/common/http/http1/codec_impl.cc:433] [C862] completed header: key=Transfer-Encoding value=chunked

So this is Transfer-Encoding: chunked twice in same http response.
According to the RFC this line appeared twice has the exact same semantic of "Transfer-Encoding: chunked, chunked`.
To avoid future confusing let's refer it as multiple-chunked regardless it's literal "chunked, chunked" or duplicated "Transfer-Encoding: chunked" :)

The same response, curl reports no error. curl has better fault tolerance？

We just spent 4 hours troubleshooting a very similiar problem like this, but there were no duplicate values at all. Just the presence of transfer-encoding: chunked alone broke Istio. We have an API that, when hit, calls out to another API to get some information. The response headers from that upstream API include transfer-encoding: chunked, just once, no duplicates.
If I port-forwarded to the pod and hit it on localhost, it works fine. If i hit it externally so traffic flows through sidecar, app gets the request, seemingly processes it just fine, but envoy shows a 502 UPE for every request. Only solution was to remove the header in code.
It seems crazy that something so trivial could blow up Envoy/Istio. Is there no solution to this?

What if we don't have access to the source code ? What then?
Doesn't seem that VirtualService remove header helps either.

declare the protocol as "TCP" and it won't try to parse it as http
On Mon, May 29, 2023, 12:17 PM Tommy Forlini @.***>
wrote:

What if we don't have access to the source code ? What then?
Doesn't seem that VirtualService remove header helps either.
—
Reply to this email directly, view it on GitHub
https://github.com/istio/istio/issues/24753#issuecomment-1567443904, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAEYGXPROLPEKHON5UTRVHTXITY3NANCNFSM4OA33SYA
.
You are receiving this because you were mentioned.Message ID:
@.***>


No CORS header

http://diff.pics/wRQkoH6jEaE0/1 doesn't load for me on FireFox 50.1 or Chromium 45.
Console (FireFox first, Chromium second):
Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at http://cdn.diff.pics/E4B174AD84B4A3C4ADFA03DFA7CC5EFC1A701405.png. (Reason: CORS header ‘Access-Control-Allow-Origin’ missing).

XMLHttpRequest cannot load http://cdn.diff.pics/E4B174AD84B4A3C4ADFA03DFA7CC5EFC1A701405.png. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://diff.pics' is therefore not allowed access.

http://diff.pics/FcJsafFaIbJb/1 works.

Finally figured out what was going on and should be fixed


Ability to tell taskbar to take all tabs from all windows terminal windows that are open and list all tabs in 1 list

Description of the new feature/enhancement
The windows taskbar shows icons. In the summary mode, clicking on the icon lists all the windows open for that application.
The question/feature suggestion is , when multiple terminal windows are open, each with their own list of tabs, to have the ability to get  1 Terminal icon in the taskbar, and that clicking on that icon shows all the tabs titles together from all the windows.
Proposed technical implementation details (optional)

IIRC we don't really have that kind of control over our taskbar entries. I believe this is basically the default behavior of Windows 11 (all the windows for an app are glommed into a single icon on the taskbar).
There's also #687 for "I want to see open tabs in the taskbar"
We do already have support for a "tray icon" / "notification area icon"

(with plans to show tab titles there too #10448). I suspect that might be the best solution for you /?


Update walls.ftl

ALOOO 4ZH
title

huh???
https://github.com/space-syndicate/space-station-14/blob/cdd349468692b085febe8bdf2369456525d3ef6b/Resources/Prototypes/Entities/Structures/Walls/walls.yml#L466-L471

https://github.com/space-syndicate/space-station-14/blob/cdd349468692b085febe8bdf2369456525d3ef6b/Resources/Prototypes/Entities/Structures/Walls/walls.yml#L466-L471

idk


Date picker widget should highlight current date at all times

When the datepicker opens on an clean date field, the modals opens on the current month, with the current day highlighted in the salmon colour.
Once a date is selected, assuming that date isn't today, the picker will never again colour-code the current date if re-opened. It will show the date selected in the field, but not the current date. As someone rarely aware of the actual date, I find this endlessly frustrating.
There's one other idiosyncrasy worth addressing when this issue is handled: When the picker is opened on a clean field, the current date e.g 15, is highlighted, but the number 15 remains highlighted in each of the months you choose, regardless of the actual current month. I can't work out why this would be useful and again this causes confusion.

Fixed by #2260


Icon color

Hi,
I integrated the image editor using via npm and everything works fine, except the icon colors at the bottom of the editor.
How can I change the color of the icons? Why is the entire menu just black?

self.editor = new ImageEditor(document.querySelector('#tui-image-editor'), {                 includeUI: {                     loadImage: {                         path: self.documentUrl,                         name: 'SampleImage'                     },                     menuBarPosition: 'bottom'                 },                 cssMaxHeight: 800             });

The theme file in the example code uses a built icon file.
Run npm build once and the build file will be in your dist  directory.

Sorry but there is no change after running npm build.
I installed the library using npm. It's available in node_modules.
I really can't follow.

Have you tested with your local development server with npm serve?

Sorry. There was a mistake in the description.
npm build -> npm bundle

Hi, unfortunately I don't have npm build and cannot upgrade. Is there a way to get just the svg-files where I can change the color and include static?

@engma969
Sorry I'm wrong again. It's not npm bundle but npm run bundle.

I'll take a try, thanks.
But I don't understand why the default styling/theme is so bad?
I would be happy if I get a styling/theme like here:
https://user-images.githubusercontent.com/35218826/41647896-7b218ae0-74b2-11e8-90db-d7805cc23e8c.gif
But the icons are the same color as the background, so you can't see them.

@enigma969
The gif image that you linked is ImageEditor's default theme. Icons of ImageEditor are SVG image that bundled via npm run bundle command.  I think you doesn't run npm run bundle. You should run that command in your repository.

I am also experiencing this issue.  Did anyone find a solution?

I was able to get ToastUI Image Editor working including styles and icons using npm. In a couple of steps here is how I did it within an Angular app using npm, typescript, scss, etc.
https://gist.github.com/coderfin/103d75152f12dad8b0bda3df73189ca9


Introduce switch case for received msg

introduce new switch case conditions on received messages
updated conditions
fixed lint checks
updated ci.yml black check and...

Codecov Report

Merging #58 (390fb14) into master (bf68a9d) will increase coverage by 0.02%.
The diff coverage is 88.98%.

@@            Coverage Diff             @@
##           master      #58      +/-   ##
==========================================
+ Coverage   82.22%   82.25%   +0.02%
==========================================
  Files          24       25       +1
  Lines        1007     1031      +24
==========================================
+ Hits          828      848      +20
- Misses        179      183       +4

Flag
Coverage Δ

unittests
82.25% <88.98%> (+0.02%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

camera/__init__.py
100.00% <ø> (ø)

common/__init__.py
100.00% <ø> (ø)

door/__init__.py
100.00% <ø> (ø)

encrypt_password_with_base64.py
0.00% <0.00%> (ø)

fdia.py
0.00% <0.00%> (ø)

messaging/__init__.py
100.00% <ø> (ø)

otp_token_helper_cli.py
0.00% <0.00%> (ø)

test/__init__.py
100.00% <ø> (ø)

camera/cam_common.py
44.68% <45.45%> (+0.23%)
:arrow_up:

camera/blink_cam.py
59.55% <75.00%> (ø)

... and 23 more

:mega: Codecov can now indicate which changes are the most critical in Pull Requests. Learn more


Add all parameters to from_tensorflow docs

TVM docs does not have the description of from_tensorflow parameters.
https://docs.tvm.ai/api/python/relay/frontend.html#tvm.relay.frontend.from_tensorflow
https://docs.tvm.ai/api/python/nnvm/frontend.html#nnvm.frontend.from_tensorflow
This PR adds it

Can you combine https://github.com/dmlc/tvm/pull/3320 with this PR, as both of them are doc fixes? Thank you

nevermind, merged as it is. Thanks @apivovarov

Tianqi, I just saw your message to combine PRs... Thank you for merging!
Question. Well the docs on website be automatically updated? is it nightly process?
I still see the old docs:
https://docs.tvm.ai/api/python/nnvm/frontend.html#nnvm.frontend.from_tensorflow
https://docs.tvm.ai/api/python/relay/frontend.html#tvm.relay.frontend.from_tensorflow

The docs will get updated immediately as soon as the master CI pipeline finishes


cannot start a program using systemctl

cannot start a program using systemctl

Hi there!
systemd is not supported on WSL: systemctl doesn't work
There is a Python script that replicates a good portion of the systemctl functionality, which may be of help: docker systemctl replacement

As systemd is now supported by wsl (https://devblogs.microsoft.com/commandline/systemd-support-is-now-available-in-wsl/) can we have it in centos as well?

@zljubisic
The required config changes for native systemd support are very minimal, and there are many other configs that can be applied: https://learn.microsoft.com/en-us/windows/wsl/wsl-config
There is nothing new to "support", because the required under-the-hood systemd changes are baked directly into WSL, and not the individual distros images. For now, you're required to opt-in into this functionality, as described in the official WSL docs.
Additionally, this repo aspires to keep a no-modifications 1:1 copy of the GenericCloud image profile for CentOS, for now.


Recursive empty object leaves empty objects

In the below example the resulting object retains a key with an empty object. Is this expected behaviour? I anticipated the return value to be {}
test('objects are recursively cleaned up', () => {
  const clean = cleanObject({ o: { b: {} } }, {
    deep: true,
    emptyObjects: true,
  });

  expect(clean).toEqual({});
});

  ● objects are recursively cleaned up

    expect(received).toEqual(expected) // deep equality

    - Expected  - 1
    + Received  + 3

    - Object {}
    + Object {
    +   "o": Object {},
    + }

Fixed in 1.2.0


feat(docs): split demo examples between demo.md and new apiExamples.md

Alaska Airlines Pull Request
Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.
Fixes: #303
Summary:
Splits demo examples into two pages:

demo.md
apiExamples.md (to be used in docsite in place of api.md)

api.md file remains as it was previously.
Type of change:
Please delete options that are not relevant.

[x] New capability
[ ] Revision of an existing capability
[ ] Infrastructure change (automation, etc.)
[ ] Other (please elaborate)

Checklist:

[x] My update follows the CONTRIBUTING guidelines of this project
[x] I have performed a self-review of my own update

By submitting this Pull Request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
Pull Requests will be evaluated by their quality of update and whether it is consistent with the goals and values of this project. Any submission is to be considered a conversation between the submitter and the maintainers of this project and may require changes to your submission.
Thank you for your submission!
-- Auro Design System Team

:tada: This PR is included in version 3.15.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Revert "Add HA/LB endpoints for kube-apiserver"

This reverts commit a70c3b661e12d75ca01f8c0b38ef6af625d3383a.

+1


Configure demo/nightly eclipse-pass.org to correctly load sample data

This PR documents how to get a locally running demo system
https://github.com/eclipse-pass/main/pull/370
There is a small problem, in that the loader can only be run after pass-core is ready.  I am duplicating the manually instructions here.
Pull Latest Docker Images
This will pull all the latest pass docker images.
./demo.sh pull

Start Pass
This will start pass in the background
./demo.sh up -d

At this point you will need to watch the logs to wait until the
pass-core shows that it has started.  This is something we
are actively working to address.
./demo.sh logs -f

It might take a while, but once the logs stop with the message below
then it is ready to load the base data.
[main] [Pass, ] INFO  org.eclipse.pass.main.Main.logStarted - Started Main in 69.863 seconds (JVM running for 79.59)

That base data can now be loaded using the following command.
./demo.sh up loader

Here we want to address the dependency so that our demo systems (demo.eclipse-pass.org and nightly.eclipse-pass.org) can properly have the loaders run once ready.
The scripts managing our demo deploys are available here
https://github.com/eclipse-pass/pass-docker/blob/main/.github/workflows/deploy_passdemo.yml
https://github.com/eclipse-pass/pass-docker/blob/main/.github/workflows/deploy_passnightly.yml
Ideally this ticket is resolved once we can log into the demo URLs with nih-user / moo and see the appropriate demo data.

I started on this but @jabrah was already working on a more detailed fix for loader

Prospective fix in this branch that Jared is currently updating, trying to update IDP in deployed instances
https://github.com/eclipse-pass/pass-docker/tree/20221104-misc-updates-and-rebranding


Uploading files to GPT@Azure

Functionality to upload files (PDF/txt for starters) to GPT-4o deployment at Azure AI Studio and getting a response back

Uploading files (pdf/txt) to API endpoint directly is not possible. We can try reading files into text and uploading those.

Switched to OpenAI direct API (using Python OpenAI library), so no need for Azure Labs.


VideoPlayer: Added started,paused,stopped,finished signals

fixes #5377
started is emitted on play()
paused is emitted when going from not paused state to paused state
stopped is emitted when the stream was playing on a call to stop
finished is emitted when the stream naturally stops(runs out)
also cleaned up the files a bit, removed some unnecessary prints, changed some fprintf(stderr prints into ERR_PRINT

Nice cleanup, though I'd prefer if the cleanup had been done in its own commit (i.e. no new signal in the first commit), and the new features added in a second commit.
Also, what is the reason for the various bits you commented out? If they're unneeded code, it's best to just drop them instead of commenting them out. If they might be needed, then it would be good to add comments in the code to explain why this stuff is commented out.

The commented out stuff was unused(there was already a large block of commented out code that caused these unused variable warnings so I just commented them out too to silence them). So I cant comment why that stuff is commented out, it just was.
I also commented out some unnecessary prints, but sure I can split it into two commits, basically the signals were added in one file and the cleanup was in another file.

That's better, thanks :)
We should really get rid of all this commented code and unused variables :)

It needs to be rebased, but the idea should probably be validated by @reduz first to check that he's fine with the new API.

closing this one, as mentioned in other similar issues, I don't think signals that are emitted when the user calls functions are needed.  By rule, signals are generally emitted when something not directly under the control of the user happens.

True, but the finished is the only one that's relevant in here. Shall I make a PR with just that signal?

I think the cleanup commit is still relevant, besides the finished signal. Better open a new PR with that.

@Keyaku Hi, do you still plan to do that PR?


Customization Idea: Change Terminal Text Position

Description of the new feature/enhancement
Set coordinates / a space in which the terminal's text is shown (for example always with 10pixels space between text and window border), so that the text can be used in a combination with a background, like I made here: My Reddit Post. Perhaps by creating two text offset variables, such as TextOffsetVertical and TextOffsetHorizontal, in which a value of pixels is placed.
Proposed technical implementation details (optional)

Can be done using "Padding", my bad.


Plugin Infrastructure

Fixes #238

[x] add entry point
[x] let plugins register signals
[ ] let plugins include urls
[ ] update builtin plugins

Coverage decreased (-0.2%) to 79.866% when pulling 0a18c444cf02960aefdae2f826576bc05840da3a on plugin-infra into d2e4a7e6b222a73b0f7ba97a1d6290b8f0a019d9 on main.

Coverage decreased (-0.2%) to 79.866% when pulling 0a18c444cf02960aefdae2f826576bc05840da3a on plugin-infra into d2e4a7e6b222a73b0f7ba97a1d6290b8f0a019d9 on main.


[#1808, #1809, #1810] Fixed Bot Inspector docs page accessibility

#1808, #1809, #1810
===
Fixes:

Cutoff focus outline border on top-level links
Header semantics of generated Markdown page (h1's downgraded to h2's, h2's to h3's, now only one h1 on the page)
Using browse mode will no longer get stuck in the dialog host component and read "blank" repeatedly

Coverage increased (+0.002%) to 65.777% when pulling 846e2845960cb8d7d13e13a880aead1df2665f52 on toanzian/acc-#1808-10 into 7bccc931257430f583e522c7d5c27afc57f8388f on master.


More compact cards

Font size scale?

Please see the difference with Feeder.

It's a good idea, I can't commit to a timeline yet. Adding this to https://github.com/jocmp/capyreader/discussions/176 for now.


.browserlistsrc should be considered even if .swcrc exists

Currently, we don't check for .browserlistsrc, but it can be confusing.

Which config file should be preferred?

Oh... Maybe we should emit a warning if there's both jsc.target/env with .browserslisttc

Looking for .browserslistrc file can be complicated, how do we check it?

@g-plane
Yeah, I also think it's a complicated task.
Does browserslist-rs supports checking .browserlistsrc or entry in the packge.json?
Maybe a command like swc doctor would be better.

Yes, but it didn't expose APIs about reading configuration files.

Is there a way to distinguish between a default value and configured value?
I think it will be enough.

What does "default value" and "configured value" mean?

If user specified anything related to browserslist, it's configured.

Maybe it can't.
There's a kind of query: defaults. If users don't specifying anything, it will use defaults as "default value". However, users can also configure it using defaults in config file manually.

Oh, I see. Then we can check if env and jsc.target are specified at the same time.

https://github.com/babel/babel/blob/cb3ebde8ceed47f4b6783c16137cc9ddc78998b7/packages/babel-helper-compilation-targets/src/index.ts#L194
I think we may need to expose an API about loading config.

@kdy1 @g-plane Hey! Are you sure browserslist config loading works fine now? The issue I have is that swc-loader doesn't load the config from package.json, so I have to do it manually, .browserslistrc doesn't seem to work either.
// webpack swc-loader options
{
  env: {
    // FIXME: swc-loader doesn't load browserslist config from package.json for some reason
    targets: require('browserslist').loadConfig({
      path: paths.appPath,
      env: nodeEnv,
    }),
    mode: 'entry',
    coreJs: '3.22',
  },
  jsc: {
    target: 'es5',
    parser: {
      syntax: 'typescript',
      dynamicImport: true,
      tsx: true,
    },
    experimental: {
      cacheRoot: paths.appSwcCache,
    },
    transform: {
      react: {
        runtime: 'automatic',
        refresh: isEnvDevelopment,
        development: isEnvDevelopment,
      },
    },
  },
  minify: false,
},


tf.train.Server session_config parameter cannot be configured by Estimator

Please make sure that this is a bug. As per our
GitHub Policy,
we only address code/doc bugs, performance issues, feature requests and
build/installation issues on GitHub. tag:bug_template
System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): No
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux CentOS7
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: No
TensorFlow installed from (source or binary): binary
TensorFlow version (use command below): 1.15
Python version: 2.7
Bazel version (if compiling from source):
GCC/Compiler version (if compiling from source):
CUDA/cuDNN version: No
GPU model and memory: No

Describe the current behavior
session_config=tf.ConfigProto(...)
run_config=tf.estimator.RunConfig(session_config=session_config)
estimator = tf.estimator.Estimator(model_fn=..., config=run_config)
tf.estimator.train_and_evaluator(estimator, train_spec, eval_spec)
ps/worker tf.Server is not configured by session_config
Describe the expected behavior
ps/worker tf.Server can be configured by Estimator
PR: https://github.com/tensorflow/estimator/pull/66

@candyzone
We see that the issue template has not been filled, could you please do so as it helps us analyse the issue [tf version, steps followed before you ran into this error or stand alone code to reproduce the issue faced]

@candyzone
We see that the issue template has not been filled, could you please do so as it helps us analyse the issue [tf version, steps followed before you ran into this error or stand alone code to reproduce the issue faced]

update

@candyzone
We see that you are using a very old version of tensor flow for which there is no support [for tf 1.x] any more, kindly upgrade to 2.x and let us know if you face any issues.

@Saduf2019
TF 2.0 move Estimator to tensorflow_estimator  as an addon. It also has this problew in TF 2.0
I make a PR in https://github.com/tensorflow/estimator/pull/66

@candyzone ,
Respective PR was closed due to lack of activity.Can you please provide the required information to debug the issue and also please  update TensorFlow to the latest stable version v.2.7 and let us know if you are facing the same error. Thanks!


Downweight service standard reports

Downweight search results for documents with format service_standard_reports.
These reports rank highly in search results because the page title is a user-friendly service names. They also cause duplicate titles in search results when a service has had multiple assessments.
https://trello.com/c/gCeAhOCh/57-make-service-standard-reports-less-prominent-and-misleading-in-search-results
Marked as DO NOT MERGE for now because I'm going to double-check the exact weighting with a performance analyst.

Tara's approved the 0.2 weighting, so this is ready to review.


Extract module is broken on windows

I hesitate to put this issue out, but I'm sure it is not just me that caused this:
I'm on Windows 10, vscode (latest version), and also latest version of rust-analyzer.
I'm writing a module in a main.rs-file, and I wanted to Ctrl+. and then extract module to file.
However, I get this error message:
cannot open file:///c%3A/Users/tpb398/Documents/GitHub/todo-cli/src/todo.rs.
Detail: Unable to read file 'c:\Users\tpb398\Documents\GitHub\todo-cli\src\todo.rs'
(Error: Unable to resolve non-existing file 'c:\Users\tpb398\Documents\GitHub\todo-cli\src\todo.rs')

Note, this file todo.rs doesn't exist, but isn't it supposed to be created here?
Maybe it is a windows thing and the c%3A is the problem.

Can confirm this also happens on my Windows machine:
cannot open file:///c%3A/Users/yoshu/Code/prisma-engines/libs/user-facing-errors/src/user_facing_error.rs.
Detail: Unable to read file 'c:\Users\yoshu\Code\prisma-engines\libs\user-facing-errors\src\user_facing_error.rs'
(Error: Unable to resolve non-existing file 'c:\Users\yoshu\Code\prisma-engines\libs\user-facing-errors\src\user_facing_error.rs')

To me this seems like the filesystem fails to create the file, or perhaps opens the file with the wrong permissions, and then tries writing into it, which fails.

I have the same issue. It's not very serious, just very inconvenient for now.

https://github.com/rust-analyzer/rust-analyzer/pull/7576 fixes this, turns out we did indeed try to fill the new module with source before actually creating its file.


Accessing in-built fragments of managed objects in json/dict form

Is there a way to get the full Cumulocity IoT managed object representation related to the ManagedObject python class representation which includes in-built/read-only fragments (e.g. lastUpdated, id, creationTime)?
I'm currently using the to_json() method, however it does not include the fragments mentioned above. Yes I could use the __dict__ method, however that would only include the python model's representation of the fields, e.g. it would have update_time rather than the Cumulocity fragmentlastUpdated.
Looking further there seems to be already some logic which looks at adding back some fragments when constructing the json dictionary, so the in-builts could potentially be added back in there. Below shows that the c8y_IsDevice is re-added into the device when marshalling to a dictionary object.
def to_json(self, only_updated=False) -> dict:
    # (no doc changes)
    object_json = super().to_json(only_updated)
    if not only_updated:
        object_json['c8y_IsDevice'] = {}
    return object_json

Note: The above example was taken from 1.5.x, so the exact implementation has changed slightly but the main point remains.
Another place where this logic could be slotted in could be the to_full_json() method, as the name might be a bit more fitting (at least from a first glance).
Background
I've been using c8yapi in another library that I've been working on to support Robot Framework tests, robotframework-c8y. Under the hood I'm using this library, and I'm mostly returning any objects as a plain dictionary to the user so they can do any followup assertions using standard python objects (and not custom classes where the user has to read through docs).
A dumbed down example would look something like this.
def assert_managed_object_existence(device_id):
    mo = client.inventory.get(device_id)
    return mo.to_json()

The idea is to get a value from Cumulocity, so some assertion, and then return the object to the user, so they could check that the managed object fragments match some expected outcome.

Honestly, the to_json and to_full_json are not intended for regular use. They provide a JSON representation which is used to update an object, hence to_json can be instrumented to only give the updated parts of an object. The __dict__ function won't really help you there as it does not have the same structure as the actual JSON.
You can't do any harm with these, hence they are not hidden. They are also sometimes helpful, especially for tests.
I was once thinking about a raw_json property or similar but I didn't go for it as it would just add memory and the use is for corner cases only. The intended audience for this package are users who are not interested in the actual JSON but work with the class' properties.
However, the session handling and everything of this package is really helpful for all kinds of scenarios, so maybe add a feature request?
For your particular case I guess it would be best to recede to the basic REST commands from the API:
def assert_managed_object_existence(device_id):
    return client.get(client.inventory.build_object_path(device_id))

Untested, but something like this should work quite nicely.

Yeah I see your point. I'll have another look at the test interface of my library to support assertions on the lastUpdated/creationTime fragments using the datetime objects. (because in the end having the lastUpdated value as a datetime object is more useful)
If I can't find a nicer interface, I'll consider doing a PR to add a raw_json() method in as I still see it as beneficial to be able to 'reconstruct' the Cumulocity IoT data representation...but maybe that is just me.


Create general “content/documentation page” component/template

Similar to a homepage/landing page (#55) we plan to create some standalone documentation pages to provide additional information to users.
Let's create a template component that we can use to create web pages via markdown files
Design is neeed here but in the meantime we could build out the functionality to create pages based off markdown files,
e.g. by adding a src/pages/about.md file or src/pages/about/index.md, this would create a webpage accessible at https://maci-rpgf.vercel.app/about
For a general layout, I'm thinking something simple like:
https://maci.pse.dev/roadmap
Nice to have

Table of contents along the right sidebar
Linkable headers (e.g. like https://maci.pse.dev/roadmap#1-maci-developer-experience-dx)

Some possible package we could use:
https://nextjs.org/learn-pages-router/basics/dynamic-routes/render-markdown


Add Tidal

Tidal is a music streaming service, directly competing with Spotify with the argument of 'we have better quality audio.

8:50AM next workday:
Hello Tidal,

Thank you for reaching out to TIDAL Support! My name is Jared and I will be assisting you today with your account deletion.

In order to move forward with your request, we will need to verify the account's ownership. In order to do so, we ask that you provide us with a minimum of three (3) of the following pieces of information:

-Username
-Email Address (Associated with the account)
-Last 4 digits and expiration date of card on account
-PayPal’s Billing Agreement Number
-Date of last charge

Best Regards,
*name removed by me*
TIDAL Member Support

8:45 the day after:
Hello,

We received your request to delete your personal data.

Please be aware that by deleting all of your personal data, you will lose access to your TIDAL account including without limitation your playlists, history and account details. You will not be able to retrieve it at a later date.

Please also note that we may need to keep some of your personal data because we have a legitimate reason to do so (for example, to comply with our legal or regulatory obligations).

TIDAL’s policy is to keep your personal data only for so long as is necessary to satisfy the processing purposes or as required by law. TIDAL will delete your personal data where the law requires TIDAL to do so.

Please confirm that would like us to proceed with the deletion of your account [yes/no].

Further information about our data handling, retention and deletion practices is detailed in our Privacy Policy located here

Best Regards,
*name removed by me*
TIDAL Member Support

Thanks for your contribution!


support for bitcoin parsing and formatting

Fixes #97.

I just found out that StyleCI automatically pushes a PR for style inconsistencies. Since I removed the inconsistencies by hand already, I removed those PRs.

LGTM. I added a few styling comments.

What is wrong with Travis? Why does it not start?

no clue, travis DERP. Restarted.

Two comments added.
This is fine from my point of view. I don't use Bitcoin, if anyone has problem with it, we can treat it as bugfix.

Ok. will add the new lines, squash and then merge.

:+1:


Model_not_found error

Hi there,
I got the below error in my first try with the ChatGPT Assistant pluggin, can you please have a look ?

Reproduce:

Open Axon.Ivy Version: 10.0.17 (Weissfluh)
Install the ChatGPT Assistant version 10.0.8 pluggin, from the designer's Axon.Ivy market.
Set apiKey as the image

Select a jsf tag on xhtml view, and click on ChatGPT Assistant icon > Explain
Finally it shows the error dialog

Thanks.

Hi @ivy-sgi
can we address this in the current sprint?
seems as if we simply have to switch our default engine.
more and more users are running into this problem https://community.axonivy.com/d/606-error-on-using-chatgpt-assistant

I assume the the "edit" context action still works; as it uses the davinci4 model: https://github.com/axonivy-market/openai-connector/blob/master/openai-assistant/src/com/axonivy/connector/openai/assistant/ChatGptRequest.java#L72
while all other actions lead to the now deprecated davinci3 model: https://github.com/axonivy-market/openai-connector/blob/master/openai-assistant/src/com/axonivy/connector/openai/assistant/ChatGptRequest.java#L51

fixed with https://github.com/axonivy-market/openai-connector/pull/43
released as 10.0.9: https://market.axonivy.com/openai-assistant


(Question) Invert single steer motor

How would i make a single steer motor on my swerve chassis un inverted  while keeping the rest inverted on mk4i imodules

This should not be needed. Either all of the modules should be inverted or none. What I think is most likely is that when determining the offsets for the wheels, the bevel gears weren't all facing the same direction (most likely). More details are available in the Tuning section of the README.

Our swerve is configured with MK four I swerve modules, however, due to a
air by our mechanical team, we have had to make two of the motors facing
upwards. These two motors are our front left steer and our front right
drive.
On Sun, Mar 5, 2023 at 10:08 PM Geoff Schmit @.***>
wrote:

This should not be needed. Either all of the modules should be inverted or
none. What I think is most likely is that when determining the offsets for
the wheels, the bevel gears weren't all facing the same direction (most
likely). More details are available in the Tuning
https://github.com/HuskieRobotics/3061-lib#tuning section of the README.
—
Reply to this email directly, view it on GitHub
https://github.com/HuskieRobotics/3061-lib/issues/35#issuecomment-1455368893,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AVILYRSP3OUAZ6CYH3XG3IDW2VILZANCNFSM6AAAAAAVOWS42Y
.
You are receiving this because you authored the thread.Message ID:
@.***>

That isn't currently supported in the code. You could modify the SwerveModuleIOTalonFX class to set the various inverted variables based on the moduleNumber parameter passed to the constructor.

ok i can try to to do something like that
On Sun, Mar 5, 2023 at 10:23 PM Geoff Schmit @.***>
wrote:

That isn't currently supported in the code. You could modify the
SwerveModuleIOTalonFX class to set the various inverted variables based on
the moduleNumber parameter passed to the constructor.
—
Reply to this email directly, view it on GitHub
https://github.com/HuskieRobotics/3061-lib/issues/35#issuecomment-1455383242,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AVILYRTJP24JTGNW3QUOSUDW2VKC5ANCNFSM6AAAAAAVOWS42Y
.
You are receiving this because you authored the thread.Message ID:
@.***>


Notifications only occur under Modern Standby when the device lid is open, not when it is shut.

https://docs.microsoft.com/en-us/windows-hardware/design/device-experiences/audio-notifications?source=docs States you receive an audio notification while the device is in modern standby. This only  occurs when the device's lid is open, not when it is shut. Same applies to Skype calls, btw. Documentation ought to reflect this.

@domcote Thanks for taking the time to leave docs feedback.
I've flagged this issue for a member of the Windows docs team for awareness and potential fix.
Appreciate your contribution to the docs community.
@GrantMeStrength Please take a look and re-assign as needed.

The documentation has been updated to reflect this. Thank you!


Add transferwise variant of tap-postgres

Migrated from GitLab: https://gitlab.com/meltano/meltano/-/issues/2410
Originally created by @DouweM on 2020-11-05 23:14:02

https://github.com/transferwise/pipelinewise-tap-postgres
https://transferwise.github.io/pipelinewise/connectors/taps/postgres.html

View 4 previous comments from the original issue on GitLab


Update shebang to /sbin/openrc-run

Use /sbin/openrc-run instead of the now deprecated /sbin/runscript.
See https://bugs.gentoo.org/show_bug.cgi?id=494220 for upstream change that deprecated runscript and https://bugs.gentoo.org/show_bug.cgi?id=573846 for additional info.

Cancelling to consolidate more changes into a single pull request.


pd_client: reduce PD reconnection

What is changed and how it works?

Issue Number: Close #14964, ref https://github.com/tikv/pd/issues/6556
What's Changed:
The current setting of reconnect PD interval is hardcode. If the request failed, it may raise lots of requests to get a member list of PD which could bring huge pressure to etcd. This PR enlarges the retry interval and makes it configurable.

Check List
Tests

Unit test

Release note
support configuring the reconnection interval for the PD client

/cc @overvenus @tabokie Would you like to take a look?

Could you describe why we need this PR?

Could you describe why we need this PR?

Done and I will create a more detailed issue later.

The change is not as straightforward as the title says. We actually have four different values to control the reconnect:

GLOBAL_RECONNECT_INTERVAL(0.1s): global reconnect interval
REQUEST_RECONNECT_INTERVAL(1s): per-request reconnect interval
REQUEST_TIMEOUT(2s): v1 request timeout, v2 request timeout & global reconnect interval
retry_interval(0.3s): reconnect interval when init or tso stream failed

This PR changes the global reconnect interval from 0.1s to 1s. I think it's too big. How about keep it within 4x, i.e. make the interval 0.4s for now.
Also in v2 we use REQUEST_TIMEOUT to control the reconnect. It should be updated as well.

These configs will be limited by GLOBAL_RECONNECT_INTERVAL in the last, How about just changing it to 300ms, which is the same as right now?

/merge

/merge

There is a question related to this PR that has been bothering me. Is the connection of PD shared by one TiKV in a connection pool, and then all regions on this TiKV share this connection pool to send heartbeats to PD?
If the GLOBAL_RECONNECT_INTERVAL is set to 200 ms, does it mean that memberlist will be attempted to be retrieved at most 5(1s/200ms) within 1 second? However, who can tell me why I see a large qps in tikv->pd->reconnect->try

@tabokie @rleungx

There is a question related to this PR that has been bothering me. Is the connection of PD shared by one TiKV in a connection pool, and then all regions on this TiKV share this connection pool to send heartbeats to PD? If the GLOBAL_RECONNECT_INTERVAL is set to 200 ms, does it mean that memberlist will be attempted to be retrieved at most 5(1s/200ms) within 1 second? However, who can tell me why I see a large qps in tikv->pd->reconnect->try  @tabokie @rleungx

They are canceled.


Include maven tests are part of test process.

Coverage remained the same at 23.007% when pulling 40299bd92a4602dfef923470c0e8e07e13691afe on ChristianMurphy:NOJIRA-additional-appveyor-testing into 81a37673b341a3ee7dc8dd8e98d357f94536909d on Jasig:master.

Coverage remained the same at 23.007% when pulling 40299bd92a4602dfef923470c0e8e07e13691afe on ChristianMurphy:NOJIRA-additional-appveyor-testing into 81a37673b341a3ee7dc8dd8e98d357f94536909d on Jasig:master.

Good news, the test setup works.
Bad news it uncovered a regression.
testDetermineImportOnUpdatePaths(org.apereo.portal.tenants.TemplateDataTenantOperationsListenerTest)  Time elapsed: 0.006 sec  <<< FAILURE!
java.lang.AssertionError: Simple path replacement incorrect expected:<[classpath:/org/apereo/portal/tenants/data/group_membership/Administrators.group-membership.xml, classpath:/org/apereo/portal/tenants/sampledata/pags-group/Members.pags-group.xml]> but was:<[classpath:\org\apereo\portal\tenants\sampledata\pags-group\Members.pags-group.xml, classpath:/org/apereo/portal/tenants/data/group_membership/Administrators.group-membership.xml]>
	at org.junit.Assert.fail(Assert.java:88)
	at org.junit.Assert.failNotEquals(Assert.java:743)
	at org.junit.Assert.assertEquals(Assert.java:118)
	at org.junit.Assert$assertEquals.callStatic(Unknown Source)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallStatic(CallSiteArray.java:53)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:157)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:173)
	at org.apereo.portal.tenants.TemplateDataTenantOperationsListenerTest.testDetermineImportOnUpdatePaths(TemplateDataTenantOperationsListenerTest.groovy:48)


[TUTORIAL] Tutorial for Vision Operators

now that #1095 and #1397 is added, we have a comprehensive set of vision operators in topi. It would be great if we can write a tutorial for these operators

cc @zhreshold @Laurawly @kevinthesun

@tqchen Since @kevinthesun has a tutorial here: https://docs.tvm.ai/tutorials/nnvm/deploy_ssd.html#sphx-glr-tutorials-nnvm-deploy-ssd-py
Should I update this one for the GPU SSD or write a new one?

We just have to update https://docs.tvm.ai/tutorials/nnvm/deploy_ssd.html#sphx-glr-tutorials-nnvm-deploy-ssd-py to introduce GPU SSD.
What I mean, however, is the topi library calls, like https://docs.tvm.ai/tutorials/topi/intro_topi.html#sphx-glr-tutorials-topi-intro-topi-py, instead of the nnvm ops.

@tqchen You mean adding a section named "vision operators" into that tutorial?

close for now due to inactive status


Support of Table of Contents

Is there a possibility to make a Table of Contents from notion work?
I am using notion-to-md for my blog and wanted to see what can be done

Not sure how the package processess the table of contents however table of contents is similar to list of links.
We can return table of contents as links so that they can be used as the user wants.
your thoughts?

Figured out a way to do this if the usecase is for blogs. Users can use something like rehype toc to convert the mdnString to a TOC or write a custom function.
For now, I wrote a Custom Function since rehype is not that flexible with styling and other things like being sticky after the scroll.
I think there is no support needed from notion-to-md


[ISSUE #4033] Skip resending async deliver schedule message when service is not available

Make sure set the target branch to develop
What is the purpose of the change
Fix #4033
Brief changelog
Skip resending async deliver schedule message when service is not available
Verifying this change
XXXX
Follow this checklist to help us incorporate your contribution quickly and easily. Notice, it would be helpful if you could finish the following 5 checklist(the last one is not necessary)before request the community to review your PR.

[x] Make sure there is a Github issue filed for the change (usually before you start working on it). Trivial changes like typos do not require a Github issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue.
[x] Format the pull request title like [ISSUE #123] Fix UnknownException when host config not exist. Each commit in the pull request should have a meaningful subject line and body.
[x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
[x] Write necessary unit-test(over 80% coverage) to verify your logic correction, more mock a little better when cross module dependency exist. If the new feature or significant change is committed, please remember to add integration-test in test module.
[x] Run mvn -B clean apache-rat:check findbugs:findbugs checkstyle:checkstyle to make sure basic checks pass. Run mvn clean install -DskipITs to make sure unit-test pass. Run mvn clean test-compile failsafe:integration-test  to make sure integration-test pass.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

May cause data loss, consider how to terminate the thread instead of skipping the message.

@HScarb #4045 Can you review it?


[Bug] Send to **** not working in more than one control net

Once you have generated a pose and set the Sent To ControlNet
Set each out put to a different ControlNet 0 and 1 for example
Once you click Send to Text to Image, it only sends data to ControlNet 0

Windows 10
Chrome
AUTOMATIC1111 v1.3.2
ControlNet v1.1.220
python: 3.10.6
torch: 2.0.1+cu118
xformers: 0.0.17
gradio: 3.32.0
checkpoint: c0d1994c73

Hi everyone!
I wanted to let you know that this issue has been resolved. You can find a simple and working solution at this https://github.com/nonnonstop/sd-webui-3d-open-pose-editor/issues/115 . I have tested it myself, and it works.


removing recorder references

In line with the new plugins, we aren't going to want the CI's to create dependency lists anymore.

@akamen @jrichardBD @psantos1113 @stevebillings

Coverage increased (+5.03%) to 75.054% when pulling 96b2db304e37a58ad97c81a468d1545c7d1ac223 on ekerwin_removing_recorder_references into da8455f1ebe2b6bf7e68dac48983bcd32711f723 on master.


Cython 3.0 support

What:
Extend the builtin testing to cover function/methods created by Cython 3.0
Exempt builtins from signature validation since its mostly broken for builtins around keyword arguments.
Changed inspect.iscoroutinefunction to asyncio.iscoroutinefunction
Why:
at Facebook I am trying to upgrade from Cython 0.29.x to Cython 3.0a6.  Many things change under the hood that cause things like inspect module to behave differently.   One thing Cython functions and methods are no longer "method_descriptors" they are a new thing called "cython_function_or_method"
Also inspect.signature on methods/functions in Cython 0.29.x would raise ValueError.  In Cython 3.0 we get a signature but it believes keyword arguments are all required.   I remember my last pull request for cpython there was this allow/deny list for opting out certain builtins for signature validation. I figure if even the stdlib can't rely on testing the signatures of builtins then testslide can make do with ignoring signature validation on builtins.
Also around the iscoroutinefunction.  The next Cython 3.0 release "3.0a7" async def functions will return True when used with asyncio.iscoroutinefunction, but will return False when used with inspect.iscoroutinefunction.  Since the asyncio variant also calls the inspect variant it seems wise to use it instead.
How:
Extend the previous builtins detection logic to include the new cython_function_or_method.  Also use this logic for exempting builtins from _validate_callable_signature.  Cleaned up some code copies by extracting the logic into a function.
Risks:
Some builtin using C-api might return valid signatures from a inspect.signature call.
Checklist:

[ ] Added tests, if you've added code that should be tested
[ ] Updated the documentation, if you've changed APIs
[X] Ensured the test suite passes
[X] Made sure your code lints
[X] Completed the Contributor License Agreement ("CLA")

With these changes I was able to get pre-written testslide based tests to pass on cython modules built under 3.0a6

Pull Request Test Coverage Report for Build 742010833

10 of 11 (90.91%)  changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.02%) to 94.135%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

testslide/lib.py
3
4
75.0%

Totals

Change from base Build 708713380:
-0.02%

Covered Lines:
2568

Relevant Lines:
2728

💛  - Coveralls


Proof of concept, argument  to export_to other than  &'static str

Hi there,
and thank you for this crate!
In addition to a 'static str as the argument to export_to attribute
we could use the following types :

an environment variable defined in .cargo/config.toml

#[ts(export_to = env("CONFIG_ENV_VAR_PATH"))]

TS::output_path method generated for the type:
    fn output_path() -> Option<&'static std::path::Path> {
        let path = if std::env!("CONFIG_ENV_VAR_PATH").ends_with('/') {
            std::concat!(std::env!("CONFIG_ENV_VAR_PATH"), "ObjB", ".ts")
        } else {
            std::env!("CONFIG_ENV_VAR_PATH")
        };
        Some(std::path::Path::new(path))
    }

name of a function (Fn(&'static str) -> Option<&'static str>) that will override the standard TS::output_path method logic.

example:

// ts_name is provided by TS macro (name of the ts type)

pub fn get_path_fn(ts_name: &'static str) -> Option<&'static str> {
    match ts_name {
        "ObjA" => Some("../ts-fn-path/ObjA.ts"),
        "ObjB" => Some("../ts-fn-path/B/ObjB.ts"),
        _ => None,
    }
}

the argument is an identifier or a path to our function
#[ts(export_to = get_path_fn)]

#[ts(export_to = crate::get_path_fn)]

TS::output_path method generated for the ( ts name ObjB):
    fn output_path() -> Option<&'static std::path::Path> {
        Some(std::path::Path::new(crate::get_path_fn("ObjB")?))
    }

For users of Rust 1.7.0 and higher const and static names

static MY_STATIC_PATH: &'static str = "../ts-static-path/";
const MY_CONST_PATH: &'static str   = "../ts-const-path/";

again as  identifier or path
#[ts(export_to=crate::MY_STATIC_PATH)]
#[ts(export_to=MY_CONST_PATH)]

TS::output_path method generated for the type:
    fn output_path() -> Option<&'static std::path::Path> {
        static PATH: std::sync::OnceLock<String> = std::sync::OnceLock::new();
        let path = PATH
            .get_or_init(|| {
                if crate::MY_CONST_PATH.ends_with('/') {
                    format!("{}{}.ts", crate ::MY_CONST_PATH, "ObjA")
                } else {
                    format!("{}", crate ::MY_CONST_PATH)
                }
            });
        Some(std::path::Path::new(path))
    }

All options are reactive, modifying the source of argument value
will make TS export to the new path without need to recompile.
Thinking about the most efficient or correct way to handle paths can be overwhelming, as there's no universally right way to do it! My proposition is to allow users to manage paths using methods already established by the programming community: constants, statics, and environment variables. Unfortunately, due to Rust's limitations for Derive macros and the 'test' runtime, this is currently restricted to those defined in .cargo/config.toml.

[ ] I have followed the steps listed in the Contributing guide.
[ ] If necessary, I have added documentation related to the changes made.
[ ] I have added or updated the tests related to the changes made.

Hey, thanks for opening this PR!
Is there a real use-case behind this feature, or is this more of a "it'd be really cool if this worked" kind of thing?
We do already allow altering the export directory by setting the TS_RS_EXPORT_DIR variable. The idea here is that the exported bindings always have the same directory structure, and we only allow users to specify where the bindings will end up.
I do think there are some advantages of keeping the directory structure static. For more exotic usecases (like custom build scripts), there's TS_RS_EXPORT_DIR.

Hi NyxCode!
It's not that providing a real-life example is difficult,
but any example could be challenged and redirected to a workflow where
the argument is a &'static str.
What is the point for such restriction in a development tool ?
One general downside of using a &'static str as a path is that in case of change
it forces the user to manually change the export_to argument for
every instance of the same path value. This limits flexibility
by ignoring the concept of a variable, which is fundamental in programming.
ts-rs is a valuable development tool, and if it requires a
path to write to, IMO it should be flexible enough to accommodate
the various ways developers handle paths.

We do already allow altering the export directory by setting the TS_RS_EXPORT_DIR variable. The idea here is that the exported bindings always have the same directory structure, and we only allow users to specify where the bindings will end up.

Looks like we're talking about two different aspects here.
I understand your concerns over TS_RS_EXPORT_DIR crate convention.
But one can still define a path (as &static str) relative to TS_RS_EXPORT_DIR that’s outside of it. The risk of the writer deleting custom code to make room for a TS-generated type is the same regardless of whether the path was provided as a const or &static str.
Honestly, I didn’t even consider any side effects while implementing this PR; my approach was "Just trust the trait and make sure TS::output_path returns what the user provided
as a path value".
The only purpose of this PR is to challenge the type of input for the export_to argument.
Especially after the latest releases, users (myself included) can now write multiple types to the same file.
So why should I have to provide the same path as a &'static str for, say, three different types if I want them written in the same file?
I this case changing my mind about TS types location, I have two options:
a) Copy and paste the new path to each of my three objects, recompile, and run test again.
b) Physically move the types file to the desired location and sort out the import paths if necessary.
However, if the path is defined as lets say an environment variable, all I need to do is:

Delete the old file with the ts types.
Change the environment variable's value and run test.

So yes, the arguments might look cooler, but there's more than that. Typically, for a group of objects, we define shared variable instead of hard-coding values. The compiler only checks these variables, and in the case of a &'static str, it will silently accept one with a typo, increasing the risk of bugs.
TS_RS_EXPORT_DIR convention is something that I simply didn't take in account, possibly it may be confusing for the user I don't know, the only point of this PR is the type of input for export_to.

This looks really interesting! 2 questions though:

You've labeled this PR as a proof of concept. Why is that? Are there any changes you think need to be made?
Can you add some tests? That'll help make sure the feature doesn't break in the future

Hi @gustavo-shigueo  !
One more thing to consider is documenting the type options. The ts-rs documentation is already quite dense with terms, and adding more might make it harder for users to remember any.
Plus the fact that macros are really bad when it comes to documenting.
I propose to introduce an easy to remeber notation like this #[ts(export_to= ? )]
Which will resolve compile-time error that includes comprehensive information about the
TS_RS_EXPORT_DIR convention and path argument types.
Bassically a quick reminder.
Compared to crate documentation this could be more efficient as the information is provided at the right place and time.

I propose to introduce an easy to remeber notation like this #[ts(export_to= ? )]
Which will resolve compile-time error that includes comprehensive information about the
TS_RS_EXPORT_DIR convention and path argument types.

That may not be necessary, the error message you've written already seems to serve that purpose quite well, so if we make sure to display it for any syntax error in export_to, that should lead the user in the right direction

Before too much effort is put into this, i think we should answer the question if we really want this feature.
I am yet to be convinced of the benefit, and I dislike the complexity this adds.

I see, I do kinda like the idea of allowing consts and statics on top of &'static str. I don't really like the function path or even the environment variables though.
As for the complexity, it would go way down if only static/const were added, but I'm also not sure it'd be worth it

Sorry for the delay, I can't actively includ myself in conversation right now ( will be back in couple of days).
Think of other proposition, I'll write later some arguments in favour of keeping all variants of Custom Path and and probably the notation I've proposed, we could just include it as a separate feature (ex."typed path") not that I've donne it but I think it should be relatively
easy to implement as Custom Path  is well modulate already, very litle friction with the rest of the crate.
Think of pros and cons in this direction, befor I write the arguments that will conclude  to this option.

This PR was initiated to demonstrate the possible use of different types for the export_to attribute argument. The current version of the crate accepts only &'static str, which may be sufficient for most users, especially for small projects. However, larger projects are likely to encounter limitations.
Naming types, modules, and variables is always important, and the right names often emerge as the project evolves. Therefore, having a flexible approach to renaming both the types of arguments and the values of those arguments is a common and necessary practice for programmers.
Since I didn't have a clear initial objective for this PR, I think it would be helpful to summarize my proposition in one clear statement.
Allowed Types:

static and const variables.
a function pointer to override the TS::output_path method.
environment variables declared in .cargo/config.toml.
No support for normal environment variables (std::env::var) due to limitations in the Derive macro and test runtime.

In my opinion, using a typed path is safer for various reasons. Limiting the crate to the use of &'static str to avoid potential bugs introduces other issues, particularly related to the inert nature of &'static str in the compiler.
The current PR aims to assist in two major ways:

Renaming a type will trigger error messages in the text editor for every module where the type was used, reducing the effort needed to resolve issues — a very common workflow.

All variants of CustomPath are directly accessed within TS::output_path, meaning any value changes will be instantly accessible by TS::output_path without needing to recompile.

The crate's responsibility is limited to two general cases:

Informing the user about the TS_RS_EXPORT_DIR crate convention and how TS::output_path modifies the initial path. This may include details about potential issues that could arise when working outside of the convention.

Note: This point is base for ? notation for export_to propossed earlier.

Preventing any misinterpretation of type variants of CustomPath

Additionally, I believe that restricting the number of variants, whatever they may be, lacks sufficient justification. Since there isn't a universally efficient approach, any type restriction is often unnecessary. The type of the path itself isn't likely to cause bugs; rather, it's the type value that could lead to issues, which is entirely the user's responsibility.
Consider either accepting Custom Path for ts-rs or including it as a separate feature that allows typed path arguments for the export_to attribute.
I've made my point, from here, it's up to you to decide what is acceptable so we can move forward with more precise code, some tests, and ensure that the PR aligns with project requirements.

There was a bug, it wasn't working for the environment variables, after refactoring. 😅
Sorry for the confusion if any!


[FEATURE] Introduce dynamic disk selection for one partition in LOCALFILE

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Search before asking

[X] I have searched in the issues and found no similar issues.

Describe the feature
Introduce the dynamic disk selection for one partition in LOCALFILE, especially when some disks reached the high-watermark.
Motivation
Currently, when mounting multiple disks in shuffle-server and using the MEMORY_LOCALFILE storage type, one app has a very large partition data and its selected disk reaches the high watermark, it will preserve the data into  the memory. This will make some app hard to get the memory.
Describe the solution
Maybe we should let the its partial big partition data store to the another disk.
Additional context
No response
Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

If you use MEMORY_LOCALFILE_HDFS, it won't be a big problem.

If you use MEMORY_LOCALFILE_HDFS, it won't be a big problem.

Yes. This is an optional solution.

@jerqi As the raid disk has been tested and shown that there is performance degradation while using raid0/LVM. I will pick this issue up to implement a multiply disk store approach for one partition.


Update readme with info on heroku deploy

On discord, @msklyarov asked about deploying origin-js changes on heroku.
This was my rough answer. Should be cleaned up and put in readme or docs.

As you discovered, a heroku instance can't get the full dev environment that we have around origin-js.  That is, change this line to point to your repo#branch:
https://github.com/OriginProtocol/demo-dapp/blob/develop/package.json#L16
If you've changed the contracts (*.sol) files you'll need to deploy those to ropsten/rinkeby. Ask me if you need help with that.
...actually, this should be in our documentation!

@wanderingstan
When I'm trying to add npm as a GitHub branch I'm getting an error:
$ cd demo-dapp
$ npm uninstall origin -S
$ npm i https://github.com/OriginProtocol/origin-js -S
$ npm start

<EMAIL_ADDRESS>start /home/maxim/WebstormProjects/Jolyy/demo-dapp
if [ "$NODE_ENV" = "production" ]; then npm run serve; else npm run dev; fi

<EMAIL_ADDRESS>dev /home/maxim/WebstormProjects/Jolyy/demo-dapp
webpack-dev-server --open

Failed to load ./.env.
clean-webpack-plugin: /home/maxim/WebstormProjects/Jolyy/demo-dapp/build has been removed.
events.js:183
throw er; // Unhandled 'error' event
^
Error: listen EADDRINUSE <IP_ADDRESS>:3000
at Object._errnoException (util.js:992:11)
at _exceptionWithHostPort (util.js:1014:20)
at Server.setupListenHandle [as _listen2] (net.js:1355:14)
at listenInCluster (net.js:1396:12)
at GetAddrInfoReqWrap.doListen [as callback] (net.js:1505:7)
at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:97:10)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR<EMAIL_ADDRESS>dev: webpack-dev-server --open
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>dev script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR!  Exit status 1
or:
$ npm start

<EMAIL_ADDRESS>start /home/maxim/WebstormProjects/Jolyy/demo-dapp
if [ "$NODE_ENV" = "production" ]; then npm run serve; else npm run dev; fi

<EMAIL_ADDRESS>dev /home/maxim/WebstormProjects/Jolyy/demo-dapp
webpack-dev-server --open

clean-webpack-plugin: /home/maxim/WebstormProjects/Jolyy/demo-dapp/build has been removed.
events.js:183
throw er; // Unhandled 'error' event
^
Error: listen EADDRINUSE <IP_ADDRESS>:3000
at Object._errnoException (util.js:992:11)
at _exceptionWithHostPort (util.js:1014:20)
at Server.setupListenHandle [as _listen2] (net.js:1355:14)
at listenInCluster (net.js:1396:12)
at GetAddrInfoReqWrap.doListen [as callback] (net.js:1505:7)
at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:97:10)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR<EMAIL_ADDRESS>dev: webpack-dev-server --open
npm ERR! Exit status 1
with .env.dev renamed to .env
===============================================
$ cat ./2018-05-23T10_25_43_232Z-debug.log
0 info it worked if it ends with ok
1 verbose cli [ '/usr/bin/node', '/usr/bin/npm', 'start' ]
2 info using<EMAIL_ADDRESS>3 info using<EMAIL_ADDRESS>4 verbose run-script [ 'prestart', 'start', 'poststart' ]
5 info lifecycle<EMAIL_ADDRESS><EMAIL_ADDRESS>6 info lifecycle<EMAIL_ADDRESS><EMAIL_ADDRESS>7 verbose lifecycle<EMAIL_ADDRESS>unsafe-perm in lifecycle true
8 verbose lifecycle<EMAIL_ADDRESS>PATH: /usr/lib/node_modules/npm/node_modules/npm-lifecycle/node-gyp-bin:/home/maxim/WebstormProjects/Jolyy/demo-dapp/node_modules/.bin:/home/maxim/bin:/home/maxim/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
9 verbose lifecycle<EMAIL_ADDRESS>CWD: /home/maxim/WebstormProjects/Jolyy/demo-dapp
10 silly lifecycle<EMAIL_ADDRESS>Args: [ '-c',
10 silly lifecycle   'if [ "$NODE_ENV" = "production" ]; then npm run serve; else npm run dev; fi' ]
11 silly lifecycle<EMAIL_ADDRESS>Returned: code: 1  signal: null
12 info lifecycle<EMAIL_ADDRESS>Failed to exec start script
13 verbose stack Error<EMAIL_ADDRESS>start: if [ "$NODE_ENV" = "production" ]; then npm run serve; else npm run dev; fi
13 verbose stack Exit status 1
13 verbose stack     at EventEmitter. (/usr/lib/node_modules/npm/node_modules/npm-lifecycle/index.js:285:16)
13 verbose stack     at emitTwo (events.js:126:13)
13 verbose stack     at EventEmitter.emit (events.js:214:7)
13 verbose stack     at ChildProcess. (/usr/lib/node_modules/npm/node_modules/npm-lifecycle/lib/spawn.js:55:14)
13 verbose stack     at emitTwo (events.js:126:13)
13 verbose stack     at ChildProcess.emit (events.js:214:7)
13 verbose stack     at maybeClose (internal/child_process.js:925:16)
13 verbose stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:209:5)
14 verbose pkgid<EMAIL_ADDRESS>15 verbose cwd /home/maxim/WebstormProjects/Jolyy/demo-dapp
16 verbose Linux 4.13.0-39-generic
17 verbose argv "/usr/bin/node" "/usr/bin/npm" "start"
18 verbose node v8.11.2
19 verbose npm  v5.6.0
20 error code ELIFECYCLE
21 error errno 1
22 error<EMAIL_ADDRESS>start: if [ "$NODE_ENV" = "production" ]; then npm run serve; else npm run dev; fi
22 error Exit status 1
23 error Failed at the<EMAIL_ADDRESS>start script.
23 error This is probably not a problem with npm. There is likely additional logging output above.
24 verbose exit [ 1, true ]
Could you please advise what I need to change?

Have you tried just editing the package.json directly? That's what we usually do.

It starts but can't find any contracts.
I replaced string in package.json:
"origin": "git+https://github.com/OriginProtocol/origin-js.git"
rm node_modules
npm i
npm start

@wanderingstan the situation is similar to previous one:

Note: This issue would be helped by debug output: https://github.com/OriginProtocol/origin-js/issues/217
Also: @msklyarov - Remind me again what it is that you're trying to accomplish?

It starts but can't find any contracts.
I replaced string in package.json:
"origin": "git+https://github.com/OriginProtocol/origin-js.git"

Ah, of course! This is referring to our github repo, but the repo does not contract addresses committed in it. The contracts have to be deployed first (usually to ropsten or rinkeby) and this will generate a /contracts/build/ directory with json files containing info for each contract.
These files are not committed to git.
That's the heart of the problem. Thanks for bringing that to our attention.
I think the only ways to make this work are:

(Preferred) manually set the contract addresses with the CONTRACT_ADDRESSES environment variable. (This can be set in heroku)
Commit /contracts/build directory to your repo. Either self-generated or copied from e.g. the published npm package. (Not 100% sure this would work.)

The CONTRACT_ADDRESSES var is not well documented. You can get the flavor of it here:
https://github.com/OriginProtocol/origin-js/blob/49fe587c5fe1380015ec35422edb2fed857312f6/test/contract-service.test.js#L102-L107
So looks like we'd need to assemble a json blob containing all of our deployed contract addresses. I don't think there's a way to get this aside from manual work right now. :(
CONTRACT_ADDRESSES
{
  userRegistryContract: { 4: { address:"0x1234567890123456789012345678901234567890" } },
  listingsRegistryContract: { 4: { address:"0x9876543210987654321098765432109876543210" } } ,
  <more contracts here>
}

@nick Is there any easy way to assemble all our contract addresses? Maybe outputting this json blob would be a good function for issue #217 ?

So far I've been assembling them manually from your contract deployment output when we do a release.
Perhaps we could have an origin.info() method which outputs things like version, contract addresses, bridge server address, IPFS address, etc which can be used in the browser console. We could even print it to the console when starting the demo-dapp with npm start

@wanderingstan
package.json:
"origin": "git+https://github.com/OriginProtocol/origin-js.git"
$ export CONTRACT_ADDRESS="{ userRegistryContract: { 4: { address:"0x1234567890123456789012345678901234567890" } },  listingsRegistryContract: { 4: { address:"0x9876543210987654321098765432109876543210" } } }"
$ npm i
$ npm start
Same issue on localhost:
The Origin Contract was not found on this network.
You may need to change networks, or deploy the contract.

@wanderingstan
package.json:
"origin": "git+https://github.com/OriginProtocol/origin-js.git"
$ export CONTRACT_ADDRESS="{ userRegistryContract: { 4: { address:"0x1234567890123456789012345678901234567890" } }, listingsRegistryContract: { 4: { address:"0x9876543210987654321098765432109876543210" } } }"
$ npm i
$ npm start
Same issue on localhost:
The Origin Contract was not found on this network.
You may need to change networks, or deploy the contract.
But it works with heroku without setting of config var.

@msklyarov

But it works with heroku without setting of config var.
I thought getting this working on heroku was the goal? I'm surprised it would work there without setting the CONTRACT_ADDRESS env var. Are you sure your heroku deploy isn't using the npm package?

I'm also not sure what contract it "tests" to determine if Origin contracts are there. I do know that there are a lot more than just those two contracts (userRegistryContract and listingsRegistryContract, however.) So that could be what's causing the problem locally.
Again, what is it that you're trying to accomplish? Or are you just testing the use of a github branch instead of the npm module?

Or are you just testing the use of a github branch instead of the npm module?
Right. If it'll work for unmodified branches then I'll move to the next step.

Error: listen EADDRINUSE :::3000
at Object._errnoException (util.js:992:11)
at _exceptionWithHostPort (util.js:1014:20)
at Server.setupListenHandle [as _listen2] (net.js:1355:14)
at listenInCluster (net.js:1396:12)

@azizsallam can you give more context on your error?
Otherwise I'm thinking we close this issue.

@azizsallam that error will occur if you have something else running on port 3000 (the port used by origin-dapp, I'm guessing you received that error when starting origin-dapp rather than origin-js).

Going to call this one closed, as there hasn't been activity in a while.


Add agriculture, forestry and fishing

Include energy demands and CO2 emissions for the agriculture, forestry and fishing sector.
Only CO2 emissions and energy uses for now. No other GHG or non-energy uses.
Can be activated with string "A" in sector_opts wildcard.
Part of the emissions (namely 1.A.4.c) was previously assigned to "industry non-elec" in the co2_totals.csv. Hence, excluding agriculture option "A" from the sector_opts will now lead to a tighter CO2 limit.
Agriculture, forestry and fishing demands are taken from JRC IDEES and are split into

electricity: lighting, ventilation, specific electricity uses, pumping devices (elec)
heat: specific heat uses, low enthalpy heat
machinery (oil): motor drives, farming machine drives, pumping devices (diesel)

Assumptions and Limitations:

heat demand assigned at "services rural heat" buses
time-series for demands are constant
nodal distribution of demands by population (inside country)
by default no fuel-switching of machinery from oil to electricity, but can be set exogenously in config.yaml as well as the fuel efficiency gains
missing values (e.g. Balkans) filled with eurostat data.
agriculture electricity demands are added to low-voltage bus

CO2 totals before and after
Previously, agriculture, forestry and fishing as per 1.A.4.c was covered in industrial non-elec.

Mt
old
new

electricity
1482
1482

residential non-elec
521
521

services non-elec
204
204

rail non-elec
14
14

road non-elec
743
743

domestic navigation
32
32

international navigation
111
111

domestic aviation
15
15

international aviation
72
72

industrial processes
335
335

agriculture
15
111

LULUCF
-290
-290

waste management
5
5

other
0
0

indirect
5
5

industrial non-elec
1071
976

Energy Demands + Emissions
Units

TWh (+ve)
Mt (-ve)

Perfect, thanks for spotting my negligence of colours!


Download selfinstall iso

SelfInstall Leap Micro test should boot iso instead of raw images.
More info in https://bugzilla.opensuse.org/show_bug.cgi?id=1197877

@jlausuch @nilxam

@andrii-suse @Martchus could you please merge?

Hm, it looks Default-SelfInstall is not synced anymore and has no isos post as well. It is certainly possible that such combination is not supported by generation, I will have a look

Ah it looks the xml has raw.xz for Default-SelfInstall , but .lst file has iso : Default-SelfInstall-Build16.23.install.iso . Which one is correct?

Made a mistake there, I should update only SelfInstall flavors. Thanks!


Consider naming improvements

Suggested naming clarifications since choices and submap are unified by the clearer term choicemap.

get_choices to get_choicemap.
set_submap to set_sub_choicemap
get_submap to get_sub_choicemap
get_submaps_shallow to get_sub_choicemaps_shallow.

Also we can make trace[:foo] implement get_sub_choicemap(trace, :foo).


readline() function in PipedGzipReader

Hello.
The readline function is not implemented in the PipedGzipReader class.
I think that implementing it is easy and it would be really useful. I think that implementation would be very similar to the read function:
def readline(self, *args):
    data = self._file.readline(*args)
    if len(args) == 0 or args[0] <= 0:
        # wait for process to terminate until we check the exit code
        self.process.wait()
    self._raise_if_error()
    return data

Correct me if I'm wrong.
Thanks and Regards.

Thanks for the suggestion! I’ve added this now exactly as you suggested, along with some unit tests. This will indeed be good to have.
If you want to contribute more and speed up getting your changes in, then you could also open a pull request. This makes it a bit easier to discuss code changes and I can just click "Merge" when I think the changes are ok.
Thanks again!


[SPARK-34116][SS][TEST] Separate state store numKeys metric test and validate metrics after committing

What changes were proposed in this pull request?

This patch proposes to pull the test of numKeys metric into a separate test in StateStoreSuite.
Why are the changes needed?

Right now in StateStoreSuite, the tests of get/put/remove/commit are mixed with numKeys metric test. I found it is flaky when I was testing with other StateStore implementation.
Current test logic is tightly bound to the in-memory map behavior of HDFSBackedStateStore. For example, put can immediately show up in the numKeys metric.
But for a StateStore implementation relying on external storage, e.g. RocksDB, the metric might be updated once the data is actually committed. And StateStoreSuite should be a common test suite for all kinds of StateStore implementations.
Specifically, we also are able to check these metrics after state store is updated (committed). So I think we can refactor the test a little bit to make it easier to incorporate other StateStore externally.
Does this PR introduce any user-facing change?

No, dev only.
How was this patch tested?

Unit test.

Test build #134062 has started for PR 31183 at commit c2bcdd8.

Test build #134062 has started for PR 31183 at commit c2bcdd8.

cc @HeartSaVioR @dongjoon-hyun @HyukjinKwon

cc @HeartSaVioR @dongjoon-hyun @HyukjinKwon

I'm fine with this change, but do you know what is the root cause of flakiness?

Current test logic is tightly bound to the in-memory map behavior of HDFSBackedStateStore. For example, put can immediately show up in the numKeys metric.
But for a StateStore implementation relying on external storage, e.g. RocksDB, the metric might be updated once the data is actually committed.

I'm fine with this change, but do you know what is the root cause of flakiness?

Current test logic is tightly bound to the in-memory map behavior of HDFSBackedStateStore. For example, put can immediately show up in the numKeys metric.
But for a StateStore implementation relying on external storage, e.g. RocksDB, the metric might be updated once the data is actually committed.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38648/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38648/

Then, could you revise the title a little more specifically by saying like Validate metrics after committing

Then, could you revise the title a little more specifically by saying like Validate metrics after committing

Updated the title. Thanks @dongjoon-hyun

Updated the title. Thanks @dongjoon-hyun

Test build #134062 has finished for PR 31183 at commit c2bcdd8.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test build #134062 has finished for PR 31183 at commit c2bcdd8.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Thank you, @viirya . Merged to master/3.1.

Thank you, @viirya . Merged to master/3.1.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134062/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134062/

LGTM2

LGTM2


OCM-10175 | test: fix id:67057

OCM-10175
$ ginkgo --focus 67057 tests/e2e/
Ginkgo detected a version mismatch between the Ginkgo CLI and the version of Ginkgo imported by your packages:
Ginkgo CLI Version:
2.11.0
Mismatched package versions found:
2.17.1 used by e2e
Ginkgo will continue to attempt to run but you may see errors (including flag
parsing errors) and should either update your go.mod or your version of the
Ginkgo CLI to match.
To install the matching version of the CLI run
go install github.com/onsi/ginkgo/v2/ginkgo
from a path that contains a go.mod file.  Alternatively you can use
go run github.com/onsi/ginkgo/v2/ginkgo
from a path that contains a go.mod file to invoke the matching version of the
Ginkgo CLI.
If you are attempting to test multiple packages that each have a different
version of the Ginkgo library with a single Ginkgo CLI that is currently
unsupported.
Running Suite: ROSA CLI e2e tests suite - /home/akanni/OCP-Repository/rosa/tests/e2e
Random Seed:<PHONE_NUMBER>
Will run 1 of 168 specs
SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS•SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
Ran 1 of 168 Specs in 38.040 seconds
SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 167 Skipped
PASS
Ginkgo ran 1 suite in 41.525803996s
Test Suite Passed

/test ci/prow/coverage

/test coverage

/retest

/retest

/retest

/lgtm

/test coverage


visual glitch on Mac

Just a quick heads up regarding strange visual behaviour on hover over a folder in safari. probably easy to fix if one knows what quirks webkit has, I don't unfortunately....
https://user-images.githubusercontent.com/5242870/142758720-96134db0-3826-4552-a629-efef9cb3e19e.mov

Can't be reproduced in firefox on macos.
Can be reproduced in safari. less buggy in safari tp.
Personally I wouldn't fix this issue. webkit always have problems where firefox and chromium work fine
https://user-images.githubusercontent.com/49815452/143240292-ff7c9543-0f08-4ae8-b3e7-b1d9772942bf.mov
https://user-images.githubusercontent.com/49815452/143240311-bfdddabf-2e08-4536-ab7b-df6ff6fbe8f5.mov