write: broken pipe

Since version 2.8.0, I'm getting the following error while running helm upgrade --install release_name chart:
E0209 11:21:52.322201   93759 portforward.go:303] error copying from remote stream to local connection: readfrom tcp4 <IP_ADDRESS>:53674-><IP_ADDRESS>:53680: write tcp4 <IP_ADDRESS>:53674-><IP_ADDRESS>:53680: write: broken pipe
Anyone got a hint, what could be a possible cause for this?
Edit: 2.8.1 does not fix this. I'm on MacOS

Same problem with linux 64bit client

Seeing the same thing; no apparent issues found in tiller log relating to the charts I'm deploying.

@albrechtsimon  @eldada   @matus-vacula  are you guys running on metal or in publicclouds?

@oivindoh I'm running on k8s in AWS. It used to be ok up until 2.7.2.

I use gke. Helm 2.8.1.

At reddit, we're seeing this happen every now and then and makes for a very bad experience. We use helm along with helmfile and it seems that when we "sync" (pretty much run a bunch of helm upgrades) we sometimes see an issue with broken pipes

So these errors might be grpc related and might not mean anything other than an overly verbose logger: https://github.com/grpc/grpc-go/issues/1362

Same issue running helm diff plugin. When I run multiple helm-diff command in a sequence I always got the error 38012 portforward.go:303] error copying from remote stream to local connection: readfrom tcp4 <IP_ADDRESS>:53526-><IP_ADDRESS>:53530: write tcp4 <IP_ADDRESS>:53526-><IP_ADDRESS>:53530: write: broken pipe. Running one by one always work

Still seeing this on 2.9.0 vs any and all clusters I have deployed on Azure via acs-engine.
E0427 10:11:49.952856   28158 portforward.go:303] error copying from remote stream to local connection: readfrom tcp4 <IP_ADDRESS>:36679-><IP_ADDRESS>:51582: write tcp4 <IP_ADDRESS>:36679-><IP_ADDRESS>:51582: write: broken pipe
/helm/2.9.0/x64/helm failed with return code: 0
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.0", GitCommit:"fc32d2f3698e36b93322a3465f63a14e9f0eaead", GitTreeState:"clean", BuildDate:"2018-03-27T00:13:02Z", GoVersion:"go1.9.4", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"9", GitVersion:"v1.9.7", GitCommit:"dd5e1a2978fd0b97d9b78e1564398aeea7e7fe92", GitTreeState:"clean", BuildDate:"2018-04-18T23:58:35Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}

$ helm version
Client: &version.Version{SemVer:"v2.9.0", GitCommit:"f6025bb9ee7daf9fee0026541c90a6f557a3e0bc", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.9.0", GitCommit:"f6025bb9ee7daf9fee0026541c90a6f557a3e0bc", GitTreeState:"clean"}

Having the exact same issue with helm-diff & helm 2.9.0

E0510 16:06:06.478944   19746 portforward.go:303] error copying from remote stream to local connection: readfrom tcp4 <IP_ADDRESS>:38750-><IP_ADDRESS>:41234: write tcp4 <IP_ADDRESS>:38750-><IP_ADDRESS>:41234: write: broken pipe
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.0", GitCommit:"fc32d2f3698e36b93322a3465f63a14e9f0eaead", GitTreeState:"clean", BuildDate:"2018-03-26T16:55:54Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.0", GitCommit:"fc32d2f3698e36b93322a3465f63a14e9f0eaead", GitTreeState:"clean", BuildDate:"2018-03-26T16:44:10Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}
$ helm version
Client: &version.Version{SemVer:"v2.8.2", GitCommit:"a80231648a1473929271764b920a8e346f6de844", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.8.2", GitCommit:"a80231648a1473929271764b920a8e346f6de844", GitTreeState:"clean"}

E0515 11:25:06.883101  170075 portforward.go:303] error copying from remote stream to local connection: readfrom tcp4 ***** write tcp4 <IP_ADDRESS>:38769-><IP_ADDRESS>:36224: write: broken pipe
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"9", GitVersion:"v1.9.7", GitCommit:"dd5e1a2978fd0b97d9b78e1564398aeea7e7fe92", GitTreeState:"clean", BuildDate:"2018-04-19T00:05:56Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"9", GitVersion:"v1.9.7", GitCommit:"dd5e1a2978fd0b97d9b78e1564398aeea7e7fe92", GitTreeState:"clean", BuildDate:"2018-04-18T23:58:35Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}
$ helm version
Client: &version.Version{SemVer:"v2.9.0", GitCommit:"f6025bb9ee7daf9fee0026541c90a6f557a3e0bc", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.9.0", GitCommit:"f6025bb9ee7daf9fee0026541c90a6f557a3e0bc", GitTreeState:"clean"}

+1

Just was dealing with similar issue and doing helm init --upgrade --history-max=0 seemed to fix it for me.
error:
E0531 14:54:27.094118   97288 portforward.go:303] error copying from remote stream to local connection: readfrom tcp4 <IP_ADDRESS>:53496-><IP_ADDRESS>:53498: write tcp4 <IP_ADDRESS>:53496-><IP_ADDRESS>:53498: write: broken pipe
Error: UPGRADE FAILED: "dev" has no deployed releases

kubectl version
Client Version: version.Info{Major:"1", Minor:"9", GitVersion:"v1.9.6", GitCommit:"9f8ebd171479bec0ada837d7ee641dec2f8c6dd1", GitTreeState:"clean", BuildDate:"2018-03-21T15:21:50Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"9", GitVersion:"v1.9.6", GitCommit:"9f8ebd171479bec0ada837d7ee641dec2f8c6dd1", GitTreeState:"clean", BuildDate:"2018-03-21T15:13:31Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}

helm version
Client: &version.Version{SemVer:"v2.8.2", GitCommit:"a80231648a1473929271764b920a8e346f6de844", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.8.2", GitCommit:"a80231648a1473929271764b920a8e346f6de844", GitTreeState:"clean"}

@ssalaues I think that only fixed your "Error: UPGRADE FAILED: "dev" has no deployed releases" issue (I'm guessing you only had a Failed release of "dev" before), not the portforward message, which appears to be very intermittent.

@oivindoh The strange thing is that I was getting the same portforward error regardless if I was doing a fresh helm install of a new deployment with new release name or trying to helm upgrade (which is what led me to this thread since it was stalling my work). In the particular case that I pasted in my previous message it was while testing a helm upgrade attempt (you're right had an issue with the previous release) but continued to get the portforward error when trying to do many other helm related commands on completely new releases.
Strange communication issue and hope it gets sorted out either way

i've seen this error just running helm list

Seeing this after upgrading from 2.7.2 to 2.9.1:
14:50:28 E0606 21:50:28.120334     271 portforward.go:303] error copying from remote stream to local connection: readfrom tcp4 <IP_ADDRESS>:44899-><IP_ADDRESS>:34138: write tcp4 <IP_ADDRESS>:44899-><IP_ADDRESS>:34138: write: broken pipe

in a 1.8.4 cluster. It didn't seem to affect the deploy, at least this time, but I'd rather not see errors in the deploy log if they're not relevant to the deploy.

Still seeing this regularly on k8s 1.10.4 + helm 1.9.1

Found in K8S v1.10.0 and helm v2.7.3
E0619 03:23:34.043302   22164 portforward.go:303] error copying from remote stream to local connection: readfrom tcp4 <IP_ADDRESS>:45405-><IP_ADDRESS>:47392: write tcp4 <IP_ADDRESS>:45405-><IP_ADDRESS>:47392: write: broken pipe
LAST DEPLOYED: Tue Jun 19 03:23:30 2018

Yes - it's still here when doing:
helm install helm/kube-prometheus --name kube-prometheus --namespace monitoring --tls
NAME:   kube-prometheus
E0709 18:06:25.276408    4526 portforward.go:303] error copying from remote stream to local connection: readfrom tcp4 <IP_ADDRESS>:56929-><IP_ADDRESS>:56931: write tcp4 <IP_ADDRESS>:56929-><IP_ADDRESS>:56931: write: broken pipe
LAST DEPLOYED: Mon Jul  9 18:06:21 2018
NAMESPACE: monitoring
STATUS: DEPLOYED

helm version --tls
Client: &version.Version{SemVer:"v2.9.1", GitCommit:"20adb27c7c5868466912eebdf6664e7390ebe710", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.9.1", GitCommit:"20adb27c7c5868466912eebdf6664e7390ebe710", GitTreeState:"clean"}

 kubectl version
Client Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.3", GitCommit:"2bba0127d85d5a46ab4b778548be28623b32d0b0", GitTreeState:"clean", BuildDate:"2018-05-28T20:03:09Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.3", GitCommit:"2bba0127d85d5a46ab4b778548be28623b32d0b0", GitTreeState:"clean", BuildDate:"2018-05-28T20:13:43Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}

helm list --tls
NAME               	REVISION	UPDATED                 	STATUS  	CHART                     	NAMESPACE
kube-prometheus    	1       	Mon Jul  9 18:06:21 2018	DEPLOYED	kube-prometheus-0.0.78    	monitoring
prometheus-operator	1       	Mon Jul  9 17:54:02 2018	DEPLOYED	prometheus-operator-0.0.25	monitoring

write: broken pipe looks like a networking error, where the tunneled connection between helm, kube-proxy and tiller gets interrupted. It's a little difficult to determine where (or what) is interrupting the connection from the error, though, which is coming from kubectl's port-forward APIs.
For anyone hitting this issue, can they try the following steps to manually set up a tunnel and see if they still see the error? This is how Helm initiates the connection to tiller.
$ TILLER_POD=$(kubectl get pods -n kube-system | grep tiller | awk '{ print $1 }')
$ kubectl -n kube-system port-forward $TILLER_POD 44134:44134
$ export HELM_HOST=:44134
$ helm list # or whatever command you were using when hitting this bug

Thanks!

@bacongobbler Not working for me

same issue as @Vhab;
using helm/tiller 2.11.0 with TLS and AKS 1.11.2.
helm client is being run via a vsts-agent docker image running in AKS; its part of several azure devops CD pipelines .I've also received the error running the helm client locally on Debian stretch.
I've only experienced the errors with helm upgrade and/or install commands and typically after the post deploy report
E1004 08:10:19.411418   14567 portforward.go:303] error copying from remote stream to local connection: readfrom tcp4 <IP_ADDRESS>:35543-><IP_ADDRESS>:41728: write tcp4 <IP_ADDRESS>:35543-><IP_ADDRESS>:41728: write: broken pipe

be good if there was a fix so my CD pipeline stops alerting a failure when the actually helm deployment works fine etc 😄

Same here, we get this error about 50% of the time when deploying to AKS from VSTS

I believe the AKS folks are aware of this bug and have hit it themselves. It appears to be an upstream issue, not necessarily a helm problem. I'll see if I can ping one of them and see if they can provide any updates on this ticket.

We are also seeing this frequently on non-AKS clusters.

I'm having the same issue with one of our tillers (3 in total) in our kubernetes cluster after upgrading from 2.9.1 to 2.11.0. All tillers are upgraded and only one has this issue.
After deleting a problematic tiller-deploy-xxxxxxx-yyyy pod which is being recreated by the deployment tiller-deploy, All helm commands are quick again. Only thing left is the error showing up 1 out of 5 times. (This is a 5 node cluster, so this might be correlated.)

@bacongobbler any update on this? Again Azure DevOps/VSTS hosted agent and AKS. Thanks.

The issue is now resolved (for now) on my side after upgrading to kubernetes v1.11.2. During this process i've also rebooted 2 of my 3 etcd/control-plane nodes. So i'm not sure whether it is the update or the reboot.

I've been trying to figure this out. So set system.debug=true in VSTS/Azure DevOps.
Since I have done that, not been able to reproduce. Can somebody else with VSTS/AKS give this a try?

still seeing this occur in our deployments, and causing some issues with a tool we're writing for Helm.
Version information if it helps at all:
ubuntu@kubenode01:/opt/flagship$ helm version
Client: &version.Version{SemVer:"v2.11.0", GitCommit:"2e55dbe1fdb5fdb96b75ff144a339489417b146b", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.11.0", GitCommit:"2e55dbe1fdb5fdb96b75ff144a339489417b146b", GitTreeState:"clean"}
ubuntu@kubenode01:/opt/flagship$ kubectl version
Client Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.3", GitCommit:"a4529464e4629c21224b3d52edfe0ea91b072862", GitTreeState:"clean", BuildDate:"2018-09-09T18:02:47Z", GoVersion:"go1.10.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.3", GitCommit:"a4529464e4629c21224b3d52edfe0ea91b072862", GitTreeState:"clean", BuildDate:"2018-09-09T17:53:03Z", GoVersion:"go1.10.3", Compiler:"gc", Platform:"linux/amd64"}
ubuntu@kubenode01:/opt/flagship$

Same issue for me, I get this error all the time (GKE k8s 1.11.5). Helm version 2.12.0.
Client: &version.Version{SemVer:"v2.12.0", GitCommit:"d325d2a9c179b33af1a024cdb5a4472b6288016a", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.12.0", GitCommit:"d325d2a9c179b33af1a024cdb5a4472b6288016a", GitTreeState:"clean"}

Any update on this?

Thanks, @technosophos, hoping gRPC upgrade fixes this as this issue causes some PITA for us.

@technosophos @bacongobbler found something interesting about this in VSTS devops. Seems it occurs after a long running install that exceeds a timeout. Anything with helm that exceeds about 150 seconds seems to result in this...for mine it's cases where IP resources have to be allocated due to a load balancer so depending on the load on K8s it'll deploy in about 90 seconds. Under load it's up to 150-160s and terminates with that error between 152 to 156 seconds. I can watch the difference...refreshing on K8s I see that the Helm job immediately ends after getting the loadbalancer external IP assignment.
Not sure if there is an easy way for you guys to check for deployments that exceed that boundary time to see if you can reliably recreate the problem.

@technosophos
Hi, I'm also facing this issue on Mac and found this issue after some googling. I've tried using helm client 2.13.0-rc.1, the error is gone, but the client just hangs.
GKE Version: 1.11.6-gke.2
Tiller Version (installed by Gitlab): v2.12.2
Output for Helm Client v2.12.3:
$ helm version --tiller-namespace gitlab-managed-apps
Client: &version.Version{SemVer:"v2.12.3", GitCommit:"eecf22f77df5f65c823aacd2dbd30ae6c65f186e", GitTreeState:"clean"}
Error: cannot connect to Tiller

$ helm ls --tiller-namespace gitlab-managed-apps
Error: transport is closing

$ helm install --name my-release --tiller-namespace gitlab-managed-apps stable/mongodb
E0216 20:08:46.435297   69434 portforward.go:331] an error occurred forwarding 58711 -> 44134: error forwarding port 44134 to pod 3fbbaff7bdeb4606f59f2e48c46544066abc658e2d0156519f7648e975ca8118, uid : exit status 1: 2019/02/16 22:08:46 socat[68417] E write(5, 0x558c5b292c50, 8192): Broken pipe
Error: transport is closing

Output for Helm Client v.2.13.0-rc.1
$ ./helm version --tiller-namespace gitlab-managed-apps
Client: &version.Version{SemVer:"v2.13.0-rc.1", GitCommit:"e0e5197f8d9b3fa13626a273ad8b3f49a8aab67e", GitTreeState:"clean"}
(... hangs ...)
^C

$ ./helm ls --tiller-namespace gitlab-managed-apps
(... hangs ...)
^C

./helm install --name my-release --tiller-namespace gitlab-managed-apps stable/mongodb
(... hangs ...)
^C

If I try to do the kubectl port-forward trick, it's all the same... The only difference, it seems, is that 2.13.0-rc.1 keeps trying to reconnect and i see several Handling connection for 44134`, instead of just one on 2.12.3.
No relevant logs on tiller pod after all this and no restarts:
$ kubectl logs tiller-deploy-5d96b489cc-rcjn8 -f -n gitlab-managed-apps
[main] 2019/02/16 20:30:07 Starting Tiller v2.12.2 (tls=true)
[main] 2019/02/16 20:30:07 GRPC listening on :44134
[main] 2019/02/16 20:30:07 Probes listening on :44135
[main] 2019/02/16 20:30:07 Storage driver is ConfigMap
[main] 2019/02/16 20:30:07 Max history per release is 0

I've also tried updating tiller image to v2.13.0-rc.1, the results are the same.

Sorry for spamming multiple messages, but it seems that when installing tiller on kube-system namespace with helm init, everything works fine.

'Fixed' this by removing --wait from my deploy command

Upgraded to 2.13.0 and this issue is still happening.
Kubectl v1.13.4
Helm 2.13.0
AKS 1.11.3
Using TLS
Using Azure Devops hosted agents

2019-03-01T06:39:43.4897111Z [command]C:\hostedtoolcache\windows\helm\2.13.0\x64\windows-amd64\helm.exe upgrade --tiller-namespace [tillerns] --namespace [snip] --install --values [values] --wait --tls --tls-ca-cert D:\a\_temp\ca.cert.pem --tls-cert D:\a\_temp\helm-vsts.cert.pem --tls-key D:\a\_temp\helm-vsts.key.pem --values [values] --values [values] --values [values] --values [values] [release] [chart]
2019-03-01T06:40:01.6728072Z E0301 06:39:44.751630    3996 portforward.go:363] error copying from remote stream to local connection: readfrom tcp4 <IP_ADDRESS>:1704-><IP_ADDRESS>:1706: write tcp4 <IP_ADDRESS>:1704-><IP_ADDRESS>:1706: wsasend: An established connection was aborted by the software in your host machine.
2019-03-01T06:40:01.6739267Z Release "[release]" has been upgraded. Happy Helming!
2019-03-01T06:40:01.6762749Z E0301 06:40:00.814924    3996 portforward.go:363] error copying from remote stream to local connection: readfrom tcp4 <IP_ADDRESS>:1704-><IP_ADDRESS>:1707: write tcp4 <IP_ADDRESS>:1704-><IP_ADDRESS>:1707: wsasend: An established connection was aborted by the software in your host machine.

Can confirm that this is still an issue in 2.13
$ helm install appscode/kubedb-catalog --name kubedb-catalog --version 0.10.0 --namespace kube-system
NAME:   kubedb-catalog
E0303 20:13:56.088186   21260 portforward.go:363] error copying from remote stream to local connection: readfrom tcp4 <IP_ADDRESS>:33299-><IP_ADDRESS>:44514: write tcp4 <IP_ADDRESS>:33299-><IP_ADDRESS>:44514: write: broken pipe
LAST DEPLOYED: Sun Mar  3 20:13:54 2019
NAMESPACE: kube-system
STATUS: DEPLOYED
...

I was just trying out kubedb right now and got this. I see a lot of people saying that removing --wait solved the problem for them, but what I think happens is that when you have that flag you are just a lot more likely to have this happen to you, as you will keep the connection up until you're done. But it does not remove the issue completely.
Just making sure it's included:
Azure AKS running k8 version 1.12.6
export TILLER_NAMESPACE="tiller"
export HELM_TLS_ENABLE="true"

Is there anything else we can provide for debugging this?

I think you need to be running tiller 2.13 and helm 2.13 for tls to work correctly.

@willejs, @simpers mentioned he was using Helm 2.13. :P

write tcp4 <IP_ADDRESS>:1704-><IP_ADDRESS>:1707: wsasend: An established connection was aborted by the software in your host machine.

@Vhab it's interesting that the error message changes based on the OS. That first error might actually confirm some of the reports here that the connection is being terminated somewhere between the client, the API server, the kubelet and tiller, but it's not identifying where (or what) is terminating the connection unfortunately.
Perhaps figuring out who's running at <IP_ADDRESS>:1704 sending requests to <IP_ADDRESS>:1707 might help us identify the problem.
Keep these reports coming! We're still unsure what is causing the underlying network failure but we'll keep trying to grok the logs we can find and see if we can identify a fix. THank you so much for the continued reports, we really appreciate it!

@simpers based on your error report:

write tcp4 <IP_ADDRESS>:33299-><IP_ADDRESS>:44514: write: broken pipe

If you want to try figuring out who's listening on ports 33299 and 44514, that might be helpful tpo diagnose where the connection's getting closed off.

I think i am actually wrong and this is still a bug. Long story, mostly with me recompiling helm-diff with 2.13 to fix it, but it doesn't. I suspect this is upstream though?

Thank you, I forgot to re-open this. Yes, right now signs are pointing towards a bug upstream affecting Helm.
On the bright side, this issue should not be a problem for Helm 3 given that we've removed tiller and interact directly with the API server. :)

Yes, right now signs are pointing towards a bug upstream affecting Helm.
It definitely feels like that.

Bit too busy to poke at it and try to get some more diagnostics at the moment, but did some googling around for possibly related issues.
https://github.com/kubernetes/kubernetes/issues/74551
https://github.com/pachyderm/pachyderm/issues/2815
These have similar symptoms.
On a side note, would there be a way to turn these errors into warnings instead? As effectively the operation was still successful.
But I suppose the source of these errors are not fine grained enough to separate out "appears to do no harm" vs "this is bad".

Hey guys, while reading thread I tried a few things out and I may've (accidentally) discovered (at least one of) the reason(s) for the error - port-forwarding to another service on the same client.  I imagine a CI client doesn't port-forward to the cluster while building (or ever), so it may be something on the cluster-side (regardless of which client actually uses that)?
Apologies if I completely misunderstood what's happening here - I just thought of posting this memo in the slim chance that it may alleviate some pain. Cheers.

For what it is worth to those trying to get to the bottom of this issue...
Upgraded to 2.13.1. I can lookup version from inside the k8s cluster from another pod.
user@app-pod-9qtnz:/opt/app$ /helm/linux-amd64/helm --tiller-namespace mynamespace version --tls --tls-ca-cert /opt/app/tiller-tls-secrets/ca.cert.pem --tls-cert /opt/app/tiller-tls-secrets/helm-cert --tls-key /opt/app/tiller-tls-secrets/helm-key
Client: &version.Version{SemVer:"v2.13.1", GitCommit:"618447cbf203d147601b4b9bd7f8c37a5d39fbb4", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.13.1", GitCommit:"618447cbf203d147601b4b9bd7f8c37a5d39fbb4", GitTreeState:"clean"}

The above assumes the local cluster config from the pod.
Cannot do it from the "outside" the cluster using a kubeconfig. Fails 100%.
Top block is port-forward, bottom is cooresponding helm cli attempt to get version.
$ kubectl -nmynamespace port-forward tiller-deploy 44134:44134
Forwarding from <IP_ADDRESS>:44134 -> 44134
Forwarding from [::1]:44134 -> 44134
Handling connection for 44134
Handling connection for 44134
Handling connection for 44134
E0325 14:48:23.291109   72782 portforward.go:316] error copying from local connection to remote stream: read tcp4 <IP_ADDRESS>:44134-><IP_ADDRESS>:65378: read: connection reset by peer
Handling connection for 44134
E0325 14:48:24.742180   72782 portforward.go:316] error copying from local connection to remote stream: read tcp4 <IP_ADDRESS>:44134-><IP_ADDRESS>:65379: read: connection reset by peer
Handling connection for 44134
E0325 14:48:26.876715   72782 portforward.go:316] error copying from local connection to remote stream: read tcp4 <IP_ADDRESS>:44134-><IP_ADDRESS>:65380: read: connection reset by peer
Handling connection for 44134
E0325 14:48:29.782902   72782 portforward.go:316] error copying from local connection to remote stream: read tcp4 <IP_ADDRESS>:44134-><IP_ADDRESS>:65381: read: connection reset by peer
Handling connection for 44134
E0325 14:48:34.851637   72782 portforward.go:316] error copying from local connection to remote stream: read tcp4 <IP_ADDRESS>:44134-><IP_ADDRESS>:65383: read: connection reset by peer
Handling connection for 44134
...

$ HELM_HOST=<IP_ADDRESS>:44134 helm version --tls --tls-verify --tls-ca-cert ~/ca-cert --tls-cert ~/helm-cert --tls-key ~/helm-key
Client: &version.Version{SemVer:"v2.13.0", GitCommit:"79d07943b03aea2b76c12644b4b54733bc5958d6", GitTreeState:"clean"}
...(hangs)

/liveness on port 44135 works fine with the port-forward.
$ kubectl -napp-ns-403 port-forward tiller-deploy-8d8cb7f47-f8mqm 44135:44135
Forwarding from <IP_ADDRESS>:44135 -> 44135
Forwarding from [::1]:44135 -> 44135
Handling connection for 44135

$ curl localhost:44135/liveness -v
*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 44135 (#0)
> GET /liveness HTTP/1.1
> Host: localhost:44135
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Mon, 25 Mar 2019 19:59:52 GMT
< Content-Length: 0
<
* Connection #0 to host localhost left intact

Kubernetes details:
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.0", GitCommit:"91e7b4fd31fcd3d5f436da26c980becec37ceefe", GitTreeState:"clean", BuildDate:"2018-06-27T22:29:25Z", GoVersion:"go1.10.3", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"10+", GitVersion:"v1.10.13-eks-g484b8", GitCommit:"484b857e3134d55ac6373fea2f51798fefa0533f", GitTreeState:"clean", BuildDate:"2019-03-08T05:32:36Z", GoVersion:"go1.10.8", Compiler:"gc", Platform:"linux/amd64"}

Here is the thing, errors in TLS validation close the connection unexpectedly and the kubectl proxy in the background complains about it without helm printing the actual error.
In my case it was as simple as adding "localhost" to the server certificate hosts and
export HELM_TLS_HOSTNAME=localhost

openssl s_client -connect was key to narrow it down, then translate it to helm flags.
This should be definitelly flagged as a bug.
The real error is completelly silent even with --debug

I'm deploying magento in an ibm cloud private and I get this error
NAME:   magento
E0416 18:45:41.935993  106642 portforward.go:303] error copying from remote stream to local connection: readfrom tcp4 <IP_ADDRESS>:35733-><IP_ADDRESS>:58668: write tcp4 <IP_ADDRESS>:35733-><IP_ADDRESS>:58668: write: broken pipe
LAST DEPLOYED: Tue Apr 16 18:45:39 2019
NAMESPACE: ace
STATUS: DEPLOYED

Here is the thing, errors in TLS validation close the connection unexpectedly and the kubectl proxy in the background complains about it without helm printing the actual error.
In my case it was as simple as adding "localhost" to the server certificate hosts and
export HELM_TLS_HOSTNAME=localhost

openssl s_client -connect was key to narrow it down, then translate it to helm flags.
This should be definitelly flagged as a bug.
The real error is completelly silent even with --debug

Confirm, that helps. I used
export HELM_TLS_HOSTNAME=tiller-server

@miguelangel-nubla @feksai Unfortunately this didn't do the trick for us.
While using --tls-verify did reproduce a similar error, which we could then make go away with --tls-verify --tls-hostname [hostname], this didn't remove the semi-random occurrence of the error during a longer helm call.
helm upgrade --tiller-namespace [tillerns] --namespace [ns] --install --values [values] --wait --tls --tls-ca-cert [cert] --tls-cert [cert] --tls-key [key] --tls-verify --tls-hostname [tillerhost] --values [values] [release] [chart]

2019-04-24T10:43:30.6030274Z E0424 10:43:29.393264    3780 portforward.go:363] error copying from remote stream to local connection: readfrom tcp4 <IP_ADDRESS>:1698-><IP_ADDRESS>:1701: write tcp4 <IP_ADDRESS>:1698-><IP_ADDRESS>:1701: wsasend: An established connection was aborted by the software in your host machine.

@Vhab yea, my excitement was short-lived. Still having same error with rare successfull results. Noticied that if helm upgrade takes more than 15 seconds then it will fail. In successfull releases it took not longer than that edge

@technosophos despite #5210 we're still having the same issue. Do you think this can be fixed or should we wait for helm 3?

The current thinking is that this is an upstream bug. So I'm not sure there is much left that we can do. But it definitely won't impact Helm 3, which no longer uses the port forwarding/tunneling aspect of Kubernetes.

agree with @technosophos.   With Helm 3 on the horizon is it worth fixing this issue ?
I found the error does not interfere with the actual helm install/upgrade, as that does work, for me its always appeared at the end of the post deployment report helm dumps out. Which I ignore and then use subsequent steps in the pipeline to double check the deployment is ok.  Not great i admit but enough of a workaround until Helm 3 comes along.
I really would not invest any more time and effort into this issue.

The biggest issue we have with this is that our CI/CD views it as a failure.
Even though the deployment was successful, the reporting says failed.
It is having the adverse behavior that it is causing people to ignore errors raised in CI/CD because “oh, that thing always errors out.L

@codejnki  agreed. I continue on error for the helm deploy step so a warning is raised instead and then I validate its worked in the next step and if that fails then its time to take notice. Not great but with Helm 3 coming and ppl educated internally on the error its still kept us using Helm.

This issue is in the way kubectl port forwarding is handled and not related to helm itself. There is an issue open on the kubernetes repo about this.

I can confirm export HELM_TLS_HOSTNAME=<tiller-deployment-name> works sufficiently, I didn't get the broken pipe error. :)

Dp you mean the name of the pod, like "tiller-deploy-d8cbf88b8-dcvjk".
But that changes on each deployment, how should i set this fixed?

This error should not be present in Helm 3 as Helm no longer relies on the portforwarder logic in Kubernetes to acquire a connection to Tiller (because Tiller has been removed).
I'm going to close this as resolved, however if there's a fix we can apply to Helm 2 to improve the situation that'd be great to hear and we'd love a PR. Thanks!

So you ending maintenance of helm 2 now, where Helm3 is even not yet stable and merging everything will also take time.?

Maybe this will help somebody still. I get same error
E1008 11:23:47.418883    4367 portforward.go:303] error copying from remote stream to local connection: readfrom tcp4 <IP_ADDRESS>:40737-><IP_ADDRESS>:38472: write tcp4 <IP_ADDRESS>:40737-><IP_ADDRESS>:38472: write: broken pipe

installing charts contained CRDs with
  annotations:
    "helm.sh/hook": crd-install

(for instance https://github.com/helm/charts/tree/master/stable/ambassador)
The error happens if on the upgrade  of the existing release, so when CRDs are already there.
Removing hook annotation helped me.
Helm v.2.11.0

I am getting the same error and increase memory or export HELM_TLS_HOSTNAME= didn't help me

Why is the issue closed? As it stands right now it is impossible to use helm with CI / CD with TLS due to this issue.
Do we have to wait for Helm 3?

Still happens:
$ helm version
Client: &version.Version{SemVer:"v2.15.1", GitCommit:"cf1de4f8ba70eded310918a8af3a96bfe8e7683b", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.15.1", GitCommit:"cf1de4f8ba70eded310918a8af3a96bfe8e7683b", GitTreeState:"clean"}

Same issue faces with K8S cluster hosted on GCP.
Command: helm upgrade  --install ******
Helm version:  v2.14.2
Kubernetes version.Info: {Major:"1", Minor:"13+", GitVersion:"v1.13.11-gke.9", GoVersion:"go1.11.13b4", Compiler:"gc", Platform:"linux/amd64"}
Error:
E1109 00:08:37.767306     269 portforward.go:372] error copying from remote stream to local connection: readfrom tcp4 <IP_ADDRESS>:44603-><IP_ADDRESS>:38070: write tcp4 <IP_ADDRESS>:44603-><IP_ADDRESS>:38070: write: broken pipe\nE1109 00:08:37.767306     269 portforward.go:372] error copying from remote stream to local connection: readfrom tcp4 <IP_ADDRESS>:44603-><IP_ADDRESS>:38070: write tcp4 <IP_ADDRESS>:44603-><IP_ADDRESS>:38070: write: broken pipe\nlog: exiting because of error: log: cannot create log: open /tmp/helm.bf54a0af24f1.root.log.ERROR.20191109-000837.269: no such file or directory\n

use safari instead of chrome

I found the problem. Issue was with storage, tillers was able at the end report problem with allocating the storage resources. But that is interesting how error handling was implemented ;)

I was able to track down the cause of this issue to expired Tiller and Helm certificates.
Issue Description
I originally secured my Tiller installation following these instructions https://v2.helm.sh/docs/using_helm/#using-ssl-between-helm-and-tiller. In the tutorial, the duration for the validity of the Helm and Tiller certificates is set to 365 days. I originally generated the certificates with this value, but over 365 days ago.
When running any Helm command (e.g. helm version or helm list), I received an error message similar to this:
an error occurred forwarding 49855 -> 44134: error forwarding port 44134 to pod 4106da54d86955cc3f88c866cf45afdaf0c6edf9f471ad669f23ba56dc77e6ab, uid : exit status 1: 2020/05/27 21:33:00 socat[15077] E write(5, 0x5642d8387150, 24): Broken pipe

Issue Resolution

Back up your existing certificates
Using your exiting Tiller key, Helm Key, CA Key, and CA certificate, generate new Helm and Tiller certificates following the original instructions described in https://v2.helm.sh/docs/using_helm/#using-ssl-between-helm-and-tiller
Upgrade your Tiller setup with the these new certificates. I did not have to to edit the Tiller secrets in the cluster manually, as the following command updated the secrets for me (note that this will also upgrade Tiller in your cluster to the corresponding version of Helm you're using):

helm init \
--service-account tiller \
--tiller-namespace tiller \
--tiller-tls \
--tiller-tls-cert tiller.crt \
--tiller-tls-key ~/.ssh/tiller.key \
--tiller-tls-verify \
--tls-ca-cert ~/.ssh/ca.helm.crt \
--upgrade

I'm getting the same issue by using and AKS cluster within Azure.


[Blazor] Use shared policies for client and server

Scenario
In my ASP.NET Core hosted client-side Blazor application, I want to use my policies in both projects.

In the server project to secure the API methods.
In the client project to appropriately show or hide certain parts of the views.

The current project template creates the following projects:

MyProject.Client (.NET Standard 2.0)
MyProject.Server (.NET Core 3.0)
MyProject.Shared (.NET Standard 2.0)

Problem
The clear approach would be to define the policies in the shared project, once.
However, policies defined in the shared project are not available in the server project.
This is partially due to the changes listed in https://github.com/aspnet/AspNetCore/issues/3756, as well as the fact that the Microsoft.AspNetCore.Identity.IdentityConstants are not available in the .NET Standard 2.0 project.
Possible options
IMHO, it should be possible to use the same policies for the server and client project, without having to define them twice. Three options I can currently come up with:

Make all projects .NET Core 3.0.
Define the policies separately in the MyProject.Server and MyProject.Client project (insufficient option).
Include the missing classes in .NET Standard (probably the worst option).

Hi. It looks like this is a question about how to use ASP.NET Core. While we do our best to look through all the issues filed here, to get a faster response we suggest posting your questions to StackOverflow using the asp.net-core-blazor tag.

@mkArtakMSFT Thanks for directing me to StackOverflow, but this isn't much of a question, more like a possible issue I see when you want to use shared policies. I'm in no rush here, just pointing this out for the 3.1.0 milestone.

Added question on StackOverflow: https://stackoverflow.com/questions/57150701/use-same-authorization-policies-for-client-and-server-in-blazor
That mixed versions and compability between Microsoft.AspNetCore.App, Microsoft.AspNetCore.Authorization (3.0.0) and Microsoft.AspNetCore.Identity (2.2.0) aren't very obvious, nor comprehensible.

@mkArtakMSFT After further investigation, this is definietly an issue with the Microsoft.AspNetCore.Identity.IdentityConstants class provided by the Microsoft.AspNetCore.Identity (2.2.0) package in the shared project.
That way, shared policies are not possible. Therefore, I'd rather call this issue a bug, not a question.
The following error is caused by adding the package, causing the client project build to fail:
2>Processing embedded resource linker descriptor: mscorlib.xml
2>Duplicate preserve in resource mscorlib.xml in mscorlib, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=7cec85d7bea7798e of System.Threading.WasmRuntime (All).  Duplicate uses (All)
2>Type Mono.ValueTuple has no fields to preserve
2>Type System.Reflection.Assembly has no fields to preserve
2>Fatal error in IL Linker
2>
2>Unhandled Exception: Mono.Linker.MarkException: Error processing method: 'System.Void System.Security.Permissions.PrincipalPermission::Demand()' in assembly: 'System.Security.Permissions.dll' ---> Mono.Cecil.ResolutionException: Failed to resolve System.Security.Principal.IPrincipal System.Threading.Thread::get_CurrentPrincipal()
2>   at Mono.Linker.Steps.MarkStep.HandleUnresolvedMethod(MethodReference reference)
2>   at Mono.Linker.Steps.MarkStep.MarkMethod(MethodReference reference)
2>   at Mono.Linker.Steps.MarkStep.MarkInstruction(Instruction instruction)
2>   at Mono.Linker.Steps.MarkStep.MarkMethodBody(MethodBody body)
2>   at Mono.Linker.Steps.MarkStep.ProcessMethod(MethodDefinition method)
2>   at Mono.Linker.Steps.MarkStep.ProcessQueue()
2>   --- End of inner exception stack trace ---
2>   at Mono.Linker.Steps.MarkStep.ProcessQueue()
2>   at Mono.Linker.Steps.MarkStep.ProcessPrimaryQueue()
2>   at Mono.Linker.Steps.MarkStep.Process()
2>   at Mono.Linker.Steps.MarkStep.Process(LinkContext context)
2>   at Mono.Linker.Pipeline.ProcessStep(LinkContext context, IStep step)
2>   at Mono.Linker.Pipeline.Process(LinkContext context)
2>   at Mono.Linker.Driver.Run(ILogger customLogger)
2>   at Mono.Linker.Driver.Execute(String[] args, ILogger customLogger)
2>   at Mono.Linker.Driver.Main(String[] args)

Followed up by this cryptic error message:
\.nuget\packages\microsoft.aspnetcore.blazor.build\3.0.0-preview6.19307.2\targets\Blazor.MonoRuntime.targets(439,5): error MSB3073: The command "dotnet "C:\Users\Stefan\.nuget\packages\microsoft.aspnetcore.blazor.mono\0.10.0-preview6.19303.4\build\netstandard1.0\../../tools/illink/illink.dll" -l none --disable-opt unreachablebodies --verbose --strip-security true --exclude-feature com --exclude-feature sre -v false -c link -u link -b true -d "C:\Users\Stefan\.nuget\packages\microsoft.aspnetcore.blazor.mono\0.10.0-preview6.19303.4\build\netstandard1.0\../../tools/mono/bcl/" -d "C:\Users\Stefan\.nuget\packages\microsoft.aspnetcore.blazor.mono\0.10.0-preview6.19303.4\build\netstandard1.0\../../tools/mono/bcl/Facades/" -o "S:\git\hou-units\src\UNITS\UNITS.Client\obj\Debug\netstandard2.0\blazor\linker/" -x "C:\Users\Stefan\.nuget\packages\microsoft.aspnetcore.blazor.build\3.0.0-preview6.19307.2\targets\BuiltInBclLinkerDescriptor.xml" -x "S:\git\hou-units\src\UNITS\UNITS.Client\obj\Debug\netstandard2.0\blazor\linker.descriptor.xml" -a "C:\Users\Stefan\.nuget\packages\jetbrains.annotations\2019.1.3\lib\netstandard2.0\JetBrains.Annotations.dll" -a "C:\Program Files\dotnet\sdk\NuGetFallbackFolder\microsoft.aspnetcore.authentication\2.2.0\lib\netstandard2.0\Microsoft.AspNetCore.Authentication.dll" -a "C:\Program Files\dotnet\sdk\NuGetFallbackFolder\microsoft.aspnetcore.authentication.abstractions\2.2.0\lib\netstandard2.0\Microsoft.AspNetCore.Authentication.Abstractions.dll" -a "C:\Program Files\dotnet\sdk\NuGetFallbackFolder\microsoft.aspnetcore.authentication.cookies\2.2.0\lib\netstandard2.0\Microsoft.AspNetCore.Authentication.Cookies.dll" -a "C:\Program Files\dotnet\sdk\NuGetFallbackFolder\microsoft.aspnetcore.authentication.core\2.2.0\lib\netstandard2.0\Microsoft.AspNetCore.Authentication.Core.dll" -a "C:\Users\Stefan\.nuget\packages\microsoft.aspnetcore.authorization\3.0.0-preview6.19307.2\lib\netstandard2.0\Microsoft.AspNetCore.Authorization.dll" -a "C:\Users\Stefan\.nuget\packages\microsoft.aspnetcore.blazor\3.0.0-preview6.19307.2\lib\netstandard2.0\Microsoft.AspNetCore.Blazor.dll" -a "C:\Users\Stefan\.nuget\packages\microsoft.aspnetcore.components\3.0.0-preview6.19307.2\lib\netstandard2.0\Microsoft.AspNetCore.Components.dll" -a "C:\Users\Stefan\.nuget\packages\microsoft.aspnetcore.components.browser\3.0.0-preview6.19307.2\lib\netstandard2.0\Microsoft.AspNetCore.Components.Browser.dll" -a "C:\Program Files\dotnet\sdk\NuGetFallbackFolder\microsoft.aspnetcore.cryptography.internal\2.2.0\lib\netstandard2.0\Microsoft.AspNetCore.Cryptography.Internal.dll" -a "C:\Program Files\dotnet\sdk\NuGetFallbackFolder\microsoft.aspnetcore.cryptography.keyderivation\2.2.0\lib\netstandard2.0\Microsoft.AspNetCore.Cryptography.KeyDerivation.dll" -a "C:\Program Files\dotnet\sdk\NuGetFallbackFolder\microsoft.aspnetcore.dataprotection\2.2.0\lib\netstandard2.0\Microsoft.AspNetCore.DataProtection.dll" -a "C:\Program Files\dotnet\sdk\NuGetFallbackFolder\microsoft.aspnetcore.dataprotection.abstractions\2.2.0\lib\netstandard2.0\Microsoft.AspNetCore.DataProtection.Abstractions.dll" -a "C:\Program Files\dotnet\sdk\NuGetFallbackFolder\microsoft.aspnetcore.hosting.abstractions\2.2.0\lib\netstandard2.0\Microsoft.AspNetCore.Hosting.Abstractions.dll" -a "C:\Program Files\dotnet\sdk\NuGetFallbackFolder\microsoft.aspnetcore.hosting.server.abstractions\2.2.0\lib\netstandard2.0\Microsoft.AspNetCore.Hosting.Server.Abstractions.dll" -a "C:\Program Files\dotnet\sdk\NuGetFallbackFolder\microsoft.aspnetcore.http\2.2.0\lib\netstandard2.0\Microsoft.AspNetCore.Http.dll" -a "C:\Program Files\dotnet\sdk\NuGetFallbackFolder\microsoft.aspnetcore.http.abstractions\2.2.0\lib\netstandard2.0\Microsoft.AspNetCore.Http.Abstractions.dll" -a "C:\Program Files\dotnet\sdk\NuGetFallbackFolder\microsoft.aspnetcore.http.extensions\2.2.0\lib\netstandard2.0\Microsoft.AspNetCore.Http.Extensions.dll" -a "C:\Program Files\dotnet\sdk\NuGetFallbackFolder\microsoft.aspnetcore.http.features\2.2.0\lib\netstandard2.0\Microsoft.AspNetCore.Http.Features.dll" -a "C:\Program Files\dotnet\sdk\NuGetFallbackFolder\microsoft.aspnetcore.identity\2.2.0\lib\netstandard2.0\Microsoft.AspNetCore.Identity.dll" -a "C:\Users\Stefan\.nuget\packages\microsoft.aspnetcore.metadata\3.0.0-preview6.19307.2\lib\netstandard2.0\Microsoft.AspNetCore.Metadata.dll" -a "C:\Program Files\dotnet\sdk\NuGetFallbackFolder\microsoft.aspnetcore.webutilities\2.2.0\lib\netstandard2.0\Microsoft.AspNetCore.WebUtilities.dll" -a "C:\Users\Stefan\.nuget\packages\microsoft.extensions.configuration\2.2.0\lib\netstandard2.0\Microsoft.Extensions.Configuration.dll" -a "C:\Users\Stefan\.nuget\packages\microsoft.extensions.configuration.abstractions\2.2.0\lib\netstandard2.0\Microsoft.Extensions.Configuration.Abstractions.dll" -a "C:\Users\Stefan\.nuget\packages\microsoft.extensions.configuration.binder\2.2.0\lib\netstandard2.0\Microsoft.Extensions.Configuration.Binder.dll" -a "C:\Users\Stefan\.nuget\packages\microsoft.extensions.dependencyinjection\3.0.0-preview6.19304.6\lib\netstandard2.0\Microsoft.Extensions.DependencyInjection.dll" -a "C:\Users\Stefan\.nuget\packages\microsoft.extensions.dependencyinjection.abstractions\3.0.0-preview6.19304.6\lib\netstandard2.0\Microsoft.Extensions.DependencyInjection.Abstractions.dll" -a "C:\Program Files\dotnet\sdk\NuGetFallbackFolder\microsoft.extensions.fileproviders.abstractions\2.2.0\lib\netstandard2.0\Microsoft.Extensions.FileProviders.Abstractions.dll" -a "C:\Program Files\dotnet\sdk\NuGetFallbackFolder\microsoft.extensions.hosting.abstractions\2.2.0\lib\netstandard2.0\Microsoft.Extensions.Hosting.Abstractions.dll" -a "C:\Program Files\dotnet\sdk\NuGetFallbackFolder\microsoft.extensions.identity.core\2.2.0\lib\netstandard2.0\Microsoft.Extensions.Identity.Core.dll" -a "C:\Users\Stefan\.nuget\packages\microsoft.extensions.logging\2.2.0\lib\netstandard2.0\Microsoft.Extensions.Logging.dll" -a "C:\Users\Stefan\.nuget\packages\microsoft.extensions.logging.abstractions\3.0.0-preview6.19304.6\lib\netstandard2.0\Microsoft.Extensions.Logging.Abstractions.dll" -a "C:\Program Files\dotnet\sdk\NuGetFallbackFolder\microsoft.extensions.objectpool\2.2.0\lib\netstandard2.0\Microsoft.Extensions.ObjectPool.dll" -a "C:\Users\Stefan\.nuget\packages\microsoft.extensions.options\3.0.0-preview6.19304.6\lib\netstandard2.0\Microsoft.Extensions.Options.dll" -a "C:\Users\Stefan\.nuget\packages\microsoft.extensions.primitives\3.0.0-preview6.19304.6\lib\netstandard2.0\Microsoft.Extensions.Primitives.dll" -a "C:\Program Files\dotnet\sdk\NuGetFallbackFolder\microsoft.extensions.webencoders\2.2.0\lib\netstandard2.0\Microsoft.Extensions.WebEncoders.dll" -a "C:\Users\Stefan\.nuget\packages\microsoft.jsinterop\3.0.0-preview6.19304.6\lib\netstandard2.0\Microsoft.JSInterop.dll" -a "C:\Program Files\dotnet\sdk\NuGetFallbackFolder\microsoft.net.http.headers\2.2.0\lib\netstandard2.0\Microsoft.Net.Http.Headers.dll" -a "C:\Program Files\dotnet\sdk\NuGetFallbackFolder\microsoft.win32.registry\4.5.0\lib\netstandard2.0\Microsoft.Win32.Registry.dll" -a "C:\Users\Stefan\.nuget\packages\mono.webassembly.interop\3.0.0-preview6.19304.6\lib\netstandard2.0\Mono.WebAssembly.Interop.dll" -a "C:\Users\Stefan\.nuget\packages\newtonsoft.json\12.0.2\lib\netstandard2.0\Newtonsoft.Json.dll" -a "C:\Users\Stefan\.nuget\packages\system.buffers\4.5.0\lib\netstandard2.0\System.Buffers.dll" -a "C:\Users\Stefan\.nuget\packages\system.componentmodel.annotations\4.6.0-preview6.19303.8\lib\netstandard2.0\System.ComponentModel.Annotations.dll" -a "C:\Users\Stefan\.nuget\packages\system.memory\4.5.3\lib\netstandard2.0\System.Memory.dll" -a "C:\Program Files\dotnet\sdk\NuGetFallbackFolder\system.numerics.vectors\4.5.0\lib\netstandard2.0\System.Numerics.Vectors.dll" -a "C:\Users\Stefan\.nuget\packages\system.runtime.compilerservices.unsafe\4.6.0-preview6.19303.8\lib\netstandard2.0\System.Runtime.CompilerServices.Unsafe.dll" -a "C:\Program Files\dotnet\sdk\NuGetFallbackFolder\system.security.accesscontrol\4.5.0\lib\netstandard2.0\System.Security.AccessControl.dll" -a "C:\Users\Stefan\.nuget\packages\system.security.cryptography.cng\4.4.0\lib\netstandard2.0\System.Security.Cryptography.Cng.dll" -a "C:\Program Files\dotnet\sdk\NuGetFallbackFolder\system.security.cryptography.pkcs\4.5.0\lib\netstandard2.0\System.Security.Cryptography.Pkcs.dll" -a "C:\Program Files\dotnet\sdk\NuGetFallbackFolder\system.security.cryptography.xml\4.5.0\lib\netstandard2.0\System.Security.Cryptography.Xml.dll" -a "C:\Program Files\dotnet\sdk\NuGetFallbackFolder\system.security.permissions\4.5.0\lib\netstandard2.0\System.Security.Permissions.dll" -a "C:\Program Files\dotnet\sdk\NuGetFallbackFolder\system.security.principal.windows\4.5.0\lib\netstandard2.0\System.Security.Principal.Windows.dll" -a "C:\Program Files\dotnet\sdk\NuGetFallbackFolder\system.text.encodings.web\4.5.0\lib\netstandard2.0\System.Text.Encodings.Web.dll" -a "C:\Users\Stefan\.nuget\packages\system.text.json\4.6.0-preview6.19303.8\lib\netstandard2.0\System.Text.Json.dll" -a "C:\Users\Stefan\.nuget\packages\system.threading.tasks.extensions\4.5.2\lib\netstandard2.0\System.Threading.Tasks.Extensions.dll" -a "S:\git\hou-units\src\UNITS\UNITS.Shared\bin\Debug\netstandard2.0\UNITS.Shared.dll" -a "S:\git\hou-units\src\UNITS\UNITS.Client\obj\Debug\netstandard2.0\UNITS.Client.dll"" exited with code -532462766.

Example project on GitHub: https://github.com/Herdo/hou-units/tree/prototype-blazor/src/UNITS

Thanks for all the details, @Herdo .
@javiercn what do you think?

Hi @Herdo thanks for contacting us. Is the IdentityConstants the only thing you are using from that assembly? (Seems like it)
I have a few suggestions:

I would define the constant in the shared assembly directly, the constant is not going to change, and if you are super worried about it changing in the future, simply write a unit test to check your value against the one in the server. Its a constant, so its gonna be replaced at compile time with the actual value.
That form of reuse is hurting you more than you are benefiting from it. You now bring an entire assembly and its dependencies to the client for a single string in the assembly.
If you are determined to do this, you can create a custom descriptor and tell the linker to link out anything you don't want and remove dependencies you don't need per https://docs.microsoft.com/en-us/aspnet/core/host-and-deploy/blazor/configure-linker?view=aspnetcore-3.0

If you ask me, going through all this trouble is not worth it.

While we target NETStandard2.0 there are things that naturally aren't going to work in the browser, like many of the ASP.NET Core bits, except for those we specifically create packages for or do work on. (most of them are shared framework in 3.0, so there aren't even packages for them). We plan to do work on the ones we think are useful, like DI, Authorization (which is a package in 3.0 as well as part of the shared framework) but it is unlikely that we do anything for other things like Identity and similar as they only apply on the server.

My final suggestion would be to follow the path of least resistance, while keeping things DRY is a good principle to follow, you need to take into account the price you are paying and the benefit you are getting from it on a per case basis.
Hope this helps.

@javiercn Yes, IdentityConstants is the only thing I'm using, because every guide and tutorial I found points out you should use this class for the authentication schemes.
Not a single one suggest hard-coding Identity.Application into the policies. Which is actually what I did as alternative to reference the Microsoft.AspNetCore.Identity (2.2.0) package. 👍
I know that this re-use is hurting a lot, and after your comment I don't plan to keep it that way, but rather go back to the hard-coded solution.
Thank you very much for your help.


Allow unresizable HTML5 apps

If the user sets the window config "resizable" to false in an html5 app, then it forces the window to always be a certain fixed size.
I just add some macro logic to the template. If resizable is true, width/height are "100%" (the current behavior). If resizable is false, width/height are the exact values specified by the user in project.xml

Significance: this will be extremely helpful in maturing HaxeFlixel for proper HTML5 support.


updated protobuf

Signed-off-by: kumarabd<EMAIL_ADDRESS>Description
This PR fixes #
Notes for Reviewers
Signed commits

[x] Yes, I signed my commits.

There are a number of conflicts to address in this PR.

Is this PR being closed in favor of the other?

Is this PR being closed in favor of the other?

@dhruv0000, I assume this should be closed. Is that right?

@dhruv0000, I assume this should be closed. Is that right?

@kumarabd @dhruv0000 I assume this PR is out of date and can be closed. Is that right?

Yes, this PR can be closed now.


Featur Request: Support duplicate query parameters

As mentioned in the stackoverflow post below, I want to use multiple, duplicate query parameters: mysite.com?id=1&id=2&id=3
https://stackoverflow.com/questions/11621477/using-duplicate-parameters-in-a-url
The usecase is to have multiple checkboxes with the same name (query parameter), so it will end up sending multiple query parameters to the backend.
This is not yet supported. Even uri.query() does remove the duplicate entries. The last one is always kept.
Is this by design? Could we add a function to (at least) show the PURE raw value?

PHP doesn't support this. If you want to support multiple ids, you need to use an array: id[] inside the input field.

Thanks


Yosys & iverilog differences in defines

I have a macro
`define INCPC()     PC<=PC+1; romA<=PC+1; IR<=romD
used like this later in an always clause
`INCPC();
This synthesises and works as expected on real hardware (xilinx 8k board)
attempting to get some simulation I can have confidence with I run:
iverilog -o test_post -D POST_SYNTHESIS atto_tb.v atto.v `yosys-config --datdir/ice40/cells_sim.v`
I get the following
atto.v:7: error: malformed `define directive.
atto.v:123: warning: macro INCPC undefined (and assumed null) at this point.
as well as a number of syntax errors etc I also see a bunch of errors in yoysys's resources
/usr/local/share/yosys/ice40/cells_sim.v:18: error: malformed statement
/usr/local/share/yosys/ice40/cells_sim.v:19: syntax error
/usr/local/share/yosys/ice40/cells_sim.v:19: Syntax in assignment statement l-value.

According to IEEE Std. 1364-2005 Syntax 19-2—Syntax for text macro definition (see sec. 19.3.1 `define), you can either define a macro without arguments and without parentheses, or with arguments and parentheses. I.e. the following two macro definitions are valid Verilog code:
`define FOO 1234
`define BAR(arg1, arg2) (arg1 + arg2)

However, macro definitions with empty parentheses are not valid Verilog code. I.e. the following macro definition is invalid:
`define FOOBAR() 1234

Icarus Verilog is more strict with things like that than Yosys and therefore rejects the macro definition with empty parentheses.

as well as a number of syntax errors etc I also see a bunch of errors in yoysys's resources

I cannot reproduce this. Which version of Icarus Verilog are you using? (What is the first line in the output of iverilog -V?) I have run ice40/cells_sim.v through Icarus Verilog 0.9.7 and current git head of Icarus Verilog, and both accepted it.

Icarus Verilog version 10.0 (stable) ()  (basically what comes with my distro (void) )
I only see issues (with cells_sim.v) when define is incorrectly used
Yosys should generate a meaningful error such as

Syntax error define used without parameters should have no brackets

can you re-poen this until Yosys does generate an error message correctly?
I believe better error reporting will improve Yosys significantly

Quoting the read_verilog help message:

Note that the Verilog frontend does a pretty good job of processing valid
verilog input, but has not very good error reporting. It generally is
recommended to use a simulator (for example Icarus Verilog) for checking
the syntax of the code, rather than to rely on read_verilog for that.

Also: If you synthesize first, simulate later, then something is seriously wrong with your design work flow.

struggling to get anything to work at all if I simulate first, they behave too differently...

You have not yet posted anything that has different behavior in simulation and synthesis, so I can't comment on that.
But whatever it is, I can ensure you: If you synthesize first, simulate later, then something is seriously wrong with your design work flow.


docs: How to import Prisma Client into your app

New page that can be linked from prisma generate success message

Would be great to document all the different import variants:
Regular Prisma Client:
TS, CJS, ESM
Edge Prisma Client:
TS, CJS, ESM
Deno Prisma Client
TS
We might not want all the possible ways, but just in case here are a few more ways.

Happy to merge and PR review suggestions. I just created this as a dummy to start the conversation.

I think we could repurpose existing pages in our docs for this content. Here are a few ideas (non-exhaustive) for pages we could use in our docs:

Working with Prisma Client (currently acts more of a landing page) or
Instantiating Prisma Client

I already added the shortlink here https://github.com/prisma/pris.ly/pull/124

The resulting page does not document how to use a custom output or how to import with Deno. (= missing documentation)
The headline "Importing Prisma Client" now also covers how to use it to send queries. (= worse than before documentation)


How to show horizontal scroll bar when there are many columns needed to be shown in one table?

The page shows scroll bar,but not in the table.

Create a container above it.
`

`
And using css do:
`#container { width:100%; overflow: auto }`

@AmitMY will hide the pager

+1，need a container for the table element

you need to surround table with div say

In css apply following style to container div
#container{white-space:nowrap; overflow: hidden;}
then apply following style to your table
.table{display:block;overflow-x:scroll;}
This will add horizontal container just below the table and before the pager div.


Understand effect of resource hints on task mapping with Slurm

Splitting this off from #1321:

This is probably on our end, but has there been a change to the way that tasks/workers/blocks work since release 0.8.0? Running in 0.8.0, I was getting 24 whole-node jobs on slurm with 384 simultaneous tasks. Now, I get 3 whole-node jobs on slurm, still with 384 simultaneous tasks. Again, I think the issue is in how we wrap Parsl, but was wondering if you were aware of any changes that could have caused this.

This may have to do with #1217.
@mattwelborn, @Lnaden: could you share your config?

Thanks for looking into this!
Here is the config from v0.8.0, which works correctly in the sense that 24 whole-node jobs get submitted to slurm.
Config(
        app_cache=True,
        checkpoint_files=None,
        checkpoint_mode=None,
        checkpoint_period=None,
        data_management_max_threads=10,
        executors=[HighThroughputExecutor(
            address='calogin1',
            cores_per_worker=4,
            heartbeat_period=30,
            heartbeat_threshold=120,
            interchange_port_range=(55000, 56000),
            label='QCFractal_Parsl_Slurm_Executor',
            launch_cmd='process_worker_pool.py {debug} {max_workers} -p {prefetch_capacity} -c {cores_per_worker} -
m {mem_per_worker} --poll {poll_period} --task_url={task_url} --result_url={result_url} --logdir={logdir} --block_i
d={{block_id}} --hb_period={heartbeat_period} --hb_threshold={heartbeat_threshold} ',
            managed=True,
            max_workers=192,
            mem_per_worker=None,
            poll_period=10,
            prefetch_capacity=0,
            provider=SlurmProvider(
                'normal_q',
                channel=LocalChannel(
                    envs={},
                    script_dir=None,
                    userhome='/home/mwelborn/qcarchive/managers/cascades_wd'
                ),
                cmd_timeout=120,
                exclusive=True,
                init_blocks=0,
                launcher=SingleNodeLauncher(),
                max_blocks=24,
                min_blocks=0,
                move_files=True,
                nodes_per_block=1,
                parallelism=1,
                scheduler_options='#SBATCH --exclusive\n#SBATCH -A themolssi\n',
                walltime='71:00:00',
                worker_init='source /home/mwelborn/qcarchive/boot_qcarchive_env.sh'
            ),
            storage_access=[],
            suppress_failure=False,
            worker_debug=False,
            worker_logdir_root=None,
            worker_port_range=(54000, 55000),
            worker_ports=None,
            working_dir=None
        )],
        lazy_errors=True,
        monitoring=None,
        retries=0,
        run_dir='runinfo',
        strategy='simple',
        usage_tracking=False
    )

And here is the config from e282e114ba43c31d9a173895e6cbbc3b4f71531c which does not work correctly, in that 3 whole-node jobs get submitted to SLURM.
    Config(
        app_cache=True,
        checkpoint_files=None,
        checkpoint_mode=None,
        checkpoint_period=None,
        data_management_max_threads=10,
        executors=[HighThroughputExecutor(
            address='calogin1',
            cores_per_worker=4,
            heartbeat_period=30,
            heartbeat_threshold=120,
            interchange_port_range=(55000, 56000),
            label='QCFractal_Parsl_Slurm_Executor',
            launch_cmd='process_worker_pool.py {debug} {max_workers} -p {prefetch_capacity} -c {cores_per_worker} -m {mem_per_worker} --poll {poll_period} --task_url={task_url} --result_url={result_url} --logdir={logdir} --block_id={{block_id}} --hb_period={heartbeat_period} --hb_threshold={heartbeat_threshold} ',
            managed=True,
            max_workers=192,
            mem_per_worker=None,
            poll_period=10,
            prefetch_capacity=0,
            provider=SlurmProvider(
                'normal_q',
                channel=LocalChannel(
                    envs={},
                    script_dir=None,
                    userhome='/home/mwelborn/qcarchive/managers/cascades_wd'
                ),
                cmd_timeout=120,
                cores_per_node=None,
                exclusive=True,
                init_blocks=0,
                launcher=SingleNodeLauncher(),
                max_blocks=24,
                mem_per_node=None,
                min_blocks=0,
                move_files=True,
                nodes_per_block=1,
                parallelism=1,
                scheduler_options='#SBATCH -A themolssi\n\n#SBATCH --exclusive\n',
                walltime='71:00:00',
                worker_init='source /home/mwelborn/qcarchive/boot_qcarchive_env.sh\n'
            ),
            storage_access=None,
            suppress_failure=False,
            worker_debug=False,
            worker_logdir_root=None,
            worker_port_range=(54000, 55000),
            worker_ports=None,
            working_dir=None
        )],
        initialize_logging=True,
        lazy_errors=True,
        max_idletime=120.0,
        monitoring=None,
        retries=0,
        run_dir='runinfo',
        strategy='simple',
        usage_tracking=False
    )

The nodes on this cluster have 32 cores, so cores_per_worker(4) * max_workers(192) = max_blocks(24) * nodes_per_block(1) * 32.

max_workers is the maximum # of workers per manager/node. I see where the confusion is coming from. We should've named it max_worker_per_manager.

The docs have it written as

max_workers (int) – Caps the number of workers launched by the manager.

So I don't think there is a need to change the variable.
But for the purposes of clarity, is there 1 Manager per block or per node?

But for the purposes of clarity, is there 1 Manager per block or per node?

@Lnaden it is 1 manager per node.
I favor removing max_workers because it is just another thing for people to grok, and it is redundant with cores_per_worker (it just overrides it if the user does not specify cores_per_worker):
           self.worker_count = min(max_workers,
                                   mem_slots,
                                   math.floor(cores_on_node / cores_per_worker))

I am still working on reproducing this. Running with this config at midway I see 24 jobs submitted:
Config(
    app_cache=True,
    checkpoint_files=None,
    checkpoint_mode=None,
    checkpoint_period=None,
    data_management_max_threads=10,
    executors=[HighThroughputExecutor(
        address='midway2-login1.rcc.local',
        cores_per_worker=4,
        heartbeat_period=30,
        heartbeat_threshold=120,
        interchange_port_range=(55000, 56000),
        label='midway_htex_multinode',
        launch_cmd='process_worker_pool.py {debug} {max_workers} -p {prefetch_capacity} -c {cores_per_worker} -m {mem_per_worker} --poll {poll_period} --task_url={task_url} --result_url={result_url} --logdir={logdir} --block_id={{block_id}} --hb_period={heartbeat_period} --hb_threshold={heartbeat_threshold} ',
        managed=True,
        max_workers=192,
        mem_per_worker=None,
        poll_period=10,
        prefetch_capacity=0,
        provider=SlurmProvider(
            'broadwl',
            channel=LocalChannel(
                envs={},
                script_dir=None,
                userhome='/home/annawoodard/resource_hints_1341'
            ),
            cmd_timeout=10,
            cores_per_node=None,
            exclusive=True,
            init_blocks=0,
            launcher=SingleNodeLauncher(),
            max_blocks=24,
            mem_per_node=None,
            min_blocks=0,
            move_files=True,
            nodes_per_block=1,
            parallelism=1,
            scheduler_options='#SBATCH --account=pi-foster\n#SBATCH qos=debug\n#SBATCH --exclusive\n',
            walltime='00:05:00',
            worker_init='source activate parsl\n'
        ),
        storage_access=None,
        suppress_failure=False,
        worker_debug=False,
        worker_logdir_root=None,
        worker_port_range=(54000, 55000),
        worker_ports=None,
        working_dir=None
    )],
    initialize_logging=True,
    lazy_errors=True,
    max_idletime=120.0,
    monitoring=None,
    retries=0,
    run_dir='runinfo',
    strategy='simple',
    usage_tracking=False
)

@mattwelborn @Lnaden could you zip up and send your logs in runinfo?

@annawoodard runinfo attached.
runinfo_e282e11.tar.gz
runinfo_v0.8.0.tar.gz

Hi @mattwelborn, Do you know how many cores are available on each node in this cluster ? If so, could you try setting say, `SlurmProvider(cores_per_node=<CORES_PER_Ners i.e 192. Which explains why the executor is under-provisioning.

@yadudoc Specifying cores_per_node to the SlurmProvider fixes the problem. Thanks!
@Lnaden Is this a fully satisfactory resolution on our end (i.e. should I close this issue)? @yadudoc  Is this a fully satisfactory resolution on your end?

I think this is evidence we should remove max_workers as it adds complexity and opportunities for misconfiguration. (I think without max_workers=192 you still would have gotten 24 blocks submitted even without cores_per_node specified.) I'll open as a separate issue.

But yes, I think this is taken care of now. Thanks everyone!


Update to Cmdliner v1.1.0

And remove some warnings

You need to update the lower bound on cmdliner in the opam file

curious, is the printbox constraint update something needed to remove warnings?

Not warnings, but to avoid compilation errors. printbox-text was moved into it's own package since the 0.6 version of printbox. With lower versions, dune complains that there is no printbox-text package. With a 0.6 and higher version, the code won't compile without adding  printbox-text as a dependancy.

also: welcome grin wave

Thank you! :smile:

Oh right, that makes sense!! I forgot about the split. Carry on then :)

for the failing test : the output has changed, but if it's just because of the unicode boxes, try:
dune runtest --auto-promote

and commit the result. The promote thing means it'll add the changes back into the source tree.

You're welcome! :grin:


Applying the model to higher sampling rates

Hi,
Thank you so much for sharing this great repository.
I was just wondering if you ever tried applying your model to unconditional audio generation using higher sampling rates (e.g. 44.1 kHz). Do you think there could be hope for it to work reasonably well, perhaps after modifying the architecture with some extra U-Net layers?

I haven't tried anything other than the two datasets documented in this repo, but I think there should be no problems with 44.1kHz. I'm guessing the architecture should work as-is, but you can certainly play with it; we didn't experiment much with the overall UNet structure so there might be room for improvement


Support org.apache.commons.lang3.StringUtils.isNotEmpty

Add method support for org.apache.commons.lang3.StringUtils.isNotEmpty
Following code rise a exception with all versions of nullaway including 0.10.24

[75,103] [NullAway] dereferenced expression this.properties.username() is @Nullable
ERROR    (see http://t.uber.com/nullaway )
ERROR   Did you mean '@SuppressWarnings("NullAway") @Override'?
[76,40] [NullAway] dereferenced expression this.properties.password() is @Nullable
ERROR   (see http://t.uber.com/nullaway )
ERROR   Did you mean '@SuppressWarnings("NullAway") @Override'?

Also I tried to use the following config but it not worked
-XepOpt:NullAway:CheckOptionalEmptiness=true \
-XepOpt:NullAway:KnownInitializers=org.apache.commons.lang3.StringUtils.isNotEmpty

same happens with org.springframework.util.StringUtils

The issue is that you call this.properties.username() (and password()) inside a lambda body.  NullAway doesn't know when the lambda function will be invoked, and it does not assume that username() and password() will remain non-null until that point.  If you store this.properties.username() and password() in local variables and do both the isNotEmpty() checks and the uses with those locals, everything will work (since the value of the local variables cannot change).

I know that the issue is closed, but it is not a similar case?
There is no lambda and nullpointer checks have been made at line 43.

KnownInitializers has been configured with org.apache.commons.collections4.isNotEmpty
-XepOpt:NullAway:KnownInitializers=org.springframework.util.Assert.notNull,org.apache.commons.collections4.isNotEmpty \
-XepOpt:NullAway:CheckOptionalEmptiness=true \

And the error still appears at line 45
[ERROR] /C:/santalucia/workspace/ams-spring-data/ams-spring-data-rest/src/main/java/com/santalucia/arq/ams/data/rest/config/repository/AmsRepositoryRestConfigurer.java:[45,52] [NullAway] dereferenced expression properties.exposeIdsFor() is @Nullable
[ERROR]     (see http://t.uber.com/nullaway )
[ERROR]   Did you mean '@SuppressWarnings("NullAway") @Override'?

This is a different case.  I don't think the KnownInitializers flag is being used correctly; that flag is for (most likely inherited) methods that behave like constructors almost and run before other methods in the class:
https://github.com/uber/NullAway/wiki/Configuration#known-initializers
For this case, we seem to be missing a model for the relevant CollectionUtils.isNotEmpty() method.  That would be added around here:
https://github.com/uber/NullAway/blob/e3a3c7672d51ec2e90c0e60c30af477ccf6f7757/nullaway/src/main/java/com/uber/nullaway/handlers/LibraryModelsHandler.java#L735-L735
A PR with a fix would be welcome!

Hello @msridhar . Would it just be sufficient to add a
.put(methodRef("org.apache.commons.collections4.CollectionUtils", "isNotEmpty(java.util.Collection<?>)"), 0)
?
P.S. I noticed that apache had a previous version with a different package name org.apache.commons.collections.CollectionUtils . Do we need to support this too ?

@pochopsp something like that should be sufficient, but we should write a test to be sure.  And yeah I'm fine with supporting org.apache.commons.collections.CollectionUtils also.


UE4 Assets

Are there are any shared memory between assets when used in game and is there anyway to have a shared volume across actors?
This is something I would like to have for the future Region Volume where it can be used as a updating minimap using the same volume data/memory.

So currently when you drag a voxel volume into the editor they all share the same memory (if it's the same asset ofc). I guess this is both good and bad, but I would like a way to have the possibility of creating a copy too so they wont share the same voxel data.


feat: set up CI

Codecov Report

:exclamation: No coverage uploaded for pull request base (main@324dc6b). Click here to learn what that means.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main       #1   +/-   ##
=======================================
  Coverage        ?   85.52%
=======================================
  Files           ?        8
  Lines           ?      228
  Branches        ?       29
=======================================
  Hits            ?      195
  Misses          ?       26
  Partials        ?        7

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 324dc6b...a850856. Read the comment docs.


"Shock and awe" is not the way to migrate a repo.

There is a note on the old repo from 8 days ago saying it moved here. That's not enough and MSFT should know better.
None of the open issues were migrated.
Nobody who had an open issue got a note with this info or a request to move the issue. I got a threat from the always offensive stalebot, which doesn't count. Actually migrating was your choice so you should be doing the move of issues anyway and not creating work for everyone else due to your choice
New issues can be reported still, which is terribly misleading.
There are PRs still outstanding.
If you are going to close the repo, finish the job in a responsible and ethical way.

Thanks for the feedback @StingyJack, we are always working to improve.

@alepauly I tagged at least one issue to bring over, and one request for you too take a look at


docs: Create link between isAcpControlled and hasAccessibleAcr

This PR improves the situation for #SDK-2463, by interlinking the two methods in the documentation.

[ ] I've added a unit test to test for potential regressions of this bug. n/a
[ ] The changelog has been updated, if applicable.
[x] Commits in this PR are minimal and have descriptive commit messages.

Closing in favour of #1481 which includes these changes, not sure why the CD / e2e tests are failing here when they're passing in #1481


Implement dict resizing

Dict resizing is working by removing all None entries, and filling up indices map from scratch
Frequency of resizing should be adjusted, I guess. Maybe it'll be useful to trigger it more often, it's hard to say without somewhat realistic benchmark
Addresses: #1531
Benchmark:
a = {0: 1}
for i in range(10000):
    del a[0]
    a[0] = 1

Results:
$ time ./target/release/rustpython-master test.py

real    0m3,316s
user    0m3,312s
sys     0m0,004s

$ time ./target/release/rustpython test.py

real    0m0,062s
user    0m0,055s
sys     0m0,008s

$ time python3 test.py

real    0m0,023s
user    0m0,015s
sys     0m0,008s

How is it possible that the benchmark is so much faster? Does this change also improve speed of other python code?

How is it possible that the benchmark is so much faster? Does this change also improve speed of other python code?

That's because reusing a key in a dictionary used to cause growth of Dict.indices. and since all of the keys have the same hash (as they are equal) each map insert used to be forced to go through hundreds of layers of hash collisions to actually find an empty slot.
Same goes for deletion.


SW-5170 Select all orgs when attempting to find org name for project

#2484  👈
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @nickgraz and the rest of your teammates on  Graphite


Correct alignment logic, ensure proper variant selection for post-mapped genomic variants

closes #69

Corrects alignment logic. For targets with gene symbols, a BlatOutput tuple is constructed containing the hsp object, the distance between the start of the alignment and the gene start, and the coverage of the alignment. The list of tuples is first sorted by increasing distance. Then, hsp objects with the same distance are grouped together, and sorted in descending order by coverage. This results in the best candidate alignment appearing at the top of the list
Ensures that only post-mapped genomic variants whose positions are known are mapped. All pre-mapped genomic variants are included as their positions are described with respect to the MAVE target sequence.

Question to consider: If we have different sized lists for the pre-mapped and post-mapped variants, how can we determine which post-mapped variant belongs to its corresponding pre-mapped variant. This metric is needed to compute the reference sequence match percentage.

We may need to update JSON schema. will double check

Just checked: no schema update needed. it's currently up to date and this PR doesn't change it.


AP-2631 Initial EKS migration work

What
Link to story
Adds annotations to live-1 ingress to allow load balancing using DNS weighting, following instructions in Step 2 here:
https://user-guide.cloud-platform.service.justice.gov.uk/documentation/other-topics/migrate-to-live.html#step-2-amend-and-apply-your-quot-live-1-quot-ingress-resource
Checklist
Before you ask people to review this PR:

Tests and rubocop should be passing: bundle exec rake
Github should not be reporting conflicts; you should have recently run git rebase main.
There should be no unnecessary whitespace changes. These make diffs harder to read and conflicts more likely.
The PR description should say what you changed and why, with a link to the JIRA story.
You should have looked at the diff against main and ensured that nothing unexpected is included in your changes.
You should have checked that the commit messages say why the change was made.
You should have run NOCOVERAGE=true rake rswag to ensure the swagger docs are up-to-date

@colinbruce The aws-weight is being set in deploy/helm/values-uat.yaml.


Gateway should handle header 'Expect 100-continue' compliant with rfc2616

If a proxy receives a request that includes an Expect request-
header field with the "100-continue" expectation, and the proxy
either knows that the next-hop server complies with HTTP/1.1 or
higher, or does not know the HTTP version of the next-hop
server, it MUST forward the request, including the Expect header
field.

Steps to reproduce problem
Target server: Spring Boot 2.1.8 (Tomcat)
@SpringBootApplication @RestController
public class MyDownstreamApp {
   public static void main(String[] args) {
      SpringApplication.run(MyDownstreamApp.class, args);
  }

  @PutMapping("/put")
  String put(@RequestParam String x) {
      System.out.println("x: " +x);
      return "hello";
  }
}

Cloud gateway: Greenwich SR3
@SpringBootApplication
public class MyGatewayApp {

    @Bean
    RouteLocator myRoutes(RouteLocatorBuilder builder) {
        return builder.routes()
                .route(p -> p
                        .path("/put")
                        .uri("http://localhost:8080"))
                .build();
    }

    public static void main(String[] args) {
        SpringApplication.run(MyGatewayApp.class, "--server.port=8082");
    }
}

When target server is called directly:
curl localhost:8080/put -X PUT -H 'Expect: 100-continue' -d x=foo1 -v
communication flow is OK

Client sends Expect: 100-continue
Server reponds 100-continue
Client sends complete request
Server responds hello

> Expect: 100-continue
> Content-Length: 6
> Content-Type: application/x-www-form-urlencoded
>
< HTTP/1.1 100
* We are completely uploaded and fine
< HTTP/1.1 200
< Content-Type: text/plain;charset=UTF-8
< Content-Length: 5

When target server is called through gateway:
curl localhost:8082/put -X PUT -H 'Expect: 100-continue' -d x=foo1 -v
communication flow is not OK. My wireshark investigation

Client sends Expect: 100-continue to gateway
Gateway responds 100-continue
Client sends complete request to gateway
I'm not sure 100% what happens in point 4
Target server responds 100-continue to gateway
Gateway sends complete request to target server
Target server responds hello
Gateway forwards 100-continue to client
Client waits for response hello but it is not forwarded

> Expect: 100-continue
> Content-Length: 6
> Content-Type: application/x-www-form-urlencoded
>
< HTTP/1.1 100 Continue
* We are completely uploaded and fine
< HTTP/1.1 100 Continue
(hang up)

Response 100 Continue is received 2 times - once generated by gateway itself, second forwarded from target server, but true response never reach client.
Worth to notice that workaround is filter our header 'Expect 100-continue` when gateway forwards request to target server

This is seriously very weird behaviour. We have spring cloud gateway in production environment and for clients which send Expect: 100-continue are not getting proper response.
I don't know how is this possible but during peak times the response for the route is also wrong.

For ex : Two routes A and B and they both call expecting 100. Route A gets 100 status code and the the payload response is sent to route B.

https://github.com/reactor/reactor-netty/issues/722

I believe this is block by the reactor netty issue listed above.

@spencergibb Reactor Netty will respond automatically with 100 Continue ONLY when the IO handler asks for the body but there was no response with 100 Continue before that.
https://github.com/reactor/reactor-netty/blob/89ae010a84e25925952e2f82ece7fae8c015742c/src/main/java/reactor/netty/http/server/HttpServerOperations.java#L292-L301
In Spring Gateway I see the following:
https://github.com/spring-cloud/spring-cloud-gateway/blob/73fd52861b8b56bd890c68d85e4c104a204ca4bc/spring-cloud-gateway-core/src/main/java/org/springframework/cloud/gateway/filter/NettyRoutingFilter.java#L147
In the implementation the incoming data is requested always without checking whether there is Expect header

To document, request.getBody() is called in webflux initialization very early on  before gateway can respond to any header here https://github.com/spring-projects/spring-framework/blob/master/spring-web/src/main/java/org/springframework/web/server/adapter/DefaultServerWebExchange.java#L131
So it still looks blocked but at the framework level now @rstoyanchev

That is not actual call but deferred initialization in case of a call to exchange.getFormData(). To trigger that something must call getFormData().

I'm able to send a 100 Continue from gateway (or let the downstream service send it), but it seems the request is terminated and the client never sends the body. I'm not sure where in the stack the problem is.

Is there any other work around in springboot to handle the Expect: 100-continue header.

Is there any other work around in springboot to handle the Expect: 100-continue header.

From issue description:

Worth to notice that workaround is filter our header 'Expect 100-continue` when gateway forwards request to target server

So you can simply add route filter that removes Expect header
.filters(f -> f.removeRequestHeader("Expect"))

or
filters:
        - RemoveRequestHeader=Expect

I just updated spring-boot to version 2.4.3 which depends on reactor-netty version 1.0.4 and netty version 4.1.59
The RemoveRequestHeader work around does not work any more.
I tried to create a gatewayFilter that sets response status to 100 and complete the response but I get:
Connection prematurely closed BEFORE response; nested exception is reactor.netty.http.client.PrematureCloseException: Connection prematurely closed BEFORE response
org.springframework.web.reactive.function.client.WebClientRequestException: Connection prematurely closed BEFORE response; nested exception is reactor.netty.http.client.PrematureCloseException: Connection prematurely closed BEFORE response
	at org.springframework.web.reactive.function.client.ExchangeFunctions$DefaultExchangeFunction.lambda$wrapException$9(ExchangeFunctions.java:137)

That is strange, because my experiment shows that  spring-boot version 2.4.3 & spring cloud version 2020.0.1 fixes the problem
The problem is still visible for spring-boot version 2.3.9.RELEASE and spring cloud Hoxton.SR9
I vote for close the problem

I reckon I haven't given enough details about what I'm trying to do.
We have several micro-services behind a spring-cloud-gateway which is accessed by mobile games.
We want the gateway to handle Expect 100-continue headers and respond with 100.
We don't want the gateway to pass the request to downstream services in this case.
Before spring-boot 2.4.3, we added RemoveRequestHeader and the gateway worked as required.
Since upgrading to spring-boot 2.4.3 it passes the request downstream which we don't want it to do.
I tried to implement a gateway filter responsible to intercept Expect 100-continue requests, set the response to 100 and complete the response.
I can't make it work; I alwys the following exception:
Connection prematurely closed BEFORE response; nested exception is reactor.netty.http.client.PrematureCloseException: Connection prematurely closed BEFORE response org.springframework.web.reactive.function.client.WebClientRequestException: Connection prematurely closed BEFORE response; nested exception is reactor.netty.http.client.PrematureCloseException: Connection prematurely closed BEFORE response 	at org.springframework.web.reactive.function.client.ExchangeFunctions$DefaultExchangeFunction.lambda$wrapException$9(ExchangeFunctions.java:137)

@paskos I'm going to say your issue is different than the original intent of this issue. Can you post your filter?

Thank you for the link.
@violetagg @spencergibb I'm far from proficient with Netty, spring-cloud-gateway and reactor or reactive programming.
Could you, please, point me in the direction on how to implement Expect 100-continue at the gateway level where the request is not sent downstream ?
Thanks a lot in advance


New term: 3beta-(2-Diethylaminoethoxy)androst-5-en-17-one hydrochloride

Hi,
Could you add 3beta-(2-Diethylaminoethoxy)androst-5-en-17-one hydrochloride to ChEBI?
Name: 3beta-(2-Diethylaminoethoxy)androst-5-en-17-one hydrochloride
Synonym: U18666A
PubChem CID: 9954082
Thank you!

Hi,
A new entry has now been created (CHEBI:190404). It should be searchable on the ChEBI website by tomorrow.


[qwt|vcpkg-qmake] Update and switch to Qt6

(other related PR which was revert #21998; reopen after this has been solved.)

Ready for review now?

any progress?

@strega-nil-ms Any results?

cc @BillyONeal

@BillyONeal: Why did you try to solve it? I mean the PR is still tagged depends:vcpkg-team. this PR probably still needs other changes from https://github.com/Neumann-A/my-vcpkg-ports/tree/master/vcpkg-qmake or https://github.com/Neumann-A/my-vcpkg-ports/tree/master/qwt

@BillyONeal: Why did you try to solve it?

I got pinged by @MehdiChinoune 's comment, saw PR with merge conflicts, so tried to fix. Feel free to force push them off.

Remove label vcpkg-team, please ping me if this PR is ready for review or needs some help.

@JackBoosY: This PR just needs a decision on how to transition qwt to qt6.

Should the qwt-qt6 port be removed ?

Should the qwt-qt6 port be removed ?

Hmmmm... I didn't realize there was already a split for this port. Does it make sense to leave this one as qt5 (and consider it morally part of qt5) and instead change callers to name qwt-qt6?

relevant for the qwt-qt6 discussion: #21825 #20921.
@BillyONeal Did you just forgot to remove the v db entry for qwt-qt6 or was that on purpose?

Thanks!

What was the resolution of https://github.com/microsoft/vcpkg/pull/20921#issuecomment-985276177 relative to this PR?  In particular, since vcpkg continues to support qt5 as separate port, what should we do in an application that's using qt5 and qwt?

What was the resolution of #20921 (comment) relative to this PR? In particular, since vcpkg continues to support qt5 as separate port, what should we do in an application that's using qt5 and qwt?

You would need to use overlays. We accepted the switch to qt6 because current versions of qt5 are no longer available from the qt project to open source customers.

I'm not sure I follow - Qt5 source is available, see https://wiki.qt.io/Building_Qt_5_from_Git#Getting_the_source_code
Since qt5 and qt6 are different ports, could there be a feature in qwt to select which one to use?  A major advantage to using vcpkg instead of system package managers is that this decision is local to the application.  We don't need to make a decision that every application on the system needs the qt5 version of libqwt, but this one does.

I'm not sure I follow - Qt5 source is available, see https://wiki.qt.io/Building_Qt_5_from_Git#Getting_the_source_code

There was more than a month where the current Qt5 was not available, although I see they may have just fixed it 8 days ago according to https://code.qt.io/cgit/qt/qt5.git/log/?h=5.15
Notably, 5.15 will be the last qt5 release and open source customers no longer receive LTS ( https://www.qt.io/blog/qt-offering-changes-2020 ), meaning open source consumption support by the qt project ended 2021-05-26.
Between the unavailability (at the time) of current qt5 sources, the public statements by the qt project that such use is no longer in support, and our general policy that the canonical version available in the catalog is that selected by upstream project owners, switching vcpkg's canonical version to qt6 was long past due.

Since qt5 and qt6 are different ports, could there be a feature in qwt to select which one to use?

Features are additive and cannot be used to control "alternatives" like this. (For example, it is always assumed that enabling a feature is 'safe' which is not true when only one may be selected)
That is, if port[a] works, and port[b] works, port[a,b] must work. See https://github.com/microsoft/vcpkg/blob/master/docs/maintainers/maintainer-guide.md#do-not-use-features-to-implement-alternatives

A major advantage to using vcpkg instead of system package managers is that this decision is local to the application

You can still do that. You just need to do it with an overlay port.

We don't need to make a decision that every application on the system needs the qt5 version of libqwt, but this one does.

vcpkg needs to make a decision on which versions of dependencies we select in the curated catalog, and test. We are only testing one outcome of what vcpkg install qwt does.

Thanks for the additional info, it's certainly helpful.  I'm not all that tuned-in to what Qt upstream has changed in their policies.
In terms of how vcpkg handles this, it seems like the ideal state would be to have a single qt port and not both qt5 and qt (the latter being v6).  However, it's reasonable that for a large framework like Qt there will be some downstream users that need to stay on v5 during a transition period, and that's what we see now realized in vcpkg (although it's not the ideal state).  In this case would a separate port qwt-qt5 be accepted?  It would need to use different directory names in include and share and a different library name.  Downstream projects would need to add some CMake code to adapt.


[WIP] agent: initial support for Mesos agent operations API

Coverage decreased (-3.4%) to 49.24% when pulling 946ca356426cc98feca81b634dc7a7551e56de3e on jdef_agent_protos into 7ef6ac766e6ae6565d082792eb10d006736fd8d1 on master.

Coverage decreased (-16.4%) to 36.248% when pulling 9bbe51ab05088ea21ebb615a84e9aba0183aef4a on jdef_agent_protos into 7ef6ac766e6ae6565d082792eb10d006736fd8d1 on master.

Coverage decreased (-17.03%) to 35.579% when pulling 9bbe51ab05088ea21ebb615a84e9aba0183aef4a on jdef_agent_protos into 7ef6ac766e6ae6565d082792eb10d006736fd8d1 on master.

Coverage remained the same at 52.609% when pulling 78226a74c2608ab1843d71ed8d6f39fe400914d5 on jdef_agent_protos into 7ef6ac766e6ae6565d082792eb10d006736fd8d1 on master.

reorganized commits for sanity

Coverage remained the same at 52.609% when pulling 9721f6193a32324c5eacd53ebbcc2b3ea992f539 on jdef_agent_protos into 7ef6ac766e6ae6565d082792eb10d006736fd8d1 on master.


add: removeRequestHeader and setRequestHeader methods

addRequestHeader(name, value) is always joining the value if the header already existed. These two new methods allow to remove a request header or replace it with a new value

thanks @theTestTube!


Update README.md

Replaced with content written for developer's site.

@zinfin Let's move this to the apps README and keep this README more about setting up the developer environment, then we can link to this doc for app context.

@zinfin We should discuss the usage of redirect uri as well as we should not require one if the user is logging into AGOL.  We have an issue to support this in the API but until then we should simplify in this app.

@doneill In the top level README in this branch https://github.com/Esri/maps-app-android/tree/build_1517

@doneill We can certainly revert to HTTP auth, thus removing any dependencies on a client id or a redirect uri, but it seems like showing how to do this would be important for non AGOL access (on premise servers)?  It's been difficult to find good documentation on how to do this correctly.

@zinfin I am not suggesting reverting back to HTTP, what I am saying is the app should default to https://www.arcgis.com which uses a default redirect uri so we should not require developers to create a redirect when working with https://www.arcgis.com.  All a developer needs is the Client ID in this case.  This will make is much easier for devs who are evaluating the app, devs who want to use a custom portal url need to create the redirect so we would document that case specifically.

@zinfin Refer to the create save map to see how our sample uses default redirect uri with default AGOL portal.

@zinfin Should we ultimately close this PR in favor of #147?

That makes sense.  I didn’t know we didn’t need to set the redirect if the url is https://www.arcgis.com.  My assumption had been that when you registered an app you always needed both a client id and a re-direct uri, based on this doc https://developers.arcgis.com/authentication/mobile-and-native-user-logins/.
From: doneill<EMAIL_ADDRESS>Reply-To: Esri/maps-app-android<EMAIL_ADDRESS>Date: Thursday, November 17, 2016 at 9:55 AM
To: Esri/maps-app-android<EMAIL_ADDRESS>Cc: Sandie Peters<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [Esri/maps-app-android] Update README.md (#142)
@zinfinhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_zinfin&d=DgMFaQ&c=n6-cguzQvX_tUIrZOS_4Og&r=Xj9mPV1Yz1Nj-lsc99Qy2Z-p2tctBvjVzwpNlPGoFHM&m=f4oFQymNsNPVhrflry8NA_5yL1vMkUUg7LEU6jVDohg&s=DWU-fpMS-VnpIuq-LnYLAqYcclwvBCzixd6b_JyQLQ0&e= I am not suggesting reverting back to HTTP, what I am saying is the app should default to https://www.arcgis.com which uses a default redirect uri so we should not require developers to create a redirect when working with https://www.arcgis.com. All a developer needs is the Client ID in this case. This will make is much easier for devs who are evaluating the app, devs who want to use a custom portal url need to create the redirect so we would document that case specifically.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_Esri_maps-2Dapp-2Dandroid_pull_142-23issuecomment-2D261319375&d=DgMFaQ&c=n6-cguzQvX_tUIrZOS_4Og&r=Xj9mPV1Yz1Nj-lsc99Qy2Z-p2tctBvjVzwpNlPGoFHM&m=f4oFQymNsNPVhrflry8NA_5yL1vMkUUg7LEU6jVDohg&s=SPaam49-vsu8BFMWFgDWvmfZlOxBanHQQwL1Zvdpwr4&e=, or mute the threadhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AHhpyIFmwTjNPgz4UcPM-5FSWec6bmkA5Kks5q-5FJUUgaJpZM4KrugC&d=DgMFaQ&c=n6-cguzQvX_tUIrZOS_4Og&r=Xj9mPV1Yz1Nj-lsc99Qy2Z-p2tctBvjVzwpNlPGoFHM&m=f4oFQymNsNPVhrflry8NA_5yL1vMkUUg7LEU6jVDohg&s=NCI2KAfD_DV7px1-vvFu1KcUowPV_tyUO7hwrPTUKJA&e=.

@zinfin Yes it can be confusing that is why I think we should be explicit in apps that show this pattern.  It is familiar to JS API devs as the API defaults this way, I am trying to get Runtime API's to do the same thing which should simplify the developer pattern. Let me know if you need assistance in documenting the approach.

@zinfin Good to merge


Don't require app.json or docker-compose for release

I don't use docker-compose or app.json for most of my Heroku apps. In such
cases, I would still like to use heroku docker:release. This is a PoC for
making those files no longer required.

Without app.json, add-on comparison is skipped
Without docker-compose.yml, docker build --tag <app> . is used

Would close #143 and #144.
If you're interested in such a use case, please let me know what it would take
to get this merge-able (there's likely a cleaner way).
Thanks

Closing out of staleness. I've moved on to the new registry deployment anyway.


[p0ng] Game over menu - Back to main menu button not working (web)

As the title says.

Not working on both Web and Windows desktop.


fix: Drawer navigation bug complete fix made

@pranshuchittora, the issue #20 , wasn't completely fixed by the PR #21 .
Issues still occuring

on clicking the links, the drawer was navigating to the link, but not closing after navigating.

I have fixed  the bug completely, Kindly check.
screen gif after fix

[x] checked the bug on ios and android as well

This change will break the desktop version. There is undesirable toggling of nav drawer whenever clicked on drawer links/items.

This change will break the desktop version. There is undesirable toggling of nav drawer whenever clicked on drawer links/items.

Thanks for your concern
but I have rechecked the changes, they are working well in desktop(browsers) as well as smart phones.

There was a need of onClick handler that on clicking will redirect the user to particular components page. Also Toggle Back the Drawer Navigation to its previous state

@pranshuchittora please review the PR, and confirm that the bug is completely fixed or not.
Thanks.

Thanks @ankushdutt, :)
Previous issue

After resolving (DESKTOP)

After resolving (SMART PHONES)

LGTM :+1:
Thanks @Uyadav207

Happy to contribute @pranshuchittora. :)


Disallow blocking nested workflow

#1167 Implements a way to run nested workflows concurrently, with the limitations that only workflows that are the last statement can be concurrent, and multiple workflows in sos_riun(['A', 'B']) can be concurrent but blocks the step executor.
However, with the new strict -j worker implementation, sos_run('A') not as the last statement will make the worker wait for the completion of A,
sos_run('A')
other_func()

So in the case of -j1, sos_run('A') will take one worker and the workflow will hang forever.
I would suggest that we disallow the execution of blocking nested workflow so that there will be no waiting process.

I'm not sure if I understand it.

multiple workflows in sos_riun(['A', 'B']) can be concurrent but blocks the step executor.

block means to hang until completion of both A and B?

I would suggest that we disallow the execution of blocking nested workflow so that there will be no waiting process.

Will it impact parallel execution of sos_riun(['A', 'B'])?

OK, In the case of
whatever
sos_run('A')

or
whatever
sos_run(['A', 'B'])

the step statement can be completed (no waiting) and the step can yield (do something else) while waiting for the completion of workflows A and B. In the case of
sos_run('A')
whatever

or
sos_run(['A', 'B'])
whatever

the statement will send out workflows to be executed, but the step is blocking so nothing else can be run on the worker. Basically I am trying to make all sos statements non-blocking so that they can use the workers more efficiently and prevent deadlock.

OK, right now I just give a warning because I have some workflows like
sos_run('A')
sos_run('B')
sos_run('C')

which should be written as
sos_run(['A', 'B', 'C'])

or
sos_run('A + B + C')

if execution matters but a warning will let the workflows run.

Final solutIon: sos will create a separate process for each blocking subworkflow, and shut it down when the workflow is completed. This means there is still a chance for sos to have more than -j + 1 processes but this is better than having sos hang forever.