edge node cannot access the service with clusterip

kubernetes1.20
kubeedge1.8.2
edgemesh:last
kube-proxy and coredns not scheduling in edge node，edgemesh deployment successful
I can't access services through podip or service

Can cloud services be accessed from edge nodes？How to configure it? How to configure? Is there a correct configuration tutorial

 edgemesh-server log
cloudcore log
edgecore log
edgecore log

need some edgemesh-agent log

I can't access services through podip or service

edgemesh only support access services by clusterip or services DNS name.

I can't access services through podip or service

edgemesh only support access services by clusterip or services DNS name.

Sorry for my late reply, I access service through clusterip, but there was no response. There was no problem accessing in the cloud node

edgemesh-agent-master log

edgemesh-server log
edgemesh-agent-slaver

edgemesh-agent-edge log

I use binary to deploy kubeedge,
Using helm to deploy edgemesh, the configuration of edgemesh has not been modified after successful deployment. Do I need to modify it?Or Kube proxy and coredns?

I also found a problem. When deleting the service endpoint, I can access the service through clusterip: curl (52) empty reply from server
However, when the endpoint exists, I can access the service directly through podid, but I cannot access the service through clusterip

I also found a problem. When deleting the service endpoint, I can access the service through clusterip: curl (52) empty reply from server
However, when the endpoint exists, I can access the service directly through podid, but I cannot access the service through clusterip

The service whose cluster ip is <IP_ADDRESS> is a very special service. This is a service automatically created by k8s to access the kube apiserver. It is also an external service, and if you want to access it at the edge, some additional configuration is required.

Let's start with some simple services, you can refer to edgemesh's test case: https://edgemesh.netlify.app/guide/test-case.html

The service whose cluster ip is <IP_ADDRESS> is a very special service. This is a service automatically created by k8s to access the kube apiserver. It is also an external service, and if you want to access it at the edge, some additional configuration is required.

I deployed the joint inference service of Sedna. Edge needs to access port 5000 of cloud, but it cannot be accessed through clusterip. DNS resolution is normal
In the service test case, I deployed TCP test, and the service endpoint was deployed on edge node
The pod in edge can be accessed, but the pod in cloud cannot
And cross-edge-cloud test failed, and no other services were tested. It is certain that there is a problem with Edge and cloud communication. Can you infer the cause of the problem？

This is a k8s knowledge, if your pod is a host network, you need to configure dnsPolicy like dnsPolicy: ClusterFirstWithHostNet , Please refer to kubernetes docs: https://kubernetes.io/zh/docs/concepts/services-networking/dns-pod-service/#pod-s-dns -policy
And how to fix your problem:
CLOUD_NODE="cloud-node-name"
EDGE_NODE="edge-node-name"

kubectl create -f - <<EOF
apiVersion: sedna.io/v1alpha1
kind: JointInferenceService
metadata:
  name: helmet-detection-inference-example
  namespace: default
spec:
  edgeWorker:
    model:
      name: "helmet-detection-inference-little-model"
    hardExampleMining:
      name: "IBT"
      parameters:
        - key: "threshold_img"
          value: "0.9"
        - key: "threshold_box"
          value: "0.9"
    template:
      spec:
        nodeName: $EDGE_NODE
        dnsPolicy: ClusterFirstWithHostNet       <----------- look at here!!!
        containers:
        - image: kubeedge/sedna-example-joint-inference-helmet-detection-little:v0.3.0
          imagePullPolicy: IfNotPresent
          name:  little-model
          env:  # user defined environments
          - name: input_shape
            value: "416,736"
          - name: "video_url"
            value: "rtsp://localhost/video"
          - name: "all_examples_inference_output"
            value: "/data/output"
          - name: "hard_example_cloud_inference_output"
            value: "/data/hard_example_cloud_inference_output"
          - name: "hard_example_edge_inference_output"
            value: "/data/hard_example_edge_inference_output"
          resources:  # user defined resources
            requests:
              memory: 64M
              cpu: 100m
            limits:
              memory: 2Gi
          volumeMounts:
            - name: outputdir
              mountPath: /data/
        volumes:   # user defined volumes
          - name: outputdir
            hostPath:
              # user must create the directory in host
              path: /joint_inference/output
              type: Directory

  cloudWorker:
    model:
      name: "helmet-detection-inference-big-model"
    template:
      spec:
        nodeName: $CLOUD_NODE
        containers:
          - image: kubeedge/sedna-example-joint-inference-helmet-detection-big:v0.3.0
            name:  big-model
            imagePullPolicy: IfNotPresent
            env:  # user defined environments
              - name: "input_shape"
                value: "544,544"
            resources:  # user defined resources
              requests:
                memory: 2Gi
EOF

May I have another question？
When I deploy the "Cloud access edge " test case, they are scheduled correctly. But when I try using the "Edge access cloud" test case, I get the error "telnet: bad address 'tcp-echo-cloud-svc.cloudzone'". What is the problem？

@Poorunga I have a similar problem, but it has nothing to do with joint inference service. I run this to deploy sedna:
curl https://raw.githubusercontent.com/kubeedge/sedna/main/scripts/installation/install.sh | SEDNA_ACTION=create bash -

root@cloud:~# kubectl get nodes -o wide && kubectl get pods -o wide -A
NAME                  STATUS   ROLES                  AGE   VERSION                   INTERNAL-IP   EXTERNAL-IP   OS-IMAGE           KERNEL-VERSION     CONTAINER-RUNTIME
cloud.kubeedge        Ready    control-plane,master   34h   v1.20.0                   <IP_ADDRESS>    <none>        Ubuntu 20.04 LTS   5.4.0-96-generic   docker://20.10.7
edge.cloud.kubeedge   Ready    agent,edge             18h   v1.19.3-kubeedge-v1.9.1   <IP_ADDRESS>      <none>        Ubuntu 20.04 LTS   5.4.0-96-generic   docker://20.10.12
NAMESPACE     NAME                                     READY   STATUS    RESTARTS   AGE     IP            NODE                  NOMINATED NODE   READINESS GATES
kube-system   coredns-7f89b7bc75-q5nzx                 1/1     Running   0          34h     <IP_ADDRESS>    cloud.kubeedge        <none>           <none>
kube-system   coredns-7f89b7bc75-wj7ch                 1/1     Running   0          34h     <IP_ADDRESS>    cloud.kubeedge        <none>           <none>
kube-system   etcd-cloud.kubeedge                      1/1     Running   0          34h     <IP_ADDRESS>    cloud.kubeedge        <none>           <none>
kube-system   kube-apiserver-cloud.kubeedge            1/1     Running   0          34h     <IP_ADDRESS>    cloud.kubeedge        <none>           <none>
kube-system   kube-controller-manager-cloud.kubeedge   1/1     Running   0          34h     <IP_ADDRESS>    cloud.kubeedge        <none>           <none>
kube-system   kube-flannel-cloud-ds-hcj2b              1/1     Running   0          34h     <IP_ADDRESS>    cloud.kubeedge        <none>           <none>
kube-system   kube-flannel-edge-ds-wlq8s               1/1     Running   1          18h     <IP_ADDRESS>      edge.cloud.kubeedge   <none>           <none>
kube-system   kube-proxy-zrspz                         1/1     Running   0          34h     <IP_ADDRESS>    cloud.kubeedge        <none>           <none>
kube-system   kube-scheduler-cloud.kubeedge            1/1     Running   0          34h     <IP_ADDRESS>    cloud.kubeedge        <none>           <none>
kubeedge      edgemesh-agent-ghfbf                     1/1     Running   0          7m48s   <IP_ADDRESS>      edge.cloud.kubeedge   <none>           <none>
kubeedge      edgemesh-agent-znjwj                     1/1     Running   0          7m48s   <IP_ADDRESS>    cloud.kubeedge        <none>           <none>
kubeedge      edgemesh-server-7cc4fbbc9b-8cm5x         1/1     Running   0          7m48s   <IP_ADDRESS>    cloud.kubeedge        <none>           <none>
sedna         gm-5bb9c898d6-tvsvx                      1/1     Running   0          7m17s   <IP_ADDRESS>   cloud.kubeedge        <none>           <none>
sedna         kb-6b7897c89-tdrhf                       1/1     Running   0          7m17s   <IP_ADDRESS>   cloud.kubeedge        <none>           <none>
sedna         lc-wt6dm                                 1/1     Running   0          7m16s   <IP_ADDRESS>      edge.cloud.kubeedge   <none>           <none>
sedna         lc-wwvwh                                 1/1     Running   0          7m16s   <IP_ADDRESS>    cloud.kubeedge        <none>           <none>

edge node sedna local controller log:
root@cloud:~# kubectl logs lc-wt6dm -n sedna
I0523 02:46:40.721346       1 server.go:120] manager dataset is started
I0523 02:46:40.721377       1 server.go:120] manager model is started
I0523 02:46:40.721392       1 server.go:120] manager jointinferenceservice is started
I0523 02:46:40.721417       1 server.go:120] manager federatedlearningjob is started
I0523 02:46:40.721424       1 server.go:120] manager incrementallearningjob is started
I0523 02:46:40.721430       1 server.go:120] manager lifelonglearningjob is started
I0523 02:46:40.721440       1 websocket.go:176] client starts to connect global manager(address: gm.sedna:9000)
I0523 02:46:40.721476       1 server.go:140] server binds port 9100 successfully
E0523 02:46:41.105524       1 websocket.go:192] client tries to connect global manager(address: gm.sedna:9000) failed, error: dial tcp: lookup gm.sedna on <IP_ADDRESS>:53: no such host
E0523 02:46:46.106480       1 websocket.go:192] client tries to connect global manager(address: gm.sedna:9000) failed, error: dial tcp: lookup gm.sedna on <IP_ADDRESS>:53: no such host
E0523 02:46:51.107468       1 websocket.go:192] client tries to connect global manager(address: gm.sedna:9000) failed, error: dial tcp: lookup gm.sedna on <IP_ADDRESS>:53: no such host
E0523 02:46:56.108435       1 websocket.go:192] client tries to connect global manager(address: gm.sedna:9000) failed, error: dial tcp: lookup gm.sedna on <IP_ADDRESS>:53: no such host
E0523 02:47:01.109633       1 websocket.go:192] client tries to connect global manager(address: gm.sedna:9000) failed, error: dial tcp: lookup gm.sedna on <IP_ADDRESS>:53: no such host
E0523 02:47:06.109766       1 websocket.go:200] max retry count reached when connecting global manager(address: gm.sedna:9000)
I0523 02:47:06.109789       1 websocket.go:176] client starts to connect global manager(address: gm.sedna:9000)
E0523 02:47:06.110507       1 websocket.go:192] client tries to connect global manager(address: gm.sedna:9000) failed, error: dial tcp: lookup gm.sedna on <IP_ADDRESS>:53: no such host
E0523 02:47:11.356621       1 websocket.go:192] client tries to connect global manager(address: gm.sedna:9000) failed, error: dial tcp: lookup gm.sedna on <IP_ADDRESS>:53: no such host
E0523 02:47:16.357424       1 websocket.go:192] client tries to connect global manager(address: gm.sedna:9000) failed, error: dial tcp: lookup gm.sedna on <IP_ADDRESS>:53: no such host
E0523 02:47:21.358357       1 websocket.go:192] client tries to connect global manager(address: gm.sedna:9000) failed, error: dial tcp: lookup gm.sedna on <IP_ADDRESS>:53: no such host
E0523 02:47:26.359281       1 websocket.go:192] client tries to connect global manager(address: gm.sedna:9000) failed, error: dial tcp: lookup gm.sedna on <IP_ADDRESS>:53: no such host
E0523 02:47:31.359426       1 websocket.go:200] max retry count reached when connecting global manager(address: gm.sedna:9000)
I0523 02:47:31.359452       1 websocket.go:176] client starts to connect global manager(address: gm.sedna:9000)
E0523 02:47:31.360392       1 websocket.go:192] client tries to connect global manager(address: gm.sedna:9000) failed, error: dial tcp: lookup gm.sedna on <IP_ADDRESS>:53: no such host
E0523 02:47:36.361284       1 websocket.go:192] client tries to connect global manager(address: gm.sedna:9000) failed, error: dial tcp: lookup gm.sedna on <IP_ADDRESS>:53: no such host
E0523 02:47:41.489029       1 websocket.go:192] client tries to connect global manager(address: gm.sedna:9000) failed, error: dial tcp: lookup gm.sedna on <IP_ADDRESS>:53: no such host
E0523 02:47:46.489923       1 websocket.go:192] client tries to connect global manager(address: gm.sedna:9000) failed, error: dial tcp: lookup gm.sedna on <IP_ADDRESS>:53: no such host
E0523 02:47:51.490771       1 websocket.go:192] client tries to connect global manager(address: gm.sedna:9000) failed, error: dial tcp: lookup gm.sedna on <IP_ADDRESS>:53: no such host
E0523 02:47:56.490911       1 websocket.go:200] max retry count reached when connecting global manager(address: gm.sedna:9000)
...

edgemesh log:
root@cloud:~# kubectl logs edgemesh-server-7cc4fbbc9b-8cm5x -n kubeedge
I0523 10:46:15.053005       1 config.go:39] Fetch public IP: <IP_ADDRESS>
I0523 10:46:15.053361       1 server.go:47] Version: v1.10.0-22+aa2119a4a1e838-dirty
I0523 10:46:15.053375       1 server.go:79] [1] New informers manager
W0523 10:46:15.053381       1 client_config.go:617] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
I0523 10:46:15.053773       1 server.go:86] [2] Register beehive modules
I0523 10:46:15.053800       1 manager.go:33] Use noCaManager to handle security
I0523 10:46:15.053814       1 nocamanager.go:44] Private key exist, skip generate
I0523 10:46:15.054957       1 module.go:34] Module tunnelserver registered successfully
I0523 10:46:15.054969       1 server.go:92] [3] Start informers manager
I0523 10:46:15.155719       1 server.go:96] [4] Start all modules
I0523 10:46:15.155783       1 core.go:24] Starting module tunnelserver
I0523 10:46:15.155906       1 tunnel.go:12] Start tunnel server success
I0523 10:46:15.155966       1 tunnel.go:14] Tunnel server addr : /ip4/<IP_ADDRESS>/tcp/20004/p2p/QmXb1kNC78vrVKYURXQuqp4SX4U8usswYwXZu3W17187MU
I0523 10:46:15.155978       1 tunnel.go:14] Tunnel server addr : /ip4/<IP_ADDRESS>/tcp/20004/p2p/QmXb1kNC78vrVKYURXQuqp4SX4U8usswYwXZu3W17187MU
I0523 10:46:15.155984       1 tunnel.go:14] Tunnel server addr : /ip4/<IP_ADDRESS>/tcp/20004/p2p/QmXb1kNC78vrVKYURXQuqp4SX4U8usswYwXZu3W17187MU
I0523 10:46:15.156172       1 tunnel.go:22] success update [edgemeshserver] addr [/ip4/<IP_ADDRESS>/tcp/20004 /ip4/<IP_ADDRESS>/tcp/20004 /ip4/<IP_ADDRESS>/tcp/20004] to secret
root@cloud:~# kubectl logs edgemesh-agent-znjwj -n kubeedge
2022/05/23 10:46:11 INFO: Install client plugin, protocol: rest
2022/05/23 10:46:11 INFO: Installed service discovery plugin: edge
I0523 10:46:11.867768       1 server.go:55] Version: v1.10.0-22+aa2119a4a1e838-dirty
I0523 10:46:11.867789       1 server.go:89] [1] New informers manager
W0523 10:46:11.867798       1 client_config.go:617] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
I0523 10:46:11.868147       1 server.go:96] [2] Prepare agent to run
I0523 10:46:11.868269       1 netif.go:98] bridge device edgemesh0 already exists
I0523 10:46:11.868314       1 server.go:100] edgemesh-agent running on CloudMode
I0523 10:46:11.868318       1 server.go:110] [3] Register beehive modules
W0523 10:46:11.868326       1 module.go:37] Module edgedns is disabled, do not register
I0523 10:46:11.877045       1 server.go:87] Using userspace Proxier.
I0523 10:46:11.890777       1 module.go:34] Module edgeproxy registered successfully
W0523 10:46:11.890796       1 module.go:37] Module edgegateway is disabled, do not register
I0523 10:46:11.890822       1 manager.go:33] Use noCaManager to handle security
I0523 10:46:11.890874       1 nocamanager.go:44] Private key exist, skip generate
I0523 10:46:11.892127       1 module.go:86] Listening on /ip4/<IP_ADDRESS>/tcp/20006/p2p/QmcwY9QWhBQ7PPDDnKMa3g4VY9xwgUXsumwWTyXNrkTv2y
I0523 10:46:11.892148       1 module.go:86] Listening on /ip4/<IP_ADDRESS>/tcp/20006/p2p/QmcwY9QWhBQ7PPDDnKMa3g4VY9xwgUXsumwWTyXNrkTv2y
I0523 10:46:11.892158       1 module.go:34] Module tunnelagent registered successfully
I0523 10:46:11.892165       1 server.go:116] [4] Start informers manager
I0523 10:46:11.992566       1 server.go:120] [5] Start all modules
I0523 10:46:11.992640       1 core.go:24] Starting module edgeproxy
I0523 10:46:11.992681       1 core.go:24] Starting module tunnelagent
I0523 10:46:11.992889       1 config.go:307] "Starting service config controller"
I0523 10:46:11.992923       1 shared_informer.go:240] Waiting for caches to sync for service config
I0523 10:46:11.992946       1 config.go:125] "Starting endpoints config controller"
I0523 10:46:11.992954       1 shared_informer.go:240] Waiting for caches to sync for endpoints config
I0523 10:46:11.992967       1 config.go:544] "Starting destination rule config controller"
I0523 10:46:11.992970       1 shared_informer.go:240] Waiting for caches to sync for destination rule config
W0523 10:46:11.993687       1 tunnel.go:28] Connection between agent and server [/ip4/<IP_ADDRESS>/tcp/20004 /ip4/<IP_ADDRESS>/tcp/20004 /ip4/<IP_ADDRESS>/tcp/20004] is not established, try connect
I0523 10:46:11.994006       1 tunnel.go:31] Tunnel agent connecting to tunnel server
W0523 10:46:11.997025       1 tunnel.go:34] Connect to server err: failed to dial QmXb1kNC78vrVKYURXQuqp4SX4U8usswYwXZu3W17187MU: all dials failed
  * [/ip4/<IP_ADDRESS>/tcp/20004] dial tcp4 <IP_ADDRESS>:20004: connect: connection refused
  * [/ip4/<IP_ADDRESS>/tcp/20004] dial tcp4 <IP_ADDRESS>:20004: connect: connection refused
  * [/ip4/<IP_ADDRESS>/tcp/20004] dial tcp4 <IP_ADDRESS>:20004: connect: connection refused
I0523 10:46:12.093284       1 shared_informer.go:247] Caches are synced for service config
I0523 10:46:12.093284       1 shared_informer.go:247] Caches are synced for destination rule config
I0523 10:46:12.093298       1 shared_informer.go:247] Caches are synced for endpoints config
I0523 10:46:13.997362       1 tunnel.go:31] Tunnel agent connecting to tunnel server
W0523 10:46:13.997535       1 tunnel.go:34] Connect to server err: dial backoff
I0523 10:46:15.055567       1 proxysocket.go:147] Dial legacy network between EMPTY_POD_NAME - {tcp EMPTY_NODE_NAME <IP_ADDRESS>:6443}
I0523 10:46:15.997881       1 tunnel.go:31] Tunnel agent connecting to tunnel server
W0523 10:46:15.998035       1 tunnel.go:34] Connect to server err: dial backoff
W0523 10:46:27.998925       1 tunnel.go:28] Connection between agent and server [/ip4/<IP_ADDRESS>/tcp/20004 /ip4/<IP_ADDRESS>/tcp/20004 /ip4/<IP_ADDRESS>/tcp/20004] is not established, try connect
I0523 10:46:27.998961       1 tunnel.go:31] Tunnel agent connecting to tunnel server
I0523 10:46:28.000643       1 tunnel.go:50] agent success connected to server [/ip4/<IP_ADDRESS>/tcp/20004 /ip4/<IP_ADDRESS>/tcp/20004 /ip4/<IP_ADDRESS>/tcp/20004]
I0523 10:46:40.491503       1 proxysocket.go:147] Dial legacy network between coredns-7f89b7bc75-wj7ch - {udp cloud.kubeedge <IP_ADDRESS>:53}
I0523 10:46:40.491687       1 proxysocket.go:147] Dial legacy network between coredns-7f89b7bc75-q5nzx - {udp cloud.kubeedge <IP_ADDRESS>:53}
I0523 10:46:40.494531       1 proxysocket.go:147] Dial legacy network between coredns-7f89b7bc75-wj7ch - {udp cloud.kubeedge <IP_ADDRESS>:53}
I0523 10:46:40.495576       1 proxysocket.go:147] Dial legacy network between coredns-7f89b7bc75-q5nzx - {udp cloud.kubeedge <IP_ADDRESS>:53}
E0523 10:46:40.496263       1 proxysocket.go:94] "Couldn't find an endpoint for service" err="missing endpoints" service="sedna/gm:tcp-0"
E0523 10:46:40.496342       1 proxysocket.go:192] "Failed to connect to balancer" err="missing endpoints"
I0523 10:46:40.534949       1 proxysocket.go:147] Dial legacy network between EMPTY_POD_NAME - {tcp EMPTY_NODE_NAME <IP_ADDRESS>:6443}
I0523 10:46:40.535084       1 proxysocket.go:147] Dial legacy network between EMPTY_POD_NAME - {tcp EMPTY_NODE_NAME <IP_ADDRESS>:6443}
I0523 10:46:40.535181       1 proxysocket.go:147] Dial legacy network between EMPTY_POD_NAME - {tcp EMPTY_NODE_NAME <IP_ADDRESS>:6443}
I0523 10:46:40.535263       1 proxysocket.go:147] Dial legacy network between EMPTY_POD_NAME - {tcp EMPTY_NODE_NAME <IP_ADDRESS>:6443}
I0523 10:46:40.535501       1 proxysocket.go:147] Dial legacy network between EMPTY_POD_NAME - {tcp EMPTY_NODE_NAME <IP_ADDRESS>:6443}
I0523 10:46:40.535877       1 proxysocket.go:147] Dial legacy network between EMPTY_POD_NAME - {tcp EMPTY_NODE_NAME <IP_ADDRESS>:6443}
I0523 10:46:40.535966       1 proxysocket.go:147] Dial legacy network between EMPTY_POD_NAME - {tcp EMPTY_NODE_NAME <IP_ADDRESS>:6443}
I0523 10:46:40.557132       1 proxysocket.go:147] Dial legacy network between EMPTY_POD_NAME - {tcp EMPTY_NODE_NAME <IP_ADDRESS>:6443}
E0523 10:46:40.565034       1 proxy.go:48] "I/O error occurred" err="readfrom tcp <IP_ADDRESS>:39484-><IP_ADDRESS>:6443: splice: connection reset by peer"
I0523 10:46:45.497219       1 proxysocket.go:147] Dial legacy network between coredns-7f89b7bc75-wj7ch - {udp cloud.kubeedge <IP_ADDRESS>:53}
I0523 10:46:45.497312       1 proxysocket.go:147] Dial legacy network between coredns-7f89b7bc75-q5nzx - {udp cloud.kubeedge <IP_ADDRESS>:53}
I0523 10:46:45.497689       1 proxysocket.go:147] Dial legacy network between coredns-7f89b7bc75-wj7ch - {udp cloud.kubeedge <IP_ADDRESS>:53}
I0523 10:46:45.497750       1 proxysocket.go:147] Dial legacy network between coredns-7f89b7bc75-q5nzx - {udp cloud.kubeedge <IP_ADDRESS>:53}
I0523 10:46:45.498213       1 proxysocket.go:147] Dial legacy network between gm-5bb9c898d6-tvsvx - {tcp cloud.kubeedge <IP_ADDRESS>:9000}
root@cloud:~# kubectl logs edgemesh-agent-ghfbf -n kubeedge
2022/05/23 10:46:11 INFO: Install client plugin, protocol: rest
2022/05/23 10:46:11 INFO: Installed service discovery plugin: edge
I0523 10:46:12.002230       1 server.go:55] Version: v1.10.0-22+aa2119a4a1e838-dirty
I0523 10:46:12.002252       1 server.go:89] [1] New informers manager
I0523 10:46:12.003755       1 server.go:96] [2] Prepare agent to run
I0523 10:46:12.004117       1 netif.go:98] bridge device edgemesh0 already exists
I0523 10:46:12.004198       1 server.go:100] edgemesh-agent running on EdgeMode
I0523 10:46:12.004212       1 server.go:110] [3] Register beehive modules
I0523 10:46:12.004558       1 module.go:34] Module edgedns registered successfully
I0523 10:46:12.028932       1 server.go:87] Using userspace Proxier.
I0523 10:46:12.041018       1 module.go:34] Module edgeproxy registered successfully
W0523 10:46:12.041035       1 module.go:37] Module edgegateway is disabled, do not register
I0523 10:46:12.041106       1 manager.go:33] Use noCaManager to handle security
I0523 10:46:12.042461       1 nocamanager.go:44] Private key exist, skip generate
I0523 10:46:12.045926       1 module.go:86] Listening on /ip4/<IP_ADDRESS>/tcp/20006/p2p/QmUw9iHnth3zMvu5STxF4yiAXPRBevbp3VfjJdAYgQ3ye1
I0523 10:46:12.045946       1 module.go:86] Listening on /ip4/<IP_ADDRESS>/tcp/20006/p2p/QmUw9iHnth3zMvu5STxF4yiAXPRBevbp3VfjJdAYgQ3ye1
I0523 10:46:12.045959       1 module.go:34] Module tunnelagent registered successfully
I0523 10:46:12.045964       1 server.go:116] [4] Start informers manager
I0523 10:46:12.146582       1 server.go:120] [5] Start all modules
I0523 10:46:12.146919       1 core.go:24] Starting module edgedns
I0523 10:46:12.147097       1 core.go:24] Starting module edgeproxy
I0523 10:46:12.147120       1 core.go:24] Starting module tunnelagent
I0523 10:46:12.147229       1 module.go:65] Runs CoreDNS v1.8.7 as a local dns
I0523 10:46:12.147990       1 config.go:307] "Starting service config controller"
I0523 10:46:12.148008       1 shared_informer.go:240] Waiting for caches to sync for service config
I0523 10:46:12.148033       1 config.go:125] "Starting endpoints config controller"
I0523 10:46:12.148038       1 shared_informer.go:240] Waiting for caches to sync for endpoints config
I0523 10:46:12.148063       1 config.go:544] "Starting destination rule config controller"
I0523 10:46:12.148066       1 shared_informer.go:240] Waiting for caches to sync for destination rule config
W0523 10:46:12.148911       1 tunnel.go:28] Connection between agent and server [/ip4/<IP_ADDRESS>/tcp/20004 /ip4/<IP_ADDRESS>/tcp/20004 /ip4/<IP_ADDRESS>/tcp/20004] is not established, try connect
I0523 10:46:12.148928       1 tunnel.go:31] Tunnel agent connecting to tunnel server
W0523 10:46:12.151186       1 tunnel.go:34] Connect to server err: failed to dial QmXb1kNC78vrVKYURXQuqp4SX4U8usswYwXZu3W17187MU: all dials failed
  * [/ip4/<IP_ADDRESS>/tcp/20004] dial tcp4 <IP_ADDRESS>:20004: connect: connection refused
  * [/ip4/<IP_ADDRESS>/tcp/20004] dial tcp4 <IP_ADDRESS>:20004: connect: connection refused
  * [/ip4/<IP_ADDRESS>/tcp/20004] dial tcp4 <IP_ADDRESS>:20004: connect: connection refused
.:53 on <IP_ADDRESS>
[INFO] plugin/reload: Running configuration MD5 = 0b3d85a9c43336189e8532480396fd1f
I0523 10:46:12.248964       1 shared_informer.go:247] Caches are synced for endpoints config
I0523 10:46:12.248969       1 shared_informer.go:247] Caches are synced for service config
I0523 10:46:12.249000       1 shared_informer.go:247] Caches are synced for destination rule config
CoreDNS-1.8.7
linux/amd64, go1.17.9,
[INFO] <IP_ADDRESS>:43904 - 42935 "HINFO IN<PHONE_NUMBER>067504715.1264191757939182058. udp 57 false 512" NXDOMAIN qr,rd,ra 132 0.022079928s
I0523 10:46:14.151964       1 tunnel.go:31] Tunnel agent connecting to tunnel server
W0523 10:46:14.152131       1 tunnel.go:34] Connect to server err: dial backoff
I0523 10:46:16.152364       1 tunnel.go:31] Tunnel agent connecting to tunnel server
W0523 10:46:16.152554       1 tunnel.go:34] Connect to server err: dial backoff
W0523 10:46:28.153911       1 tunnel.go:28] Connection between agent and server [/ip4/<IP_ADDRESS>/tcp/20004 /ip4/<IP_ADDRESS>/tcp/20004 /ip4/<IP_ADDRESS>/tcp/20004] is not established, try connect
I0523 10:46:28.153941       1 tunnel.go:31] Tunnel agent connecting to tunnel server
I0523 10:46:28.158105       1 tunnel.go:50] agent success connected to server [/ip4/<IP_ADDRESS>/tcp/20004 /ip4/<IP_ADDRESS>/tcp/20004 /ip4/<IP_ADDRESS>/tcp/20004]
[INFO] <IP_ADDRESS>:36951 - 33603 "AAAA IN gm.sedna.global.kubeedge. udp 42 false 512" NXDOMAIN qr,rd,ra 117 0.112335301s
[INFO] <IP_ADDRESS>:34080 - 46614 "A IN gm.sedna.global.kubeedge. udp 42 false 512" NXDOMAIN qr,rd,ra 117 0.29052987s
[INFO] <IP_ADDRESS>:42954 - 51897 "A IN gm.sedna. udp 26 false 512" FORMERR qr,rd 26 0.000685112s
[INFO] <IP_ADDRESS>:35543 - 158 "AAAA IN gm.sedna. udp 26 false 512" NXDOMAIN qr,rd,ra 101 0.055532415s
[INFO] <IP_ADDRESS>:36260 - 61498 "A IN gm.sedna. udp 26 false 512" NXDOMAIN qr,rd,ra 101 0.091239033s
[INFO] <IP_ADDRESS>:44168 - 3869 "AAAA IN gm.sedna.global.kubeedge. udp 42 false 512" NXDOMAIN qr,aa,rd,ra 117 0.00009085s
[INFO] <IP_ADDRESS>:48637 - 27971 "A IN gm.sedna.global.kubeedge. udp 42 false 512" NXDOMAIN qr,aa,rd,ra 117 0.000108263s
[INFO] <IP_ADDRESS>:59817 - 43359 "AAAA IN gm.sedna. udp 26 false 512" NXDOMAIN qr,aa,rd,ra 101 0.000043943s
[INFO] <IP_ADDRESS>:50366 - 47237 "A IN gm.sedna. udp 26 false 512" NXDOMAIN qr,aa,rd,ra 101 0.000031429s
[INFO] <IP_ADDRESS>:41017 - 24445 "A IN gm.sedna.global.kubeedge. udp 42 false 512" NXDOMAIN qr,aa,rd,ra 117 0.000072676s
[INFO] <IP_ADDRESS>:50437 - 3054 "AAAA IN gm.sedna.global.kubeedge. udp 42 false 512" NXDOMAIN qr,aa,rd,ra 117 0.000045966s
[INFO] <IP_ADDRESS>:53813 - 13315 "AAAA IN gm.sedna. udp 26 false 512" NXDOMAIN qr,aa,rd,ra 101 0.00003747s
[INFO] <IP_ADDRESS>:53874 - 55188 "A IN gm.sedna. udp 26 false 512" NXDOMAIN qr,aa,rd,ra 101 0.000038963s
[INFO] <IP_ADDRESS>:51476 - 41984 "A IN gm.sedna.global.kubeedge. udp 42 false 512" NXDOMAIN qr,aa,rd,ra 117 0.000182331s
[INFO] <IP_ADDRESS>:56501 - 38419 "AAAA IN gm.sedna.global.kubeedge. udp 42 false 512" NXDOMAIN qr,aa,rd,ra 117 0.00020319s
[INFO] <IP_ADDRESS>:55399 - 8208 "AAAA IN gm.sedna. udp 26 false 512" NXDOMAIN qr,aa,rd,ra 101 0.00003751s
[INFO] <IP_ADDRESS>:45889 - 39990 "A IN gm.sedna. udp 26 false 512" NXDOMAIN qr,aa,rd,ra 101 0.000059671s
[INFO] <IP_ADDRESS>:54435 - 40874 "A IN gm.sedna.global.kubeedge. udp 42 false 512" NXDOMAIN qr,aa,rd,ra 117 0.000076373s
[INFO] <IP_ADDRESS>:47847 - 44440 "AAAA IN gm.sedna.global.kubeedge. udp 42 false 512" NXDOMAIN qr,aa,rd,ra 117 0.000091801s
...

I have a similar issue. Cloud -> Edge connection is okay, but edge -> cloud works only for resolving and that is all. After that I've just an error: Connection closed by foreign host


Inconsistent syntax highlighting

atom v1.8.0 (package: angualrjs v0.3.4)
I noticed inconsistencies in angular's {{}} markup highlighting. There are three different cases in this HTML and they are highlighted differently

I use built-in language-javascript to highlight javascript code in angular expression.

language-javascript highlight the code like same. I'm not sure whether implementing it as own is better or not.

I see. I just found this weird and i thought it was good idea to report it.

I have the same issue but with the attribute called "title" ex: (post.title)


Suggestions for website improvements

Suggestions made by @kellijohnson-NOAA

[x] Review fix-grammar branch (@kellijohnson-NOAA made some grammar changes on the index.qmd page)
[ ] The website is not fully accessible to those with disabilities given that there is no alternative text for any of the images. I would recommend first fixing the lack of alternative text and then potentially running it through an accessibility checker.
[x] Another small thing, having clickable text that says "here" is frowned upon from those that cannot see given the word "here" is not informative about what they will be navigating to. I tried to change some of these in the branch called fix-grammar but there are probably some that I missed.
[x] In the "## Asking questions about Stock Synthesis" section on the front page it suggests that users can open an issue. I would suggest not including that as an option. Questions should be directed to the Discussion page.
[ ] Furthermore, rather than suggesting that users look to the VLAB page, I think it would be better to try to navigate the information from VLAB to the Discussion pages much like what we did for old issues.
[x] In the "## Contributing to Stock Synthesis" it suggests that future developers can either open a pull request or submit an issue when it should only be the latter. Pull requests should not be encouraged prior to creating an issue.
[x] In the "## Tools for working with Stock Synthesis" I suggest breaking out each of the Data Limited tools into its own line. And, it is not clear where readers should open an issue if they want their tool listed under this heading. Again, I suggest that you have open an issue rather than submit a pull request.
[ ] The page with how to cite the SS3 user manual should also have a link to where users can find the user manual, as well as information for what versions the manual would be helpful for. I am also not sure if we really want a link to the user manual on the front page of the website given the pdf is out of date with the current version of SS3. Also, I do not think that this link to citing the old SS3 user manual should be the first topic in the drop down menu for SS3 User Manuals at the top of the page.
[x] The small icons at the top of the home page next to the three drop down menus are too small to recognize what they are and do not really add definitive help for each of the three topics. Thus, I suggest removing them.
[x] When you click on the link to the html version of the SS3 User Manual in the drop down it should open a new page and not change the current page because there is no way to navigate back to the website.

@kellijohnson-NOAA do you have any suggestions for an accessibility checker to use?


Request for help tracking down missing "add" event

I'm not seeing an "add" event for changes to the filesystem.  I think this is unlikely to be a problem with chokidar, but I'm hoping someone may spot what I'm missing here, so I can understand what's going on.
Problem
Inside an Atom package, I'm waiting for two files to appear.  The files are created, by an external process.
The code I have is:
// test.js
var chokidar = require('chokidar');

files = [
  "/Users/richard/Developer/wip/slash/.ensime_cache/port",
  "/Users/richard/Developer/wip/slash/.ensime_cache/http"
];

chokidar.watch(files, {
  persistent: true
}).on('add', function(path) {
  return console.log('WATCH', path);
});

Inside Atom, that code only sees the "port" file appear.  That is, I expect to see in the console...
WATCH /Users/richard/Developer/wip/slash/.ensime_cache/port
WATCH /Users/richard/Developer/wip/slash/.ensime_cache/http

...but I actually only see:
WATCH /Users/richard/Developer/wip/slash/.ensime_cache/port

The files are created on disk.
However, if I run that code outside of Atom ($ node test.js, on a Mac, using node 4.2.1) I see the two files appear as expected.
Work around
If I watch for one file, and when it is "add"-ed go watch for the second one, I see both add events.  Example code: https://github.com/ensime/ensime-atom/pull/150/files
Request for help
I've become stuck in trying to understand why I see only one of the files appear when running inside Atom. Any suggestions for investigating this would be much appreciated.

Instead of watching the two file paths directly, using a glob pattern like /Users/richard/Developer/wip/slash/.ensime_cache/(http|port) may help since it will cause a watcher to be attached at the .ensime_cache directory level. Picking up add events can be spotty when setting watchers on specific file paths.

Thank you @es128 - I couldn't get that glob to match, but it did suggest another direction: watching the directory for add add events, and picking out the ones I want. Thank you.


Implement onSaveInstanceState()/onRestoreInstanceState() methods of ScrollableLayout

This PR fixed the issue #1.
Implemented onSaveInstanceState()/onRestoreInstanceState() methods of ScrollableLayout.

Implemented in latest (1.1.0) version


Fix GenerateJsonSchema.generate_definitions signature

This massively improves the GenerateJsonSchema.generate_definitions API — discovered the need for this while working with @tiangolo through FastAPI test failures with pydantic v2.
In particular, prior to this change, the first item returned by the GenerateJsonSchema.generate_definitions would only include a mapping to schemas that were "pure" definition schemas. Now, it will include every set of inputs as a mapped output.
In particular, we were able to make this usable from FastAPI for the purpose of exposing the mode='serialization' capability, fixing some bugs with the FastAPI generation, and significantly reduce the overhead of generating JSON schema — it previously needed to be computed twice because the map returned by the function didn't have all (or even most) of the data necessary; now it does. (This is used on the main-pv2-serialization-schemas.)
If there are any objections to making this change to the generate_definitions method in pydantic, I would suggest we add a new method with a new name that does exactly this. However, considering how much more useful this method is and the fact that we currently view GenerateJsonSchema as "unstable", I would be in favor of just making this change to that method's signature. Open to disagreement on this.

amazing 🚀 .


[docs] Describe passing the reporter object through the API

Fixes https://github.com/DevExpress/testcafe/issues/4723

@testcafe-build-bot \retest


Add Concordia University (Montreal, QC, Canada) [Solved]

Institution/School Name
{Concordia University, Montreal, QC, Canada}
Instiution/School Website
{http://www.concordia.ca}
Email Users
Please place an x between the square brackets if yes

[ x] Students
[ x] Academic/Teaching Staff
[x ] Other/Support Staff

Education Levels
Please place an x between the square brackets if yes

[ x] Post-graduate research
[x ] Graduate School
[x ] College (University) or Undergraduate School or equivalent
[ ] Community College or equivalent
[ ] High School or equivalent
[ ] Elementary School or equivalent

Hi,
It would appear that you have opened a pull-request containing commits from JetBrains/swot which is a fork of this repository (leereilly/swot).
If you intended to create a pull request for JetBrains/swot, may I suggest you create a pull request there.
We cannot accept your pull request if it contains more than one institution, as is the case here.
If you want your institution to be included please follow the following steps:

Delete your fork of swot
Create a new fork of swot based upon this repository
Create the file (and directories if necessary) to add the institution to your fork
Open a new pull request against this repository to merge your fork.

I am closing this pull request, as it is for the wrong repository.


404 on https://github.com/wodby/drupal-apache

Hi guys,
The documentation seems to point to this repo: https://github.com/wodby/drupal-apache but this a 404, however the dockerhub image for wodby/drupal-apache is available.
Has the git repo been deleted?

The image has been replaced with https://github.com/wodby/php-apache, TBA with the next d4d release

Links updated with 2.4.0 release


fix(material/datepicker): avoid re-rendering mat-calendar when minDat…

…e or maxDate is at a different time on the same day
Avoid re-rendering   when it receives a new value for
either the [minDate] or [maxDate] Input, and the value is at a different time on the same day.
Make two code changes to MatCalendar to accomplish this behavior change.

Normalize _minDate and _maxDate to midnight of the given day
Add check in ngOnChanges to re-render only if the current minDate
is not equal to the previous minDate.

@wagnermaciel @crisbeto this PR is ready for your eyes again 👀 . I've addressed the PR comments and it is no longer a draft. This is the most readable solution I could come up with, curious to know what you think.


Fix custom model pickling

Previous custom model pickle routine was using Cloudpickle which was causing errors to load a custom model in different python versions. Now we use plain pickle to serialize and load the model.

should we document the new requirement of saving the custom model into its own file ?

Yes, i will write a detailed documentation in our docs in a follow up PR

bors r+


Bug in PathPrefix's relative_join

This is an issue i tracked back to from the router.
So given this prefix:
prefix = Lotus::Utils::PathPrefix.new('fruits')

when joined with the same string
prefix.join('fruits') # => /fruits

will get rid of one of them only keeping the above /fruits instead of /fruits/fruits
Obviously that shouldn't be the case because we should be able to have paths such as /fruits/fruits/fruits/fruits if we wanted to.
Basically the issue is because PathPrefix uses Lotus::Kernel.Array which calls .uniq! on itself and removes any duplicate items.
I was able to fix the issue by using the standard array for the relative_join + calling chomp("/") on the result of the join to make some tests happy.
I'm not entirely sold on this solution because it seems to me that even though this fixed the problem entirely as far as I can tell (all tests pass*), Kernel.Array was put there for a reason beyond my understanding at this point. So please let me know what you think!
*While all tests pass on Utils, Router, Lotus and the example app with the initial problem (linked below), I had to amend 2 tests on Router which probably needs review since i'm not very happy with changing tests without knowing exactly what it is that it's testing.
I've added failing tests and solution to both Utils and Router. Please let me know what you think and if you need me to clarify anything.
Link to the app with the initial routing issue + failing tests:
https://github.com/theocodes/routertest
Thanks!

this is ok for me
@jodosha @joneslee85

@theocodes @AlfonsoUceda Thank you :+1: Merging this.


When two or more detail rows are open, errors occur by sorting table headers or clicking pagination buttons

Since Buefy version 0.6.1 issue #246 is back again.
Buefy version: 0.6.1
Vuejs version: 2.5.5
OS/Browser: MacOS 10.13.1 / Chrome 62
Although a detail-key is available, the following errors show up when opening the first detail row and sorting a column:

[Vue warn]: It seems there are duplicate keys that is causing an update error. Make sure each v-for item has a unique key.
[Vue warn]: Error in nextTick: "TypeError: Cannot read property 'key' of undefined"
TypeError: Cannot read property 'key' of undefined

A second issue that occurs is that rows disappear after sorting columns.

@wanxe can you take a look?

Of course @rafaelpimpa I think is fixed on  #426

As you can see on the video I can't reproduce the error, is possible to attach any fiddle? @sanderheilbron

Hello in my mind is not the correct solution to remove the :key from detail row.
Possible solution is to add a pre or postfix to the keys. Here is the closed Vue issue #7323 with the same explained problem.
Best regards Florian

Yes is a possible solution. Let me test it. Thanks sow much @fdrews :)

Your welcome and thank you for your great work.
If you test it, you will get another errors for duplicate keys in the docs components.


multiChoice: refresh only if it is necessary

I have strange glitches sometimes and I hope it can be fixed with this PR.

Thanks for this.


A ldap(s) authentification plugin

It could be great to have a ldap authentification plugin that give back principal with group details, that could then be use to whitelist an api.
best regards
serge

Currently we are working on LDAP Plugin which authenticate user using Bind Authentication.  We still defining the scope of the LDAP plugin and Authorization going to be part of it eventually.

@shashiranjan84 How would this integrate with Consumer?  Or would it be a replacement?

So far the plugin implementation validate user using the credential(username:password) sent in the request against the LDAP server using Bind Authentication and cache the credential for future use.
Please give it a try if possible https://github.com/Mashape/kong/tree/plugin/ldap

Closed with #1133.


maintainers: remove Ryan Cragun

This hasn't been true for a while, lets make it official.
Signed-off-by: Ryan Cragun<EMAIL_ADDRESS>

Wait, I'm still in @chef/chef-server-maintainers myself? That'll need to be updated.

@srenatus Updated

@ryancragun No worries if not, but if you would like feel free to move yourself to the "Past Maintainers" section.


Switch from yargs to minimist

For our simple usecase we do not need the extra functionality of yargs.
This removes 4 dependencies from our dependency graph.
https://github.com/bcoe/yargs
https://github.com/substack/minimist

:+1:


Possible hardcoded value returning a 404 Not Found resource error

Hi,
I believe there is forgotten userID hardcoded here: https://github.com/ugurkocde/IntuneAssignmentChecker/blob/ca3aa95ad3783807c8f0694ea293e9cf6348070d/IntuneAssignmentChecker_v2.ps1#L269
This obviously throws a "404 Not Found" error on the resource.
See:

In the v1, this line was instead:
$groupsUri = "https://graph.microsoft.com/v1.0/users/$userid/transitiveMemberOf?`$select=id,displayName"
I tested reverting this change manually, and it worked.
I'm sorry I can't make a PR as I didn't clone the repo. I'm using the script in standalone, by downloading it manually from the repo.

Hi Batiste,
Thank you! It’s fixed now.
Wishing you a great day from sunny Germany.
Best regards,
Ugur


Push pagination into database

Resolves https://github.com/ascoderu/opwen-webapp/issues/184

Codecov Report

Merging #185 into master will decrease coverage by 0.29%.
The diff coverage is 76.66%.

@@            Coverage Diff            @@
##           master     #185     +/-   ##
=========================================
- Coverage   69.65%   69.35%   -0.3%
=========================================
  Files          29       28      -1
  Lines        1687     1664     -23
=========================================
- Hits         1175     1154     -21
+ Misses        512      510      -2

Impacted Files
Coverage Δ

opwen_email_client/webapp/ioc.py
95.65% <ø> (ø)
:arrow_up:

opwen_email_client/webapp/views.py
47.36% <0%> (+0.15%)
:arrow_up:

opwen_email_client/domain/email/store.py
97.1% <100%> (ø)
:arrow_up:

opwen_email_client/webapp/config.py
100% <100%> (ø)
:arrow_up:

opwen_email_client/domain/email/sql_store.py
100% <100%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 53eb581...a1a8398. Read the comment docs.


warning at start - GDK XSetErrorHandler()

(java:214103): Gdk-WARNING **: 09:29:38.975: XSetErrorHandler() called with a GDK error trap pushed. Don't do that.
In terminal when starting AsciidocFX, can't say if important.
Kali ( debian ) Linux 2024-1, recent versions of all packages, AsciidocFX 1.8.8, 1.8.9

Is it just a warning or preventing the app functioning?

'just a warning', the program works, it sometimes switches to a 'mode' where it has
problems in positioning, accepting characters as typed ( jumps or produces wrong
characters on screen or the like ), can't say if related ... the problems likely pop up when
more than one ~400k files are open, not! actually short in CPU or mem, but on opening
touched roof of both. Only solution yet is restart.
Had hoped the warning would explain itself to you which I can't.

still observed in ver. 1.8.10


Istio Sidecar consuming high CPU Istio 1.30-1.3.3

Bug description
We are running Istio v1.3.3 (this also happend on v1.3.0), most of our applications are running fine. However, in one of our deployments the CPU on the sidecar is pegged at 1.0 for the life of the pod. This happens for the majority of the pods in the same deployment but not all. The pod contains nginx, istio-proxy and an application container all of which have normal CPU use except istio-proxy. I turned on debugging on the sidecar however there are not really interesting.
 istio-proxy [2019-09-25 09:45:06.322][27][debug][client] [external/envoy/source/common/http/codec_client.cc:95] [C1145026] response complete
 istio-proxy [2019-09-25 09:45:06.322][27][debug][filter] [src/envoy/http/mixer/filter.cc:214] Called Mixer::Filter : onDestroy state: 2
 istio-proxy [2019-09-25 09:45:06.322][27][debug][pool] [external/envoy/source/common/http/http1/conn_pool.cc:199] [C1145026] response complete
 istio-proxy [2019-09-25 09:45:06.322][27][debug][pool] [external/envoy/source/common/http/http1/conn_pool.cc:237] [C1145026] moving to ready
 istio-proxy [2019-09-25 09:45:06.322][27][debug][filter] [src/envoy/http/mixer/filter.cc:228] Called Mixer::Filter : log
 istio-proxy [2019-09-25 09:45:06.322][27][debug][filter] [./src/envoy/http/mixer/report_data.h:144] No dynamic_metadata found for filter envoy.filters.http.rbac
 istio-proxy [2019-09-25 09:45:06.323][27][debug][filter] [src/istio/control/client_context_base.cc:138] Report attributes: attributes {
 istio-proxy   key: "connection.mtls"
 istio-proxy   value {
 istio-proxy     bool_value: false
 istio-proxy   }
 istio-proxy }
 istio-proxy attributes {
 istio-proxy   key: "context.protocol"
 istio-proxy   value {
 istio-proxy     string_value: "http"
 istio-proxy   }
 istio-proxy }
 istio-proxy attributes {
 istio-proxy   key: "context.prox
 istio-proxy [2019-09-25 09:45:06.324][27][debug][connection] [external/envoy/source/common/network/connection_impl.cc:520] [C1145208] remote close
 istio-proxy [2019-09-25 09:45:06.324][27][debug][connection] [external/envoy/source/common/network/connection_impl.cc:190] [C1145208] closing socket: 0
 istio-proxy [2019-09-25 09:45:06.324][27][debug][main] [external/envoy/source/server/connection_handler_impl.cc:80] [C1145208] adding to cleanup list
 istio-proxy [2019-09-25 09:45:06.324][27][debug][filter] [src/envoy/tcp/mixer/filter.cc:100] [C1138233] Called tcp filter onRead bytes: 17
 istio-proxy [2019-09-25 09:45:06.324][28][debug][router] [external/envoy/source/common/router/router.cc:1002] [C637330][S2924433164096281138] upstream headers complete: end_stream=false
 istio-proxy [2019-09-25 09:45:06.324][28][debug][filter] [src/envoy/http/mixer/filter.cc:141] Called Mixer::Filter : encodeHeaders 2
 istio-proxy [2019-09-25 09:45:06.324][28][debug][http] [external/envoy/source/common/http/conn_manager_impl.cc:1435] [C637330][S2924433164096281138] encoding headers via codec (end_stream=false):

I was able to collect an strace from the PID of the problematic pod
sudo strace -c -f -p 5296
strace: Process 5296 attached with 15 threads
^Cstrace: Process 5296 detached
strace: Process 5329 detached
strace: Process 5330 detached
strace: Process 5331 detached
strace: Process 5332 detached
strace: Process 5338 detached
strace: Process 5340 detached
strace: Process 5341 detached
strace: Process 5342 detached
strace: Process 5343 detached
strace: Process 5344 detached
strace: Process 5345 detached
strace: Process 5346 detached
strace: Process 5347 detached
strace: Process 6356 detached
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 73.39    0.176384         256       687        44 futex
  9.04    0.021728          49       436           nanosleep
  8.82    0.021201         122       173           epoll_pwait
  6.45    0.015504        3100         5         2 restart_syscall
  0.64    0.001548          30        50           write
  0.50    0.001208          11       109        50 read
  0.31    0.000753          75        10           sched_yield
  0.30    0.000715          14        50        25 accept4
  0.24    0.000567          22        25           close
  0.16    0.000385           7        50           epoll_ctl
  0.08    0.000181           7        25           getsockname
  0.07    0.000178           7        25           setsockopt
------ ----------- ----------- --------- --------- ----------------
100.00    0.240352                  1645       121 total

Here is the output of
kubectl -n $namespace exec -it $pod -c istio-proxy -- top on a pod with busy-envoy
top - 15:33:18 up 18:25,  0 users,  load average: 1.58, 1.56, 1.53
Tasks:   3 total,   1 running,   2 sleeping,   0 stopped,   0 zombie
%Cpu(s): 22.5 us, 15.0 sy,  0.0 ni, 62.0 id,  0.0 wa,  0.0 hi,  0.4 si,  0.0 st
KiB Mem :  7807704 total,  1493152 free,  3275896 used,  3038656 buff/cache
KiB Swap:        0 total,        0 free,        0 used.  4147820 avail Mem

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
   21 istio-p+  20   0<PHONE_NUMBER>56  19180 S 104.7  1.4  51:18.67 envoy
    1 istio-p+  20   0  138296  25400  20892 S   0.0  0.3   0:07.84 pilot-agent
   33 istio-p+  20   0   36836   3016   2584 R   0.0  0.0   0:00.00 top

There has been a forum thread about it, but this is now ongoing without any resolution. So far the only workaround has been to downgrade the sidecar to 1.2.6
https://discuss.istio.io/t/istio-sidecar-consuming-high-cpu/3894
Any ideas on how I can troubleshoot further?
Affected product area (please put an X in all that apply)
[ ] Configuration Infrastructure
[ ] Docs
[ ] Installation
[ ] Networking
[x ] Performance and Scalability
[ ] Policies and Telemetry
[ ] Security
[ ] Test and Release
[ ] User Experience
[ ] Developer Infrastructure
Expected behavior
CPU not going high
Steps to reproduce the bug
Unable to reproduce
Version (include the output of istioctl version --remote and kubectl version)
citadel version: 1.3.0-1.3.3
galley version: 1.3.0-1.3.3
ingressgateway version: 1.3.0-1.3.3
pilot version: 1.3.0-1.3.3
policy version: 1.3.0-1.3.3
sidecar-injector version: 1.3.0-1.3.3
telemetry version: 1.3.0-1.3.3
How was Istio installed?
helm chart
Environment where bug was observed (cloud vendor, OS, etc)
amazon eks

from a different deployment with same issue
sudo strace -c -f -p 13144
strace: Process 13144 attached with 15 threads
^Cstrace: Process 13144 detached
strace: Process 13207 detached
strace: Process 13208 detached
strace: Process 13209 detached
strace: Process 13210 detached
strace: Process 13211 detached
strace: Process 13213 detached
strace: Process 13218 detached
strace: Process 13219 detached
strace: Process 13220 detached
strace: Process 13221 detached
strace: Process 13222 detached
strace: Process 13225 detached
strace: Process 13226 detached
strace: Process 15569 detached
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 63.45    0.008818          90        97         6 futex
 15.72    0.002184          28        77           nanosleep
 11.23    0.001561          57        27           epoll_pwait
  6.82    0.000948         189         5         2 restart_syscall
  1.17    0.000162          20         8           write
  0.50    0.000070          17         4           close
  0.42    0.000058           3        19         8 read
  0.36    0.000050           6         8         4 accept4
  0.21    0.000029           3         8           epoll_ctl
  0.09    0.000012           3         4           setsockopt
  0.04    0.000005           1         4           getsockname
  0.00    0.000000           0         1           sched_yield
------ ----------- ----------- --------- --------- ----------------
100.00    0.013897                   262        20 total

same issue

@elvizlai  / @crhuber Would you mind sharing some of your configuration dump?
Possibly using the dump_kubernetes.sh tool of this PR:  the dump_kubernetes.sh tools that we're working on: (c.f. this Pull Request) : https://github.com/istio/istio/pull/18013 ?

I'm also experiencing this, istio is using the whole CPU, and it autoscaled to 5 (using 5 vCPU).
I made a simple install on GKE using this parameters (no further customization)
helm install install/kubernetes/helm/istio-init --name istio-init --namespace istio-system --set certmanager.enabled=true

helm install install/kubernetes/helm/istio-cni --name istio-cni --namespace kube-system \
    --set cniBinDir=/home/kubernetes/bin \

helm install install/kubernetes/helm/istio --name istio --namespace istio-system \
    --set istio_cni.enabled=true \
    --set kiali.enabled=true \
    --set grafana.enabled=true \
    --set tracing.enabled=true \
    --set gateways.istio-ingressgateway.sds.enabled=true \
    --set gateways.istio-ingressgateway.resources.requests.cpu=500m \
    --set gateways.istio-ingressgateway.resources.requests.memory=256Mi \
    --set pilot.traceSampling=30 \
    --set sidecarInjectorWebhook.rewriteAppHTTPProbe=true \
    --set gateways.istio-ingressgateway.loadBalancerIP=""

We have recently upgraded from 1.2.0 to 1.3.1 and experiencing the same issue.
Did some investigation and it seems like the issue for us only appears when Istio acts as an HTTP proxy.
Sidecar is using anywhere between 1-2 CPU cores if the service definition has the name: http at the port section (as the documentation recommends it). If the sidecar acts only as a TCP proxy there is no issue.
For example, we only have at the moment liveness and readiness checks running, no real traffic.
In the proxy logs I'm seeing the switch when I add 'name: http' to the service definition port section;
`   ports:

name: http
nodePort: 30920
port: 80
protocol: TCP
targetPort: 6450
`

Log section:
[2019-11-04T10:44:56.676Z] "- - -" 0 - "-" "-" 0 0 3997 - "-" "-" "-" "-" "<IP_ADDRESS>:6450" inbound|80||service2.service2.svc.cluster.local <IP_ADDRESS>:44050 172.XX.XX.56:6450 172.XX.XX.35:59856 - - [2019-11-04T10:44:57.965Z] "- - -" 0 - "-" "-" 0 0 4002 - "-" "-" "-" "-" "<IP_ADDRESS>:6450" inbound|80||service2.service2.svc.cluster.local <IP_ADDRESS>:44098 172.XX.XX.56:6450 172.XX.XX.4:61737 - - [2019-11-04T10:45:03.751Z] "GET /liveness HTTP/1.1" 200 - "-" "-" 0 32 6 6 "-" "kube-probe/1.14" "3254795b-0327-469e-9fec-866184147f52" "172.XX.XX.56:6450" "<IP_ADDRESS>:6450" inbound|80|http|service2.service2.svc.cluster.local - 172.XX.XX.56:6450 172.XX.XX..35:53446 - default [2019-11-04T10:45:03.870Z] "GET /readiness HTTP/1.1" 200 - "-" "-" 0 215 5 4 "-" "kube-probe/1.14" "85ef2b38-03c3-4519-8601-ff0fc28745fd" "172.XX.XX.56:6450" "<IP_ADDRESS>:6450" inbound|80|http|service2.service2.svc.cluster.local - 172.XX.XX.56:6450 172.XX.XX.35:53452 - default
Service2 and service3 has 'name: http' at the port definition the others not.
CPU usage under with Istio 1.3.1:
NAMESPACE          NAME                                      CPU(cores)   MEMORY(bytes) service1           service1-65f7d9d8c4-26cd4                      8m           111Mi service2           service2-7f75b699d4-29q85                1115m        166Mi service3           service3-6fc5b956cf-sbmbt                     1502m        108Mi service4           service4-6f594f98c-zdq5p                      10m          169Mi
Downgraded back to Istio 1.2.0 and the issue disappeared.
CPU usage under 1.2.0:
NAMESPACE          NAME                                      CPU(cores)   MEMORY(bytes) service1       service1-65f7d9d8c4-zdq5p                      8m           109Mi service2      service2-7f75b699d4-hp5dq                      7m        169Mi service3           service3-6fc5b956cf-xt9bc                2m        100Mi service4           service4-6f594f98c-s9np6                     7m          170Mi
istio is getting deployed with the following helm template generation:
helm template /istio-${ISTIO_VERSION}/install/kubernetes/helm/istio \                 --name istio \                 --namespace istio-system \                 --set gateways.istio-ingressgateway.serviceAnnotations."service\.beta\.kubernetes\.io/azure-dns-label-name"="$DNS_LABEL_NAME" \                 --set gateways.istio-egressgateway.enabled='true' \                 --set global.outboundTrafficPolicy.mode="REGISTRY_ONLY" \                 --set global.proxy.accessLogFile="/dev/stdout" \                 --set global.controlPlaneSecurityEnabled=true \                 --set mixer.adapters.useAdapterCRDs=false \                 --set grafana.enabled=true \                 --set tracing.enabled=true \                 --set kiali.enabled=true \                 --set "kiali.dashboard.jaegerURL=http://jaeger-query:16686" \                 --set "kiali.dashboard.grafanaURL=http://grafana:3000" \                 > /home/admin/istio-$ISTIO_VERSION.yaml

We are also facing the issue after an upgrade from 1.2.4 to 1.3.3; as a temporary fix, we used the sidecar.istio.io/proxyImage annotation to downgrade the istio-proxy image to 1.2.4. This has been working out for the past 24 hours without other issues.
Furthermore, and I double checked it, the ingress gateways are not affected on our clusters.

Everyone, would you mind experimenting with a different proxy image that would include gdb and Envoy's symbols so we could try to see if some kind of infinite loop is identifiable in a backtrace?
If you're manually injecting Istio, you could replace in the injected sidecar (as a reminder, this could be generated by istioctl kube-inject -f yourconf.yaml, c.f.  bookinfo for example).
If you're running istio.1.3.3, it will look like replacing:
        image: docker.io/istio/proxyv2:1.3.3

by
        image: docker.io/francoispesce/proxyv2:1.3.3-gdb

You need to add to the existing securityContext, the capabilities required for gdb (SYS_PTRACE) as shown here:
        securityContext:
          readOnlyRootFilesystem: true
          runAsUser: 1337
          capabilities:
            add:
            - SYS_PTRACE

Once this is done and your pods running, the goal would be to attach a gdb to the running pid of envoy in the istio-proxy container, and share the output of info threads, to see if there's a pattern in your different use cases. You could also Ctrl-C and continue to figure out if one of the thread is looping abnormally (everything described here look like some kind of infinite loop).
Here's a quick example on how it would look like to run gdb and get these infos:
$ kubectl exec -it yourpod -c istio-proxy -- bash

istio-proxy@yourpod:/$ ps auxw
USER        PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
istio-p+      1  1.2  0.1<PHONE_NUMBER>6 ?        Ssl  07:17   0:00 /usr/local/bin/pilot-agent proxy sidecar --domain default.svc.cluster.local --configPath /etc/istio/proxy --binaryPath /usr/local/bin/envoy --ser
istio-p+     17 11.4  0.2<PHONE_NUMBER>2 ?        Sl   07:17   0:02 /usr/local/bin/envoy -c /etc/istio/proxy/envoy-rev0.json --restart-epoch 0 --drain-time-s 45 --parent-shutdown-time-s 60 --service-cluster produc
istio-p+     31  1.0  0.0  18260  3300 pts/0    Ss   07:17   0:00 bash
istio-p+     40  0.0  0.0  34424  2792 pts/0    R+   07:17   0:00 ps auxw
istio-proxy@productpage-v1-7895b8f545-nxrdg:/$ sudo gdb -p 17

(gdb) info threads
  Id   Target Id         Frame
* 1    Thread 0x7fcbde882d00 (LWP 17) "envoy" 0x00007fcbdda71a13 in epoll_wait () from /lib/x86_64-linux-gnu/libc.so.6
  2    Thread 0x7fcbdd969700 (LWP 20) "default-executo" 0x00007fcbddd41360 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
  3    Thread 0x7fcbdd168700 (LWP 21) "resolver-execut" 0x00007fcbddd41360 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
  4    Thread 0x7fcbdc967700 (LWP 22) "grpc_global_tim" 0x00007fcbddd41709 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
  5    Thread 0x7fcbdc166700 (LWP 23) "envoy" 0x00007fcbdda71a13 in epoll_wait () from /lib/x86_64-linux-gnu/libc.so.6
  6    Thread 0x7fcbdb965700 (LWP 24) "envoy" 0x00007fcbdda71a13 in epoll_wait () from /lib/x86_64-linux-gnu/libc.so.6
  7    Thread 0x7fcbdb164700 (LWP 25) "envoy" 0x00007fcbdda71a13 in epoll_wait () from /lib/x86_64-linux-gnu/libc.so.6
  8    Thread 0x7fcbda963700 (LWP 26) "envoy" 0x00007fcbdda71a13 in epoll_wait () from /lib/x86_64-linux-gnu/libc.so.6
  9    Thread 0x7fcbda162700 (LWP 27) "envoy" 0x00007fcbddd41360 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
  10   Thread 0x7fcbd9961700 (LWP 28) "envoy" 0x00007fcbddd41360 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
  11   Thread 0x7fcbd9160700 (LWP 29) "envoy" 0x00007fcbdda6b4d9 in syscall () from /lib/x86_64-linux-gnu/libc.so.6

I've built a bunch of these image for the various 1.3.x proxyv2, you can find them  on docker.io hub.
Just FYI, you can build the same following this simple Dockerfile template:
FROM istio/proxyv2:1.3.4
RUN apt-get update && \
    apt-get install --no-install-recommends -y \
      gdb \
   && apt-get upgrade -y \
   && apt-get clean \
   && rm -rf  /var/log/*log /var/lib/apt/lists/* /var/log/apt/* /var/lib/dpkg/*-old /var/cache/debconf/*-old

ADD https://storage.googleapis.com/istio-build/proxy/envoy-symbol-c33dc49585e5e7b5f616c8b5377a5f1f52505e20.tar.gz /tmp/
RUN rm /usr/local/bin/envoy \
  && tar -xvzf /tmp/envoy-symbol-c33dc49585e5e7b5f616c8b5377a5f1f52505e20.tar.gz --directory / \
  && rm /tmp/envoy-symbol-c33dc49585e5e7b5f616c8b5377a5f1f52505e20.tar.gz

You have to adjust for your version in the FROM tag (1.3.4 in this example) and pick the envoy symbol hash as indicated in istio.deps of the same version, e.g. https://github.com/istio/istio/blob/1.3.4/istio.deps would show c33dc49585e5e7b5f616c8b5377a5f1f52505e20 for the proxy "lastStableSHA": .

"> Once this is done and your pods running, the goal would be to attach a gdb to the running pid of envoy in the istio-proxy container, and share the output of info threads, to see if there's a pattern in your different use cases. You could also Ctrl-C and continue to figure out if one of the thread is looping abnormally (everything described here look like some kind of infinite loop)."

I did the test on one of our services in an environment of ours with the Image: docker.io/francoispesce/proxyv2:1.3.1-gdb:
(gdb) info threads
  Id   Target Id         Frame
* 1    Thread 0x7f358f17dd00 (LWP 15) "envoy" 0x00007f358e36ca13 in epoll_wait () from /lib/x86_64-linux-gnu/libc.so.6
  2    Thread 0x7f358e264700 (LWP 18) "default-executo" 0x00007f358e63c360 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
  3    Thread 0x7f358da63700 (LWP 19) "resolver-execut" 0x00007f358e63c360 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
  4    Thread 0x7f358d262700 (LWP 20) "grpc_global_tim" 0x00007f358e63c709 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
  5    Thread 0x7f358ca61700 (LWP 22) "envoy" 0x00007f358e36ca13 in epoll_wait () from /lib/x86_64-linux-gnu/libc.so.6
  6    Thread 0x7f358c260700 (LWP 23) "envoy" 0x00007f358e36ca13 in epoll_wait () from /lib/x86_64-linux-gnu/libc.so.6
  7    Thread 0x7f358ba5f700 (LWP 25) "envoy" 0x00007f358e3664d9 in syscall () from /lib/x86_64-linux-gnu/libc.so.6
  8    Thread 0x7f358b25e700 (LWP 26) "envoy" 0x00007f358e36ca13 in epoll_wait () from /lib/x86_64-linux-gnu/libc.so.6
  9    Thread 0x7f358a25c700 (LWP 27) "envoy" 0x00007f358e63c360 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
  10   Thread 0x7f358aa5d700 (LWP 28) "envoy" 0x00007f358e36ca13 in epoll_wait () from /lib/x86_64-linux-gnu/libc.so.6
  11   Thread 0x7f3589a5b700 (LWP 29) "envoy" 0x00007f358e63c360 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
  12   Thread 0x7f358925a700 (LWP 32) "envoy" 0x00007f358e3664d9 in syscall () from /lib/x86_64-linux-gnu/libc.so.6

I did the test on my production env where this issue happens, using docker.io/francoispesce/proxyv2:1.3.3-gdb
on a pod with high-cpu usage envoy:
(gdb) info threads
  Id   Target Id         Frame
* 1    Thread 0x7f1a5998ed00 (LWP 24) "envoy" 0x00007f1a58b7da13 in epoll_wait () from /lib/x86_64-linux-gnu/libc.so.6
  2    Thread 0x7f1a58a75700 (LWP 25) "default-executo" 0x00007f1a58e4d360 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
  3    Thread 0x7f1a58274700 (LWP 26) "resolver-execut" 0x00007f1a58e4d360 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
  4    Thread 0x7f1a57a73700 (LWP 27) "grpc_global_tim" 0x00007f1a58e4d709 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
  5    Thread 0x7f1a57272700 (LWP 28) "envoy" 0x00007f1a58b7da13 in epoll_wait () from /lib/x86_64-linux-gnu/libc.so.6
  6    Thread 0x7f1a56a71700 (LWP 29) "envoy" 0x00007f1a58b7da13 in epoll_wait () from /lib/x86_64-linux-gnu/libc.so.6
  7    Thread 0x7f1a56270700 (LWP 30) "envoy" 0x00007f1a58b7da13 in epoll_wait () from /lib/x86_64-linux-gnu/libc.so.6
  8    Thread 0x7f1a55a6f700 (LWP 31) "envoy" 0x00007f1a58b7da13 in epoll_wait () from /lib/x86_64-linux-gnu/libc.so.6
  9    Thread 0x7f1a5526e700 (LWP 32) "envoy" 0x00007f1a58e4d360 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
  10   Thread 0x7f1a54a6d700 (LWP 33) "envoy" 0x00007f1a58e4d360 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
  11   Thread 0x7f1a5426c700 (LWP 34) "envoy" 0x00007f1a58b774d9 in syscall () from /lib/x86_64-linux-gnu/libc.so.6

for comparison, on a a pod with no high-cpu usage envoy I've got this:
(gdb) info threads
  Id   Target Id         Frame
* 1    Thread 0x7f940423ed00 (LWP 22) "envoy" 0x00007f940342da13 in epoll_wait () from /lib/x86_64-linux-gnu/libc.so.6
  2    Thread 0x7f9403325700 (LWP 23) "default-executo" 0x00007f94036fd360 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
  3    Thread 0x7f9402b24700 (LWP 24) "resolver-execut" 0x00007f94036fd360 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
  4    Thread 0x7f9402323700 (LWP 25) "grpc_global_tim" 0x00007f94036fd709 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
  5    Thread 0x7f9401b22700 (LWP 26) "envoy" 0x00007f940342da13 in epoll_wait () from /lib/x86_64-linux-gnu/libc.so.6
  6    Thread 0x7f9401321700 (LWP 27) "envoy" 0x00007f940342da13 in epoll_wait () from /lib/x86_64-linux-gnu/libc.so.6
  7    Thread 0x7f9400b20700 (LWP 28) "envoy" 0x00007f94034274d9 in syscall () from /lib/x86_64-linux-gnu/libc.so.6
  8    Thread 0x7f940031f700 (LWP 29) "envoy" 0x00007f940342da13 in epoll_wait () from /lib/x86_64-linux-gnu/libc.so.6
  9    Thread 0x7f93ffb1e700 (LWP 30) "envoy" 0x00007f940342da13 in epoll_wait () from /lib/x86_64-linux-gnu/libc.so.6
  10   Thread 0x7f93ff31d700 (LWP 31) "envoy" 0x00007f94036fd360 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
  11   Thread 0x7f93feb1c700 (LWP 32) "envoy" 0x00007f94036fd360 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0

Strangely, after quitting the gdb process, envoy process was killed (did not show up on ps auxw anymore).
Moreover, on previous debugging, in our case, we were able to see (using strace on the node) that envoy was in a loop of epoll_wait() on a socket (from an established connection from another pod in the mesh). A weird thing was that on that connection there were only 2 segments sent in/out.
strace -c -f -p $pid
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 70.69   17.048433          38    448634           epoll_wait
 29.14    7.028070       42594       165        27 futex
  0.17    0.040000       40000         1         1 restart_syscall
  0.00    0.000000           0        58           read
  0.00    0.000000           0        59           write
  0.00    0.000000           0         1           close
  0.00    0.000000           0        41        20 readv
  0.00    0.000000           0        20           writev
  0.00    0.000000           0         9           epoll_ctl
------ ----------- ----------- --------- --------- ----------------
100.00   24.116503                448988        48 total

strace -f -r -p $pid on the node where the busy envoy runs, we see lots of
[pid 26651]      0.000036 epoll_wait(15, [{EPOLLOUT, {u32=603, u64=603}}], 128, 5) = 1
[pid 26651]      0.000036 epoll_wait(15, [{EPOLLOUT, {u32=603, u64=603}}], 128, 5) = 1
[pid 26651]      0.000037 epoll_wait(15, [{EPOLLOUT, {u32=603, u64=603}}], 128, 5) = 1
[pid 26651]      0.000038 epoll_wait(15, [{EPOLLOUT, {u32=603, u64=603}}], 128, 5) = 1
[pid 26651]      0.000042 epoll_wait(15, [{EPOLLOUT, {u32=603, u64=603}}], 128, 5) = 1
[pid 26651]      0.000045 epoll_wait(15, [{EPOLLOUT, {u32=603, u64=603}}], 128, 5) = 1
[pid 26651]      0.000562 epoll_wait(15, [{EPOLLOUT, {u32=603, u64=603}}], 128, 5) = 1
[pid 26651]      0.000051 epoll_wait(15, [{EPOLLOUT, {u32=603, u64=603}}], 128, 1) = 1
[pid 26651]      0.000038 epoll_wait(15, [{EPOLLOUT, {u32=603, u64=603}}], 128, 1) = 1

tcp    ESTAB      0      0      x.x.x.202:15006              x.x.x.203:36810               users:(("envoy",pid=29,fd=603))
     skmem:(r0,rb12582912,t0,tb12582912,f0,w0,o0,bl0) ts sack cubic wscale:9,9 rto:204 rtt:0.013/0.006 mss:8899 cwnd:10 segs_in:2 send 54763.1Mbps lastsnd:38913588 lastrcv:38913588 lastack:38913588 rcv_space:26697

Everyone, thanks again for your help, I'd like to ask again if you can share some of your configuration dump?
You can send it to me at [my-github-handle] @ google.com .
I'm curious about any network details (is there any UDP listeners configured for example).

@StefanCenusa
Could you share your config_dump of the busy sidecar proxy? I started to think about you have non-trivial configs/filters so that I cannot reproduce

@lambdai found the bug tonight.  fix will be out shortly.
In the meantime, we think setting global.proxy.protocolDetectionTimeout = 0 will address the issue for unpatched 1.3 versions.  See https://istio.io/docs/reference/config/installation-options/#global-options

@duderino I've set to 0ms but still having the issue. moreover, my pilot was already running with
  # if protocol sniffing is enabled for outbound
  enableProtocolSniffingForOutbound: false
  # if protocol sniffing is enabled for inbound
  enableProtocolSniffingForInbound: false

I guess I'll wait for the fix to try it out...

@duderino didn't work for me either

Try --set pilot.env.PILOT_INBOUND_PROTOCOL_DETECTION_TIMEOUT=0s --set global.proxy.protocolDetectionTimeout=0s. Somehow we manged to make this extremely complicated to configure...

@howardjohn @lambdai can you elaborate on the underlying root cause? Is it that the memory buffers allocated for protocol detection are not freed appropriately after timeout?

@sergii-s and everyone, make sure to apply the protocol detection timeout to the inbound filter for the workaround to be effective.
--set pilot.env.PILOT_INBOUND_PROTOCOL_DETECTION_TIMEOUT=0s --set global.proxy.protocolDetectionTimeout=0s

Try --set pilot.env.PILOT_INBOUND_PROTOCOL_DETECTION_TIMEOUT=0s --set global.proxy.protocolDetectionTimeout=0s
I have redeployed Istio 1.3.1 into one of our environments with these settings and can confirm that the high CPU usage disappears on the sidecars.

Experienced the same issue with istio 1.3.4. Fix with disabling protocol detection timeout worked for me.

This is fixed in 1.3.5 (released today), 1.4.0 (releasing soon), and mitigations provided for other releases, so I think its safe to close this. Thanks everyone for helping us track this down

This is fixed in 1.3.5 (released today), 1.4.0 (releasing soon), and mitigations provided for other releases, so I think its safe to close this. Thanks everyone for helping us track this down

I'll be honest, a little new to istio here, but I was having (i think) similar issues described in this thread.  I updated from 1.3.2 to 1.3.5 and still had high CPU, but after applying the Helm values here the CPU usage seems to have come way down.  I'll continue to monitor/verify, but wanted to chime in here hoping it's helpful

@jl-gogovapps its possible you did not update the sidecars? See https://istio.io/docs/setup/upgrade/steps/#sidecar-upgrade you need to actually restart the pods

Awesome news that it is fixed in 1.3.5. It would be appreciated if 1.3.5's release note mentioned about it.

@nak3 see

Security update
ISTIO-SECURITY-2019-006 A DoS vulnerability has been discovered in Envoy.
CVE-2019-18817: An infinite loop can be triggered in Envoy if the option continue_on_listener_filters_timeout is set to True, which is the case in Istio. This vulnerability could be leveraged for a DoS attack.

Oh! That is this one. I found the CVE was linked to this now. Thank you and sorry for bothering you @howardjohn

Thanks! I'm not sure how to know unfortunately.  I applied the changes (1.3.2 -> 1.3.5) through helm upgrade.  However I did not observe if the pods restarted, or if only the new istio services were rolled out.  I did observe the pods roll when I applied the configuration changes, which I did after the upgrade.
Either way, thats good to know!

So, in release 1.3.5 and above, it's not necessary keep these two settings?
pilot.env.PILOT_INBOUND_PROTOCOL_DETECTION_TIMEOUT=0s
global.proxy.protocolDetectionTimeout=0s

I need to use istio 1.0.5 because I am working on OPENSHIFT cluster. can anybody help me how do I setup
pilot.env.PILOT_INBOUND_PROTOCOL_DETECTION_TIMEOUT=0s
global.proxy.protocolDetectionTimeout=0s
these values in that case as manifest command is not there istioctl 1.0.5 version.

that feature isn't in 1.0, so you can't disable it since it doesn't exist
On Wed, Jan 8, 2020, 3:38 AM dudiakash<EMAIL_ADDRESS>wrote:

I need to use istio 1.0.5 because I am working on OPENSHIFT cluster. can
anybody help me how do I setup
pilot.env.PILOT_INBOUND_PROTOCOL_DETECTION_TIMEOUT=0s
global.proxy.protocolDetectionTimeout=0s
these values in that case as manifest command is not there istioctl 1.0.5
version.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/istio/istio/issues/18229?email_source=notifications&email_token=AAEYGXIIMNNDIOZKNUTMAGTQ4W3L7A5CNFSM4JEDXZXKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEIMDIZY#issuecomment-572011623,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAEYGXNCLTXF6OQS4THCCD3Q4W3L7ANCNFSM4JEDXZXA
.

I tried to use     --set gateways.istio-ingressgateway.resources.requests.cpu=500m but it doesn't works with istio 1.5.2


Added support for overriding column length: Resolves issue 173

This is an MR adding support for overriding the terminal column width when rendering ASCII graphs (e.g. dependencyTree)
This uses the inbuilt sbt 1.0 functionality of asciiGraphWidth to set this value (ensuring consistency with the sbt tasks that also render ascii graphs). There is backported support, which depends on an environment variable SBT_ASCII_GRAPH_WIDTH instead (to avoid complex setting requirements).

@jrudolph Hi, any plan to add this feature? Actually, I am eager to use it in my project.


Stats API

This will supplement/complement Filtering.
GetCards API:

Takes an optional CardFilter.cs which is essentially a mirror of Card and implements IEquatable<Card>.  Its props are all nullable, and unifies common string props as .Text.
Returns Dump.Where(CardFilter), a simple Card[]

Filter UI:

Consumes the Card[]
BUT ALSO:
Collates the filtered matches by property & value to provide match count data to display.

Collating this data is somewhat expensive.  This will still be a two-step process done server-side, but doing so:

Eliminates redundant CardFilter.ts and "clever" lifecycle & template logic
Allows for a direct mapping of API data to template.
Resembles future graphQL work.

done


Merge pull request #1 from yahoo/master

Merging updates from yahoo

Sorry, meant to pull into my fork. Great library!!

Hey! Not a problem! We are heavily working on Mendel 2.0 with a lot of great stuff incoming. I am publishing a lot of new documentation and will post the new architecture design soon.
It is a pleasure to know we have people interested in Mendel!

Don't worry, most stuff will be retro-compatible, so it is safe to use the current version right now if you'd like.

Yeah for sure! We have started playing with Mendel and I don't know of anything else like it. The flat folder/file structure, the server side require resolution, the dynamic bundle resolution and creation, variation inheritance, the total decoupling from assignment & tracking, and I know it does a ton more... really amazing work!! Thanks for open sourcing this!!
One question, if you don't mind. I'd like to know if I'm starting to understand layers correctly... Am I correct to think that a layer is a collection of mutually exclusive variations; where variations are defined, in a .mendelrc file, as one or more folders from the variationsdir (using variation inheritance, if desired)?
Let's say we have a file structure like this:
bash> tree

├── base
│   ├── file1.js
│   └── file2.js
└── variations
    ├── bucket_A
    │   └── file1.js
    ├── bucket_B
    │   └── file1.js
    ├── bucket_C
    │   └── file2.js
    └── bucket_D
        └── file2.js

And in our .mendelrc:
# ... #
variationsdir: variations
variations:
    bucket_A:
    bucket_B:
    bucket_C:
    bucket_D:

And now we want to do the assignment part... Would the following, conceptual example, make sense?

// false as a placeholder for base
const layer1 = [false, "bucket_A", "bucket_B"];
const layer2 = [false, "bucket_C", "bucket_D"];

const userId = 1234;

const variations = [
    hashUserIdToSingleElementInArray(userId, layer1),
    hashUserIdToSingleElementInArray(userId, layer2)
].filter(selection => !!selection)

var resolver = req.mendel.resolver(variations);

/* ... */

Does this seem like the right direction to be thinking in?

Yeah, that seems to make sense. I am not sure using userId will give you the correct uniform distribution you want, specially if userId is an incremental integer, this could mean new users get on an experiment consistently and old users consistently on base, and that's not a neutral distribution. But assuming the function hashUserIdToSingleElementInArray is a fair random assignment implementation, balanced the way you would like, your code should work fine.
Also — I think you already realized based on your example — once you go multilayer you can have conflicts, and if you poke around the code enough you will find that mendel-core already detects that. For instance, If in you next revision bucket_C starts implementing a variation of file1.js, it will conflict with bucket_A or bucket_B. MendelMiddleware currently ignores the conflicts, but you can get a list of runtime conflicts if you use findTreeForVariations directly.
PS: .filter(selection => !!selection) might read better as .filter(Boolean) 😉

Great, thanks for the confirmation and elaboration about conflicts.  I had noticed that with conflicts Mendel seemed to choose the file based on order in the .mendelrc file. I guess this is what you mean by MendelMiddleware ignores conflicts. So it seems the issues with getting conflicts is not that Mendel will break, but that an experiment could inadvertently not be the combination we intend. I played around with findTreeForVariations. That seems very useful. Thanks for pointing it out! Something like this, yeah?
require('mendel-core/trees')()
    .findTreeForVariations('main', ['bucket_A', 'bucket_C'])
    .conflictList

Yes, in production we'd do our best to make sure it's a fair random assignment and also that we don't get correlations between layers by only hashing on the userId. Probably in this case we'd hash on a string of userId uuid.v4() and the layer name ${userId}.${layerName}.  We have been working with the Planout library from facebook. So the real thing would probably end up looking more like this, assuming we wanted a uniform assignment:
const userId = uuid.v4();
const layerName = "layer1";
const layer = [false, "bucket_A", "bucket_B"];
const hash = sha1(`${userId}.${layerName}`);

const randIndex = new BigNumber(hash.substr(0, 15), 16).modulo(layer.length).toNumber();
const assignment = layer[randIndex];

That seem better?
PS: .filter(Boolean) nice one :+1:

LGTM
I guess you can use crypto directly and get this sha1 as buffer and slice the binary without turning it to string before converting to number, but I digress with over-optimizations =)
Here at Yahoo we use our proprietary assignment backend and I don't know much of the internals. In theory Mendel should work very well with Planout, but I didn't have time to try it out. This would be an interesting example to have on the repo and in the docs.

:+1: for over-optimization
would you like a PR that adds a Planout example?

I would be very please indeed! Any small contribution is very welcome, and an example with Planout is specially welcome, since we don't have any example with random assignment solutions out there!


Need to upgrade @opentelemetry/intstrumentation

"@opentelemetry/instrumentation" has some problem from "require-in-the-middle" that is hooking "require".
It is fixed in this pr. but is not deployed.
It makes so many performance problem.
And @opentelemetry/instrumentation version is greater than @splunk/otel's dependency version (^0.33.0).
I want to use fixed @opentelemetry/instrumentation with @splunk/otel.
Because I want to solve this problem, If you let me know how to know that the package upgrade.
Thanks.

Hi, which version of the distro are you using? We just released 2.0, which is using the latest instrumentation packages.
As for the fix for require in the middle, this will be brought in when it's published

I checked 2.0 rc.
But, It describe instrumentation package 0.33.0.
Does will it change?

Updated @opentelemetry/instrumentation to 0.34.0. Release coming soon.

Good news! Thanks


Temperature and Humidity Sensor not loading on hombridge

Hello there,
I've recently added a temperature and humidity sensor on Smart Life/tuya hub. On app, the sensor showed up and worked well, but not loaded on homebridge plugin. Any ideas how to solve it?

Hi @Mendelyeev ,This is the category currently supported.

Hi @Mendelyeev @Hanh94, I'm working on this and plan on opening a pull request soon :)
I'm adding support to device type wsdcg. This is the sensor I own and I'm using for the tests.

Hi @Mendelyeev @Hanh94, I'm working on this and plan on opening a pull request soon :)
I'm adding support to device type wsdcg. This is the sensor I own and I'm using for the tests.

HI @macecchi , this sensor is the same of mine!

This would be great


catkin_virtualenv: 0.6.0-1 in 'noetic/distribution.yaml' [bloom]

Increasing version of package(s) in repository catkin_virtualenv to 0.6.0-1:

upstream repository: https://github.com/locusrobotics/catkin_virtualenv.git
release repository: https://github.com/locusrobotics/catkin_virtualenv-release.git
distro file: noetic/distribution.yaml
bloom version: 0.9.7
previous version for package: null

catkin_virtualenv
* Don't require catkin_package to be called before catkin_generate_virtualenv (#67 <https://github.com/locusrobotics/catkin_virtualenv/issues/67>)
* Revert "Downgrade docutils so that boto works (#66 <https://github.com/locusrobotics/catkin_virtualenv/issues/66>)"
  This reverts commit 998cd6add2e43e12036d0db15a7c4d58fe3411cf.
* Downgrade docutils so that boto works (#66 <https://github.com/locusrobotics/catkin_virtualenv/issues/66>)
  See https://github.com/boto/botocore/issues/1942 and related threads.
* Make regex for Python bytecode more selective (#65 <https://github.com/locusrobotics/catkin_virtualenv/issues/65>)
  Fix regex to match only files ending in ".py[co]" and not files ending
  in "py[co]".
* Remove user specific paths (#63 <https://github.com/locusrobotics/catkin_virtualenv/issues/63>)
  * Remove user specific paths
  * Change working directory of venv_lock command
  * Lock catkin_virtualenv base requirements
  Co-authored-by: Paul Bovbel<EMAIL_ADDRESS>* RST-3172 Check that requirements file is locked (#62 <https://github.com/locusrobotics/catkin_virtualenv/issues/62>)
* Two helpful hints (#61 <https://github.com/locusrobotics/catkin_virtualenv/issues/61>)
* Fix input requirements warning (#58 <https://github.com/locusrobotics/catkin_virtualenv/issues/58>)
  * Only warn about INPUT_REQUIREMENTS if a package exports requirements to begin with
  * Update catkin_virtualenv/cmake/catkin_generate_virtualenv.cmake
  Co-authored-by: Andrew Blakey<EMAIL_ADDRESS>  Co-authored-by: Andrew Blakey<EMAIL_ADDRESS>* Preserve symlinks during copy (#57 <https://github.com/locusrobotics/catkin_virtualenv/issues/57>)
* Don't ignore unknown args
* RST-3172 Refactor catkin_virtualenv to allow locking dependencies (#55 <https://github.com/locusrobotics/catkin_virtualenv/issues/55>)
  * Remove unused options
  * Fix regex for comments
  * Migrate scripts
  * Remove old code
  * Move common requirements to an export file
  * Minor cleanup
  * Remove requirement-parsing unit tests
  * Fix logging config
  * Fix test builds
  * Generate lock files
  * Fix tests
  * Move dh-virtualenv functions into separate file
  * Fix roslint
  * Update docs
  * Update requirements
  * CMake comments
  * Fix pip-args
  * README fixup
  * Correct ARG_LOCK_FILE handling
  * Remove headers
  * Use set comprehension
  * Add migration doc
  * Respin
* Use exec to dive into python (#51 <https://github.com/locusrobotics/catkin_virtualenv/issues/51>)
* First python2 issue of 2020 (#49 <https://github.com/locusrobotics/catkin_virtualenv/issues/49>)
  * Clean up options, virtualenv installs setuptools by default
  * Make sure we install a compatible setuptools version for py2 venv
* catkin-pkg-modules has disappeared off pypi (#46 <https://github.com/locusrobotics/catkin_virtualenv/issues/46>)
  * catkin-pkg-modules has disappeared off pypi, but catkin-pkg is still there
  * Version all requirements
* Contributors: David V. Lu!!, Michael Johnson, Paul Bovbel, abencz

holding for the upcoming Noetic sync

@sloretz it is intentional - catkin_virtualenv allows a package to bundle either a python2 or python3 virtualenv, hence depending on both trees.

holding for the upcoming Noetic sync

missed that before requesting review, my bad.


Flutter hot reload on certain files crashes Dart compiler

Description
I ran into a situation where a Flutter app would build and run without any issues until hot reloading certain files that were changed.
First discovered when migrating the Flutter Gallery to make use of state restoration: https://github.com/flutter/gallery/pull/389.
This also only happens upon hot reload involves changed Dart files containing the class that extends RestorableListenable. The sample app only has one Dart file, but in the gallery app, this is more apparent since changes to files that do not contain the extended RestorableListenable class do not trigger the crash.
This issue was originally discovered while migrating the Reply and Shrine mini apps to use state restoration, both of which use state management packages (provider and scoped_model).
Steps to Reproduce
Reproducible sample: https://github.com/shihaohong/state_restoration_provider_crash
Steps to reproduce:

Run the Flutter application
Make any changes to the code containing the _RestorableCounterState class in the IDE (ie. adding spaces after a line of code) and save
Hot Reload and the Dart compiler crashes

Stack trace:
Unhandled exception:
root::package:hot_reload_crash/main.dart::_RestorableCounterState::@methods::package:flutter/src/widgets/restoration.dart::_debugAssertNotDisposed is already bound
#0      CanonicalName.bindTo (package:kernel/canonical_name.dart:192:7)
#1      Class.computeCanonicalNames (package:kernel/ast.dart:1201:51)
#2      Library.computeCanonicalNames (package:kernel/ast.dart:559:14)
#3      Component.computeCanonicalNamesForLibrary (package:kernel/ast.dart:10821:13)
#4      Component.computeCanonicalNames (package:kernel/ast.dart:10783:7)
#5      sortComponent (package:vm/kernel_front_end.dart:679:13)

#6      FrontendCompiler.writeDillFile (package:frontend_server/frontend_server.dart:656:5)
#7      FrontendCompiler.recompileDelta (package:frontend_server/frontend_server.dart:766:13)

/cc @johnniwinther @jensjoha

I guess this is the same issue as https://github.com/flutter/flutter/issues/72700, right?

It is -- I filed an issue both here and there because at the time I wasn't sure if it was a Dart or Flutter issue

I can't reproduce. What version (commit) of flutter did this happen on?

I can't reproduce. What version (commit) of flutter did this happen on?

Scratch that. When I said flutter run it did a web-version (which does other things with mixins) and it didn't reproduce. Specifying linux and I can reproduce (I suppose android/ios would also 'work' (i.e. crash)). I'll investigate!

Scratch that. When I said flutter run it did a web-version (which does other things with mixins) and it didn't reproduce. Specifying linux and I can reproduce (I suppose android/ios would also 'work' (i.e. crash)). I'll investigate!


TestNG @AfterClass step injected as the 1st step of the following test case

Test suite consists of 3 test classes.
1st test class contains: @Test  @AfterClass
2nd test class contains: @BeforeClass @Test  @AfterClass
3rd test class contains: @BeforeClass @Test @AfterClass
Issue: allure report put step from @AfterClass of 1st test class as a first step of 2nd test class just before the step from @BeforeClass;
Step from @AfterClass of 2nd test class as a first step if 3rd test class. Step from @AfterClass of 3rd test class is missing.
Expected result: @Step from TestNG @AfterClass is injected at the end of the same test class where @AfterClass method is placed. (Same as it works for @BeforeClass)

Reproduced on Allure v 1.4.19

@dmlemeshko thanks for suggestion. There is a lot of issues about TestNg. In few words - Allure doesn't support test class fixtures.
You can see: https://github.com/allure-framework/allure-core/issues/517 and https://github.com/allure-framework/allure-core/issues/570

Is there an ETA resolving this bug? Looks like it has been open for a year.

We are working on a new TestNG adaptor that will fix that issue. You can track the work on https://github.com/allure-framework/allure2/issues/108

fixed in new TestNG adaptor https://github.com/allure-framework/allure-java


User/Charity relationship, Email update

Fix models and migrations for user/charity relationship.
Use instance variable to display all users in admin dashboard
Fix spelling of Spare Change in emails.
Add scheduler for weekly emails
replace with better foodbank logo

Since last pull request I also fixed the charity routes to nest them properly.

@joshuacroff @tapley can you look over this and pull it? I ended up needing to fix this stuff before proceeding on other fronts. The tests we already have don't all pass. I am trying to get them passing before moving forward to make sure I am testing and breaking tests strategically. @joshuacroff if you want to push up a pull request for your stuff first, I can approve it first before you approve mine so I can resolve the conflicts. I just don't want this pull to get to big before merging.


cmake之后，提示错误，是什么原因？

Please try building vips from source as mentioned in the README:
sudo apt-get install build-essential pkg-config libglib2.0-dev libexpat1-dev libjpeg-dev libexif-dev libpng-dev libtiff-dev
wget https://github.com/jcupitt/libvips/releases/download/v8.6.1/vips-8.6.1.tar.gz
tar xzf vips-8.6.1.tar.gz
cd vips-8.6.1
./configure
make
sudo make install

Hope that works out for you


hciattach times out sometimes on first boot for rpi0w

Description
I'm using this layer with yocto to build an image for the rpi0-w, and when I first boot the image on the device sometimes it works flawlessly but other times it doesn't set up the bluetooth properly. If it doesn't work on the first boot then rebooting will usually fix it eventually, sometimes running hciattach manually fixes the problem as well. When I dig through the logs I find that the hciattach has timed out during boot.
Steps to reproduce the issue:

bitbake rpi-hwup-image
dd image onto sd card
boot

Describe the results you received:
Sometimes Bluetooth isn't working. There are no devices under hcitool dev or hciconfig. Also rfkill doesn't list the bluetooth device.
Describe the results you expected:
For bluetooth to setup the first time.
Additional information you deem important (e.g. issue happens only occasionally):
here is the output of systemctl status brcm43438:
Active: [[0;1;31mfailed[[0m (Result: exit-code) since Mon 2017-10-30 18:21:54 UTC; 28s ago
Process: 162 ExecStart=/usr/bin/hciattach -n /dev/ttyAMA0 bcm43xx 921600 noflow -[[0;1;31m(code=exited, status=1/FAILURE)[[0m
Main PID: 162 (code=exited, status=1/FAILURE)

systemd[1]: Started Broadcom BCM43438 bluetooth HCI.
hciattach[162]: Initialization timed out.
hciattach[162]: bcm43xx_init
systemd[1]: [[0;1;39mbrcm43438.service: Main process exited, code=exited, status=1/FAILURE[[0m
systemd[1]: [[0;1;39mbrcm43438.service: Unit entered failed state.[[0m
systemd[1]: [[0;1;39mbrcm43438.service: Failed with result `exit-code`.[[0m

Additional details (revisions used, host distro, etc.):
local.conf
BB_DISKMON_DIRS = "\
    STOPTASKS,${TMPDIR},1G,100K \
    STOPTASKS,${DL_DIR},1G,100K \
    STOPTASKS,${SSTATE_DIR},1G,100K \
    STOPTASKS,/tmp,100M,100K \
    ABORT,${TMPDIR},100M,1K \
    ABORT,${DL_DIR},100M,1K \
    ABORT,${SSTATE_DIR},100M,1K \
    ABORT,/tmp,10M,1K"
CONF_VERSION = "1"
CORE_IMAGE_EXTRA_INSTALL += "openssh rsync sensorhub"
IMAGE_INSTALL_append += "nodejs nodejs-npm bluez5 networkmanager linux-firmware-bcm43430 config systemd"
IMAGE_INSTALL_append += " crda i2c-tools hostapd linux-firmware-bcm43430 linux-firmware-ralink linux-firmware-rtl8192ce linux-firmware-rtl8192cu linux-firmware-rtl8192su wireless-tools dhcp-client iw wpa-supplicant"
DISTRO_FEATURES_append += " bluez5 bluetooth wifi systemd"```

I've done some more testing, it seems that systemctl restart brcm43438 fixes the problem consistently without a reboot. My band-aid solution is going to be adding a script on startup that checks if bluetooth is broken and if it is it will run that to fix it. Hopefully, that works until we can find a permanent solution :).

That sounds like a hardware issue. Are you able to reproduce this on a raspbian?

I never had this issue on raspbian. I also tested it on 5+ pi's. For what it's worth, once I got the reset script working I never had any more problems. The key is to make sure that nothing tries to use bluetooth before it has been set up properly, otherwise, you have to reboot before it'll work again.

If you stop the service and run it manually after everything boots up, does it always work? I'm wondering if there is a dependency that we are not aware of.

Yeah, that fixes it fairly consistently. I'll do some tests to see if anything have changed over the last month, but here is the script that I've been using that has worked consistently:
if [ ! -d "/sys/class/bluetooth/hci0" ]; then
        sleep 20
        if [ ! -d "/sys/class/bluetooth/hci0" ]; then
                systemctl restart brcm43438
                sleep 20
                systemctl restart start-sensorhub
        fi
fi
echo nameserver <IP_ADDRESS> > /etc/resolv.conf```

It's definitely something that we need to fix for stabolity.

Also, I tried removing my fix and the problem returned.

i have the same issue, tested with systemv and systemd, i can not get the device to get initialized.

i tried to reproduce the colleagues error. First I am using rpi0w, in the image, there is no /etc/firmware folder, the folder is installed in lib/firmware/ second it does not have the bcm43430A1.hcd file. As far as i know, the file is needed to reflash the chip. When i created the folders manually in mkdir -p /etc/firmware/brcm/ and copied the files to these locations, i get intialization time out.  I then copied the file LICENSE.broadcom_bcm43xx from /lib/firmware to /etc/firmware and i ran hciattach again, i get patch not found .. which patch does it mean? may be the firmware file BCM43430A1.hcd ? thank you

Moving https://github.com/agherzan/meta-raspberrypi/issues/174 discussion here. As per @sherifomran investigation this sounds like an outdated firmware. Is anybody able to provide a patch with an updated version of it to master? @keeslinp are you able to test this as well?


Complete version of compose00 crashes

even though the Horn version typechecks.

This is caused by multiple problems.  For one, #collect:ing over TCon bogusly left r unchanged; this is fixed in de1978e.  Others are fixed in #131.


Bug in FxTagTracker::UpdateTagHistory

I found an error in the FxTagTracker :: UpdateTagHistory method. If the m_CurRefHistory counter overflows, the pos% TAG_HISTORY_DEPTH value becomes negative and points outside the m_TagHistory array. This causes BugCheck.
See https://github.com/microsoft/Windows-Driver-Frameworks/blob/168a37f03e6498f483cb8eddf700b62680ec8d1d/src/framework/shared/object/fxtagtracker.cpp#L177

Right after it there is pos %= TAG_HISTORY_DEPTH; and comment that it would never be negative.