CI: K8sFQDNTest Restart Cilium validate that FQDN is still working: Error reaching kube-dns before test

https://jenkins.cilium.io/job/cilium-master-k8s-1.17-kernel-4.9/132/testReport/Suite-k8s-1/17/K8sFQDNTest_Restart_Cilium_validate_that_FQDN_is_still_working/
3205f837_K8sFQDNTest_Restart_Cilium_validate_that_FQDN_is_still_working.zip
Stacktrace
/home/jenkins/workspace/cilium-master-k8s-1.17-kernel-4.9/src/github.com/cilium/cilium/test/ginkgo-ext/scopes.go:465
Error reaching kube-dns before test: error looking up kube-dns.kube-system.svc.cluster.local from default/app2-5cc5d58844-nv6wr: ;; connection timed out; no servers could be reached

command terminated with exit code 1

Expected
    <*errors.errorString | 0xc0024981b0>: {
        s: "error looking up kube-dns.kube-system.svc.cluster.local from default/app2-5cc5d58844-nv6wr: ;; connection timed out; no servers could be reached\n\ncommand terminated with exit code 1\n",
    }
to be nil
/home/jenkins/workspace/cilium-master-k8s-1.17-kernel-4.9/src/github.com/cilium/cilium/test/k8sT/fqdn.go:89

Standard Output
Cilium pods: [cilium-mxkd7 cilium-zggcq]
Netpols loaded:
CiliumNetworkPolicies loaded:
Endpoint Policy Enforcement:
Pod                          Ingress   Egress
grafana-7fd557d749-qs865
prometheus-d87f8f984-7rqc2
app1-7b6ddb776f-9q5nl
app1-7b6ddb776f-n4vvx
app2-5cc5d58844-nv6wr
app3-6c7856c5b5-fs6n9
coredns-767d4c6dd7-jsfcl
Cilium agent 'cilium-mxkd7': Status: Ok  Health: Ok Nodes "" ContinerRuntime:  Kubernetes: Ok KVstore: Ok Controllers: Total 36 Failed 0
Cilium agent 'cilium-zggcq': Status: Ok  Health: Ok Nodes "" ContinerRuntime:  Kubernetes: Ok KVstore: Ok Controllers: Total 35 Failed 0

Standard Error
Click to show
15:21:47 STEP: Running BeforeAll block for EntireTestsuite K8sFQDNTest
15:21:47 STEP: Ensuring the namespace kube-system exists
15:21:47 STEP: WaitforPods(namespace="kube-system", filter="-l k8s-app=cilium-test-logs")
15:21:47 STEP: WaitforPods(namespace="kube-system", filter="-l k8s-app=cilium-test-logs") => <nil>
15:21:47 STEP: Installing Cilium
15:21:48 STEP: Waiting for Cilium to become ready
15:22:18 STEP: Validating if Kubernetes DNS is deployed
15:22:18 STEP: Checking if deployment is ready
15:22:18 STEP: Checking if kube-dns service is plumbed correctly
15:22:18 STEP: Checking if DNS can resolve
15:22:18 STEP: Checking if pods have identity
15:22:20 STEP: Kubernetes DNS is up and operational
15:22:20 STEP: Validating Cilium Installation
15:22:20 STEP: Performing Cilium controllers preflight check
15:22:20 STEP: Performing Cilium health check
15:22:20 STEP: Performing Cilium status preflight check
15:22:26 STEP: Performing Cilium service preflight check
15:22:26 STEP: Performing K8s service preflight check
15:22:26 STEP: Cilium is not ready yet: connectivity health is failing: Cluster connectivity is unhealthy on 'cilium-mxkd7': Exitcode: 1
Err: exit status 1
Stdout:

Stderr:
 	 Error: Cannot get status/probe: Put "http://%2Fvar%2Frun%2Fcilium%2Fhealth.sock/v1beta/status/probe": dial unix /var/run/cilium/health.sock: connect: no such file or directory

	 command terminated with exit code 1

15:22:26 STEP: Performing Cilium controllers preflight check
15:22:26 STEP: Performing Cilium status preflight check
15:22:26 STEP: Performing Cilium health check
15:22:28 STEP: Performing Cilium service preflight check
15:22:28 STEP: Performing K8s service preflight check
15:22:28 STEP: Performing Cilium controllers preflight check
15:22:28 STEP: Performing Cilium health check
15:22:28 STEP: Performing Cilium status preflight check
15:22:30 STEP: Performing Cilium service preflight check
15:22:30 STEP: Performing K8s service preflight check
15:22:30 STEP: Performing Cilium controllers preflight check
15:22:30 STEP: Performing Cilium status preflight check
15:22:30 STEP: Performing Cilium health check
15:22:31 STEP: Performing Cilium service preflight check
15:22:31 STEP: Performing K8s service preflight check
15:22:31 STEP: Performing Cilium status preflight check
15:22:31 STEP: Performing Cilium controllers preflight check
15:22:31 STEP: Performing Cilium health check
15:22:34 STEP: Performing Cilium service preflight check
15:22:34 STEP: Performing K8s service preflight check
15:22:34 STEP: Performing Cilium controllers preflight check
15:22:34 STEP: Performing Cilium status preflight check
15:22:34 STEP: Performing Cilium health check
15:22:35 STEP: Performing Cilium service preflight check
15:22:35 STEP: Performing K8s service preflight check
15:22:35 STEP: Performing Cilium controllers preflight check
15:22:35 STEP: Performing Cilium health check
15:22:35 STEP: Performing Cilium status preflight check
15:22:38 STEP: Performing Cilium service preflight check
15:22:38 STEP: Performing K8s service preflight check
15:22:38 STEP: Cilium is not ready yet: connectivity health is failing: Cluster connectivity is unhealthy on 'cilium-mxkd7': Exitcode: 1
Err: exit status 1
Stdout:

Stderr:
 	 Error: Cannot get status/probe: Put "http://%2Fvar%2Frun%2Fcilium%2Fhealth.sock/v1beta/status/probe": dial unix /var/run/cilium/health.sock: connect: no such file or directory

	 command terminated with exit code 1

15:22:38 STEP: Performing Cilium controllers preflight check
15:22:38 STEP: Performing Cilium status preflight check
15:22:38 STEP: Performing Cilium health check
15:22:40 STEP: Performing Cilium service preflight check
15:22:40 STEP: Performing K8s service preflight check
15:22:40 STEP: Performing Cilium controllers preflight check
15:22:40 STEP: Performing Cilium status preflight check
15:22:40 STEP: Performing Cilium health check
15:22:41 STEP: Performing Cilium service preflight check
15:22:41 STEP: Performing K8s service preflight check
15:22:41 STEP: Performing Cilium status preflight check
15:22:41 STEP: Performing Cilium controllers preflight check
15:22:41 STEP: Performing Cilium health check
15:22:43 STEP: Performing Cilium service preflight check
15:22:43 STEP: Performing K8s service preflight check
15:22:43 STEP: Performing Cilium controllers preflight check
15:22:43 STEP: Performing Cilium status preflight check
15:22:43 STEP: Performing Cilium health check
15:22:45 STEP: Performing Cilium service preflight check
15:22:45 STEP: Performing K8s service preflight check
15:22:45 STEP: Performing Cilium controllers preflight check
15:22:45 STEP: Performing Cilium health check
15:22:45 STEP: Performing Cilium status preflight check
15:22:47 STEP: Performing Cilium service preflight check
15:22:47 STEP: Performing K8s service preflight check
15:22:47 STEP: Performing Cilium controllers preflight check
15:22:47 STEP: Performing Cilium status preflight check
15:22:47 STEP: Performing Cilium health check
15:22:48 STEP: Performing Cilium service preflight check
15:22:48 STEP: Performing K8s service preflight check
15:22:48 STEP: Cilium is not ready yet: connectivity health is failing: Cluster connectivity is unhealthy on 'cilium-mxkd7': Exitcode: 1
Err: exit status 1
Stdout:

Stderr:
 	 Error: Cannot get status/probe: Put "http://%2Fvar%2Frun%2Fcilium%2Fhealth.sock/v1beta/status/probe": dial unix /var/run/cilium/health.sock: connect: no such file or directory

	 command terminated with exit code 1

15:22:48 STEP: Performing Cilium controllers preflight check
15:22:48 STEP: Performing Cilium health check
15:22:48 STEP: Performing Cilium status preflight check
15:22:50 STEP: Performing Cilium service preflight check
15:22:50 STEP: Performing K8s service preflight check
15:22:50 STEP: Performing Cilium controllers preflight check
15:22:50 STEP: Performing Cilium status preflight check
15:22:50 STEP: Performing Cilium health check
15:22:51 STEP: Performing Cilium service preflight check
15:22:51 STEP: Performing K8s service preflight check
15:22:51 STEP: Performing Cilium controllers preflight check
15:22:51 STEP: Performing Cilium health check
15:22:51 STEP: Performing Cilium status preflight check
15:22:54 STEP: Performing Cilium service preflight check
15:22:54 STEP: Performing K8s service preflight check
15:22:54 STEP: Performing Cilium status preflight check
15:22:54 STEP: Performing Cilium controllers preflight check
15:22:54 STEP: Performing Cilium health check
15:22:57 STEP: Performing Cilium service preflight check
15:22:57 STEP: Performing K8s service preflight check
15:22:57 STEP: Performing Cilium controllers preflight check
15:22:57 STEP: Performing Cilium status preflight check
15:22:57 STEP: Performing Cilium health check
15:23:00 STEP: Performing Cilium service preflight check
15:23:00 STEP: Performing K8s service preflight check
15:23:00 STEP: Performing Cilium controllers preflight check
15:23:00 STEP: Performing Cilium health check
15:23:00 STEP: Performing Cilium status preflight check
15:23:01 STEP: Performing Cilium service preflight check
15:23:01 STEP: Performing K8s service preflight check
15:23:01 STEP: Cilium is not ready yet: connectivity health is failing: Cluster connectivity is unhealthy on 'cilium-mxkd7': Exitcode: 1
Err: exit status 1
Stdout:

Stderr:
 	 Error: Cannot get status/probe: Put "http://%2Fvar%2Frun%2Fcilium%2Fhealth.sock/v1beta/status/probe": dial unix /var/run/cilium/health.sock: connect: no such file or directory

	 command terminated with exit code 1

15:23:01 STEP: Performing Cilium controllers preflight check
15:23:01 STEP: Performing Cilium health check
15:23:01 STEP: Performing Cilium status preflight check
15:23:02 STEP: Performing Cilium service preflight check
15:23:02 STEP: Performing K8s service preflight check
15:23:02 STEP: Performing Cilium controllers preflight check
15:23:02 STEP: Performing Cilium health check
15:23:02 STEP: Performing Cilium status preflight check
15:23:03 STEP: Performing Cilium service preflight check
15:23:03 STEP: Performing K8s service preflight check
15:23:03 STEP: Performing Cilium controllers preflight check
15:23:03 STEP: Performing Cilium status preflight check
15:23:03 STEP: Performing Cilium health check
15:23:05 STEP: Performing Cilium service preflight check
15:23:05 STEP: Performing K8s service preflight check
15:23:05 STEP: Performing Cilium controllers preflight check
15:23:05 STEP: Performing Cilium health check
15:23:05 STEP: Performing Cilium status preflight check
15:23:08 STEP: Performing Cilium service preflight check
15:23:08 STEP: Performing K8s service preflight check
15:23:08 STEP: Performing Cilium controllers preflight check
15:23:08 STEP: Performing Cilium status preflight check
15:23:08 STEP: Performing Cilium health check
15:23:10 STEP: Performing Cilium service preflight check
15:23:10 STEP: Performing K8s service preflight check
15:23:10 STEP: Performing Cilium controllers preflight check
15:23:10 STEP: Performing Cilium status preflight check
15:23:10 STEP: Performing Cilium health check
15:23:11 STEP: Performing Cilium service preflight check
15:23:11 STEP: Performing K8s service preflight check
15:23:11 STEP: Cilium is not ready yet: connectivity health is failing: Cluster connectivity is unhealthy on 'cilium-mxkd7': Exitcode: 1
Err: exit status 1
Stdout:

Stderr:
 	 Error: Cannot get status/probe: Put "http://%2Fvar%2Frun%2Fcilium%2Fhealth.sock/v1beta/status/probe": dial unix /var/run/cilium/health.sock: connect: no such file or directory

	 command terminated with exit code 1

15:23:11 STEP: Performing Cilium controllers preflight check
15:23:11 STEP: Performing Cilium health check
15:23:11 STEP: Performing Cilium status preflight check
15:23:13 STEP: Performing Cilium service preflight check
15:23:13 STEP: Performing K8s service preflight check
15:23:13 STEP: Performing Cilium controllers preflight check
15:23:13 STEP: Performing Cilium health check
15:23:13 STEP: Performing Cilium status preflight check
15:23:16 STEP: Performing Cilium service preflight check
15:23:16 STEP: Performing K8s service preflight check
15:23:16 STEP: Performing Cilium status preflight check
15:23:16 STEP: Performing Cilium controllers preflight check
15:23:16 STEP: Performing Cilium health check
15:23:18 STEP: Performing Cilium service preflight check
15:23:18 STEP: Performing K8s service preflight check
15:23:19 STEP: Waiting for cilium-operator to be ready
15:23:19 STEP: WaitforPods(namespace="kube-system", filter="-l name=cilium-operator")
15:23:19 STEP: WaitforPods(namespace="kube-system", filter="-l name=cilium-operator") => <nil>
15:23:19 STEP: Applying demo manifest
15:23:19 STEP: WaitforPods(namespace="default", filter="-l zgroup=testapp")
15:23:29 STEP: WaitforPods(namespace="default", filter="-l zgroup=testapp") => <nil>
FAIL: Error reaching kube-dns before test: error looking up kube-dns.kube-system.svc.cluster.local from default/app2-5cc5d58844-nv6wr: ;; connection timed out; no servers could be reached

command terminated with exit code 1

Expected
    <*errors.errorString | 0xc0024981b0>: {
        s: "error looking up kube-dns.kube-system.svc.cluster.local from default/app2-5cc5d58844-nv6wr: ;; connection timed out; no servers could be reached\n\ncommand terminated with exit code 1\n",
    }
to be nil
===================== TEST FAILED =====================
15:27:29 STEP: Running AfterFailed block for EntireTestsuite K8sFQDNTest
cmd: kubectl get pods -o wide --all-namespaces
Exitcode: 0
Stdout:
 	 NAMESPACE           NAME                               READY   STATUS    RESTARTS   AGE     IP              NODE   NOMINATED NODE   READINESS GATES
	 cilium-monitoring   grafana-7fd557d749-qs865           1/1     Running   0          72m     <IP_ADDRESS>       k8s2   <none>           <none>
	 cilium-monitoring   prometheus-d87f8f984-7rqc2         1/1     Running   0          72m     <IP_ADDRESS>        k8s2   <none>           <none>
	 default             app1-7b6ddb776f-9q5nl              2/2     Running   0          4m13s   <IP_ADDRESS>       k8s1   <none>           <none>
	 default             app1-7b6ddb776f-n4vvx              2/2     Running   0          4m13s   <IP_ADDRESS>       k8s1   <none>           <none>
	 default             app2-5cc5d58844-nv6wr              1/1     Running   0          4m13s   <IP_ADDRESS>      k8s1   <none>           <none>
	 default             app3-6c7856c5b5-fs6n9              1/1     Running   0          4m13s   <IP_ADDRESS>      k8s1   <none>           <none>
	 kube-system         cilium-mxkd7                       1/1     Running   0          5m45s   <IP_ADDRESS>   k8s1   <none>           <none>
	 kube-system         cilium-operator-5f99fccbd8-2dbr9   1/1     Running   0          5m44s   <IP_ADDRESS>   k8s1   <none>           <none>
	 kube-system         cilium-operator-5f99fccbd8-dvpg7   1/1     Running   0          5m44s   <IP_ADDRESS>   k8s2   <none>           <none>
	 kube-system         cilium-zggcq                       1/1     Running   0          5m44s   <IP_ADDRESS>   k8s2   <none>           <none>
	 kube-system         coredns-767d4c6dd7-jsfcl           1/1     Running   0          7m6s    <IP_ADDRESS>      k8s2   <none>           <none>
	 kube-system         etcd-k8s1                          1/1     Running   0          75m     <IP_ADDRESS>   k8s1   <none>           <none>
	 kube-system         kube-apiserver-k8s1                1/1     Running   0          75m     <IP_ADDRESS>   k8s1   <none>           <none>
	 kube-system         kube-controller-manager-k8s1       1/1     Running   0          75m     <IP_ADDRESS>   k8s1   <none>           <none>
	 kube-system         kube-proxy-2tcqm                   1/1     Running   0          75m     <IP_ADDRESS>   k8s1   <none>           <none>
	 kube-system         kube-proxy-dx6pr                   1/1     Running   0          73m     <IP_ADDRESS>   k8s2   <none>           <none>
	 kube-system         kube-scheduler-k8s1                1/1     Running   0          75m     <IP_ADDRESS>   k8s1   <none>           <none>
	 kube-system         log-gatherer-kcfvr                 1/1     Running   0          72m     <IP_ADDRESS>   k8s1   <none>           <none>
	 kube-system         log-gatherer-qcb6s                 1/1     Running   0          72m     <IP_ADDRESS>   k8s2   <none>           <none>
	 kube-system         registry-adder-26hfp               1/1     Running   0          72m     <IP_ADDRESS>   k8s1   <none>           <none>
	 kube-system         registry-adder-ldqph               1/1     Running   0          72m     <IP_ADDRESS>   k8s2   <none>           <none>

Stderr:

Fetching command output from pods [cilium-mxkd7 cilium-zggcq]
cmd: kubectl exec -n kube-system cilium-mxkd7 -- cilium service list
Exitcode: 0
Stdout:
 	 ID   Frontend             Service Type   Backend
	 1    <IP_ADDRESS>:9090   ClusterIP      1 => <IP_ADDRESS>:9090
	 2    <IP_ADDRESS>:80      ClusterIP      1 => <IP_ADDRESS>:80
	                                          2 => <IP_ADDRESS>:80
	 3    <IP_ADDRESS>:443        ClusterIP      1 => <IP_ADDRESS>:6443
	 4    <IP_ADDRESS>:53        ClusterIP      1 => <IP_ADDRESS>:53
	 5    <IP_ADDRESS>:9153      ClusterIP      1 => <IP_ADDRESS>:9153
	 6    <IP_ADDRESS>:3000   ClusterIP      1 => <IP_ADDRESS>:3000
	 7    <IP_ADDRESS>:69      ClusterIP      1 => <IP_ADDRESS>:69
	                                          2 => <IP_ADDRESS>:69

Stderr:

cmd: kubectl exec -n kube-system cilium-mxkd7 -- cilium endpoint list
Exitcode: 0
Stdout:
 	 ENDPOINT   POLICY (ingress)   POLICY (egress)   IDENTITY   LABELS (source:key[=value])                            IPv6        IPv4         STATUS
	            ENFORCEMENT        ENFORCEMENT
	 880        Disabled           Disabled          13212      k8s:appSecond=true                                     fd02::1af   <IP_ADDRESS>   ready
	                                                            k8s:id=app2
	                                                            k8s:io.cilium.k8s.policy.cluster=default
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=app2-account
	                                                            k8s:io.kubernetes.pod.namespace=default
	                                                            k8s:zgroup=testapp
	 1129       Disabled           Disabled          2894       k8s:id=app1                                            fd02::1d3   <IP_ADDRESS>    ready
	                                                            k8s:io.cilium.k8s.policy.cluster=default
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=app1-account
	                                                            k8s:io.kubernetes.pod.namespace=default
	                                                            k8s:zgroup=testapp
	 1749       Disabled           Disabled          4          reserved:health                                        fd02::157   <IP_ADDRESS>   ready
	 1868       Disabled           Disabled          2894       k8s:id=app1                                            fd02::16a   <IP_ADDRESS>    ready
	                                                            k8s:io.cilium.k8s.policy.cluster=default
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=app1-account
	                                                            k8s:io.kubernetes.pod.namespace=default
	                                                            k8s:zgroup=testapp
	 2876       Disabled           Disabled          1          k8s:cilium.io/ci-node=k8s1                                                      ready
	                                                            k8s:node-role.kubernetes.io/master
	                                                            reserved:host
	 3081       Disabled           Disabled          48485      k8s:id=app3                                            fd02::19a   <IP_ADDRESS>   ready
	                                                            k8s:io.cilium.k8s.policy.cluster=default
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=default
	                                                            k8s:io.kubernetes.pod.namespace=default
	                                                            k8s:zgroup=testapp

Stderr:

cmd: kubectl exec -n kube-system cilium-zggcq -- cilium service list
Exitcode: 0
Stdout:
 	 ID   Frontend             Service Type   Backend
	 1    <IP_ADDRESS>:53        ClusterIP      1 => <IP_ADDRESS>:53
	 2    <IP_ADDRESS>:9153      ClusterIP      1 => <IP_ADDRESS>:9153
	 3    <IP_ADDRESS>:3000   ClusterIP      1 => <IP_ADDRESS>:3000
	 4    <IP_ADDRESS>:9090   ClusterIP      1 => <IP_ADDRESS>:9090
	 5    <IP_ADDRESS>:69      ClusterIP      1 => <IP_ADDRESS>:69
	                                          2 => <IP_ADDRESS>:69
	 6    <IP_ADDRESS>:443        ClusterIP      1 => <IP_ADDRESS>:6443
	 7    <IP_ADDRESS>:80      ClusterIP      1 => <IP_ADDRESS>:80
	                                          2 => <IP_ADDRESS>:80

Stderr:

cmd: kubectl exec -n kube-system cilium-zggcq -- cilium endpoint list
Exitcode: 0
Stdout:
 	 ENDPOINT   POLICY (ingress)   POLICY (egress)   IDENTITY   LABELS (source:key[=value])                              IPv6       IPv4         STATUS
	            ENFORCEMENT        ENFORCEMENT
	 27         Disabled           Disabled          2560       k8s:app=grafana                                          fd02::1    <IP_ADDRESS>    ready
	                                                            k8s:io.cilium.k8s.policy.cluster=default
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=default
	                                                            k8s:io.kubernetes.pod.namespace=cilium-monitoring
	 112        Disabled           Disabled          1          k8s:cilium.io/ci-node=k8s2                                                       ready
	                                                            reserved:host
	 2245       Disabled           Disabled          4          reserved:health                                          fd02::81   <IP_ADDRESS>   ready
	 2411       Disabled           Disabled          471        k8s:app=prometheus                                       fd02::7a   <IP_ADDRESS>     ready
	                                                            k8s:io.cilium.k8s.policy.cluster=default
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=prometheus-k8s
	                                                            k8s:io.kubernetes.pod.namespace=cilium-monitoring
	 3246       Disabled           Disabled          28075      k8s:io.cilium.k8s.policy.cluster=default                 fd02::51   <IP_ADDRESS>   ready
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=coredns
	                                                            k8s:io.kubernetes.pod.namespace=kube-system
	                                                            k8s:k8s-app=kube-dns

Stderr:

===================== Exiting AfterFailed =====================
15:27:50 STEP: Running AfterEach for block EntireTestsuite K8sFQDNTest
15:27:50 STEP: Running AfterEach for block EntireTestsuite

</details

All DNS connections from default/app2-5cc5d58844-nv6wr seem to be hanging. We don't see the requests in the CoreDNS logs, so it probably wasn't received.
We can trace one request for example. On the source node:
$ cat ~/Downloads/tmp/cilium-mxkd7-hubble_observe.json | ./hubble observe --from-port 55145 --to-port 53
Jun 28 15:27:21.649: default/app2-5cc5d58844-nv6wr:55145 <> kube-system/coredns-767d4c6dd7-jsfcl:53 to-overlay FORWARDED (UDP)
Jun 28 15:27:26.651: default/app2-5cc5d58844-nv6wr:55145 <> kube-system/coredns-767d4c6dd7-jsfcl:53 to-overlay FORWARDED (UDP)

On the destination node:
$ cat ~/Downloads/tmp/cilium-zggcq-hubble_observe.json | ./hubble observe --from-port 55145 --to-port 53
Jun 28 15:27:21.656: default/app2-5cc5d58844-nv6wr:55145 -> kube-system/coredns-767d4c6dd7-jsfcl:53 to-endpoint FORWARDED (UDP)
Jun 28 15:27:26.658: default/app2-5cc5d58844-nv6wr:55145 -> kube-system/coredns-767d4c6dd7-jsfcl:53 to-endpoint FORWARDED (UDP)

Monitor aggregation is enabled, so only to-xxx traces are collected.
Endpoint routes are disabled, so these to-endpoint traces were emitted from the cilium_vxlan device after a tail call from bpf_overlay to bpf_lxc. Then, the packet should be redirected to the lxc device and enter the container. Let's confirm that.
We can first get the lxc device for the destination CoreDNS container:
$ jq '.[].status | select(."external-identifiers"."k8s-pod-name" == "coredns-767d4c6dd7-jsfcl").networking' cilium-zggcq-endpoint_list.txt
{
  "addressing": [
    {
      "ipv4": "<IP_ADDRESS>",
      "ipv6": "fd02::51"
    }
  ],
  "host-mac": "9a:b7:ea:b0:e5:82",
  "interface-index": 207,
  "interface-name": "lxc3e1dbd546d93",
  "mac": "8a:52:f4:b8:28:b8"
}

We can then check the BPF programs attached to the node:
$ cat bugtool-cilium-zggcq/cmd/bpftool-net-show.md
Error: Netlink error reporting not supported
xdp:

tc:
cilium_net(9) clsact/ingress bpf_host_cilium_net.o:[to-host]
cilium_host(10) clsact/ingress bpf_host.o:[to-host]
cilium_host(10) clsact/egress bpf_host.o:[from-host]
lxc899cabc800bc(17) clsact/ingress bpf_lxc.o:[from-container]
lxcbea3fce3587e(19) clsact/ingress bpf_lxc.o:[from-container]
lxc3e1dbd546d93(207) clsact/ingress bpf_lxc.o:[from-container]
lxc3e1dbd546d93(207) clsact/egress bpf_lxc.o:[to-container]
cilium_vxlan(212) clsact/ingress bpf_overlay.o:[from-overlay]
cilium_vxlan(212) clsact/egress bpf_overlay.o:[to-overlay]
lxc_health(214) clsact/ingress bpf_lxc.o:[from-container]

flow_dissector:

Here, we see that, contrary to other containers, the CoreDNS pod actually has two BPF programs attached, one at ingress and one at egress. That should be the case only when endpoint routes are enabled. Similarly, in the routes, we can see that it's the only endpoint with a route:
$ grep lxc3e1dbd546d93 bugtool-cilium-zggcq/cmd/ip--4-r.md
<IP_ADDRESS> dev lxc scope link

Therefore the DNS packet is sent to the stack. It flows through netfilter and hits the FORWARD-filter table:
Chain FORWARD (policy DROP 84 packets, 8322 bytes)
 pkts bytes target     prot opt in     out     source               destination
 2162 3076K CILIUM_FORWARD  all  --  any    any     anywhere             anywhere             /* cilium-feeder: CILIUM_FORWARD */
   96 13716 KUBE-FORWARD  all  --  any    any     anywhere             anywhere             /* kubernetes forwarding rules */
   48  5280 KUBE-SERVICES  all  --  any    any     anywhere             anywhere             ctstate NEW /* kubernetes service portals */
[...]
Chain CILIUM_FORWARD (1 references)
 pkts bytes target     prot opt in     out     source               destination
 1204 2923K ACCEPT     all  --  any    cilium_host  anywhere             anywhere             /* cilium: any->cluster on cilium_host forward accept */
    0     0 ACCEPT     all  --  cilium_host any     anywhere             anywhere             /* cilium: cluster->any on cilium_host forward accept (nodeport) */
  864  140K ACCEPT     all  --  lxc+   any     anywhere             anywhere             /* cilium: cluster->any on lxc+ forward accept */
    0     0 ACCEPT     all  --  cilium_net any     anywhere             anywhere             /* cilium: cluster->any on cilium_net forward accept (nodeport) */

Since endpoint routes are disabled in the agent, rules installed in CILIUM_FORWARD don't match the packet because they assume it will come out of cilium_host. As per FORWARD's default policy, the packet is dropped.
This is actually a known limitation of Cilium since https://github.com/cilium/cilium/pull/16227 (changing the status of endpoint routes on an existing Cilium installation is not supported, even though we do it in CI). We usually work around it by deleting existing pods so that their routes are reinstalled from scratch. That would be a short solution here.

This would only happen in our 4.9 CI job because of the following condition: https://github.com/cilium/cilium/blob/e6f34c3c98fe2e247fde581746e552d8cb18c33c/bpf/bpf_lxc.c#L1149-L1156
So if ENABLE_NODEPORT is defined (only undefined on 4.9), we redirect the packet to the lxc device instead of passing to the stack and we therefore skip the FORWARD-filter table.

I believe that K8sDatapathConfig is the only suite which may switch up endpoint routes mode on/off, and in this particular failure case the K8sDatapathConfig tests ran immediately prior to K8sFQDN. I wonder if we're just not cleaning up the environment properly enough in the AfterAll / BeforeAll steps of one of these two contexts. This could explain why we don't see the failure more often - it requires particular groups of tests to be run in a particular order.

This could explain why we don't see the failure more often - it requires particular groups of tests to be run in a particular order.

Yep, but checking if DNS resolves is the first thing we do after any Cilium deployment AFAIK. So any test without endpoint routes running after a test with endpoints routes should fail.

Seems like that should be established as part of this path:
https://github.com/cilium/cilium/blob/eb9a5c4dcb47e866749ae21ae31420a5c882c739/test/k8sT/fqdn.go#L76
...
https://github.com/cilium/cilium/blob/eb9a5c4dcb47e866749ae21ae31420a5c882c739/test/k8sT/assertionHelpers.go#L177
...
https://github.com/cilium/cilium/blob/eb9a5c4dcb47e866749ae21ae31420a5c882c739/test/helpers/kubectl.go#L2048
...
https://github.com/cilium/cilium/blob/eb9a5c4dcb47e866749ae21ae31420a5c882c739/test/helpers/kubectl.go#L1973
However the failing line is later than this, so whatever check we did above was functionally different from the actual DNS lookup.
https://github.com/cilium/cilium/blob/eb9a5c4dcb47e866749ae21ae31420a5c882c739/test/k8sT/fqdn.go#L89

I don't know off-hand how different host DNS resolution is but it may provide some hints here.

Hm. DNS resolution from a hostns pod should be the same as long as it's on a different node than the CoreDNS pod. Once the request reaches the destination node via the tunnel it's basically indistinguishable from a request from a pod (except from policy point of view, but we're not concerned with that here).

I've looked into the following failures and also observed that there are per-endpoint routes for the DNS pod:
https://jenkins.cilium.io/job/cilium-master-k8s-1.21-kernel-4.9/544/testReport/junit/Suite-k8s-1/21/K8sDemosTest_Tests_Star_Wars_Demo/
https://jenkins.cilium.io/job/cilium-master-k8s-1.17-kernel-4.9/129/testReport/junit/Suite-k8s-1/17/K8sDemosTest_Tests_Star_Wars_Demo/
https://jenkins.cilium.io/job/cilium-master-k8s-1.21-kernel-4.9/546/testReport/junit/Suite-k8s-1/21/K8sDemosTest_Tests_Star_Wars_Demo/
https://jenkins.cilium.io/job/cilium-master-k8s-1.21-kernel-4.9/539/testReport/junit/Suite-k8s-1/21/K8sKafkaPolicyTest_Kafka_Policy_Tests_KafkaPolicies/
Here's a one-liner I've been using to establish which endpoints are configured with endpoint-routes mode in a CI sysdump:
$ for dir in $(find test_results/*/*/bugtool-cilium-*/cmd/state/[0-9]* -type d); do base64_decode $(grep BASE64 $dir/ep_config.h) | gron | grep -i -e req -e PodName | norg; done
{"K8sPodName":"xwing-6f56868789-gddzk"}
{"K8sPodName":"spaceship-6567c9b4bd-6gmg6"}
{"K8sPodName":"xwing-6f56868789-h8zzz"}
{"K8sPodName":""}
{"K8sPodName":""}
{"K8sPodName":"deathstar-595989bc5b-g92b4"}
{"K8sPodName":"xwing-6f56868789-cnvmc"}
{"K8sPodName":"spaceship-6567c9b4bd-9hm98"}
{"K8sPodName":"spaceship-6567c9b4bd-btd2x"}
{"K8sPodName":""}
{"K8sPodName":"deathstar-595989bc5b-l6w9h"}
{"K8sPodName":"spaceship-6567c9b4bd-9cxzj"}
{"DatapathConfiguration":{"require-egress-prog":true,"require-routing":false},"K8sPodName":"coredns-755cd654d4-6658j"}
{"K8sPodName":"prometheus-655fb888d7-dv5z4"}
{"K8sPodName":"deathstar-595989bc5b-phz25"}
{"K8sPodName":""}
{"K8sPodName":"grafana-5747bcc8f9-zqjbv"}

The fix at https://github.com/cilium/cilium/pull/16767 was not sufficient; https://github.com/cilium/cilium/pull/16835 should fix it.


Some fixes and improvements

Hi LatchBio. First of all thanks for creating and sharing this repo. While we at Plural were trying to use the DaemonSet installation method from the Sysbox docs we ran into a number of problems and the custom AMI you've built was much more stable.
However, when trying to use the AMI from this repo to run KinD we ran into problems that were caused by the crio.conf not being setup properly. Note that other docker containers were able to run within the pod on our cluster, but specifically not KinD. This lead us down a journey to understand each step being executed in the packer build, as well as all the scripts used by the sysbox daemonset deployment script.
What we ended up doing is rewriting most of the build script so that it is equivalent to the script from sysbox, along with comments for the relevant function so upstream changes by sysbox can more easily be found and maintained. Another reason for doing this is because possible incorrect configurations might lead to security vulnerabilities in the system so we wanted to be absolutely sure CRI-O and sysbox are configured properly. The biggest changes here are that we use the sysbox installer container image to grab the relevant binaries and configs for sysbox and CRI-O and install them the same as the upstream installer script does. It took some time to test all the changes but eventually the images were fully working and we were able to run KinD. We were even able to launch a container, run KinD within that container, then use CAPI to deploy another 6 node Kubernetes cluster with the docker provider and install Calico on that cluster. It's a bit container-inception but I hope it's somewhat clear.
Finally, we setup some GitHub actions that use semantic release, configure-aws-credentials that uses OIDC so GitHub can securely authenticate against AWS without static credentials and of course packer to automatically build AMIs for multiple k8s version and system architectures. To setup the OIDC configuration between AWS and GitHub you can use this terraform module. The action also caches the files we are sourcing from the sysbox installer container so builds are pretty fast.
What we will still add very shortly is a small configuration that will allow us to use Renovate to keep the environment variable that sets the sysbox version in the GitHub action YAML updated automatically (by creating PRs, but can be set to auto-merge). This way there will be very little maintenance overhead for us in the future as new versions of Sysbox are released.
If any of this sounds interesting to you, you can have a look at our fork. We'd also be happy to contribute this back upstream in a PR if you like, or feel free to use our changes and implement them here yourselves. Note that our AMIs are public, so if you'd like to use them directly (for testing or otherwise) you are able to. We're pushing them to all the regular AWS regions.
In the future we might also expand our repo to include images for Azure and GCP, which probably is less interesting to you but I thought I'd mention it.

Hi! This is awesome to hear. Very glad you got some use out of this repo.
I'm going to open a PR to look at the diff, very curious what we got wrong on our end. I tried to match the installer as closely as possible too, but my understanding of Kubernetes setups and container runtimes is far from perfect. Might also be that the upstream installation script has changed a lot over the past year.
The only issues we have experienced are related to using host devices for FUSE and GPU through a bind-mount, so I will test against your fork to see if it's incidentally fixed as well. If not, we'll probably have pretty low interest in adopting your version given that everything works OK for us (barring any security implications that I see from the diff).

I'll add a banner on top of the readme linking to your version. All the CI/CD you've set up is certainly enough to prefer using it over our script. We're certainly less dedicated to actually supporting this as an open-source product :)


Customize format for GitHub release date badge

:clipboard: Description

Currently it basically only shows month, and month year when it's not the running year.
I highly like to see a YYYY-MM-DD option for example, or even make that the default as that seems most natural and useful. Like the badge itself states it's a release date, not a release month or rough release time frame. :)
Linking #2744

Just came back here after a while to see if there is a solution in between.
This actually applies to both available release date badges:
/github/release-date/:user/:repo: https://shields.io/badges/git-hub-release-date-published-at
/github/release-date-pre/:user/:repo: https://shields.io/badges/git-hub-pre-release-date


Added a variable for adding GPU Operator Namespace Labels

This MR creates a new variable called gpu_operator_namespace_labels which allows user to add any additional labels they want to add on the GPU operator namespace.
Without this, any changes to the GPU operator namespace labels can easily be overwritten by Terraform commands.

Should we instead use the GPU operator value daemonsets.labels?
https://github.com/NVIDIA/gpu-operator/blob/73d41d465188543f3ba093889b728e50564e74c7/deployments/gpu-operator/values.yaml#L28C5-L28C5

Should we instead use the GPU operator value daemonsets.labels? https://github.com/NVIDIA/gpu-operator/blob/73d41d465188543f3ba093889b728e50564e74c7/deployments/gpu-operator/values.yaml#L28C5-L28C5

@evberrypi  The link you have is labeling the Daemonset, what we need is the label on the GPU Operator namespace, which the GPU operator helm chart does not control, and is instead controlled by TF here. Please review again.


How do I I need to use cookbook with vaadin version 24.0.0 & hilla 2.0.0

Hi,
I tried to increase the version of cookbook to the last version of vaadin to use spring 3 version & I increased also the version of hilla 2.0.0 supported by vaddin 24.0.0
but when I try to launch the application I have an issue relate to loading the servlet.
did you think the project will be increase to the last version of vaadin soon ?
thanks in advance

Fixed in #288


Added Ibex as a target (under Verilator)

Also modified the prolog code for targets not supporting supervisor mode
certain CSR's are only written if privilege modes are supported

I suspect this needs breaking into two pull requests, one for Ibex and
one for some general tidying up. Could you let me know your thoughts
on the above and I'll then review again.

No problem, let me take another look
Thx
Lee
On 12/08/2019 14:46, Jeremy Bennett wrote:

@jeremybennett commented on this pull request.
Hi Lee,
Thanks for adding Ibex Verilator support. I assume you have tested the
patch and that it doesn't break anything existing.
I think some extra changes have crept in beyond what is needed for Ibex.
In general anything in the |riscv-ovpsim| directory, I regard as your
territory, and if you are happy, I'm happy to merge. The same applies
to source OVPsim specific files elsewhere, such as
|riscv-target/riscvOVPsim/compliance_io.h|
Since this patch is about adding ibex, you've also added files in
|riscv-target/golden/device/ibex_rv32imc| and |riscv-target/ibex|, and
I am presuming you have tested these against the Ibex Verilator model
and they work.
However, I also see some files changed, that seem to be generic. Here
are some comments on those files:

|riscv-target/golden/README.md|. Is it correct to say that OVPsim
is part of this repository. Surely it is a separate repository,
|riscv-ovpsim| (https://github.com/riscv/riscv-ovpsim)? In any
case I don't think this is an Ibex specific change.
|riscv-target/golden/compliance_io.h|. This may be valid, but it
is a new file and I couldn't see how it related to Ibex.
|riscv-target/golden/compliance_test.h|. Likewise.
|riscv-test-env/p/link.ld|. This is a generic file, and you seem
to be adding a new rule to position sections named |.text.trap|.
I'm don't believe this section naming is generic, so I don't think
it belongs here.
|riscv-test-env/p/riscv_test.h|. This seems to be adding changes
related to privileged mode testing. Is it needed for Ibex testing?
|riscv-test-suite/rv32i/Makefrag|. This seems to be a change
related to unaligned jumps.

I suspect this needs breaking into two pull requests, one for Ibex and
one for some general tidying up. Could you let me know your thoughts
on the above and I'll then review again.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/riscv/riscv-compliance/pull/61?email_source=notifications&email_token=ACVJMXE46PUGFWQSR7GMFVLQEFSSDA5CNFSM4IKR5UH2YY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOCBIFZYY#pullrequestreview-273702115,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACVJMXDHXXINM3BYJN67JYLQEFSSDANCNFSM4IKR5UHQ.

need to resubmit


[WIP Restrict eth offer items

Disallow ETH offer items except in matchOrders and matchAdvancedOrders.
WIP still fixing tests

Working on getting coverage back to 100%

MagicModulus = 69 ftw 🔥


HAP HDR missing docs

Hello,
With the recent add of the HAP HDR flavour, a part of the doc is not clear about what is allow in term of texture.
The choice was made to use one name, for two kind of texture (unsigned and signed).
Does it means, an HAP HDR can store depending of the frame Unsigned and signed texture ?
Or is an HAP HDR file, need to always have the same kind of texture for each frame ? (in that case checking the texture format for the first frame, will be enough to know which kind of texture is used inside the file)

Hi - thanks, yes this should be tightened up. I think the utility of encoding signed and unsigned is probably outweighed by the convenience of decoding a single format, so I will add something along the lines of:
If an identifier describes more than one format or combination of formats, a single stream should be limited to a single format or a single combination of formats.

Hi,
Thanks for your answer.
When you talk about combination of formats, you talk about HAPQA (or HAP HDR Alpha in the future) ?
In other words, for HAP HDR, all the frames need to have same texture (is not allow to have some frames in unsigned, and some frames in signed) ? (and i think it's better to restrict it like this, it made decoding easy, without a need to change texture format for the same file)

When you talk about combination of formats, you talk about HAPQA (or HAP HDR Alpha in the future) ?

Correct

In other words, for HAP HDR, all the frames need to have same texture (is not allow to have some frames in unsigned, and some frames in signed) ?

also correct

Perfect, Thanks Tom !


feat: add slackv2 notification

SUMMARY

SeeSIP
This PR adds a new Notification SlackV2Notification that uses the Slack upload files v1 method. It is required for slack workspace owners to add the channels:read scope to their workspace in order to use this api. It also fetches all slack channels in order to convert user-inputed names to api-required ids. Because of this, I added a UI to support this.
Per the SIP, the both the api and UI will attempt to convert the notification to SlackV2 if the scopes are in place.
BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

After:
With feature flag on:
https://github.com/apache/superset/assets/5186919/64e0b816-fe6a-4bb3-b882-ecd08949eb21
With feature flag off (after it has been turned on once, so the existing v2 slack reports do not convert back to v1):
https://github.com/apache/superset/assets/5186919/1f04fa0a-bd15-4fc5-bfc2-85ef0a38be24
TESTING INSTRUCTIONS

Create a new slack application and connect it to Superset.
ADDITIONAL INFORMATION

[ ] Has associated issue:
[ ] Required feature flags:
[ ] Changes UI
[ ] Includes DB Migration (follow approval process in SIP-59)

[ ] Migration is atomic, supports rollback & is backwards-compatible
[ ] Confirm DB migration upgrade and downgrade tested
[ ] Runtime estimates and downtime expectations provided

[ ] Introduces new feature or API
[ ] Removes existing feature or API

/testenv up

/testenv up FEATURE_ALERT_REPORTS=true


Taming transformers

I have been training a lot a models before but now I get this error message about taming transformers. Anyone can help ?
Warning: Taming Transformers not found at path /content/gdrive/MyDrive/sd/stablediffusion/src/taming-transformers/taming
Traceback (most recent call last):
  File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/webui.py", line 14, in <module>
    from modules import shared, devices, sd_samplers, upscaler, extensions, localization, ui_tempdir
  File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/sd_samplers.py", line 12, in <module>
    from modules import prompt_parser, devices, processing, images
  File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/processing.py", line 15, in <module>
    import modules.sd_hijack
  File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/sd_hijack.py", line 10, in <module>
    import modules.textual_inversion.textual_inversion
  File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/textual_inversion/textual_inversion.py", line 13, in <module>
    from modules import shared, devices, sd_hijack, processing, sd_models, images, sd_samplers
  File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/sd_models.py", line 15, in <module>
    from modules.sd_hijack_inpainting import do_inpainting_hijack, should_hijack_inpainting
  File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/sd_hijack_inpainting.py", line 6, in <module>
    import ldm.models.diffusion.ddpm
  File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/ldm/models/diffusion/ddpm.py", line 24, in <module>
    from ldm.models.autoencoder import VQModelInterface, IdentityFirstStage, AutoencoderKL
  File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/ldm/models/autoencoder.py", line 6, in <module>
    from taming.modules.vqvae.quantize import VectorQuantizer2 as VectorQuantizer
ModuleNotFoundError: No module named 'taming'

Had to rename the SD file and update to the new one


How is this different from Q?

What is difference between this library and Q? What is the motivation to do another package and not continue previous work?

The difference is that this library expects every quantum to be dataizable to bytes only, while Q allows using other types. I am experimenting to find the best approach for OO. At this moment I suppose that in OOP must be no basic types (atoms in EO) but bytes, and a program must only operate these. Once I find what fits OO best I'll release 1.0.0 version of either this or Q.

I am experimenting to find the best approach for OOP.

Good luck to find out!


NO-JIRA clean up docs

Remove some out-dated material, fix a few links, & clean up a few
random bits.

@jbertram nice one!


Problem with Persistent notification

Hi Team,
I have added Hypertrack sdk in my application. But i am facing an issue with persistent notifications while live location is shared. I am not able to set launch mode for the activity class which should be opened when this notification is tapped.
In ServiceNotificationParamsBuilder we can only set activity class name (setContentIntentActivityClass()) and extra data with intent (setContentIntentExtras()). Is there any method to set launch modes for intent?
Please help.
Thanks,
kajol

Updated to the new version


Fix overridden promise $super calls in async component factory

1. Why is this change necessary?
Without the change, overriding a component which was already overridden was not entirely possible if the first override makes a $super call inside a promise chain. The second override itself worked, but calling $super from it to the overridden method in the first override caused an infinite loop as the virtual call stack was reset every iteration.
2. What does this change do, exactly?
Removes a virtual call stack reset that caused the problem described above. Also adds a test for the described case.
3. Describe each step to reproduce the issue or behaviour.

Register a regular component with any method
Register an override for the component that overrides this method and returns a promise chain which at some point calls $super to the regular component method.
Register another override for the component that also overrides the method and calls $super to the first override. This second $super call does not need to be inside a promise chain, but can be.
Profit

Or alternatively:

Execute the given new unit test without the change.

4. Please link to the relevant issues (if any).
5. Checklist

[x] I have rebased my changes to remove merge conflicts
[x] I have written tests and verified that they fail without my change
[x] I have created a changelog file with all necessary information about my changes
[x] I have written or adjusted the documentation according to my changes
[x] This change has comments for package types, values, functions, and non-obvious lines of code
[x] I have read the contribution requirements and fulfil them.

🤖 Generated by Copilot at fb2ee08
This pull request fixes a bug in the async-component.factory.ts that caused the $super calls inside promise chains to be overridden by the wrong component. It also adds a changelog entry and a unit test for the fix.

🤖 Generated by Copilot at fb2ee08

Fix overridden $super calls in promise chains in admin components (link)
Simplify the _inheritedFrom method in the async-component.factory.ts (link)
Add a unit test for the async-component.factory.ts (link)

Please note that this bug may be eligible for porting back to 6.4 (which would be trivial, as the changes would also need be applied to component.factory.ts which is to those ends a more or less exact copy of async-component.factory.ts.

Running into the same problem. Any updates here @shyim ?

Hey maximilianruesch,
is the problem only happening when mixing synchronous methods with asynchronous methods?

As far as i am aware, it is happening every time there is any asynchronous handling (any handling that returns a promise) inside the $super call stack. I will add a test that concerns asynchronous base methods too (with asynchronous methods in the stack).

Hello,
thank you for creating this pull request.
I have opened an issue on our Issue Tracker for you. See the issue link: https://issues.shopware.com/issues/NEXT-30599
Please use this issue to track the state of your pull request.

thanks for your contribution @maximilianruesch  👍🎉💙


feat(transport): implement TLS record fragmentation

Thank a lot for the contribution! Could you please sign the Google CLA to make the CI job pass:

📝 If you are not currently covered under a CLA, please visit https://cla.developers.google.com/. Once you've signed, follow the "New Contributors" link at the bottom of this page to update this check.

@jyyi1  How can I get a non-Chinese phone number to register Google account?

@Lanius-collaris Ugh, I see the issue. The CLA system requires Google account, which in turn seems to require a phone number.
They require Google accounts because they are used to link individual and corporate CLAs to covered contributors and provide authentication for CLA management.
Have you tried the "use existing email" option when creating the account?

Google needs a backup contact information, that's why you need a phone number. But if you provide some email, then you may not need a phone.

Have you tried the "use existing email" option when creating the account?

@fortuna Yes, I have tried. But I get unknown error.

@fortuna I tried again today, after I entered the verification code from mail, Google asked for a number.


haproxy misconfiguration after restarting VPC with cleanup

Steps to reproduce the issue

Create 1 VPC
create 2 VPC tiers, and a VM on each VPC tier
acquire 2 public IPs
create LB rules to the VM in each tier

The VR has correct haproxy configuration

restart VPC with cleanup

The new VR has haproxy configuration for LBs of only 1 tier

health check failing

ISSUE TYPE

Bug Report

COMPONENT NAME

VPC

CLOUDSTACK VERSION

4.18/4.19

CONFIGURATION

OS / ENVIRONMENT

SUMMARY

STEPS TO REPRODUCE

EXPECTED RESULTS

ACTUAL RESULTS

just reviewed both solutions (@JoaoJandre ) and both seem to do the job
#8881 does it by overloading the call to create the LB commands
#8765 does it by checking which list to load from the DB
If I understand correctly this creates the list multiple times in #8765, while in #8881 it is created only once. I see no problem with either code but I think Wei has a point and we should go with the overloading solution.
Note that this is purely from a code analysis point of view. I have not had time to test either yet but have started building the envs to check them.

@JoaoJandre both tested and checked /etc/haproxy/haproxy.cfg to be exactly the same as before restart. (cc @weizhouapache @SadiJr )

@JoaoJandre both tested and checked /etc/haproxy/haproxy.cfg to be exactly the same as before restart. (cc @weizhouapache @SadiJr )
less work then I thought.

@DaanHoogland I've also just finished testing both solutions. Both fix #8745. However, #8765 does indeed produce more commands than necessary, while #8881 will only send one command for all LBs. Thus I think we should go with #8881.

Fixed by #8881


[DOCFIX] Add a bootstrap-conf command snippet in the Chinese version of docs/en/Configuring-Alluxio-with-HDFS.md

This PR fixes issue with JIRA ticket ALLUXIO-1979
In docs/en/Configuring-Alluxio-with-HDFS.md of the English version there is a bootstrap-conf command snippet added in the Configuring Alluxio section. This PR translate it into Chinese and add it in the corresponding page of the Chinese version.

Can one of the admins verify this patch?

ok to test

LGTM, thanks for the translation!

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/10250/
Test PASSed.

Merged build finished. Test PASSed.


Failed to inject filter ebpf for kprobe_skb_2: register r8 used twice

With the static arm64 build of pwru v1.0.1, trying to use a filter results in the following error:
# ./pwru  'dst host <IP_ADDRESS>'
2023/09/04 22:24:06 Failed to inject filter ebpf for kprobe_skb_2: register r8 used twice
# ./pwru  'port 80'
2023/09/04 22:26:28 Failed to inject filter ebpf for kprobe_skb_5: register r8 used twice

Kernel version: 6.4.14
Distro: Alpine edge

Thanks for the issue. Does the same appear with v1.0.0?

v1.0.0 seems to get past the filter generation, but fails while attaching kprobes because it doesn't have 5e6011d624f72d406df1dba7d974314c9c7dca92.

@jschwinger233 Mind taking a look?

Update: Just built v1.0.0 with the rlimit increased (as a workaround) and it seems to work fine.

The pwru v1.0.1 amd64 build also has this issue.

Interesting, if I build v1.0.1 pwru from scratch (Archlinux, LLVM 16, gcc 13), I get it working. But with the released version (built by https://github.com/cilium/pwru/actions/runs/6071926301/job/16470942060, LLVM 13, gcc 12) I'm bumping into the same problem:
> sudo ./pwru 'port 80'
2023/09/05 09:49:58 Failed to inject filter ebpf for kprobe_skb_3: register r8 used twice

Checkouting upstream locally and running make local-release seems fine.

I think it could be caused by wrong registers found for skb->data and skb->date_end.
In a good kprobemultipwru_bpfel_x86.o, we have
; 	bpf_printk("%d %d", data, data_end);
      79:	r1 = 16 ll
      81:	r2 = 6
      82:	r3 = r9
      83:	r4 = r8
      84:	call 6

In a bad one, we have
      76:	r3 = r8
      77:	r3 += r6
; 	void *data_end = skb_head + l4_off + len;
      78:	r8 += r7
; 	bpf_printk("%d %d", data, data_end);
      79:	r1 = 16 ll
      81:	r2 = 6
      82:	r4 = r8
      83:	call 6

In the pwru, the searching for registers is implemented inflexibly:
https://github.com/cilium/pwru/blob/80d79736633a8c3ce0dac8abb4ae80bb0a5e9a8d/internal/libpcap/inject.go#L126-L145
So pwru found r8 as both skb->data and skb->data_end, which explains error register r8 used twice

@kdrag0n @dakehero Please try https://github.com/cilium/pwru/releases/tag/v1.0.2.

v1.0.2 works for me on arm64. Thanks for the quick fix!

v1.0.2 amd64 build also works for me


Setter for Value Types in element.go?

Version
Pre-release v1.0.0-alpha.2
Background
One of the variables for the Element struct is Value, which is itself a Value struct (see here). This struct presumably stores the main data for the element (e.g. patient name). Each Value struct (e.g. StringsValue) contains a single variable called value. value, however, is an unexported value because it's lowercase and the implementer of the library has no access to it.
Problem
Unable to create a new element with an actual value because we don't have access to the value variable in any *Value struct outside the dicom package. For example, attempting to create an element with this:
studyDescElem := dicom.Element{
		Tag: dicomtag.Tag{0x0008, 0x1030},
		RawValueRepresentation:  "LO",
		Value: &dicom.StringsValue{
			value: []string{studyDescription},
		},
	}

Yields the error unknown field 'value' in struct literal of type dicom.StringsValue
Expected behavior
It is useful for the implementer to be able to create an Element struct (e.g. for dicom generation during unit testing), like this:
e := &Element{
		Tag:                    tag.Rows,
		ValueRepresentation:    tag.VRInt32List,
		RawValueRepresentation: "US",
		Value: &intsValue{
			value: []int{100},
		},
	}

Recommendation
Add setter functions for each *Value struct (e.g. func (s *StringsValue) Set([]string)) or capitalize the value variable in each *Value struct.
@suyashkumar let me know what you think, and if I'm missing something or if there's another way to create an element with a value :)

@IChocked thank you for the good detailed request and suggestion!
Indeed you are correct that at some point we need to support for creating elements, mostly to support testing downstream (and a write package, which hasn't materialized yet in the rewrite). I'd say right now the alpha is mostly ready to test for data parsing, but some of these other use cases still need to be built out.
This is something I've been thinking about recently--I think I'd like to keep all of the Value implementations and their members private (e.g. I recently sent a PR to make the Value implementations private: #103), hopefully there's no reason that you as a Caller need to know about the details of the Value implementation. This keeps the dicom namespace less crowded, and you as a user just need to focus on the public API of the Value (and I'll have more examples on how to work with it). I'm not 100% sure if I like this yet or not, but would love to hear your thoughts as well!
What I will probably do to address your use case is to have an exported function like:
// NewValue creates a new DICOM value for the supplied data.
// Data must be one of the following types, otherwise and error will be returned.
// Acceptable types: []int, []string, []*Element, PixelDataInfo, ...
func NewValue(data interface{}) (Value, error)

And probably an accompanying func NewElement(tag tag.Tag, vr string, vl int, data interface{}) (*Element, error)
What do you think?

Opened up #108 to add such a NewValue function--does this address your use case?
Maybe I can also add a MustNewValue function that doesn't return an error (but panics) if the wrong type is sent in, which might be easier to use in tests since it can return a single Value (and probably not a huge deal).

Also I should probably set a new tag on the rewrite branch after submitting this, cause a bunch of commits have gone in since v1.0.0-alpha.2, though v1.0.0-alpha.2 should still parse dicoms correctly.

@suyashkumar A little bit of a background: our CLI uses your library, and since you've showed me a way to write DICOM, I've encompassed it in segmed CLI. Brian's request is a result of us not being able to compile the CLI b/c of the missing support for writing. So instead of addressing writing now, I guess what we could do is to use dcmutil and run it on a bunch of data and tell you what we get.

@wkoszek makes sense, thank you for the context! I will make it a priority to include a working package to write DICOMs in the full 1.0 release -- it's just not there in the alpha version yet (just the DICOM parsing component is complete). Would be great if we could test the parsing component of it while everything else is in progress (feel free to use the dicomutil CLI as you mentioned), and once things are more baked it can be integrated into the segmed CLI :). I am hoping that in about a month everything should be good to go, but we shall see what issues come up, etc.
For the write package, I will probably be able to reuse a lot of the existing write package API wise, and then internally rewrite/replace some of the logic over time as well, if that's the fastest way to unblock users and the release. Will think on it more later this week and start to add in some of that!

Closing this out as I think #108 should address this, along with new full-fledged write support! Writes even for sequences are here once #140 is merged :).


Fixed uninitialized values reported by Valgrind

Coverage decreased (-0.1%) to 36.598% when pulling 584a71e09cb8bb0cc32dd393297490f4e069cc19 on gpakosz:valgrind into a3a7ba9fdca0226275e7acdbd5f2cb777a4c4651 on akrennmair:master.

Thanks! The fix gives an error with g++ 4.9, though. Can you look into it?

Coverage decreased (-0.6%) to 36.123% when pulling 3a25d14d5ec67bee151bfae3fbf4af8ff9a3f471 on gpakosz:valgrind into a3a7ba9fdca0226275e7acdbd5f2cb777a4c4651 on akrennmair:master.

Coverage decreased (-0.6%) to 36.123% when pulling 3a25d14d5ec67bee151bfae3fbf4af8ff9a3f471 on gpakosz:valgrind into a3a7ba9fdca0226275e7acdbd5f2cb777a4c4651 on akrennmair:master.

Coverage decreased (-0.6%) to 36.123% when pulling 3a25d14d5ec67bee151bfae3fbf4af8ff9a3f471 on gpakosz:valgrind into a3a7ba9fdca0226275e7acdbd5f2cb777a4c4651 on akrennmair:master.

Coverage increased (+0.009%) to 36.741% when pulling 0ef4b1b3354066aeb464887bc4282a0f9653b78e on gpakosz:valgrind into a3a7ba9fdca0226275e7acdbd5f2cb777a4c4651 on akrennmair:master.

Thank you!

Damn I put Travis to work :(


[8.x] Fix #33620 Simplify testNow handling

As mentioned in https://github.com/laravel/framework/issues/33620#issuecomment-662710044 there is no particular reason to handle differently cases with or without testNow instance and return different type of instance regarding if it's relative time + testNow in use. The getDateTime() could simply always return a Carbon instance (or the class chosen by the user via the Date facade). And getDateTimestamp() could simply rely of this previous method getDateTime() to calculate the timestamp from it.
So we could get rid of isTestingRelativeDateTime() which is redundant with what Carbon does internally to detect the testNow instance and reduce the methods complexity.
This could be picked in 7.x and 6.x.
Fix #33620

Is this breaking? If not this can go to 6.x

Nope, it's not breaking, all output should be compatible. The only thing is about the isTestingRelativeDateTime() method removal, if called via parent::isTestingRelativeDateTime in a sub-class that extends a class that uses ValidatesAttributes trait, then it will raise an exception that the method is undefined.
Is the protected scope considered as part of the public API for BC consideration?

Is the protected scope considered as part of the public API for BC consideration?

Technically it is. I don't suppose there's a way to leave it somehow if we moved this to 6.x? Then again I don't think much people are extending that.


Stuck cursor

I'm using zulip via the webapp. Once or twice a day, when the page reloads, the "all messages" cursor resets me to a particular message from a year ago (November 3rd, 2022). I assumed that it would resolve eventually, but it's been happening for months, always with the same message. As you can imagine, this makes "all messages" unusable, because my team has had many thousands of messages since then.
If I read all my messages in individual streams and then go back to "All messages", the cursor is back to normal and I get the most recent messages again.
Other possibly-relevant information: I have several muted streams with tens of thousands of unread messages. I don't interact with these and they don't affect the "all messages" problem I'm having.
Please let me know what other debugging information would be useful for you.

Zulip Server and web app version:

[x] Zulip Cloud (*.zulipchat.com)
[ ] Zulip Server 7.0+
[ ] Zulip Server 6.0+
[ ] Zulip Server 5.0 or older
[ ] Other or not sure

Thanks for the report @akaptur! I think this is a weird symptom of #23812; basically, the "All messages" view is special for historical reasons, and loads data in starting from your oldest unread message forwards to the present... and it is incorrectly taking you to the closest loaded messages (from a year ago). The problem only affects you for the first 1-2 minutes after reloading the app, because the fetching process catches up to the present eventually.
There are a couple (unpleasant) workarounds that may help you until we can fix this:

Marking as read messages in muted streams.
Searching for in:home near:1000000000 whenever this happens; this is basically equivalent to "All messages" but will always start from the latest messages.

I'm not sure we can fix this in a clean way without completing #23812, but that is a near-term priority for us.

Thanks, Tim! I marked all my muted messages as read, but I'm still seeing the same result on a hard refresh. Sounds like I should wait for #23812 to be complete, but if there's any information you need from me, I'm happy to provide it.

@akaptur stop by chat.zulip.org and send me a DM -- there's some debugging we can do interactively.

@akaptur I think there's a very good chance that this we fixed this bug this spring; can you see if it still reproduces?


LineageOS for microG 17.1

How come there are no LineageOS for microG 17.1 builds, while the website states

We ship OTA updates from upstream LineageOS weekly. In this way you always receive new features and security updates just few days after they are released mainline.

I thought this was an automated process?

I've read that the guy who runs Lineage for microG is copletley different guy, and for now, he's doing something else, and he's known for that kind of "floating between multiple projects".
Kinda sucks, open source shouldn't look like this :/

@TheLastGimbus You’re mixing things. The person you describe is @mar-v-in, who develops microG. However @corna, who handles Lineage for microG, is indeed a completely different guy.
@m54fjt6910tt3o948b1zi840oanas5uty4j3nq2 See https://github.com/lineageos4microg/docker-lineage-cicd/pull/70. But as said above, @corna is not answering anymore regarding this project, so…

@ArchangeGabriel Thanks. I wasn't aware of the PR. Let's hope somebody will take over the project.

@m54fjt6910tt3o948b1zi840oanas5uty4j3nq2 For what it’s worth, I got an answer from @corna, he is currently overly busy, but still hopes to return to the project at some point or at least find someone to take it over. But it could be some times before that happens.

I plan to give the docker build image a refresh, but for me it was still not clear if its worth it, as microg development has been stalled for quite some time and there are a few glitches with LOS 17 and current Apps.. There were also open questions on, whether we might want to use microg fork from e.foundation, which seems to be currently better maintained.

@spolack It'll definitely be worth it, especially since @mar-v-in resumed development lately: https://gist.github.com/mar-v-in/c8254168c63361c5094de13a7afea344#gistcomment-3311911

May you have a look here: https://www.reddit.com/r/MicroG/comments/h8cwpg/i_created_a_tool_to_build_los_171_with_signature/
or here: https://gitlab.com/fremontt/minimal-docker-for-lineageos/image
I had to change a few things, because the final zip signing always failed for me:

In the Dockerfile add python2 to the packages to be installed.
In utils/run.sh I'm pointing /usr/bin/python to python2
rm /usr/bin/python
ln /usr/bin/python2 /usr/bin/python
In utils/build-los.sh in function syncSources I added a --force-sync to fix some syncing problems.

I'm using the restricted sigspoof patch and installed Magisk and MinMicroG (NoGoolage) via Magisk.

It's great to see the 17.1 builds being released - thank you everyone!
I think this issue can be closed now :-)
However, I'd like to mention: https://github.com/lineageos4microg/l4m_website/issues/52


⚠ WSINGRESOS - RECIBOPAGO has degraded performance

In 10d6757, WSINGRESOS - RECIBOPAGO (https://srvshyweb.yucatan.gob.mx/cgi-bin/wspd_cgi.sh/WService=wsingresos/internet/servicios/wsRecibosPagosWeb.r) experienced degraded performance:

HTTP code: 200
Response time: 2195 ms

Resolved: WSINGRESOS - RECIBOPAGO performance has improved in 52ad2ca after 21 minutes.


feat: ASIO_NO_DEPRECATED option to disable Asio depecated interfaces and functionality. Default value is true

Part of PR #17

This is no breaking change because the definition was added without any option to disable it.
Now it can be set to false.


How to change default currency?

edit setting file:
DEFAULT_COUNTRY = 'CN' #'US'
DEFAULT_CURRENCY = 'CNY' #'CNH' #'USD'
when click orders #18 in dashboard:
[23/Aug/2017 10:19:35] "GET /jsi18n/ HTTP/1.1" 200 3331
Internal Server Error: /dashboard/orders/18/
Traceback (most recent call last):
File "/home/long/wagtailTranslation/local/lib/python2.7/site-packages/django/core/handlers/exception.py", line 42, in inner
response = get_response(request)
File "/home/long/wagtailTranslation/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 249, in _legacy_get_response
response = self._get_response(request)
File "/home/long/wagtailTranslation/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 187, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/home/long/wagtailTranslation/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 185, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/home/long/wagtailTranslation/local/lib/python2.7/site-packages/django/contrib/auth/decorators.py", line 23, in _wrapped_view
return view_func(request, *args, **kwargs)
File "/home/long/wagtailTranslation/saleor-master/saleor/dashboard/order/views.py", line 65, in order_details
balance = captured - order.get_total()
File "/home/long/wagtailTranslation/local/lib/python2.7/site-packages/prices/init.py", line 138, in sub
(other.currency, self.currency))
ValueError: Cannot subtract prices in u'CNY' from u'USD'
[23/Aug/2017 10:20:06] "GET /dashboard/orders/18/ HTTP/1.1" 500 102609

When you change default currency, you need to delete all orders with the old default currency to make this work.


test: use a random db name when an env var is not given

WHAT
Assign a random UUID database name in tests when an env var is not given.
WHY
Somehow, it can happen that a database is deleted from another test and got the following error:
    client_test.go:197: failed to apply mutation: spanner: code = "NotFound", desc = "Database not found: projects/*********************/instances/***************************/databases/***********"
=

Using a random UUID database name should resolve this issue.

@hengfengli Thank you for following up this problem 🎉


Is there any way to change current location programmatically over the api?

Or would I need something like selenium to make it work. Since if I change my location in chrome, it works fine.

/user/ping Endpoint (described in the documentation) works for me


Fix font issues

Fixes issues related to the import and the usages of the fonts.

Adds font assets for the Dharma Gothic font.
Adds font-face rules for the Dharma Gothic font.
Uses css variables to define fonts:

--fonts-body
--font-headlines

Fixes Google font link
Removes the /sass folder since it is not being used. As per this
issue, https://github.com/getzola/zola/issues/837, only the static
and templates folder can be used to override themes elements.
Add a new scss file for storing custom fonts if any.
Use the head block to load custom fonts.

Drive-by:

Adds CI task to run all the CI tasks at once.
Fixes SASS linter/formatter tasks.

Thanks @lalver1. I am going to merge it then. Once your account will be be created this will be done automatically by Kodiak :)


Improve build instructions

Our build instruction currently recommend to clone the repo from master when building. This has caused some confusion since models converted with the stable pip package might not be compatible with the the runtime built from master (E.g. #408, #388, and spectrum comment).
We should change the build instructions to recommend pulling the latest stable release tag and should simplify the cross-compilation instructions to recommend using the published docker image for v0.3.1 which already includes all required dependencies and the correct version of the code.

from build instructions https://docs.larq.dev/compute-engine/build_arm/

Thanks @honglh, we'll add that to the build instructions!

Note to self: the build instructions for the converter have to be updated. By default they build for the wrong python version, see https://github.com/larq/compute-engine/issues/408.

Note to self: the build instructions for the converter have to be updated. By default they build for the wrong python version, see https://github.com/larq/compute-engine/issues/408.

Good point.
I think it might also be useful to add a big note that recommends using the stable converter from pypi if possible since building it from source can be a bit of a pain.

The build instructions could also use some description of how to get bazelisk on MacOS, they currently only specify what to do for linux:
sudo wget -O /usr/local/bin/bazel \
    https://github.com/bazelbuild/bazelisk/releases/download/v1.2.1/bazelisk-linux-amd64
sudo chmod +x /usr/local/bin/bazel

(@leonoverweel )

Yeah just running brew install bazel did not work:
ERROR: The project you're trying to build requires Bazel 3.0.0 (specified in /Users/leon/Repositories/Plumerai/vww-demo/lce-micro/.bazelversion), but it wasn't found in /usr/local/Cellar/bazel/3.2.0/libexec/bin.

You can brew install bazelisk I believe. And if not, you can install through npm.

(Personally, I install bazelisk through brew, and then symlink /usr/local/bin/bazel -> /usr/local/bin/bazelisk.)

(Personally, I install bazelisk through brew, and then symlink /usr/local/bin/bazel -> /usr/local/bin/bazelisk.)

Nice, this seems to have worked for me.

Note to self: make sure that the bug in #411 does not occur when the build instructions are updated.

Note to self: I think it makes sense to also have docs for a "development" setup which goes into detail on how to run the tests and install LCE from master.

Here I'm collecting the build issues that people have run into, with a green circle if they've been fixed.

:red_circle: In general, people had trouble with mismatching compiler/runtime versions (https://github.com/larq/compute-engine/issues/408 but I think also other users). The docs should reflect this version-matching more clearly.

:green_circle:  https://github.com/larq/compute-engine/issues/411  compiling the pip package in the Docker container without any setup --> fixed by updating docs to tell users to build outside of docker or to use the release_linux.sh script inside Docker

Issue https://github.com/larq/compute-engine/issues/396

:orange_circle: ar command not found. We've not been able to reproduce this issue, but the user fixed it themselves by using gcc-ar instead.
:green_circle: broken path to libtensorflow-lite.a . Was fixed by a tensorflow commit, see also https://github.com/larq/compute-engine/pull/480

Issue https://github.com/larq/compute-engine/issues/408

Mismatching converter/runtime versions, already listed above.
:green_circle: Building the converter for particular python versions --> docs have been updated to explain this
:green_circle: broken path to libtensorflow-lite.a is now fixed, see above

Closed via https://github.com/larq/docs/pull/124


fix typeahead in find star in database

Under search, the menu says “Find stars in database”  everywhere else we call it a dataset. Please fix.

done

done


Adding a single contribution to a memory

Hello,
First of all, I would like to thank you for the amazing job you've done so far.
Into the issue.
I created a new engine (en-nl) and started it.
I tried to add a single contribution to an existing memory. Following what can be found here, I used the following query:
curl -X POST http://localhost:8045/memories/1/corpus -d "source=en" -d "target=nl" -d "sentence=Hello Mike!" -d "translation=Hallo Mike!" | python -mjson.tool
The problem is that the server's response does not return the proper memory id:
{     "data": {         "begin": 1,         "dataChannel": 1,         "end": 1,         "id": "80000001-0000-0000-0000-000000000001",         "memory": 0,         "progress": 0.0,         "size": 1     },     "status": 200 }
I did check that the provided memory id existed for that engine.

Hi @frdg
so, regardless the odd output, the addition just completed successfully. Unfortunately we have a limitation with the current version of MMT that cannot return the proper Memory id for a single contribution (only for a corpus import, like a TMX).
This is actually not very well handled (we use a memory of 0 as a null value) and not properly explained in the documentation so your doubts are perfectly reasonable.
We will work in improving the API response and the documentation as well.
Thank you!

Thank you for your answer, indeed, I forgot to add that the newly inserted translation was available. I was wondering whether or not it was stored in right place.

Thanks @frdg
I'm keeping this issue open so that it will be used as a reminder, we will close it as soon as we fixed the odd output.


Migrate to hatch/pyproject.toml

Should roughly use the setup provided here: https://github.com/pimoroni/boilerplate-python
This includes some changes to how linting/tests are handled and some other "improvements" that would be nice here.

I, uh, should probably get around to this sometime.
Since this is also a systemy tool we should look into things like uv, since uv run 32blit Just Works (I think).


DP: Investigate and start building code for Geo DP to use in our proxy

@ingastrelnikova assigning to you, as you already started to investigate.
(I can help if needed)

Pending:
Clean script
Generate new points (noise) for all polygons (all cells)
Add command to send file to vm: either scpor copy-files
User queries request? -> we filter the dataset (per date or county/state) and then we do DP with our implementation and then with the new table generated and then we can do the spatial functions..

scp command to send file to vm:
gcloud compute scp test-vm.txt vm-postgis-03:~
where test-vm.txt is the local path/file..
This assumes the project is already set to ours, we can create a quick script for it, if needed, it should do:
gcloud config set project peng-geodp
gcloud config set compute/region europe-west3
gcloud config set compute/zone europe-west3-c


Change toolbar so it appears only when the mouse is over it or it gets the focus

Currently the toolbar disappears after a short interval but immediately re-appears if the mouse is moved.  this is mildly annoying to some people (and "super annoying" to others...).
See: http://idpf.org/forum/topic-1755
iBooks only makes the toolbar appear if the mouse pointer is actually over the toolbar area OR if the keyboard shortcut is used to give a toolbar control the focus.
The Chrome App should behave the same way.

Duplicate, with screenshots: https://github.com/readium/readium-js-viewer/issues/389

We also need to make certain that tabbing into the toolbar forces it to become visible.  For example, if focus is on the close icon of the table of contents and the user presses shift-tab, I believe the toolbar should appear (if it is not already visible) with focus on the enter full screen icon.  This could be confusing to sighted keyboard users who might be surprised at focus going to something they couldn't initially see on the screen but I think that consistent behavior for non-sighted users is also important. They will learn that the toolbar is always in the focus ring.  I'm open to other interpretations, though. I can't find good documentation for using iBook with the keyboard when VoiceOver is NOT running.  I don't believe you can fully navigate iBooks with only the keyboard unless VO is on.  Thus, I couldn't test how iBooks handles this situation.

Pushed a fix to develop. Want to get some people trying it out before I declare this fixed.
@becka11y Tabbing from the ToC isn't an issue since the navbar never disappears when the ToC is open. This is the current behavior so it's nothing new. I'm wondering if we should focus and display the navbar when someone hits tab while the navbar is hidden. Thoughts?

We probably shouldn't focus the tab bar when a user presses the tab key because they could want to tab to the next taxable item within the content.  Although, as I am playing with this, I often don't see the nav bar disappear at all? It seems to stay visible even when the toc is not displayed?  I'm playing with it more now in Safari.

The updated navbar show/hide behaviour looks fine to me. Mouse usage is intuitive, and keyboard navigation works as usual: e.g. x and v single-key accelerators to hide / show the navbar (respectively), and tab-cycling into the navbar automatically reveals it. As usual, when the keyboard focus is within the navbar, the navbar remains visible.
I agree with @becka11y that once I start using keyboard navigation, the navbar show/hide behaviour somehow feels less "predictable. Sometimes I am not able to switch back to auto-hide mode (even after clicking within the displayed document text).

@danielweck @becka11y Thanks for testing. This is what I was worried about. I must have missed an edge case that stops the show hide loop.


[Bug]:  WEB:  Unsupported operation: Platform._operatingSystem

Package Version
2.7.2
Flutter Version
3.19
Platforms
Web
How to reproduce?
make an emoji and drag it around.  Drag is also kind of broken.
Logs (optional)
══╡ EXCEPTION CAUGHT BY GESTURE ╞═══════════════════════════════════════════════════════════════════
The following UnsupportedError was thrown while handling a gesture:
Unsupported operation: Platform._operatingSystem

Example code (optional)
mouse is not visible, but it's on the right side

Also, adding text makes a full screen barrier, but the image isn't maximized like drawing.

Device Model (optional)
linux

I assume you need to use: universal_io: ^2.2.2

Thanks again, Steve for reporting an issue.
I found the problem with the platform error. I had a part in the code that looks like this if (Platform.isAndroid) and you right universal_io will resolve this issue and make it a bit easier for me to program the package. But I try to use not many other packages, so I just changed the code to this if(!kIsWeb && Platform.isAndroid) which should also resolve this issue.
To the part with “mouse is not visible, but it's on the right side”:
Yes, in the current version I had the problem that layers do not detect correct gestures when they are overlaid on the background image. However, in the dev branch I have already completely rewritten the way they are detected, which should solve this problem. I can't say when I will merge the dev branch into the stable branch because I still have to write more code for the new crop-rotate editor, but I hope to release it in the next weeks.

cool.  A few minutes ago I had the same thought about doing if(!kIsWeb && Platform.isAndroid), I was going to suggest that.  Thanks.


Increase checkerboard texture contrast

Increases the contrast of the checkerboard background used at several places in the editor (restores it to the value used some time ago). Also removed the unused color_picker_sample icon and cleaned up redundant linecap attributes in the SVGs.
This makes it easier to distinguish small transparency gradations:
Closes #57891.
Current master:

This PR:

You should also remove the theme property's entry from the docs.

Should probably add GuiMiniCheckerboard to the list of color exceptions so that it doesn't get converted. Also may be worth to still use a similar color from the palette, #808080, instead of a custom color.

@pycbouh Implemented the suggested changes.

Thanks!


Backported fixes for tests that were not prepared for TASK_STARTING.

Mostly a cherry-pick of 31c4d688f9a571b103e73d8ea7be4af02ae109ac without including changes to common/shell.hpp as those are not test related.

Seems we need yet another fix as per CI results....
../tests/journald_tests.cpp:482
Value of: strings::contains(secondQuery.get(), specialChildString)
  Actual: true
Expected: false


urllib3: Add mechanisms to fall back to TLS 1.0 and 1.1 with urllib3 version 2

About
The upcoming urllib3-2.0.0 will drop support for TLS 1.0 and 1.1 by default, however it can be re-enabled on demand.

If you still need to use TLS 1.0 or 1.1 in your application you can still upgrade to v2.0, you’ll only need to set ssl_minimum_version to the proper value to continue using legacy TLS versions.
-- https://urllib3.readthedocs.io/en/latest/v2-migration-guide.html#modern-security-by-default

Proposal
crate-python should offer corresponding fallback adjustments, for both connecting via DBAPI, and for connecting via SQLAlchemy.
References

GH-480
GH-524

crate-python should offer corresponding fallback adjustments to re-enable support for TLS 1.0 and TLS 1.1.

The reworked patch now takes care of that.

GH-527


Update notification made Firefox loose all my tabs

When I started Firefox toay it came with those instructions on how to set Proxyswitcher (never had seen it before), but Ff also lost data on all my more than 1000 open tabs.
It may have been some bad interaction with Tab Groups, my most beloved and necessary addon.
So cheers, I'll see if I recover my tabs.

Just recovered them from a recent backup.


React Forget

There might be a significant change coming about the React.memo called React Forget.
If they'd release it, I'd advise adding a question regarding that topic.
Source: https://www.youtube.com/watch?v=lGEMwh32soc

Okay. Thanks for the suggestion

The question related to this feature will be once available in ReactJS. We will close this issue for now.


Sealed promise not waiting for the finally clause in a previous sealed promise

Currently if we chain a sealed promise, containing a .finally clause with another sealed promise, then the execution of the second promise will start immediately after the last .then clause of the first promise.

Please provide sample code for this.

From what I could discern from your description I made this and you can see the order is as expected:

Did I understand wrongly?

I am currently using PromiseKit 3 and Xcode 7.1
What I actually refer to is the finally clause.
I am also including a code to show what is my point. In short - there is no guarantee that the finally clause of getUsers will be finished before the next promise getMessagesForUsers start execution.
The case that I describe is that we should create a token which is used to execute a query and then destroy it. If the finally logic is not executed on time, the next createToken will fail, as the server already has a token created.
- (AnyPromise *)getUsers
{
    __block NSString *serverToken = nil;

    return [AnyPromise promiseWithResolverBlock:^(PMKResolver  _Nonnull resolve) {
        [self createTokenOnServer:^(NSString *token) {
            serverToken = token;
            resolve(token);
        }];
    }].then(^(NSString *token) {
        return [self fetchServerUsersUsingToken:token];
    }).finally(^{
        if (serverToken) {
            [self destroyToken:serverToken];
        }
    });
}

- (AnyPromise *)getMessagesForUsers:(NSArray *)users
{
    __block NSString *serverToken = nil;

    return [AnyPromise promiseWithResolverBlock:^(PMKResolver  _Nonnull resolve) {
        [self createTokenOnServer:^(NSString *token) {
            serverToken = token;
            resolve(token);
        }];
    }].then(^(NSString *token) {
        return [self fetchServerMessagesForUsers:users usingToken:token];
    }).finally(^{
        if (serverToken) {
            [self destroyToken:serverToken];
        }
    });
}

- (AnyPromise *)doTheWork
{
    return [self getUsers].then(^(NSArray *users) {
        [self getMessagesForUsers:users];
    });
}

I am doing this whole logic, because I should follow the rules on the server - to create a token, use it for one request only and then destroy it. If there is any other mechanism to achieve this logic, please advise.

I just wrote this test that proves the finally finishes before any subsequent executions can run:

Can you provide a test project that demonstrates your bug?

Is your problem that you are not returning [self destroyToken:serverToken];?
Because then that asynchronous operation will not have completed before getMessagesForUsers runs.
Perhaps you want:
- (AnyPromise *)getUsers
{
    __block NSString *serverToken = nil;

    return [AnyPromise promiseWithResolverBlock:^(PMKResolver  _Nonnull resolve) {
        [self createTokenOnServer:^(NSString *token) {
            serverToken = token;
            resolve(token);
        }];
    }].then(^(NSString *token) {
        return [self fetchServerUsersUsingToken:token];
    }).finally(^id{
        if (serverToken) {
            return [self destroyToken:serverToken];
        } else {
            return nil;
        }
    });
}

Probably what you mention is the exact cause of the problem - that I am not returning anything from the finally block.
With the current version that I use - PromiseKit 3 there is an error if I try to return a result from the finally:
Incompatible block pointer types passing 'id (^)(void)' to parameter of type 'dispatch_block_t  _Nonnull __strong' (aka 'void (^__strong)(void)')

Since the declaration is (AnyPromise *(^)(dispatch_block_t))finally, the finally clause returns void. Is there something that I am missing here? How is it possible to return something from the finally?

Indeed, you cannot. I’m not sure why I thought this feature existed. The reason it does not is this:
somePromise1.then(^{
   return @1;
}).finally(^{
    // can't interfere with chain
}).then(^(id obj){
   // obj is @1
});

finally is for book-keeping, it just passes through the @1 and thus can't interfere with the chain. If we made it possible to wait on a promise it would introduce ambiguity. Does the promise influence subsequent then values? If not does that mean that users must always manually pipe the value in future?

Still I don't think it is necessarily what you want, you want to ensure create always follows a destroy, so really you need to do something like this:
- (AnyPromise *)getUsers
{
    __block NSString *serverToken = nil;
    __block static AnyPromise *destroy = [AnyPromise promiseWithValue:@1];

    return [AnyPromise promiseWithResolverBlock:^(PMKResolver  _Nonnull resolve) {
        destroy.finally(^{
            [self createTokenOnServer:^(NSString *token) {
                serverToken = token;
                resolve(token);
            }];
        });
    }].then(^(NSString *token) {
        return [self fetchServerUsersUsingToken:token];
    }).finally(^id{
        if (serverToken) {
            destroy = [self destroyToken:serverToken];
        }
    });
}

Actually my case is even trickier - destroy should always happen if a token was successfully created, but not before executing the real request - i.e. fetchServerUsersUsingToken. If the destroy happens before the request - the token actually does not exist anymore on the server.
I though for a different scenario:

Create the token
then
Make the request with the token
then
Destroy the token, passing through the previous token result (the result from 2)
then
Do anything additional

The only drawback of this logic is that the destroy might not get called if 2. fails

This is not elegant, but without a better idea of how to approach API for the pattern, this is what we have:
return [AnyPromise promiseWithResolverBlock:^(PMKResolver resolve) {
    [self createToken].then(^(id token){
        AnyPromise *requestPromise =  [self makeRequest:token]
        requestPromise.finally(^{
             [self destroyToken:token].finally(^{ resolve(requestPromise.value) });
        });
    });
}];

Thanks you for the good idea. Although not very elegant, it will be sufficient for my needs.


parameterize index in civis.io.dataframe_to_civis

@keithing I use civis.io.dataframe_to_civis to write to redshift, and had meaningfull indexes but these weren't written and it doesn't appear there's the ability at all to write them. We may want to make this a parameter (which would get passed to to_csv).

@nickahamed , if you call dataframe_to_civis with df.reset_index() instead of df (assuming your DataFrame is named df), that will include your index as the first column of the table. That seems simpler than adding a parameter to the dataframe_to_civis function.

yeah, I got there eventually. Just not as intuitive for the intro python user. I don't know if we can add that recommendation into the docs somehow so folks aren't left puzzled.


chore(deps): upgrade configuration

Upgrades project dependencies. See details in workflow run.

Automatically created by projen via the "upgrade-configuration-2.x" workflow

@Mergifyio refresh

⚪ Backport skipped
The pull request was not backported as there were no branches to backport to. If this is a mistake, please apply the desired version labels or run the backport tool manually.
Manual backport
To create the backport manually run:
backport --pr 2586

Questions ?
Please refer to the Backport tool documentation


SubFactory: weird behavior

In the snippet below, only test_stub fails. Is that expected?
from django.db import models
import factory
from factory.containers import StubObject

class B(models.Model):
    pass

class A(models.Model):
    b = models.ForeignKey(B)

class BFactory(factory.Factory):
    class Meta:
        model = B

class AFactory(factory.Factory):
    class Meta:
        model = A

    b = factory.SubFactory(BFactory)

def test_stub():
    a = AFactory.stub()
    assert isinstance(a, StubObject)
    assert isinstance(a.b, StubObject)  # this assertion fails. Is that per design?

def test_create():
    a = AFactory.create()
    assert isinstance(a, A)
    assert isinstance(a.b, B)

def test_build():
    a = AFactory.build()
    assert isinstance(a, A)
    assert isinstance(a.b, B)

Well, that's indeed a bug.
However, the "stub" strategy is less useful now that build() works properly, so improving "stub" support has not been high in the todo-list.
However, I'd be more than happy to merge a patch about this, and even to guide you in writing such a fix :)


Additions in Issue Template

Description:
I noticed that the repository does not have an appropriate issue template.
Issue Template: I want to add  some more points to your issue template like styling changes etc
Please assign this issue to me under Gssoc'23 @LakshaySK106

Assigned you!


python3.pkgs.ezdxf: 0.18.1 -> 1.3.2

Description of changes
The previous package is both broken (pytest failures) & out of date.
The only dependent (meerk40t) is broken on Python 3.12 already, where it's only an optional dependency.
Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Because the existing package is broken this can only improve things. Self-merging.


[AutoPR eventhub/resource-manager] EventHub : corrected typo

Created to sync https://github.com/Azure/azure-rest-api-specs/pull/5360

(message created by the CI based on PR content)

This PR has been merged into https://github.com/Azure/azure-sdk-for-python/pull/4529