Gateway API cannot access outside of cluster node

Is there an existing issue for this?

[X] I have searched the existing issues

What happened?
I setup a multi node k8s environment on bare metal machine, use cilium as cni, use metallb support loadbalancer service ip allocate. run example gateway api test, the service is accessible inner k8s node; while outside k8s node, for example other server which in the same network range or local pc, the service is un-accessible.
some test env
(base)  ➜ ~  kubectl get nodes
NAME           STATUS                     ROLES                      AGE    VERSION
<IP_ADDRESS>   Ready                      worker                     24d    v1.27.11
<IP_ADDRESS>   Ready,SchedulingDisabled   worker                     11d    v1.27.11
<IP_ADDRESS>   Ready                      worker                     24d    v1.27.11
<IP_ADDRESS>   Ready                      controlplane,etcd,worker   235d   v1.27.11
<IP_ADDRESS>   Ready                      controlplane,etcd,worker   418d   v1.27.11
<IP_ADDRESS>   Ready                      controlplane,etcd,worker   314d   v1.27.11
<IP_ADDRESS>   Ready,SchedulingDisabled   worker                     157d   v1.27.11
<IP_ADDRESS>    Ready,SchedulingDisabled   worker                     136d   v1.27.11
<IP_ADDRESS>    Ready                      worker                     75d    v1.27.11

(base) ➜ ~  kubectl get gc -A
NAME     CONTROLLER                                      ACCEPTED   AGE
cilium   io.cilium/gateway-controller                    True       11d
eg       gateway.envoyproxy.io/gatewayclass-controller   True       44d
kong     konghq.com/kic-gateway-controller               True       77d

(base)  ➜ ~  kubectl get gateway -A
NAMESPACE   NAME        CLASS    ADDRESS        PROGRAMMED   AGE
default     cilium-gw   cilium   <IP_ADDRESS>   True         11d

(base)  ➜ ~  kubectl get gateway -o yaml
apiVersion: v1
items:
- apiVersion: gateway.networking.k8s.io/v1
  kind: Gateway
  metadata:
    annotations:
      kubectl.kubernetes.io/last-applied-configuration: |
        {"apiVersion":"gateway.networking.k8s.io/v1","kind":"Gateway","metadata":{"annotations":{},"name":"cilium-gw","namespace":"default"},"spec":{"gatewayClassName":"cilium","listeners":[{"name":"web-gw-echo","port":30988,"protocol":"HTTP"}]}}
    creationTimestamp: "2024-04-24T06:25:58Z"
    generation: 2
    name: cilium-gw
    namespace: default
    resourceVersion: "264290055"
    uid: 031185f3-663f-4fc5-b5e1-0ac6d8b92038
  spec:
    gatewayClassName: cilium
    listeners:
    - allowedRoutes:
        namespaces:
          from: All
      name: web-gw-echo
      port: 30988
      protocol: HTTP
  status:
    addresses:
    - type: IPAddress
      value: <IP_ADDRESS>
    conditions:
    - lastTransitionTime: "2024-04-26T02:30:03Z"
      message: Gateway successfully scheduled
      observedGeneration: 2
      reason: Accepted
      status: "True"
      type: Accepted
    - lastTransitionTime: "2024-05-05T12:10:12Z"
      message: Gateway successfully reconciled
      observedGeneration: 2
      reason: Programmed
      status: "True"
      type: Programmed
    listeners:
    - attachedRoutes: 1
      conditions:
      - lastTransitionTime: "2024-05-05T12:10:12Z"
        message: Listener Programmed
        observedGeneration: 2
        reason: Programmed
        status: "True"
        type: Programmed
      - lastTransitionTime: "2024-05-05T12:10:12Z"
        message: Listener Accepted
        observedGeneration: 2
        reason: Accepted
        status: "True"
        type: Accepted
      - lastTransitionTime: "2024-05-05T12:10:12Z"
        message: Resolved Refs
        reason: ResolvedRefs
        status: "True"
        type: ResolvedRefs
      name: web-gw-echo
      supportedKinds:
      - group: gateway.networking.k8s.io
        kind: HTTPRoute
kind: List
metadata:
  resourceVersion: ""

(base) ➜ ~  kubectl get httproute -o yaml
apiVersion: v1
items:
- apiVersion: gateway.networking.k8s.io/v1
  kind: HTTPRoute
  metadata:
    annotations:
      kubectl.kubernetes.io/last-applied-configuration: |
        {"apiVersion":"gateway.networking.k8s.io/v1","kind":"HTTPRoute","metadata":{"annotations":{},"name":"example-route-1","namespace":"default"},"spec":{"parentRefs":[{"name":"cilium-gw"}],"rules":[{"backendRefs":[{"kind":"Service","name":"echo-1","port":8080,"weight":50},{"kind":"Service","name":"echo-2","port":8090,"weight":25},{"kind":"Service","name":"echo-3","port":8090,"weight":25}],"matches":[{"path":{"type":"PathPrefix","value":"/echo"}}]}]}}
    creationTimestamp: "2024-04-24T06:25:58Z"
    generation: 3
    name: example-route-1
    namespace: default
    resourceVersion: "264333662"
    uid: 4096d5d1-44f0-47c5-b695-d6efd25532fc
  spec:
    parentRefs:
    - group: gateway.networking.k8s.io
      kind: Gateway
      name: cilium-gw
    rules:
    - backendRefs:
      - group: ""
        kind: Service
        name: echo-1
        port: 8080
        weight: 1
      matches:
      - path:
          type: PathPrefix
          value: /echo
  status:
    parents:
    - conditions:
      - lastTransitionTime: "2024-05-05T13:45:52Z"
        message: Accepted HTTPRoute
        observedGeneration: 3
        reason: Accepted
        status: "True"
        type: Accepted
      - lastTransitionTime: "2024-05-05T13:45:52Z"
        message: Service reference is valid
        observedGeneration: 3
        reason: ResolvedRefs
        status: "True"
        type: ResolvedRefs
      controllerName: io.cilium/gateway-controller
      parentRef:
        group: gateway.networking.k8s.io
        kind: Gateway
        name: cilium-gw
kind: List
metadata:
  resourceVersion: ""

(base)  ➜ ~  kubectl get svc | grep echo
echo-1                          ClusterIP      <IP_ADDRESS>   <none>                                              8080/TCP                                             11d
echo-1-lb                       LoadBalancer   <IP_ADDRESS>     <IP_ADDRESS>                                        8080:32337/TCP                                       133m
echo-2                          ClusterIP      <IP_ADDRESS>    <none>                                              8090/TCP                                             11d
echo-3                          ClusterIP      <IP_ADDRESS>     <none>                                              8090/TCP                                             10d

gateway access compare of inner k8s and outside k8s
curl success from inner k8s node
# from <IP_ADDRESS>
(base) ➜ ~  curl -vvv http://<IP_ADDRESS>:30988/echo
*   Trying <IP_ADDRESS>:30988...
* TCP_NODELAY set
* Connected to <IP_ADDRESS> (<IP_ADDRESS>) port 30988 (#0)
> GET /echo HTTP/1.1
> Host: <IP_ADDRESS>:30988
> User-Agent: curl/7.68.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< date: Sun, 05 May 2024 14:40:03 GMT
< content-type: text/plain
< server: envoy
< x-envoy-upstream-service-time: 0
< transfer-encoding: chunked
<

Hostname: echo-1-78b66687b5-bfj4s

Pod Information:
        node name:      <IP_ADDRESS>
        pod name:       echo-1-78b66687b5-bfj4s
        pod namespace:  default
        pod IP: <IP_ADDRESS>

Server values:
        server_version=nginx: 1.12.2 - lua: 10010

Request Information:
        client_address=<IP_ADDRESS>
        method=GET
        real path=/echo
        query=
        request_version=1.1
        request_scheme=http
        request_uri=http://<IP_ADDRESS>:8080/echo

Request Headers:
        accept=*/*
        host=<IP_ADDRESS>:30988
        user-agent=curl/7.68.0
        x-envoy-expected-rq-timeout-ms=15000
        x-envoy-internal=true
        x-forwarded-for=<IP_ADDRESS>
        x-forwarded-proto=http
        x-request-id=df402da0-2963-4c07-90a4-c64dd7e955a4

Request Body:
        -no body in request-

*

curl failed from outside k8s server
# from <IP_ADDRESS>
 ✘  ~  curl -vvv http://<IP_ADDRESS>:30988/echo --connect-timeout 2
*   Trying <IP_ADDRESS>...
* TCP_NODELAY set
* Connection timed out after 2000 milliseconds
* stopped the pause stream!
* Closing connection 0
curl: (28) Connection timed out after 2000 milliseconds

The direct loadbalancer service is worked
curl from outside k8s server
# from <IP_ADDRESS>
 ✘ qiang.qi@sjhltj-plat-aigc-dev-005  ~  curl -vvv http://<IP_ADDRESS>:8080/echo --connect-timeout 2
*   Trying <IP_ADDRESS>...
* TCP_NODELAY set
* Connected to <IP_ADDRESS> (<IP_ADDRESS>) port 8080 (#0)
> GET /echo HTTP/1.1
> Host: <IP_ADDRESS>:8080
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Sun, 05 May 2024 14:42:45 GMT
< Content-Type: text/plain
< Transfer-Encoding: chunked
< Connection: keep-alive
< Server: echoserver
<

Hostname: echo-1-78b66687b5-bfj4s

Pod Information:
        node name:      <IP_ADDRESS>
        pod name:       echo-1-78b66687b5-bfj4s
        pod namespace:  default
        pod IP: <IP_ADDRESS>

Server values:
        server_version=nginx: 1.12.2 - lua: 10010

Request Information:
        client_address=<IP_ADDRESS>
        method=GET
        real path=/echo
        query=
        request_version=1.1
        request_scheme=http
        request_uri=http://<IP_ADDRESS>:8080/echo

Request Headers:
        accept=*/*
        host=<IP_ADDRESS>:8080
        user-agent=curl/7.58.0

Request Body:
        -no body in request-

* Connection #0 to host <IP_ADDRESS> left intact

some config view or test result
(base) » cilium status                                                                                [16:39:27]
    /¯¯\
 /¯¯\__/¯¯\    Cilium:             1 warnings
 \__/¯¯\__/    Operator:           OK
 /¯¯\__/¯¯\    Envoy DaemonSet:    OK
 \__/¯¯\__/    Hubble Relay:       OK
    \__/       ClusterMesh:        disabled

Deployment             hubble-ui          Desired: 1, Ready: 1/1, Available: 1/1
DaemonSet              cilium-envoy       Desired: 9, Ready: 9/9, Available: 9/9
DaemonSet              cilium             Desired: 9, Ready: 9/9, Available: 9/9
Deployment             cilium-operator    Desired: 1, Ready: 1/1, Available: 1/1
Deployment             hubble-relay       Desired: 1, Ready: 1/1, Available: 1/1
Containers:            cilium             Running: 9
                       cilium-operator    Running: 1
                       hubble-relay       Running: 1
                       hubble-ui          Running: 1
                       cilium-envoy       Running: 9
Cluster Pods:          260/307 managed by Cilium
Helm chart version:
Image versions         cilium-envoy       quayproxy/cilium/cilium-envoy@sha256:d52f476c29a97c8b250fdbfbb8472191a268916f6a8503671d0da61e323b02cc: 9
                       cilium             quayproxy/cilium/cilium@sha256:b760a4831f5aab71c711f7537a107b751d0d0ce90dd32d8b358df3c5da385426: 1
                       cilium             quay.io/cilium/cilium:v1.15.4@sha256:b760a4831f5aab71c711f7537a107b751d0d0ce90dd32d8b358df3c5da385426: 8
                       cilium-operator    quayproxy/cilium/operator-generic@sha256:404890a83cca3f28829eb7e54c1564bb6904708cdb7be04ebe69c2b60f164e9a: 1
                       hubble-relay       quay.io/cilium/hubble-relay:v1.15.4@sha256:03ad857feaf52f1b4774c29614f42a50b370680eb7d0bfbc1ae065df84b1070a: 1
                       hubble-ui          quayproxy/cilium/hubble-ui@sha256:7d663dc16538dd6e29061abd1047013a645e6e69c115e008bee9ea9fef9a6666: 1
                       hubble-ui          quayproxy/cilium/hubble-ui-backend@sha256:1e7657d997c5a48253bb8dc91ecee75b63018d16ff5e5797e5af367336bc8803: 1
Warnings:              cilium             cilium-p7fqp    Hubble: Server not initialized

(base) ➜ ~  cilium config view                                               <<<
agent-not-ready-taint-key                         node.cilium.io/agent-not-ready
arping-refresh-period                             30s
auto-direct-node-routes                           false
bpf-lb-acceleration                               disabled
bpf-lb-external-clusterip                         false
bpf-lb-map-max                                    65536
bpf-lb-sock                                       false
bpf-map-dynamic-size-ratio                        0.0025
bpf-policy-map-max                                16384
bpf-root                                          /sys/fs/bpf
cgroup-root                                       /run/cilium/cgroupv2
cilium-endpoint-gc-interval                       5m0s
cluster-id                                        0
cluster-name                                      local
cluster-pool-ipv4-cidr                            <IP_ADDRESS>/16
cluster-pool-ipv4-mask-size                       24
cni-exclusive                                     true
cni-log-file                                      /var/run/cilium/cilium-cni.log
custom-cni-conf                                   false
debug                                             true
debug-verbose                                     flow
dnsproxy-enable-transparent-mode                  true
egress-gateway-reconciliation-trigger-interval    1s
enable-auto-protect-node-port-range               true
enable-bgp-control-plane                          false
enable-bpf-clock-probe                            false
enable-endpoint-health-checking                   true
enable-envoy-config                               true
enable-gateway-api                                true
enable-gateway-api-secrets-sync                   true
enable-health-check-loadbalancer-ip               false
enable-health-check-nodeport                      true
enable-health-checking                            true
enable-host-legacy-routing                        false
enable-hubble                                     true
enable-ingress-controller                         true
enable-ingress-proxy-protocol                     false
enable-ingress-secrets-sync                       true
enable-ipv4                                       true
enable-ipv4-big-tcp                               false
enable-ipv4-masquerade                            true
enable-ipv6                                       false
enable-ipv6-big-tcp                               false
enable-ipv6-masquerade                            true
enable-k8s-networkpolicy                          true
enable-k8s-terminating-endpoint                   true
enable-l2-announcements                           true
enable-l2-neigh-discovery                         true
enable-l7-proxy                                   true
enable-local-redirect-policy                      false
enable-masquerade-to-route-source                 false
enable-metrics                                    true
enable-policy                                     default
enable-remote-node-identity                       true
enable-sctp                                       false
enable-svc-source-range-check                     true
enable-vtep                                       false
enable-well-known-identities                      false
enable-xt-socket-fallback                         true
enforce-ingress-https                             true
external-envoy-proxy                              true
gateway-api-secrets-namespace                     cilium-secrets
hubble-disable-tls                                false
hubble-export-file-max-backups                    5
hubble-export-file-max-size-mb                    10
hubble-listen-address                             :4244
hubble-socket-path                                /var/run/cilium/hubble.sock
hubble-tls-cert-file                              /var/lib/cilium/tls/hubble/server.crt
hubble-tls-client-ca-files                        /var/lib/cilium/tls/hubble/client-ca.crt
hubble-tls-key-file                               /var/lib/cilium/tls/hubble/server.key
identity-allocation-mode                          crd
identity-gc-interval                              15m0s
identity-heartbeat-timeout                        30m0s
ingress-default-lb-mode                           dedicated
ingress-lb-annotation-prefixes                    service.beta.kubernetes.io service.kubernetes.io cloud.google.com
ingress-secrets-namespace                         cilium-secrets
ingress-shared-lb-service-name                    cilium-ingress
install-no-conntrack-iptables-rules               false
ipam                                              cluster-pool
ipam-cilium-node-update-rate                      15s
k8s-client-burst                                  20
k8s-client-qps                                    10
kube-proxy-replacement                            true
kube-proxy-replacement-healthz-bind-address
max-connected-clusters                            255
mesh-auth-enabled                                 true
mesh-auth-gc-interval                             5m0s
mesh-auth-queue-size                              1024
mesh-auth-rotated-identities-queue-size           1024
monitor-aggregation                               medium
monitor-aggregation-flags                         all
monitor-aggregation-interval                      5s
node-port-bind-protection                         true
nodes-gc-interval                                 5m0s
operator-api-serve-addr                           <IP_ADDRESS>:9234
operator-prometheus-serve-addr                    :9963
policy-cidr-match-mode
preallocate-bpf-maps                              false
procfs                                            /host/proc
proxy-connect-timeout                             30
proxy-max-connection-duration-seconds             0
proxy-max-requests-per-connection                 0
remove-cilium-node-taints                         true
routing-mode                                      tunnel
service-no-backend-response                       reject
set-cilium-is-up-condition                        true
set-cilium-node-taints                            true
sidecar-istio-proxy-image                         cilium/istio_proxy
skip-cnp-status-startup-clean                     false
synchronize-k8s-nodes                             true
tofqdns-dns-reject-response-code                  refused
tofqdns-enable-dns-compression                    true
tofqdns-endpoint-max-ip-per-hostname              50
tofqdns-idle-connection-grace-period              0s
tofqdns-max-deferred-connection-deletes           10000
tofqdns-proxy-response-max-delay                  100ms
tunnel-port                                       8573
tunnel-protocol                                   vxlan
unmanaged-pod-watcher-interval                    15
vtep-cidr
vtep-endpoint
vtep-mac
vtep-mask
write-cni-conf-when-ready                         /host/etc/cni/net.d/05-cilium.conflist

(base)  ➜ ~  kubectl -n kube-system exec ds/cilium -- cilium-dbg status --verbose
Defaulted container "cilium-agent" out of: cilium-agent, config (init), mount-cgroup (init), apply-sysctl-overwrites (init), mount-bpf-fs (init), clean-cilium-state (init), install-cni-binaries (init)
KVStore:                Ok   Disabled
Kubernetes:             Ok   1.27 (v1.27.11) [linux/amd64]
Kubernetes APIs:        ["EndpointSliceOrEndpoint", "cilium/v2::CiliumClusterwideEnvoyConfig", "cilium/v2::CiliumClusterwideNetworkPolicy", "cilium/v2::CiliumEndpoint", "cilium/v2::CiliumEnvoyConfig", "cilium/v2::CiliumNetworkPolicy", "cilium/v2::CiliumNode", "cilium/v2alpha1::CiliumCIDRGroup", "core/v1::Namespace", "core/v1::Pods", "core/v1::Secrets", "core/v1::Service", "networking.k8s.io/v1::NetworkPolicy"]
KubeProxyReplacement:   True   [enp96s0f0   <IP_ADDRESS> fe80::6e92:bfff:fed6:8e80 (Direct Routing), tunl0  <IP_ADDRESS>]
Host firewall:          Disabled
SRv6:                   Disabled
CNI Chaining:           none
Cilium:                 Ok   1.15.4 (v1.15.4-9b3f9a8c)
NodeMonitor:            Listening for events on 64 CPUs with 64x4096 of shared memory
Cilium health daemon:   Ok
IPAM:                   IPv4: 23/254 allocated from <IP_ADDRESS>/24,
Allocated addresses:
  <IP_ADDRESS> (kube-system/seaweedfs-csi-driver-node-6lg5f [restored])
  <IP_ADDRESS> (longhorn-system/longhorn-csi-plugin-db45q)
  <IP_ADDRESS> (longhorn-system/instance-manager-e-eb075fabca1262a6ebdfa0c5f7ac8ee0 [restored])
  <IP_ADDRESS> (arachnid/llm-sphinx-8154e41fae8d42afa1b1f644887e6351-7c9b97bdfc-79lhz [restored])
  <IP_ADDRESS> (longhorn-system/csi-provisioner-65dd48db7b-xpb25 [restored])
  <IP_ADDRESS> (arachnid/llm-sphinx-d4660d9fbbe2453eb8ce81cc5a0c682f-7cdb7c6f85-qvnfg [restored])
  <IP_ADDRESS> (longhorn-system/csi-attacher-689fdb475f-vjqhc [restored])
  <IP_ADDRESS> (longhorn-system/csi-snapshotter-59ff78b47b-88t78 [restored])
  <IP_ADDRESS> (ingress)
  <IP_ADDRESS> (router)
  <IP_ADDRESS> (monitoring/alertmanager-main-0 [restored])
  <IP_ADDRESS> (longhorn-system/csi-resizer-847d645569-mlc6h [restored])
  <IP_ADDRESS> (ingress-nginx/nginx-ingress-controller-blbsr [restored])
  <IP_ADDRESS> (monitoring/promtail-zxf5s [restored])
  <IP_ADDRESS> (default/goldpinger-jvmk8)
  <IP_ADDRESS> (health)
  <IP_ADDRESS> (longhorn-system/longhorn-admission-webhook-66676d6947-7p9rp [restored])
  <IP_ADDRESS> (coroot/coroot-node-agent-tdwxk)
  <IP_ADDRESS> (arachnid/llm-serve-222ba757-0dd7-4acc-9c37-cc271a094b52-8978fcbf5-4x78b [restored])
  <IP_ADDRESS> (longhorn-system/longhorn-manager-x8hd4 [restored])
  <IP_ADDRESS> (longhorn-system/instance-manager-r-eb075fabca1262a6ebdfa0c5f7ac8ee0 [restored])
  <IP_ADDRESS> (default/dcgm-exporter-6kb8l [restored])
  <IP_ADDRESS> (athena/deployment-zhongshihua-7574bb5656-zh2td [restored])
IPv4 BIG TCP:           Disabled
IPv6 BIG TCP:           Disabled
BandwidthManager:       Disabled
Host Routing:           Legacy
Masquerading:           IPTables [IPv4: Enabled, IPv6: Disabled]
Clock Source for BPF:   ktime
Controller Status:      140/140 healthy
  Name                                                                                Last success     Last error       Count   Message
  bpf-map-sync-cilium_lxc                                                                  3s ago           never            0       no error
  cilium-health-ep                                                                         50s ago          never            0       no error
  dns-garbage-collector-job                                                                17s ago          never            0       no error
  endpoint-1053-regeneration-recovery                                                      never            never            0       no error
  endpoint-11-regeneration-recovery                                                        never            never            0       no error
  endpoint-1292-regeneration-recovery                                                      never            never            0       no error
  endpoint-1335-regeneration-recovery                                                      never            never            0       no error
  endpoint-1728-regeneration-recovery                                                      never            never            0       no error
  endpoint-1777-regeneration-recovery                                                      never            never            0       no error
  endpoint-1789-regeneration-recovery                                                      never            never            0       no error
  endpoint-1912-regeneration-recovery                                                      never            never            0       no error
  endpoint-2085-regeneration-recovery                                                      never            never            0       no error
  endpoint-2739-regeneration-recovery                                                      never            never            0       no error
  endpoint-303-regeneration-recovery                                                       never            never            0       no error
  endpoint-3032-regeneration-recovery                                                      never            never            0       no error
  endpoint-3277-regeneration-recovery                                                      never            never            0       no error
  endpoint-3296-regeneration-recovery                                                      never            never            0       no error
  endpoint-344-regeneration-recovery                                                       never            never            0       no error
  endpoint-46-regeneration-recovery                                                        never            never            0       no error
  endpoint-476-regeneration-recovery                                                       never            never            0       no error
  endpoint-487-regeneration-recovery                                                       never            never            0       no error
  endpoint-491-regeneration-recovery                                                       never            never            0       no error
  endpoint-495-regeneration-recovery                                                       never            never            0       no error
  endpoint-792-regeneration-recovery                                                       never            never            0       no error
  endpoint-92-regeneration-recovery                                                        never            never            0       no error
  endpoint-935-regeneration-recovery                                                       never            never            0       no error
  endpoint-gc                                                                              1m17s ago        never            0       no error
  ep-bpf-prog-watchdog                                                                     9s ago           never            0       no error
  ipcache-inject-labels                                                                    1m1s ago         274h21m17s ago   0       no error
  k8s-heartbeat                                                                            22s ago          never            0       no error
  link-cache                                                                               13s ago          never            0       no error
  neighbor-table-refresh                                                                   21s ago          never            0       no error
  resolve-identity-1292                                                                    1m9s ago         never            0       no error
  resolve-identity-1728                                                                    4m45s ago        never            0       no error
  resolve-identity-3277                                                                    3m16s ago        never            0       no error
  resolve-identity-46                                                                      1m10s ago        never            0       no error
  resolve-identity-92                                                                      14s ago          never            0       no error
  resolve-labels-/                                                                         274h21m12s ago   never            0       no error
  resolve-labels-arachnid/llm-serve-222ba757-0dd7-4acc-9c37-cc271a094b52-8978fcbf5-4x78b   274h21m10s ago   never            0       no error
  resolve-labels-arachnid/llm-sphinx-8154e41fae8d42afa1b1f644887e6351-7c9b97bdfc-79lhz     274h21m9s ago    never            0       no error
  resolve-labels-arachnid/llm-sphinx-d4660d9fbbe2453eb8ce81cc5a0c682f-7cdb7c6f85-qvnfg     274h21m9s ago    never            0       no error
  resolve-labels-athena/deployment-zhongshihua-7574bb5656-zh2td                            274h21m9s ago    never            0       no error
  resolve-labels-coroot/coroot-node-agent-tdwxk                                            175h19m46s ago   never            0       no error
  resolve-labels-default/dcgm-exporter-6kb8l                                               274h21m10s ago   never            0       no error
  resolve-labels-default/goldpinger-jvmk8                                                  172h38m18s ago   never            0       no error
  resolve-labels-ingress-nginx/nginx-ingress-controller-blbsr                              274h21m9s ago    never            0       no error
  resolve-labels-kube-system/seaweedfs-csi-driver-node-6lg5f                               274h21m9s ago    never            0       no error
  resolve-labels-longhorn-system/csi-attacher-689fdb475f-vjqhc                             274h21m9s ago    never            0       no error
  resolve-labels-longhorn-system/csi-provisioner-65dd48db7b-xpb25                          274h21m10s ago   never            0       no error
  resolve-labels-longhorn-system/csi-resizer-847d645569-mlc6h                              274h21m10s ago   never            0       no error
  resolve-labels-longhorn-system/csi-snapshotter-59ff78b47b-88t78                          274h21m9s ago    never            0       no error
  resolve-labels-longhorn-system/instance-manager-e-eb075fabca1262a6ebdfa0c5f7ac8ee0       274h21m10s ago   never            0       no error
  resolve-labels-longhorn-system/instance-manager-r-eb075fabca1262a6ebdfa0c5f7ac8ee0       274h21m10s ago   never            0       no error
  resolve-labels-longhorn-system/longhorn-admission-webhook-66676d6947-7p9rp               274h21m9s ago    never            0       no error
  resolve-labels-longhorn-system/longhorn-csi-plugin-db45q                                 251h15m16s ago   never            0       no error
  resolve-labels-longhorn-system/longhorn-manager-x8hd4                                    274h21m10s ago   never            0       no error
  resolve-labels-monitoring/alertmanager-main-0                                            274h21m9s ago    never            0       no error
  resolve-labels-monitoring/promtail-zxf5s                                                 274h21m10s ago   never            0       no error
  restoring-ep-identity (1053)                                                             274h21m12s ago   never            0       no error
  restoring-ep-identity (11)                                                               274h21m12s ago   never            0       no error
  restoring-ep-identity (1335)                                                             274h21m12s ago   never            0       no error
  restoring-ep-identity (1777)                                                             274h21m12s ago   never            0       no error
  restoring-ep-identity (1789)                                                             274h21m12s ago   never            0       no error
  restoring-ep-identity (1912)                                                             274h21m12s ago   never            0       no error
  restoring-ep-identity (2085)                                                             274h21m12s ago   never            0       no error
  restoring-ep-identity (2739)                                                             274h21m12s ago   never            0       no error
  restoring-ep-identity (303)                                                              274h21m12s ago   never            0       no error
  restoring-ep-identity (3032)                                                             274h21m12s ago   never            0       no error
  restoring-ep-identity (3296)                                                             274h21m12s ago   never            0       no error
  restoring-ep-identity (344)                                                              274h21m12s ago   never            0       no error
  restoring-ep-identity (476)                                                              274h21m12s ago   never            0       no error
  restoring-ep-identity (487)                                                              274h21m12s ago   never            0       no error
  restoring-ep-identity (491)                                                              274h21m12s ago   never            0       no error
  restoring-ep-identity (495)                                                              274h21m12s ago   never            0       no error
  restoring-ep-identity (792)                                                              274h21m12s ago   never            0       no error
  restoring-ep-identity (935)                                                              274h21m12s ago   never            0       no error
  sync-host-ips                                                                            1m1s ago         never            0       no error
  sync-lb-maps-with-k8s-services                                                           274h21m12s ago   never            0       no error
  sync-policymap-1053                                                                      6m9s ago         never            0       no error
  sync-policymap-11                                                                        6m9s ago         never            0       no error
  sync-policymap-1292                                                                      6m9s ago         never            0       no error
  sync-policymap-1335                                                                      6m9s ago         never            0       no error
  sync-policymap-1728                                                                      4m45s ago        never            0       no error
  sync-policymap-1777                                                                      6m9s ago         never            0       no error
  sync-policymap-1789                                                                      6m9s ago         never            0       no error
  sync-policymap-1912                                                                      6m9s ago         never            0       no error
  sync-policymap-2085                                                                      6m9s ago         never            0       no error
  sync-policymap-2739                                                                      6m9s ago         never            0       no error
  sync-policymap-303                                                                       6m9s ago         never            0       no error
  sync-policymap-3032                                                                      6m9s ago         never            0       no error
  sync-policymap-3277                                                                      8m16s ago        never            0       no error
  sync-policymap-3296                                                                      6m9s ago         never            0       no error
  sync-policymap-344                                                                       6m9s ago         never            0       no error
  sync-policymap-476                                                                       6m9s ago         never            0       no error
  sync-policymap-487                                                                       6m9s ago         never            0       no error
  sync-policymap-491                                                                       6m9s ago         never            0       no error
  sync-policymap-495                                                                       6m9s ago         never            0       no error
  sync-policymap-792                                                                       6m9s ago         never            0       no error
  sync-policymap-92                                                                        15s ago          never            0       no error
  sync-policymap-935                                                                       6m9s ago         never            0       no error
  sync-to-k8s-ciliumendpoint (1053)                                                        10s ago          never            0       no error
  sync-to-k8s-ciliumendpoint (11)                                                          10s ago          never            0       no error
  sync-to-k8s-ciliumendpoint (1335)                                                        10s ago          never            0       no error
  sync-to-k8s-ciliumendpoint (1728)                                                        9s ago           never            0       no error
  sync-to-k8s-ciliumendpoint (1777)                                                        10s ago          never            0       no error
  sync-to-k8s-ciliumendpoint (1789)                                                        10s ago          never            0       no error
  sync-to-k8s-ciliumendpoint (1912)                                                        10s ago          never            0       no error
  sync-to-k8s-ciliumendpoint (2085)                                                        10s ago          never            0       no error
  sync-to-k8s-ciliumendpoint (2739)                                                        10s ago          never            0       no error
  sync-to-k8s-ciliumendpoint (303)                                                         10s ago          never            0       no error
  sync-to-k8s-ciliumendpoint (3032)                                                        10s ago          never            0       no error
  sync-to-k8s-ciliumendpoint (3277)                                                        12s ago          never            0       no error
  sync-to-k8s-ciliumendpoint (3296)                                                        10s ago          never            0       no error
  sync-to-k8s-ciliumendpoint (344)                                                         10s ago          never            0       no error
  sync-to-k8s-ciliumendpoint (476)                                                         10s ago          never            0       no error
  sync-to-k8s-ciliumendpoint (487)                                                         10s ago          never            0       no error
  sync-to-k8s-ciliumendpoint (491)                                                         10s ago          never            0       no error
  sync-to-k8s-ciliumendpoint (792)                                                         10s ago          never            0       no error
  sync-to-k8s-ciliumendpoint (92)                                                          10s ago          never            0       no error
  sync-to-k8s-ciliumendpoint (935)                                                         10s ago          never            0       no error
  sync-utime                                                                               52s ago          never            0       no error
  template-dir-watcher                                                                     never            never            0       no error
  waiting-initial-global-identities-ep (1053)                                              274h21m12s ago   never            0       no error
  waiting-initial-global-identities-ep (11)                                                274h21m12s ago   never            0       no error
  waiting-initial-global-identities-ep (1335)                                              274h21m12s ago   never            0       no error
  waiting-initial-global-identities-ep (1777)                                              274h21m12s ago   never            0       no error
  waiting-initial-global-identities-ep (1789)                                              274h21m12s ago   never            0       no error
  waiting-initial-global-identities-ep (1912)                                              274h21m12s ago   never            0       no error
  waiting-initial-global-identities-ep (2085)                                              274h21m12s ago   never            0       no error
  waiting-initial-global-identities-ep (2739)                                              274h21m12s ago   never            0       no error
  waiting-initial-global-identities-ep (303)                                               274h21m12s ago   never            0       no error
  waiting-initial-global-identities-ep (3032)                                              274h21m12s ago   never            0       no error
  waiting-initial-global-identities-ep (3296)                                              274h21m12s ago   never            0       no error
  waiting-initial-global-identities-ep (344)                                               274h21m12s ago   never            0       no error
  waiting-initial-global-identities-ep (476)                                               274h21m12s ago   never            0       no error
  waiting-initial-global-identities-ep (487)                                               274h21m12s ago   never            0       no error
  waiting-initial-global-identities-ep (491)                                               274h21m12s ago   never            0       no error
  waiting-initial-global-identities-ep (792)                                               274h21m12s ago   never            0       no error
  waiting-initial-global-identities-ep (935)                                               274h21m12s ago   never            0       no error
  write-cni-file                                                                           274h21m28s ago   never            0       no error
Proxy Status:            OK, ip <IP_ADDRESS>, 0 redirects active on ports 10000-20000, Envoy: embedded
Global Identity Range:   min 256, max 65535
Hubble:                  Ok   Current/Max Flows: 4095/4095 (100.00%), Flows/s: 79.36   Metrics: Disabled
KubeProxyReplacement Details:
  Status:                 True
  Socket LB:              Enabled
  Socket LB Tracing:      Enabled
  Socket LB Coverage:     Full
  Devices:                enp96s0f0   <IP_ADDRESS> fe80::6e92:bfff:fed6:8e80 (Direct Routing), tunl0  <IP_ADDRESS>
  Mode:                   SNAT
  Backend Selection:      Random
  Session Affinity:       Enabled
  Graceful Termination:   Enabled
  NAT46/64 Support:       Disabled
  XDP Acceleration:       Disabled
  Services:
  - ClusterIP:      Enabled
  - NodePort:       Enabled (Range: 30000-32767)
  - LoadBalancer:   Enabled
  - externalIPs:    Enabled
  - HostPort:       Enabled
BPF Maps:   dynamic sizing: on (ratio: 0.002500)
  Name                          Size
  Auth                          524288
  Non-TCP connection tracking   2371333
  TCP connection tracking       4742666
  Endpoint policy               65535
  IP cache                      512000
  IPv4 masquerading agent       16384
  IPv6 masquerading agent       16384
  IPv4 fragmentation            8192
  IPv4 service                  65536
  IPv6 service                  65536
  IPv4 service backend          65536
  IPv6 service backend          65536
  IPv4 service reverse NAT      65536
  IPv6 service reverse NAT      65536
  Metrics                       1024
  NAT                           4742666
  Neighbor table                4742666
  Global policy                 16384
  Session affinity              65536
  Sock reverse NAT              2371333
  Tunnel                        65536
Encryption:                       Disabled
Cluster health:                   9/9 reachable   (2024-05-05T14:21:45Z)
  Name                            IP              Node        Endpoints
  local/<IP_ADDRESS> (localhost)   <IP_ADDRESS>     reachable   reachable
  local/<IP_ADDRESS>              <IP_ADDRESS>    reachable   reachable
  local/<IP_ADDRESS>              <IP_ADDRESS>    reachable   reachable
  local/<IP_ADDRESS>              <IP_ADDRESS>    reachable   reachable
  local/<IP_ADDRESS>              <IP_ADDRESS>    reachable   reachable
  local/<IP_ADDRESS>              <IP_ADDRESS>    reachable   reachable
  local/<IP_ADDRESS>              <IP_ADDRESS>    reachable   reachable
  local/<IP_ADDRESS>              <IP_ADDRESS>    reachable   reachable
  local/<IP_ADDRESS>               <IP_ADDRESS>     reachable   reachable
Modules Health:
agent
├── controlplane
│   ├── envoy-proxy
│   │   └── timer-job-version-check                         [OK] OK (51.321218ms) (11d, x1)
│   ├── node-manager
│   │   ├── background-sync                                 [OK] Node validation successful (11d, x7154)
│   │   ├── nodes-add                                       [OK] Node adds successful (11d, x9)
│   │   └── nodes-update                                    [OK] Node updates successful (11d, x16)
│   ├── l2-announcer
│   │   └── leader-election                                 [OK]  (11d, x1)
│   ├── stale-endpoint-cleanup                              [OK]  (11d, x1)
│   ├── daemon
│   │   └── ep-bpf-prog-watchdog                            [OK] ep-bpf-prog-watchdog (11d, x32896)
│   ├── endpoint-manager
│   │   ├── cilium-endpoint-1777 (longhorn-system/csi-attacher-689fdb475f-vjqhc)
│   │   │   ├── cep-k8s-sync                                [OK] sync-to-k8s-ciliumendpoint (1777) (11d, x98761)
│   │   │   ├── datapath-regenerate                         [OK] Endpoint regeneration successful (11d, x120)
│   │   │   └── policymap-sync                              [OK] sync-policymap-1777 (11d, x1098)
│   │   ├── cilium-endpoint-495
│   │   │   ├── datapath-regenerate                         [OK] Endpoint regeneration successful (11d, x120)
│   │   │   └── policymap-sync                              [OK] sync-policymap-495 (11d, x1098)
│   │   ├── cilium-endpoint-303 (longhorn-system/instance-manager-r-eb075fabca1262a6ebdfa0c5f7ac8ee0)
│   │   │   ├── cep-k8s-sync                                [OK] sync-to-k8s-ciliumendpoint (303) (11d, x98761)
│   │   │   ├── datapath-regenerate                         [OK] Endpoint regeneration successful (11d, x120)
│   │   │   └── policymap-sync                              [OK] sync-policymap-303 (11d, x1098)
│   │   ├── cilium-endpoint-487 (longhorn-system/csi-resizer-847d645569-mlc6h)
│   │   │   ├── cep-k8s-sync                                [OK] sync-to-k8s-ciliumendpoint (487) (11d, x98761)
│   │   │   ├── datapath-regenerate                         [OK] Endpoint regeneration successful (11d, x120)
│   │   │   └── policymap-sync                              [OK] sync-policymap-487 (11d, x1098)
│   │   ├── cilium-endpoint-2739 (kube-system/seaweedfs-csi-driver-node-6lg5f)
│   │   │   ├── cep-k8s-sync                                [OK] sync-to-k8s-ciliumendpoint (2739) (11d, x98761)
│   │   │   ├── datapath-regenerate                         [OK] Endpoint regeneration successful (11d, x120)
│   │   │   └── policymap-sync                              [OK] sync-policymap-2739 (11d, x1098)
│   │   ├── endpoint-gc                                     [OK] endpoint-gc (11d, x3293)
│   │   ├── cilium-endpoint-1912 (longhorn-system/longhorn-manager-x8hd4)
│   │   │   ├── cep-k8s-sync                                [OK] sync-to-k8s-ciliumendpoint (1912) (11d, x98761)
│   │   │   ├── datapath-regenerate                         [OK] Endpoint regeneration successful (11d, x120)
│   │   │   └── policymap-sync                              [OK] sync-policymap-1912 (11d, x1098)
│   │   ├── cilium-endpoint-344 (longhorn-system/csi-provisioner-65dd48db7b-xpb25)
│   │   │   ├── cep-k8s-sync                                [OK] sync-to-k8s-ciliumendpoint (344) (11d, x98761)
│   │   │   ├── datapath-regenerate                         [OK] Endpoint regeneration successful (11d, x120)
│   │   │   └── policymap-sync                              [OK] sync-policymap-344 (11d, x1098)
│   │   ├── cilium-endpoint-3296 (athena/deployment-zhongshihua-7574bb5656-zh2td)
│   │   │   ├── cep-k8s-sync                                [OK] sync-to-k8s-ciliumendpoint (3296) (11d, x98761)
│   │   │   ├── datapath-regenerate                         [OK] Endpoint regeneration successful (11d, x120)
│   │   │   └── policymap-sync                              [OK] sync-policymap-3296 (11d, x1098)
│   │   ├── cilium-endpoint-476 (arachnid/llm-sphinx-d4660d9fbbe2453eb8ce81cc5a0c682f-7cdb7c6f85-qvnfg)
│   │   │   ├── cep-k8s-sync                                [OK] sync-to-k8s-ciliumendpoint (476) (11d, x98761)
│   │   │   ├── datapath-regenerate                         [OK] Endpoint regeneration successful (11d, x120)
│   │   │   └── policymap-sync                              [OK] sync-policymap-476 (11d, x1098)
│   │   ├── cilium-endpoint-2085 (longhorn-system/instance-manager-e-eb075fabca1262a6ebdfa0c5f7ac8ee0)
│   │   │   ├── cep-k8s-sync                                [OK] sync-to-k8s-ciliumendpoint (2085) (11d, x98761)
│   │   │   ├── datapath-regenerate                         [OK] Endpoint regeneration successful (11d, x120)
│   │   │   └── policymap-sync                              [OK] sync-policymap-2085 (11d, x1098)
│   │   ├── cilium-endpoint-1335 (longhorn-system/csi-snapshotter-59ff78b47b-88t78)
│   │   │   ├── cep-k8s-sync                                [OK] sync-to-k8s-ciliumendpoint (1335) (11d, x98761)
│   │   │   ├── datapath-regenerate                         [OK] Endpoint regeneration successful (11d, x120)
│   │   │   └── policymap-sync                              [OK] sync-policymap-1335 (11d, x1098)
│   │   ├── cilium-endpoint-11 (monitoring/alertmanager-main-0)
│   │   │   ├── cep-k8s-sync                                [OK] sync-to-k8s-ciliumendpoint (11) (11d, x98761)
│   │   │   ├── datapath-regenerate                         [OK] Endpoint regeneration successful (11d, x120)
│   │   │   └── policymap-sync                              [OK] sync-policymap-11 (11d, x1098)
│   │   ├── cilium-endpoint-935 (longhorn-system/longhorn-admission-webhook-66676d6947-7p9rp)
│   │   │   ├── cep-k8s-sync                                [OK] sync-to-k8s-ciliumendpoint (935) (11d, x98761)
│   │   │   ├── datapath-regenerate                         [OK] Endpoint regeneration successful (11d, x120)
│   │   │   └── policymap-sync                              [OK] sync-policymap-935 (11d, x1098)
│   │   ├── cilium-endpoint-3032 (arachnid/llm-serve-222ba757-0dd7-4acc-9c37-cc271a094b52-8978fcbf5-4x78b)
│   │   │   ├── cep-k8s-sync                                [OK] sync-to-k8s-ciliumendpoint (3032) (11d, x98761)
│   │   │   ├── datapath-regenerate                         [OK] Endpoint regeneration successful (11d, x120)
│   │   │   └── policymap-sync                              [OK] sync-policymap-3032 (11d, x1098)
│   │   ├── cilium-endpoint-1789 (default/dcgm-exporter-6kb8l)
│   │   │   ├── cep-k8s-sync                                [OK] sync-to-k8s-ciliumendpoint (1789) (11d, x98761)
│   │   │   ├── datapath-regenerate                         [OK] Endpoint regeneration successful (11d, x120)
│   │   │   └── policymap-sync                              [OK] sync-policymap-1789 (11d, x1098)
│   │   ├── cilium-endpoint-792 (ingress-nginx/nginx-ingress-controller-blbsr)
│   │   │   ├── datapath-regenerate                         [OK] Endpoint regeneration successful (11d, x120)
│   │   │   ├── cep-k8s-sync                                [OK] sync-to-k8s-ciliumendpoint (792) (11d, x98761)
│   │   │   └── policymap-sync                              [OK] sync-policymap-792 (11d, x1098)
│   │   ├── cilium-endpoint-46
│   │   │   └── datapath-regenerate                         [OK] Endpoint regeneration successful (11d, x120)
│   │   ├── cilium-endpoint-1292
│   │   │   ├── datapath-regenerate                         [OK] Endpoint regeneration successful (11d, x120)
│   │   │   └── policymap-sync                              [OK] sync-policymap-1292 (11d, x1098)
│   │   ├── cilium-endpoint-1053 (arachnid/llm-sphinx-8154e41fae8d42afa1b1f644887e6351-7c9b97bdfc-79lhz)
│   │   │   ├── datapath-regenerate                         [OK] Endpoint regeneration successful (11d, x120)
│   │   │   ├── cep-k8s-sync                                [OK] sync-to-k8s-ciliumendpoint (1053) (11d, x98761)
│   │   │   └── policymap-sync                              [OK] sync-policymap-1053 (11d, x1098)
│   │   ├── cilium-endpoint-491 (monitoring/promtail-zxf5s)
│   │   │   ├── cep-k8s-sync                                [OK] sync-to-k8s-ciliumendpoint (491) (11d, x98761)
│   │   │   ├── datapath-regenerate                         [OK] Endpoint regeneration successful (11d, x120)
│   │   │   └── policymap-sync                              [OK] sync-policymap-491 (11d, x1098)
│   │   ├── cilium-endpoint-92 (longhorn-system/longhorn-csi-plugin-db45q)
│   │   │   ├── cep-k8s-sync                                [OK] sync-to-k8s-ciliumendpoint (92) (10d, x90447)
│   │   │   ├── datapath-regenerate                         [OK] Endpoint regeneration successful (10d, x96)
│   │   │   └── policymap-sync                              [OK] sync-policymap-92 (10d, x1006)
│   │   ├── cilium-endpoint-1728 (coroot/coroot-node-agent-tdwxk)
│   │   │   ├── cep-k8s-sync                                [OK] sync-to-k8s-ciliumendpoint (1728) (7d7h, x63115)
│   │   │   ├── datapath-regenerate                         [OK] Endpoint regeneration successful (7d7h, x63)
│   │   │   └── policymap-sync                              [OK] sync-policymap-1728 (7d7h, x702)
│   │   └── cilium-endpoint-3277 (default/goldpinger-jvmk8)
│   │       ├── datapath-regenerate                         [OK] Endpoint regeneration successful (7d4h, x50)
│   │       ├── policymap-sync                              [OK] sync-policymap-3277 (7d4h, x691)
│   │       └── cep-k8s-sync                                [OK] sync-to-k8s-ciliumendpoint (3277) (7d4h, x62147)
│   ├── auth
│   │   ├── observer-job-auth request-authentication        [OK] Primed (11d, x1)
│   │   ├── observer-job-auth gc-identity-events            [OK] OK (15.323µs) [277] (11d, x1)
│   │   └── timer-job-auth gc-cleanup                       [OK] OK (26.905µs) (11d, x1)
│   └── bgp-cp
│       └── job-diffstore-events                            [OK] Running (11d, x2)
└── datapath
    ├── node-address
    │   └── job-node-address-update                         [OK] Running (11d, x1)
    ├── l2-responder
    │   └── job-l2-responder-reconciler                     [OK] Running (11d, x1)
    └── agent-liveness-updater
        └── timer-job-agent-liveness-updater                [OK] OK (35.486µs) (11d, x1)

Cilium Version
(base) ➜ ~  cilium version
cilium-cli: v0.16.4 compiled with go1.22.1 on linux/amd64
cilium image (default): v1.15.3
cilium image (stable): v1.15.4
cilium image (running): 1.15.4

Kernel Version
Linux sjhltj-plat-aigc-dev-002-RL2206A0082 5.15.0-92-generic #102~20.04.1-Ubuntu SMP Mon Jan 15 13:09:14 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Kubernetes Version
(base)  ➜ ~  kubectl version
Client Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.2", GitCommit:"8b5a19147530eaac9476b0ab82980b4088bbc1b2", GitTreeState:"clean", BuildDate:"2021-09-15T21:38:50Z", GoVersion:"go1.16.8", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"27", GitVersion:"v1.27.11", GitCommit:"b9e2ad67ad146db566be5a6db140d47e52c8adb2", GitTreeState:"clean", BuildDate:"2024-02-14T10:31:56Z", GoVersion:"go1.21.7", Compiler:"gc", Platform:"linux/amd64"}
WARNING: version difference between client (1.22) and server (1.27) exceeds the supported minor version skew of +/-1

Regression
No response
Sysdump
No response
Relevant log output
No response
Anything else?
No response
Cilium Users Document

[ ] Are you a user of Cilium? Please add yourself to the Users doc

Code of Conduct

[x] I agree to follow this project's Code of Conduct

Thanks for this issue @baildagq, you've added quite a bit of info, which is excellent.
That said, I agree that from what you've provided, this looks like it should work. A couple of troubleshooting questions for you though:

Can you check the Envoy access logs and see if the requests from the external source are being logged there?
Can you attach a sysdump using the process at https://docs.cilium.io/en/stable/operations/troubleshooting/#automatic-log-state-collection, preferably just after you've tried doing connections from both internal and external nodes? That will let me check the logs myself.

(I'm on triage this week, and normally wouldn't be troubleshooting on this issue, but this is assigned to my team anyway).

Thank you very much! @youngnick
In order to see envoy access logs, I upgrade the cilium deploy with dedicated envoy daemonset and open debug level log. (config and status is updated in previous issue description)
sysmdump file

https://gofile.io/d/A2B5Sx

Envoy access logs
curl load balancer ip on k8s node, normal echo ( from <IP_ADDRESS>)
(base)  ➜ ~  kubectl -n kube-system logs ds/cilium-envoy --tail 1 -f
[2024-05-06 08:44:12.100][7][debug][main] [external/envoy/source/server/server.cc:263] flushing stats
[2024-05-06 08:44:13.703][101][debug][misc] [cilium/bpf_metadata.cc:224] EGRESS POD IP: <IP_ADDRESS>, destination IP: <IP_ADDRESS>
[2024-05-06 08:44:13.703][101][debug][filter] [cilium/conntrack.cc:178] cilium.bpf_metadata: Using conntrack map global
[2024-05-06 08:44:13.704][101][debug][filter] [cilium/conntrack.cc:128] cilium.bpf_metadata: Opened IPv4 conntrack map global
[2024-05-06 08:44:13.704][101][debug][filter] [cilium/conntrack.cc:229] cilium.bpf_metadata: IPv4 conntrack map global lookup failed: No such file or directory
[2024-05-06 08:44:13.704][101][debug][filter] [cilium/ipcache.cc:90] cilium.ipcache: <IP_ADDRESS> has ID 2
[2024-05-06 08:44:13.704][101][debug][filter] [cilium/ipcache.cc:90] cilium.ipcache: <IP_ADDRESS> has ID 2
[2024-05-06 08:44:13.704][101][debug][filter] [cilium/ipcache.cc:90] cilium.ipcache: <IP_ADDRESS> has ID 8
[2024-05-06 08:44:13.704][101][debug][filter] [./cilium/socket_option.h:236] Cilium SocketOption(): source_identity: 8, ingress: false, port: 14438, pod_ip: <IP_ADDRESS>, source_addresses: /<IP_ADDRESS>:0/, mark: 80b00 (magic mark: b00, cluster: 0, ID: 8), proxy_id: 0
[2024-05-06 08:44:13.704][101][debug][filter] [cilium/network_filter.cc:76] cilium.network: onNewConnection
[2024-05-06 08:44:13.704][101][debug][conn_handler] [external/envoy/source/extensions/listener_managers/listener_manager/active_tcp_listener.cc:157] [Tags: "ConnectionId":"412"] new connection from <IP_ADDRESS>:41456
[2024-05-06 08:44:13.705][101][debug][http] [external/envoy/source/common/http/conn_manager_impl.cc:379] [Tags: "ConnectionId":"412"] new stream
[2024-05-06 08:44:13.705][101][debug][http] [external/envoy/source/common/http/conn_manager_impl.cc:1166] [Tags: "ConnectionId":"412","StreamId":"9768031735628631749"] request headers complete (end_stream=true):
':authority', '<IP_ADDRESS>:30988'
':path', '/echo'
':method', 'GET'
'user-agent', 'curl/7.68.0'
'accept', '*/*'

[2024-05-06 08:44:13.705][101][debug][http] [external/envoy/source/common/http/conn_manager_impl.cc:1149] [Tags: "ConnectionId":"412","StreamId":"9768031735628631749"] request end stream
[2024-05-06 08:44:13.705][101][debug][connection] [external/envoy/source/common/network/connection_impl.h:98] [C412] current connecting state: false
[2024-05-06 08:44:13.705][101][debug][router] [external/envoy/source/common/router/router.cc:478] [Tags: "ConnectionId":"412","StreamId":"9768031735628631749"] cluster 'default/cilium-gateway-cilium-gw/default:echo-3:8090' match for URL '/echo'
[2024-05-06 08:44:13.705][101][debug][filter] [cilium/ipcache.cc:90] cilium.ipcache: <IP_ADDRESS> has ID 10783
[2024-05-06 08:44:13.705][101][debug][router] [external/envoy/source/common/router/router.cc:698] [Tags: "ConnectionId":"412","StreamId":"9768031735628631749"] router decoding headers:
':authority', '<IP_ADDRESS>:30988'
':path', '/echo'
':method', 'GET'
':scheme', 'http'
'user-agent', 'curl/7.68.0'
'accept', '*/*'
'x-forwarded-for', '<IP_ADDRESS>'
'x-forwarded-proto', 'http'
'x-envoy-internal', 'true'
'x-request-id', '5d84537f-8d7a-45be-a2b2-e150d1d41311'
'x-envoy-expected-rq-timeout-ms', '15000'

[2024-05-06 08:44:13.705][101][debug][pool] [external/envoy/source/common/http/conn_pool_base.cc:78] queueing stream due to no available connections (ready=0 busy=0 connecting=0)
[2024-05-06 08:44:13.705][101][debug][pool] [external/envoy/source/common/conn_pool/conn_pool_base.cc:291] trying to create new connection
[2024-05-06 08:44:13.705][101][debug][pool] [external/envoy/source/common/conn_pool/conn_pool_base.cc:145] creating a new connection (connecting=0)
[2024-05-06 08:44:13.705][101][debug][connection] [external/envoy/source/common/network/connection_impl.h:98] [C413] current connecting state: true
[2024-05-06 08:44:13.705][101][debug][client] [external/envoy/source/common/http/codec_client.cc:57] [Tags: "ConnectionId":"413"] connecting
[2024-05-06 08:44:13.705][101][debug][connection] [external/envoy/source/common/network/connection_impl.cc:948] [C413] connecting to <IP_ADDRESS>:8080
[2024-05-06 08:44:13.706][101][debug][connection] [external/envoy/source/common/network/connection_impl.cc:967] [C413] connection in progress
[2024-05-06 08:44:13.706][101][debug][connection] [external/envoy/source/common/network/connection_impl.cc:695] [C413] connected
[2024-05-06 08:44:13.706][101][debug][client] [external/envoy/source/common/http/codec_client.cc:88] [Tags: "ConnectionId":"413"] connected
[2024-05-06 08:44:13.706][101][debug][pool] [external/envoy/source/common/conn_pool/conn_pool_base.cc:328] [Tags: "ConnectionId":"413"] attaching to next stream
[2024-05-06 08:44:13.706][101][debug][pool] [external/envoy/source/common/conn_pool/conn_pool_base.cc:182] [Tags: "ConnectionId":"413"] creating stream
[2024-05-06 08:44:13.706][101][debug][router] [external/envoy/source/common/router/upstream_request.cc:580] [Tags: "ConnectionId":"412","StreamId":"9768031735628631749"] pool ready
[2024-05-06 08:44:13.706][101][debug][filter] [cilium/ipcache.cc:90] cilium.ipcache: <IP_ADDRESS> has ID 10783
[2024-05-06 08:44:13.706][101][debug][filter] [cilium/l7policy.cc:144] cilium.l7policy: Ingress from 2 policy lookup for endpoint <IP_ADDRESS> for port 14438: ALLOW
[2024-05-06 08:44:13.706][101][debug][filter] [cilium/l7policy.cc:152] cilium.l7policy: egress (8->10783) policy lookup for endpoint <IP_ADDRESS> for port 8080: ALLOW
[2024-05-06 08:44:13.706][101][debug][client] [external/envoy/source/common/http/codec_client.cc:141] [Tags: "ConnectionId":"413"] encode complete
[2024-05-06 08:44:13.707][101][debug][router] [external/envoy/source/common/router/router.cc:1446] [Tags: "ConnectionId":"412","StreamId":"9768031735628631749"] upstream headers complete: end_stream=false
[2024-05-06 08:44:13.707][101][debug][http] [external/envoy/source/common/http/conn_manager_impl.cc:1820] [Tags: "ConnectionId":"412","StreamId":"9768031735628631749"] encoding headers via codec (end_stream=false):
':status', '200'
'date', 'Mon, 06 May 2024 08:44:13 GMT'
'content-type', 'text/plain'
'server', 'envoy'
'x-envoy-upstream-service-time', '0'

[2024-05-06 08:44:13.707][101][debug][client] [external/envoy/source/common/http/codec_client.cc:128] [Tags: "ConnectionId":"413"] response complete
[2024-05-06 08:44:13.707][101][debug][http] [external/envoy/source/common/http/conn_manager_impl.cc:1925] [Tags: "ConnectionId":"412","StreamId":"9768031735628631749"] Codec completed encoding stream.
[2024-05-06 08:44:13.707][101][debug][pool] [external/envoy/source/common/http/http1/conn_pool.cc:53] [Tags: "ConnectionId":"413"] response complete
[2024-05-06 08:44:13.707][101][debug][pool] [external/envoy/source/common/conn_pool/conn_pool_base.cc:215] [Tags: "ConnectionId":"413"] destroying stream: 0 remaining
[2024-05-06 08:44:13.708][101][debug][connection] [external/envoy/source/common/network/connection_impl.cc:663] [Tags: "ConnectionId":"412"] remote close
[2024-05-06 08:44:13.708][101][debug][connection] [external/envoy/source/common/network/connection_impl.cc:250] [Tags: "ConnectionId":"412"] closing socket: 0

curl load balancer ip outside k8s cluster,  timeout ( from <IP_ADDRESS>)
I think there is nothing access logs, following some log I think not belong to '/echo' request
(base) @sjhltj-plat-aigc-dev-002-RL2206A0082 ➜ ~  kubectl -n kube-system logs ds/cilium-envoy --tail 10 -f
Found 9 pods, using pod/cilium-envoy-9jgbd
[2024-05-06 08:46:35.065][117][debug][connection] [external/envoy/source/common/network/connection_impl.cc:727] [Tags: "ConnectionId":"426"] write flush complete
[2024-05-06 08:46:35.065][68][debug][connection] [external/envoy/source/common/network/connection_impl.cc:727] [Tags: "ConnectionId":"425"] write flush complete
[2024-05-06 08:46:35.065][117][debug][connection] [external/envoy/source/common/network/connection_impl.cc:250] [Tags: "ConnectionId":"426"] closing socket: 1
[2024-05-06 08:46:35.065][68][debug][connection] [external/envoy/source/common/network/connection_impl.cc:250] [Tags: "ConnectionId":"425"] closing socket: 1
[2024-05-06 08:46:35.065][68][debug][conn_handler] [external/envoy/source/extensions/listener_managers/listener_manager/active_stream_listener_base.cc:121] [Tags: "ConnectionId":"425"] adding to cleanup list
[2024-05-06 08:46:35.065][117][debug][conn_handler] [external/envoy/source/extensions/listener_managers/listener_manager/active_stream_listener_base.cc:121] [Tags: "ConnectionId":"426"] adding to cleanup list
[2024-05-06 08:46:37.138][7][debug][main] [external/envoy/source/server/server.cc:263] flushing stats
[2024-05-06 08:46:42.139][7][debug][main] [external/envoy/source/server/server.cc:263] flushing stats
[2024-05-06 08:46:47.140][7][debug][main] [external/envoy/source/server/server.cc:263] flushing stats
[2024-05-06 08:46:52.142][7][debug][main] [external/envoy/source/server/server.cc:263] flushing stats

[2024-05-06 08:46:57.140][7][debug][main] [external/envoy/source/server/server.cc:263] flushing stats
[2024-05-06 08:47:02.142][7][debug][main] [external/envoy/source/server/server.cc:263] flushing stats
[2024-05-06 08:47:05.064][103][debug][conn_handler] [external/envoy/source/extensions/listener_managers/listener_manager/active_tcp_listener.cc:157] [Tags: "ConnectionId":"427"] new connection from <IP_ADDRESS>:60070
[2024-05-06 08:47:05.064][103][debug][http] [external/envoy/source/common/http/conn_manager_impl.cc:379] [Tags: "ConnectionId":"427"] new stream
[2024-05-06 08:47:05.064][103][debug][http] [external/envoy/source/common/http/conn_manager_impl.cc:1166] [Tags: "ConnectionId":"427","StreamId":"4984062691940870373"] request headers complete (end_stream=true):
':authority', '<IP_ADDRESS>:9878'
':path', '/healthz'
':method', 'GET'
'user-agent', 'kube-probe/1.27'
'accept', '*/*'
'connection', 'close'

[2024-05-06 08:47:05.064][78][debug][conn_handler] [external/envoy/source/extensions/listener_managers/listener_manager/active_tcp_listener.cc:157] [Tags: "ConnectionId":"428"] new connection from <IP_ADDRESS>:60072
[2024-05-06 08:47:05.064][103][debug][http] [external/envoy/source/common/http/conn_manager_impl.cc:1149] [Tags: "ConnectionId":"427","StreamId":"4984062691940870373"] request end stream
[2024-05-06 08:47:05.064][103][debug][connection] [external/envoy/source/common/network/connection_impl.h:98] [C427] current connecting state: false
[2024-05-06 08:47:05.064][78][debug][http] [external/envoy/source/common/http/conn_manager_impl.cc:379] [Tags: "ConnectionId":"428"] new stream
[2024-05-06 08:47:05.064][103][debug][router] [external/envoy/source/common/router/router.cc:478] [Tags: "ConnectionId":"427","StreamId":"4984062691940870373"] cluster '/envoy-admin' match for URL '/healthz'
[2024-05-06 08:47:05.064][78][debug][http] [external/envoy/source/common/http/conn_manager_impl.cc:1166] [Tags: "ConnectionId":"428","StreamId":"7629558170557722876"] request headers complete (end_stream=true):
':authority', '<IP_ADDRESS>:9878'
':path', '/healthz'
':method', 'GET'
'user-agent', 'kube-probe/1.27'
'accept', '*/*'
'connection', 'close'

[2024-05-06 08:47:05.064][103][debug][router] [external/envoy/source/common/router/router.cc:698] [Tags: "ConnectionId":"427","StreamId":"4984062691940870373"] router decoding headers:
':authority', '<IP_ADDRESS>:9878'
':path', '/ready'
':method', 'GET'
':scheme', 'http'
'user-agent', 'kube-probe/1.27'
'accept', '*/*'
'x-forwarded-proto', 'http'
'x-request-id', 'fc9f66d1-de4d-4929-87bd-30a6a1733b63'
'x-envoy-expected-rq-timeout-ms', '15000'
'x-envoy-original-path', '/healthz'

[2024-05-06 08:47:05.064][78][debug][http] [external/envoy/source/common/http/conn_manager_impl.cc:1149] [Tags: "ConnectionId":"428","StreamId":"7629558170557722876"] request end stream
[2024-05-06 08:47:05.064][78][debug][connection] [external/envoy/source/common/network/connection_impl.h:98] [C428] current connecting state: false
[2024-05-06 08:47:05.064][103][debug][pool] [external/envoy/source/common/conn_pool/conn_pool_base.cc:265] [Tags: "ConnectionId":"338"] using existing fully connected connection
[2024-05-06 08:47:05.064][103][debug][pool] [external/envoy/source/common/conn_pool/conn_pool_base.cc:182] [Tags: "ConnectionId":"338"] creating stream
[2024-05-06 08:47:05.064][78][debug][router] [external/envoy/source/common/router/router.cc:478] [Tags: "ConnectionId":"428","StreamId":"7629558170557722876"] cluster '/envoy-admin' match for URL '/healthz'
[2024-05-06 08:47:05.064][103][debug][router] [external/envoy/source/common/router/upstream_request.cc:580] [Tags: "ConnectionId":"427","StreamId":"4984062691940870373"] pool ready
[2024-05-06 08:47:05.064][78][debug][router] [external/envoy/source/common/router/router.cc:698] [Tags: "ConnectionId":"428","StreamId":"7629558170557722876"] router decoding headers:
':authority', '<IP_ADDRESS>:9878'
':path', '/ready'
':method', 'GET'
':scheme', 'http'
'user-agent', 'kube-probe/1.27'
'accept', '*/*'
'x-forwarded-proto', 'http'
'x-request-id', '647ce180-958c-4488-a632-89f54dec1158'
'x-envoy-expected-rq-timeout-ms', '15000'
'x-envoy-original-path', '/healthz'

[2024-05-06 08:47:05.064][103][debug][client] [external/envoy/source/common/http/codec_client.cc:141] [Tags: "ConnectionId":"338"] encode complete
[2024-05-06 08:47:05.064][78][debug][pool] [external/envoy/source/common/conn_pool/conn_pool_base.cc:265] [Tags: "ConnectionId":"177"] using existing fully connected connection
[2024-05-06 08:47:05.064][78][debug][pool] [external/envoy/source/common/conn_pool/conn_pool_base.cc:182] [Tags: "ConnectionId":"177"] creating stream
[2024-05-06 08:47:05.064][78][debug][router] [external/envoy/source/common/router/upstream_request.cc:580] [Tags: "ConnectionId":"428","StreamId":"7629558170557722876"] pool ready
[2024-05-06 08:47:05.064][78][debug][client] [external/envoy/source/common/http/codec_client.cc:141] [Tags: "ConnectionId":"177"] encode complete
[2024-05-06 08:47:05.064][7][debug][http] [external/envoy/source/common/http/conn_manager_impl.cc:379] [Tags: "ConnectionId":"339"] new stream
[2024-05-06 08:47:05.065][7][debug][http] [external/envoy/source/common/http/conn_manager_impl.cc:1166] [Tags: "ConnectionId":"339","StreamId":"17497885548340367174"] request headers complete (end_stream=true):
':authority', '<IP_ADDRESS>:9878'
':path', '/ready'
':method', 'GET'
'user-agent', 'kube-probe/1.27'
'accept', '*/*'
'x-forwarded-proto', 'http'
'x-request-id', 'fc9f66d1-de4d-4929-87bd-30a6a1733b63'
'x-envoy-expected-rq-timeout-ms', '15000'
'x-envoy-original-path', '/healthz'

[2024-05-06 08:47:05.065][7][debug][http] [external/envoy/source/common/http/conn_manager_impl.cc:1149] [Tags: "ConnectionId":"339","StreamId":"17497885548340367174"] request end stream
[2024-05-06 08:47:05.065][7][debug][connection] [external/envoy/source/common/network/connection_impl.h:98] [C339] current connecting state: false
[2024-05-06 08:47:05.065][7][debug][admin] [external/envoy/source/server/admin/admin_filter.cc:85] [Tags: "ConnectionId":"339","StreamId":"17497885548340367174"] request complete: path: /ready
[2024-05-06 08:47:05.065][7][debug][http] [external/envoy/source/common/http/conn_manager_impl.cc:1820] [Tags: "ConnectionId":"339","StreamId":"17497885548340367174"] encoding headers via codec (end_stream=false):
':status', '200'
'content-type', 'text/plain; charset=UTF-8'
'cache-control', 'no-cache, max-age=0'
'x-content-type-options', 'nosniff'
'date', 'Mon, 06 May 2024 08:47:05 GMT'
'server', 'envoy'

[2024-05-06 08:47:05.065][7][debug][misc] [external/envoy/source/server/admin/admin_filter.cc:101] nextChunk: response.length=5 more_data=false end_stream=true
[2024-05-06 08:47:05.065][7][debug][http] [external/envoy/source/common/http/conn_manager_impl.cc:1925] [Tags: "ConnectionId":"339","StreamId":"17497885548340367174"] Codec completed encoding stream.
[2024-05-06 08:47:05.065][7][debug][http] [external/envoy/source/common/http/conn_manager_impl.cc:379] [Tags: "ConnectionId":"178"] new stream
[2024-05-06 08:47:05.066][7][debug][http] [external/envoy/source/common/http/conn_manager_impl.cc:1166] [Tags: "ConnectionId":"178","StreamId":"1671841335709348664"] request headers complete (end_stream=true):
':authority', '<IP_ADDRESS>:9878'
':path', '/ready'
':method', 'GET'
'user-agent', 'kube-probe/1.27'
'accept', '*/*'
'x-forwarded-proto', 'http'
'x-request-id', '647ce180-958c-4488-a632-89f54dec1158'
'x-envoy-expected-rq-timeout-ms', '15000'
'x-envoy-original-path', '/healthz'

[2024-05-06 08:47:05.066][7][debug][http] [external/envoy/source/common/http/conn_manager_impl.cc:1149] [Tags: "ConnectionId":"178","StreamId":"1671841335709348664"] request end stream
[2024-05-06 08:47:05.066][7][debug][connection] [external/envoy/source/common/network/connection_impl.h:98] [C178] current connecting state: false
[2024-05-06 08:47:05.066][7][debug][admin] [external/envoy/source/server/admin/admin_filter.cc:85] [Tags: "ConnectionId":"178","StreamId":"1671841335709348664"] request complete: path: /ready
[2024-05-06 08:47:05.066][7][debug][http] [external/envoy/source/common/http/conn_manager_impl.cc:1820] [Tags: "ConnectionId":"178","StreamId":"1671841335709348664"] encoding headers via codec (end_stream=false):
':status', '200'
'content-type', 'text/plain; charset=UTF-8'
'cache-control', 'no-cache, max-age=0'
'x-content-type-options', 'nosniff'
'date', 'Mon, 06 May 2024 08:47:05 GMT'
'server', 'envoy'

[2024-05-06 08:47:05.066][7][debug][misc] [external/envoy/source/server/admin/admin_filter.cc:101] nextChunk: response.length=5 more_data=false end_stream=true
[2024-05-06 08:47:05.066][7][debug][http] [external/envoy/source/common/http/conn_manager_impl.cc:1925] [Tags: "ConnectionId":"178","StreamId":"1671841335709348664"] Codec completed encoding stream.
[2024-05-06 08:47:05.066][103][debug][router] [external/envoy/source/common/router/router.cc:1446] [Tags: "ConnectionId":"427","StreamId":"4984062691940870373"] upstream headers complete: end_stream=false
[2024-05-06 08:47:05.066][78][debug][router] [external/envoy/source/common/router/router.cc:1446] [Tags: "ConnectionId":"428","StreamId":"7629558170557722876"] upstream headers complete: end_stream=false
[2024-05-06 08:47:05.066][103][debug][http] [external/envoy/source/common/http/conn_manager_impl.cc:1751] [Tags: "ConnectionId":"427","StreamId":"4984062691940870373"] closing connection due to connection close header
[2024-05-06 08:47:05.066][103][debug][http] [external/envoy/source/common/http/conn_manager_impl.cc:1820] [Tags: "ConnectionId":"427","StreamId":"4984062691940870373"] encoding headers via codec (end_stream=false):
':status', '200'
'content-type', 'text/plain; charset=UTF-8'
'cache-control', 'no-cache, max-age=0'
'x-content-type-options', 'nosniff'
'date', 'Mon, 06 May 2024 08:47:05 GMT'
'server', 'envoy'
'x-envoy-upstream-service-time', '1'
'connection', 'close'

[2024-05-06 08:47:05.066][78][debug][http] [external/envoy/source/common/http/conn_manager_impl.cc:1751] [Tags: "ConnectionId":"428","StreamId":"7629558170557722876"] closing connection due to connection close header
[2024-05-06 08:47:05.066][103][debug][client] [external/envoy/source/common/http/codec_client.cc:128] [Tags: "ConnectionId":"338"] response complete
[2024-05-06 08:47:05.066][78][debug][http] [external/envoy/source/common/http/conn_manager_impl.cc:1820] [Tags: "ConnectionId":"428","StreamId":"7629558170557722876"] encoding headers via codec (end_stream=false):
':status', '200'
'content-type', 'text/plain; charset=UTF-8'
'cache-control', 'no-cache, max-age=0'
'x-content-type-options', 'nosniff'
'date', 'Mon, 06 May 2024 08:47:05 GMT'
'server', 'envoy'
'x-envoy-upstream-service-time', '1'
'connection', 'close'

[2024-05-06 08:47:05.066][103][debug][http] [external/envoy/source/common/http/conn_manager_impl.cc:1925] [Tags: "ConnectionId":"427","StreamId":"4984062691940870373"] Codec completed encoding stream.
[2024-05-06 08:47:05.066][78][debug][client] [external/envoy/source/common/http/codec_client.cc:128] [Tags: "ConnectionId":"177"] response complete
[2024-05-06 08:47:05.066][103][debug][connection] [external/envoy/source/common/network/connection_impl.cc:139] [C427] closing data_to_write=277 type=0
[2024-05-06 08:47:05.066][103][debug][connection] [external/envoy/source/common/network/connection_impl_base.cc:47] [Tags: "ConnectionId":"427"] setting delayed close timer with timeout 1000 ms
[2024-05-06 08:47:05.066][78][debug][http] [external/envoy/source/common/http/conn_manager_impl.cc:1925] [Tags: "ConnectionId":"428","StreamId":"7629558170557722876"] Codec completed encoding stream.
[2024-05-06 08:47:05.066][103][debug][pool] [external/envoy/source/common/http/http1/conn_pool.cc:53] [Tags: "ConnectionId":"338"] response complete
[2024-05-06 08:47:05.066][103][debug][pool] [external/envoy/source/common/conn_pool/conn_pool_base.cc:215] [Tags: "ConnectionId":"338"] destroying stream: 0 remaining
[2024-05-06 08:47:05.066][78][debug][connection] [external/envoy/source/common/network/connection_impl.cc:139] [C428] closing data_to_write=277 type=0
[2024-05-06 08:47:05.066][78][debug][connection] [external/envoy/source/common/network/connection_impl_base.cc:47] [Tags: "ConnectionId":"428"] setting delayed close timer with timeout 1000 ms
[2024-05-06 08:47:05.066][78][debug][pool] [external/envoy/source/common/http/http1/conn_pool.cc:53] [Tags: "ConnectionId":"177"] response complete
[2024-05-06 08:47:05.066][78][debug][pool] [external/envoy/source/common/conn_pool/conn_pool_base.cc:215] [Tags: "ConnectionId":"177"] destroying stream: 0 remaining
[2024-05-06 08:47:05.066][103][debug][connection] [external/envoy/source/common/network/connection_impl.cc:727] [Tags: "ConnectionId":"427"] write flush complete
[2024-05-06 08:47:05.066][103][debug][connection] [external/envoy/source/common/network/connection_impl.cc:250] [Tags: "ConnectionId":"427"] closing socket: 1
[2024-05-06 08:47:05.066][103][debug][conn_handler] [external/envoy/source/extensions/listener_managers/listener_manager/active_stream_listener_base.cc:121] [Tags: "ConnectionId":"427"] adding to cleanup list
[2024-05-06 08:47:05.066][78][debug][connection] [external/envoy/source/common/network/connection_impl.cc:727] [Tags: "ConnectionId":"428"] write flush complete
[2024-05-06 08:47:05.066][78][debug][connection] [external/envoy/source/common/network/connection_impl.cc:250] [Tags: "ConnectionId":"428"] closing socket: 1
[2024-05-06 08:47:05.066][78][debug][conn_handler] [external/envoy/source/extensions/listener_managers/listener_manager/active_stream_listener_base.cc:121] [Tags: "ConnectionId":"428"] adding to cleanup list

Besides,  curl the loadbalancer ip + port from intertal node succeed and from external node failed (timeout),  curl the cluster's node IP + nodePort has the same perform, from internal node succeed and from external node failed(connection refused)
(base) » kubectl get svc -A | grep -i load                                                                                [16:59:40]
default                         cilium-gateway-cilium-gw                         LoadBalancer   <IP_ADDRESS>   <IP_ADDRESS>                                        30988:30226/TCP                                                                                56m
default                         echo-1-lb                                        LoadBalancer   <IP_ADDRESS>   <IP_ADDRESS>                                        8080:32337/TCP                                                                                50m
kube-system                     cilium-ingress                                   LoadBalancer   <IP_ADDRESS>   <IP_ADDRESS>                                        80:32757/TCP,443:32291/TCP

cilium connectivity test not pass all test, logout:
(base) » cilium connectivity test                                                                                [16:03:00]
ℹ  Monitor aggregation detected, will skip some flow validation steps
ℹ  Skipping tests that require a node Without Cilium
⌛ [local] Waiting for deployment cilium-test/client to become ready...
⌛ [local] Waiting for deployment cilium-test/client2 to become ready...
⌛ [local] Waiting for deployment cilium-test/echo-same-node to become ready...
⌛ [local] Waiting for deployment cilium-test/client3 to become ready...
⌛ [local] Waiting for deployment cilium-test/echo-other-node to become ready...
⌛ [local] Waiting for pod cilium-test/client3-7557dd665c-rjtnh to reach DNS server on cilium-test/echo-same-node-57bb597f97-z5ggv pod...
⌛ [local] Waiting for pod cilium-test/client-56f8968958-zk4l9 to reach DNS server on cilium-test/echo-same-node-57bb597f97-z5ggv pod...
⌛ [local] Waiting for pod cilium-test/client2-5668f9f59b-dsv2c to reach DNS server on cilium-test/echo-same-node-57bb597f97-z5ggv pod...
⌛ [local] Waiting for pod cilium-test/client-56f8968958-zk4l9 to reach DNS server on cilium-test/echo-other-node-bd6cd689f-4snll pod...
⌛ [local] Waiting for pod cilium-test/client2-5668f9f59b-dsv2c to reach DNS server on cilium-test/echo-other-node-bd6cd689f-4snll pod...
⌛ [local] Waiting for pod cilium-test/client3-7557dd665c-rjtnh to reach DNS server on cilium-test/echo-other-node-bd6cd689f-4snll pod...
⌛ [local] Waiting for pod cilium-test/client-56f8968958-zk4l9 to reach default/kubernetes service...
⌛ [local] Waiting for pod cilium-test/client2-5668f9f59b-dsv2c to reach default/kubernetes service...
⌛ [local] Waiting for pod cilium-test/client3-7557dd665c-rjtnh to reach default/kubernetes service...
⌛ [local] Waiting for Service cilium-test/echo-other-node to become ready...
⌛ [local] Waiting for Service cilium-test/echo-other-node to be synchronized by Cilium pod kube-system/cilium-5w6lg
⌛ [local] Waiting for Service cilium-test/echo-other-node to be synchronized by Cilium pod kube-system/cilium-rbqvh
⌛ [local] Waiting for Service cilium-test/echo-same-node to become ready...
⌛ [local] Waiting for Service cilium-test/echo-same-node to be synchronized by Cilium pod kube-system/cilium-rbqvh
⌛ [local] Waiting for Service cilium-test/echo-same-node to be synchronized by Cilium pod kube-system/cilium-5w6lg
⌛ [local] Retrieving service cilium-test/cilium-ingress-ingress-service ...
⌛ [local] Waiting for NodePort <IP_ADDRESS>:31691 (cilium-test/echo-other-node) to become ready...
⌛ [local] Waiting for NodePort <IP_ADDRESS>:32426 (cilium-test/echo-same-node) to become ready...
⌛ [local] Waiting for NodePort <IP_ADDRESS>:31691 (cilium-test/echo-other-node) to become ready...
⌛ [local] Waiting for NodePort <IP_ADDRESS>:32426 (cilium-test/echo-same-node) to become ready...
⌛ [local] Waiting for NodePort <IP_ADDRESS>:31691 (cilium-test/echo-other-node) to become ready...
⌛ [local] Waiting for NodePort <IP_ADDRESS>:32426 (cilium-test/echo-same-node) to become ready...
⌛ [local] Waiting for NodePort <IP_ADDRESS>:31691 (cilium-test/echo-other-node) to become ready...
⌛ [local] Waiting for NodePort <IP_ADDRESS>:32426 (cilium-test/echo-same-node) to become ready...
⌛ [local] Waiting for NodePort <IP_ADDRESS>:31691 (cilium-test/echo-other-node) to become ready...
⌛ [local] Waiting for NodePort <IP_ADDRESS>:32426 (cilium-test/echo-same-node) to become ready...
⌛ [local] Waiting for NodePort <IP_ADDRESS>:31691 (cilium-test/echo-other-node) to become ready...
⌛ [local] Waiting for NodePort <IP_ADDRESS>:32426 (cilium-test/echo-same-node) to become ready...
⌛ [local] Waiting for NodePort <IP_ADDRESS>:32426 (cilium-test/echo-same-node) to become ready...
⌛ [local] Waiting for NodePort <IP_ADDRESS>:31691 (cilium-test/echo-other-node) to become ready...
⌛ [local] Waiting for NodePort <IP_ADDRESS>:31691 (cilium-test/echo-other-node) to become ready...
⌛ [local] Waiting for NodePort <IP_ADDRESS>:32426 (cilium-test/echo-same-node) to become ready...
⌛ [local] Waiting for NodePort <IP_ADDRESS>:31691 (cilium-test/echo-other-node) to become ready...
⌛ [local] Waiting for NodePort <IP_ADDRESS>:32426 (cilium-test/echo-same-node) to become ready...
⌛ [local] Waiting for DaemonSet cilium-test/host-netns-non-cilium to become ready...
⌛ [local] Waiting for DaemonSet cilium-test/host-netns to become ready...
ℹ  Skipping IPCache check
🔭 Enabling Hubble telescope...
⚠  Unable to contact Hubble Relay, disabling Hubble telescope and flow validation: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp <IP_ADDRESS>:4245: connect: connection refused"
ℹ  Expose Relay locally with:
   cilium hubble enable
   cilium hubble port-forward&
ℹ  Cilium version: 1.15.4
🏃 Running 75 tests ...
[=] Test [no-unexpected-packet-drops] [1/75]
.
  [-] Scenario [no-unexpected-packet-drops/no-unexpected-packet-drops]
  [.] Action [no-unexpected-packet-drops/no-unexpected-packet-drops/local/<IP_ADDRESS>]
  ❌ Found unexpected packet drops:
{
  "labels": {
    "direction": "INGRESS",
    "reason": "Invalid packet"
  },
  "name": "cilium_drop_count_total",
  "value": 21
}
  [.] Action [no-unexpected-packet-drops/no-unexpected-packet-drops/local/<IP_ADDRESS>]
  [.] Action [no-unexpected-packet-drops/no-unexpected-packet-drops/local/<IP_ADDRESS>]
  [.] Action [no-unexpected-packet-drops/no-unexpected-packet-drops/local/<IP_ADDRESS>]
  [.] Action [no-unexpected-packet-drops/no-unexpected-packet-drops/local/<IP_ADDRESS>]
  [.] Action [no-unexpected-packet-drops/no-unexpected-packet-drops/local/<IP_ADDRESS>]
  [.] Action [no-unexpected-packet-drops/no-unexpected-packet-drops/local/<IP_ADDRESS>]
  [.] Action [no-unexpected-packet-drops/no-unexpected-packet-drops/local/<IP_ADDRESS>]
  ❌ Found unexpected packet drops:
{
  "labels": {
    "direction": "EGRESS",
    "reason": "Missed tail call"
  },
  "name": "cilium_drop_count_total",
  "value": 2
}
  [.] Action [no-unexpected-packet-drops/no-unexpected-packet-drops/local/<IP_ADDRESS>]
  ❌ Found unexpected packet drops:
{
  "labels": {
    "direction": "INGRESS",
    "reason": "First logical datagram fragment not found"
  },
  "name": "cilium_drop_count_total",
  "value": 5
}
[=] Test [no-policies] [2/75]
..............
  [-] Scenario [no-policies/client-to-client]
  [.] Action [no-policies/client-to-client/ping-ipv4-0: cilium-test/client-56f8968958-zk4l9 (<IP_ADDRESS>) -> cilium-test/client2-5668f9f59b-dsv2c (<IP_ADDRESS>:0)]
  [.] Action [no-policies/client-to-client/ping-ipv4-1: cilium-test/client-56f8968958-zk4l9 (<IP_ADDRESS>) -> cilium-test/client3-7557dd665c-rjtnh (<IP_ADDRESS>:0)]
  [.] Action [no-policies/client-to-client/ping-ipv4-2: cilium-test/client2-5668f9f59b-dsv2c (<IP_ADDRESS>) -> cilium-test/client3-7557dd665c-rjtnh (<IP_ADDRESS>:0)]
  [.] Action [no-policies/client-to-client/ping-ipv4-3: cilium-test/client2-5668f9f59b-dsv2c (<IP_ADDRESS>) -> cilium-test/client-56f8968958-zk4l9 (<IP_ADDRESS>:0)]
  [.] Action [no-policies/client-to-client/ping-ipv4-4: cilium-test/client3-7557dd665c-rjtnh (<IP_ADDRESS>) -> cilium-test/client2-5668f9f59b-dsv2c (<IP_ADDRESS>:0)]
  [.] Action [no-policies/client-to-client/ping-ipv4-5: cilium-test/client3-7557dd665c-rjtnh (<IP_ADDRESS>) -> cilium-test/client-56f8968958-zk4l9 (<IP_ADDRESS>:0)]
  [-] Scenario [no-policies/pod-to-hostport]
  [.] Action [no-policies/pod-to-hostport/curl-0: cilium-test/client2-5668f9f59b-dsv2c (<IP_ADDRESS>) -> cilium-test/echo-other-node-bd6cd689f-4snll (<IP_ADDRESS>:4000)]
  [.] Action [no-policies/pod-to-hostport/curl-1: cilium-test/client2-5668f9f59b-dsv2c (<IP_ADDRESS>) -> cilium-test/echo-same-node-57bb597f97-z5ggv (<IP_ADDRESS>:4000)]
  [.] Action [no-policies/pod-to-hostport/curl-2: cilium-test/client3-7557dd665c-rjtnh (<IP_ADDRESS>) -> cilium-test/echo-other-node-bd6cd689f-4snll (<IP_ADDRESS>:4000)]
  [.] Action [no-policies/pod-to-hostport/curl-3: cilium-test/client3-7557dd665c-rjtnh (<IP_ADDRESS>) -> cilium-test/echo-same-node-57bb597f97-z5ggv (<IP_ADDRESS>:4000)]
  [.] Action [no-policies/pod-to-hostport/curl-4: cilium-test/client-56f8968958-zk4l9 (<IP_ADDRESS>) -> cilium-test/echo-other-node-bd6cd689f-4snll (<IP_ADDRESS>:4000)]
  [.] Action [no-policies/pod-to-hostport/curl-5: cilium-test/client-56f8968958-zk4l9 (<IP_ADDRESS>) -> cilium-test/echo-same-node-57bb597f97-z5ggv (<IP_ADDRESS>:4000)]
  [-] Scenario [no-policies/pod-to-world]
  [.] Action [no-policies/pod-to-world/http-to-one.one.one.one.-0: cilium-test/client-56f8968958-zk4l9 (<IP_ADDRESS>) -> one.one.one.one.-http (one.one.one.one.:80)]
  [.] Action [no-policies/pod-to-world/https-to-one.one.one.one.-0: cilium-test/client-56f8968958-zk4l9 (<IP_ADDRESS>) -> one.one.one.one.-https (one.one.one.one.:443)]
  ❌ command "curl -w %{local_ip}:%{local_port} -> %{remote_ip}:%{remote_port} = %{response_code} --silent --fail --show-error --output /dev/null --connect-timeout 2 --max-time 10 -H Host: one.one.one.one --retry 3 --retry-all-errors --retry-delay 3 https://one.one.one.one.:443" failed: error with exec request (pod=cilium-test/client-56f8968958-zk4l9, container=client): command terminated with exit code 28
  ℹ  curl output:
  :0 -> :0 = 000

  [.] Action [no-policies/pod-to-world/https-to-one.one.one.one.-index-0: cilium-test/client-56f8968958-zk4l9 (<IP_ADDRESS>) -> one.one.one.one.-https-index (one.one.one.one.:443)]
  ❌ command "curl -w %{local_ip}:%{local_port} -> %{remote_ip}:%{remote_port} = %{response_code} --silent --fail --show-error --output /dev/null --connect-timeout 2 --max-time 10 -H Host: one.one.one.one --retry 3 --retry-all-errors --retry-delay 3 https://one.one.one.one.:443/index.html" failed: error with exec request (pod=cilium-test/client-56f8968958-zk4l9, container=client): command terminated with exit code 28
  ℹ  curl output:
  :0 -> :0 = 000

  [.] Action [no-policies/pod-to-world/http-to-one.one.one.one.-1: cilium-test/client2-5668f9f59b-dsv2c (<IP_ADDRESS>) -> one.one.one.one.-http (one.one.one.one.:80)]
  ❌ command "curl -w %{local_ip}:%{local_port} -> %{remote_ip}:%{remote_port} = %{response_code} --silent --fail --show-error --output /dev/null --connect-timeout 2 --max-time 10 -H Host: one.one.one.one --retry 3 --retry-all-errors --retry-delay 3 http://one.one.one.one.:80" failed: command failed (pod=cilium-test/client2-5668f9f59b-dsv2c, container=client2): "curl: (28) Resolving timed out after 2000 milliseconds\n"
  ℹ  curl output:
  <IP_ADDRESS>:44598 -> <IP_ADDRESS>:80 = 301

  [.] Action [no-policies/pod-to-world/https-to-one.one.one.one.-1: cilium-test/client2-5668f9f59b-dsv2c (<IP_ADDRESS>) -> one.one.one.one.-https (one.one.one.one.:443)]
  ❌ command "curl -w %{local_ip}:%{local_port} -> %{remote_ip}:%{remote_port} = %{response_code} --silent --fail --show-error --output /dev/null --connect-timeout 2 --max-time 10 -H Host: one.one.one.one --retry 3 --retry-all-errors --retry-delay 3 https://one.one.one.one.:443" failed: error with exec request (pod=cilium-test/client2-5668f9f59b-dsv2c, container=client2): command terminated with exit code 28
  ℹ  curl output:
  :0 -> :0 = 000

  [.] Action [no-policies/pod-to-world/https-to-one.one.one.one.-index-1: cilium-test/client2-5668f9f59b-dsv2c (<IP_ADDRESS>) -> one.one.one.one.-https-index (one.one.one.one.:443)]
  ❌ command "curl -w %{local_ip}:%{local_port} -> %{remote_ip}:%{remote_port} = %{response_code} --silent --fail --show-error --output /dev/null --connect-timeout 2 --max-time 10 -H Host: one.one.one.one --retry 3 --retry-all-errors --retry-delay 3 https://one.one.one.one.:443/index.html" failed: error with exec request (pod=cilium-test/client2-5668f9f59b-dsv2c, container=client2): command terminated with exit code 28
  ℹ  curl output:
  :0 -> :0 = 000

  [.] Action [no-policies/pod-to-world/http-to-one.one.one.one.-2: cilium-test/client3-7557dd665c-rjtnh (<IP_ADDRESS>) -> one.one.one.one.-http (one.one.one.one.:80)]
  [.] Action [no-policies/pod-to-world/https-to-one.one.one.one.-2: cilium-test/client3-7557dd665c-rjtnh (<IP_ADDRESS>) -> one.one.one.one.-https (one.one.one.one.:443)]
  ❌ command "curl -w %{local_ip}:%{local_port} -> %{remote_ip}:%{remote_port} = %{response_code} --silent --fail --show-error --output /dev/null --connect-timeout 2 --max-time 10 -H Host: one.one.one.one --retry 3 --retry-all-errors --retry-delay 3 https://one.one.one.one.:443" failed: error with exec request (pod=cilium-test/client3-7557dd665c-rjtnh, container=client3): command terminated with exit code 28
  ℹ  curl output:
  :0 -> :0 = 000

  [.] Action [no-policies/pod-to-world/https-to-one.one.one.one.-index-2: cilium-test/client3-7557dd665c-rjtnh (<IP_ADDRESS>) -> one.one.one.one.-https-index (one.one.one.one.:443)]
  ❌ command "curl -w %{local_ip}:%{local_port} -> %{remote_ip}:%{remote_port} = %{response_code} --silent --fail --show-error --output /dev/null --connect-timeout 2 --max-time 10 -H Host: one.one.one.one --retry 3 --retry-all-errors --retry-delay 3 https://one.one.one.one.:443/index.html" failed: error with exec request (pod=cilium-test/client3-7557dd665c-rjtnh, container=client3): command terminated with exit code 28
  ℹ  curl output:
  :0 -> :0 = 000

  [-] Scenario [no-policies/pod-to-host]
  [.] Action [no-policies/pod-to-host/ping-ipv4-internal-ip: cilium-test/client-56f8968958-zk4l9 (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-hostname: cilium-test/client-56f8968958-zk4l9 (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-internal-ip: cilium-test/client-56f8968958-zk4l9 (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-hostname: cilium-test/client-56f8968958-zk4l9 (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-internal-ip: cilium-test/client-56f8968958-zk4l9 (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-hostname: cilium-test/client-56f8968958-zk4l9 (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-internal-ip: cilium-test/client-56f8968958-zk4l9 (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-hostname: cilium-test/client-56f8968958-zk4l9 (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-internal-ip: cilium-test/client-56f8968958-zk4l9 (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-hostname: cilium-test/client-56f8968958-zk4l9 (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-internal-ip: cilium-test/client-56f8968958-zk4l9 (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-hostname: cilium-test/client-56f8968958-zk4l9 (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-internal-ip: cilium-test/client-56f8968958-zk4l9 (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-hostname: cilium-test/client-56f8968958-zk4l9 (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-internal-ip: cilium-test/client-56f8968958-zk4l9 (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-hostname: cilium-test/client-56f8968958-zk4l9 (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-internal-ip: cilium-test/client-56f8968958-zk4l9 (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-hostname: cilium-test/client-56f8968958-zk4l9 (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-internal-ip: cilium-test/client2-5668f9f59b-dsv2c (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-hostname: cilium-test/client2-5668f9f59b-dsv2c (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-internal-ip: cilium-test/client2-5668f9f59b-dsv2c (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-hostname: cilium-test/client2-5668f9f59b-dsv2c (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-internal-ip: cilium-test/client2-5668f9f59b-dsv2c (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-hostname: cilium-test/client2-5668f9f59b-dsv2c (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-internal-ip: cilium-test/client2-5668f9f59b-dsv2c (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-hostname: cilium-test/client2-5668f9f59b-dsv2c (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-internal-ip: cilium-test/client2-5668f9f59b-dsv2c (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-hostname: cilium-test/client2-5668f9f59b-dsv2c (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-internal-ip: cilium-test/client2-5668f9f59b-dsv2c (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-hostname: cilium-test/client2-5668f9f59b-dsv2c (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-internal-ip: cilium-test/client2-5668f9f59b-dsv2c (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-hostname: cilium-test/client2-5668f9f59b-dsv2c (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-internal-ip: cilium-test/client2-5668f9f59b-dsv2c (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-hostname: cilium-test/client2-5668f9f59b-dsv2c (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-internal-ip: cilium-test/client2-5668f9f59b-dsv2c (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-hostname: cilium-test/client2-5668f9f59b-dsv2c (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-internal-ip: cilium-test/client3-7557dd665c-rjtnh (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-hostname: cilium-test/client3-7557dd665c-rjtnh (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-internal-ip: cilium-test/client3-7557dd665c-rjtnh (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-hostname: cilium-test/client3-7557dd665c-rjtnh (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-internal-ip: cilium-test/client3-7557dd665c-rjtnh (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-hostname: cilium-test/client3-7557dd665c-rjtnh (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-internal-ip: cilium-test/client3-7557dd665c-rjtnh (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-hostname: cilium-test/client3-7557dd665c-rjtnh (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-internal-ip: cilium-test/client3-7557dd665c-rjtnh (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-hostname: cilium-test/client3-7557dd665c-rjtnh (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-internal-ip: cilium-test/client3-7557dd665c-rjtnh (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-hostname: cilium-test/client3-7557dd665c-rjtnh (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-internal-ip: cilium-test/client3-7557dd665c-rjtnh (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-hostname: cilium-test/client3-7557dd665c-rjtnh (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-internal-ip: cilium-test/client3-7557dd665c-rjtnh (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-hostname: cilium-test/client3-7557dd665c-rjtnh (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-internal-ip: cilium-test/client3-7557dd665c-rjtnh (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [.] Action [no-policies/pod-to-host/ping-ipv4-hostname: cilium-test/client3-7557dd665c-rjtnh (<IP_ADDRESS>) -> <IP_ADDRESS> (<IP_ADDRESS>:0)]
  [-] Scenario [no-policies/host-to-pod]
  [.] Action [no-policies/host-to-pod/curl-ipv4-0: cilium-test/host-netns-89llg (<IP_ADDRESS>) -> cilium-test/echo-other-node-bd6cd689f-4snll (<IP_ADDRESS>:8080)]
  [.] Action [no-policies/host-to-pod/curl-ipv4-1: cilium-test/host-netns-89llg (<IP_ADDRESS>) -> cilium-test/echo-same-node-57bb597f97-z5ggv (<IP_ADDRESS>:8080)]
  [.] Action [no-policies/host-to-pod/curl-ipv4-2: cilium-test/host-netns-dzws4 (<IP_ADDRESS>) -> cilium-test/echo-other-node-bd6cd689f-4snll (<IP_ADDRESS>:8080)]
  [.] Action [no-policies/host-to-pod/curl-ipv4-3: cilium-test/host-netns-dzws4 (<IP_ADDRESS>) -> cilium-test/echo-same-node-57bb597f97-z5ggv (<IP_ADDRESS>:8080)]
  [.] Action [no-policies/host-to-pod/curl-ipv4-4: cilium-test/host-netns-hwt4p (<IP_ADDRESS>) -> cilium-test/echo-same-node-57bb597f97-z5ggv (<IP_ADDRESS>:8080)]
  [.] Action [no-policies/host-to-pod/curl-ipv4-5: cilium-test/host-netns-hwt4p (<IP_ADDRESS>) -> cilium-test/echo-other-node-bd6cd689f-4snll (<IP_ADDRESS>:8080)]
  [.] Action [no-policies/host-to-pod/curl-ipv4-6: cilium-test/host-netns-qksw8 (<IP_ADDRESS>) -> cilium-test/echo-other-node-bd6cd689f-4snll (<IP_ADDRESS>:8080)]
  [.] Action [no-policies/host-to-pod/curl-ipv4-7: cilium-test/host-netns-qksw8 (<IP_ADDRESS>) -> cilium-test/echo-same-node-57bb597f97-z5ggv (<IP_ADDRESS>:8080)]
  [.] Action [no-policies/host-to-pod/curl-ipv4-8: cilium-test/host-netns-vfnhm (<IP_ADDRESS>) -> cilium-test/echo-other-node-bd6cd689f-4snll (<IP_ADDRESS>:8080)]
  [.] Action [no-policies/host-to-pod/curl-ipv4-9: cilium-test/host-netns-vfnhm (<IP_ADDRESS>) -> cilium-test/echo-same-node-57bb597f97-z5ggv (<IP_ADDRESS>:8080)]
  [.] Action [no-policies/host-to-pod/curl-ipv4-10: cilium-test/host-netns-6z9pf (<IP_ADDRESS>) -> cilium-test/echo-other-node-bd6cd689f-4snll (<IP_ADDRESS>:8080)]
  [.] Action [no-policies/host-to-pod/curl-ipv4-11: cilium-test/host-netns-6z9pf (<IP_ADDRESS>) -> cilium-test/echo-same-node-57bb597f97-z5ggv (<IP_ADDRESS>:8080)]
  [.] Action [no-policies/host-to-pod/curl-ipv4-12: cilium-test/host-netns-rggmn (<IP_ADDRESS>) -> cilium-test/echo-other-node-bd6cd689f-4snll (<IP_ADDRESS>:8080)]
  [.] Action [no-policies/host-to-pod/curl-ipv4-13: cilium-test/host-netns-rggmn (<IP_ADDRESS>) -> cilium-test/echo-same-node-57bb597f97-z5ggv (<IP_ADDRESS>:8080)]
  [.] Action [no-policies/host-to-pod/curl-ipv4-14: cilium-test/host-netns-svvfm (<IP_ADDRESS>) -> cilium-test/echo-other-node-bd6cd689f-4snll (<IP_ADDRESS>:8080)]
  [.] Action [no-policies/host-to-pod/curl-ipv4-15: cilium-test/host-netns-svvfm (<IP_ADDRESS>) -> cilium-test/echo-same-node-57bb597f97-z5ggv (<IP_ADDRESS>:8080)]
  [.] Action [no-policies/host-to-pod/curl-ipv4-16: cilium-test/host-netns-9g256 (<IP_ADDRESS>) -> cilium-test/echo-other-node-bd6cd689f-4snll (<IP_ADDRESS>:8080)]
  [.] Action [no-policies/host-to-pod/curl-ipv4-17: cilium-test/host-netns-9g256 (<IP_ADDRESS>) -> cilium-test/echo-same-node-57bb597f97-z5ggv (<IP_ADDRESS>:8080)]
  [-] Scenario [no-policies/pod-to-external-workload]
  [-] Scenario [no-policies/pod-to-pod]
  [.] Action [no-policies/pod-to-pod/curl-ipv4-0: cilium-test/client-56f8968958-zk4l9 (<IP_ADDRESS>) -> cilium-test/echo-same-node-57bb597f97-z5ggv (<IP_ADDRESS>:8080)]
  [.] Action [no-policies/pod-to-pod/curl-ipv4-1: cilium-test/client-56f8968958-zk4l9 (<IP_ADDRESS>) -> cilium-test/echo-other-node-bd6cd689f-4snll (<IP_ADDRESS>:8080)]
  [.] Action [no-policies/pod-to-pod/curl-ipv4-2: cilium-test/client2-5668f9f59b-dsv2c (<IP_ADDRESS>) -> cilium-test/echo-same-node-57bb597f97-z5ggv (<IP_ADDRESS>:8080)]
  [.] Action [no-policies/pod-to-pod/curl-ipv4-3: cilium-test/client2-5668f9f59b-dsv2c (<IP_ADDRESS>) -> cilium-test/echo-other-node-bd6cd689f-4snll (<IP_ADDRESS>:8080)]
  [.] Action [no-policies/pod-to-pod/curl-ipv4-4: cilium-test/client3-7557dd665c-rjtnh (<IP_ADDRESS>) -> cilium-test/echo-same-node-57bb597f97-z5ggv (<IP_ADDRESS>:8080)]
  [.] Action [no-policies/pod-to-pod/curl-ipv4-5: cilium-test/client3-7557dd665c-rjtnh (<IP_ADDRESS>) -> cilium-test/echo-other-node-bd6cd689f-4snll (<IP_ADDRESS>:8080)]
  [-] Scenario [no-policies/pod-to-cidr]
  [.] Action [no-policies/pod-to-cidr/external-1111-0: cilium-test/client-56f8968958-zk4l9 (<IP_ADDRESS>) -> external-1111 (<IP_ADDRESS>:443)]
  ❌ command "curl -w %{local_ip}:%{local_port} -> %{remote_ip}:%{remote_port} = %{response_code} --silent --fail --show-error --output /dev/null --connect-timeout 2 --max-time 10 --retry 3 --retry-all-errors --retry-delay 3 https://<IP_ADDRESS>:443" failed: error with exec request (pod=cilium-test/client-56f8968958-zk4l9, container=client): command terminated with exit code 28
  ℹ  curl output:
  :0 -> :0 = 000

  [.] Action [no-policies/pod-to-cidr/external-1111-1: cilium-test/client2-5668f9f59b-dsv2c (<IP_ADDRESS>) -> external-1111 (<IP_ADDRESS>:443)]
  ❌ command "curl -w %{local_ip}:%{local_port} -> %{remote_ip}:%{remote_port} = %{response_code} --silent --fail --show-error --output /dev/null --connect-timeout 2 --max-time 10 --retry 3 --retry-all-errors --retry-delay 3 https://<IP_ADDRESS>:443" failed: error with exec request (pod=cilium-test/client2-5668f9f59b-dsv2c, container=client2): command terminated with exit code 28
  ℹ  curl output:
  :0 -> :0 = 000

  [.] Action [no-policies/pod-to-cidr/external-1111-2: cilium-test/client3-7557dd665c-rjtnh (<IP_ADDRESS>) -> external-1111 (<IP_ADDRESS>:443)]
  ❌ command "curl -w %{local_ip}:%{local_port} -> %{remote_ip}:%{remote_port} = %{response_code} --silent --fail --show-error --output /dev/null --connect-timeout 2 --max-time 10 --retry 3 --retry-all-errors --retry-delay 3 https://<IP_ADDRESS>:443" failed: error with exec request (pod=cilium-test/client3-7557dd665c-rjtnh, container=client3): command terminated with exit code 28
  ℹ  curl output:
  :0 -> :0 = 000

  [.] Action [no-policies/pod-to-cidr/external-1001-0: cilium-test/client-56f8968958-zk4l9 (<IP_ADDRESS>) -> external-1001 (<IP_ADDRESS>:443)]
  ❌ command "curl -w %{local_ip}:%{local_port} -> %{remote_ip}:%{remote_port} = %{response_code} --silent --fail --show-error --output /dev/null --connect-timeout 2 --max-time 10 --retry 3 --retry-all-errors --retry-delay 3 https://<IP_ADDRESS>:443" failed: error with exec request (pod=cilium-test/client-56f8968958-zk4l9, container=client): command terminated with exit code 28
  ℹ  curl output:
  :0 -> :0 = 000

  [.] Action [no-policies/pod-to-cidr/external-1001-1: cilium-test/client2-5668f9f59b-dsv2c (<IP_ADDRESS>) -> external-1001 (<IP_ADDRESS>:443)]
  ❌ command "curl -w %{local_ip}:%{local_port} -> %{remote_ip}:%{remote_port} = %{response_code} --silent --fail --show-error --output /dev/null --connect-timeout 2 --max-time 10 --retry 3 --retry-all-errors --retry-delay 3 https://<IP_ADDRESS>:443" failed: error with exec request (pod=cilium-test/client2-5668f9f59b-dsv2c, container=client2): command terminated with exit code 28
  ℹ  curl output:
  :0 -> :0 = 000

  [.] Action [no-policies/pod-to-cidr/external-1001-2: cilium-test/client3-7557dd665c-rjtnh (<IP_ADDRESS>) -> external-1001 (<IP_ADDRESS>:443)]
  ❌ command "curl -w %{local_ip}:%{local_port} -> %{remote_ip}:%{remote_port} = %{response_code} --silent --fail --show-error --output /dev/null --connect-timeout 2 --max-time 10 --retry 3 --retry-all-errors --retry-delay 3 https://<IP_ADDRESS>:443" failed: error with exec request (pod=cilium-test/client3-7557dd665c-rjtnh, container=client3): command terminated with exit code 28
  ℹ  curl output:
  :0 -> :0 = 000

  [-] Scenario [no-policies/pod-to-service]
  [.] Action [no-policies/pod-to-service/curl-0: cilium-test/client-56f8968958-zk4l9 (<IP_ADDRESS>) -> cilium-test/echo-other-node (echo-other-node.cilium-test:8080)]
  [.] Action [no-policies/pod-to-service/curl-1: cilium-test/client-56f8968958-zk4l9 (<IP_ADDRESS>) -> cilium-test/echo-same-node (echo-same-node.cilium-test:8080)]
  [.] Action [no-policies/pod-to-service/curl-2: cilium-test/client2-5668f9f59b-dsv2c (<IP_ADDRESS>) -> cilium-test/echo-other-node (echo-other-node.cilium-test:8080)]
  [.] Action [no-policies/pod-to-service/curl-3: cilium-test/client2-5668f9f59b-dsv2c (<IP_ADDRESS>) -> cilium-test/echo-same-node (echo-same-node.cilium-test:8080)]
  [.] Action [no-policies/pod-to-service/curl-4: cilium-test/client3-7557dd665c-rjtnh (<IP_ADDRESS>) -> cilium-test/echo-other-node (echo-other-node.cilium-test:8080)]
  [.] Action [no-policies/pod-to-service/curl-5: cilium-test/client3-7557dd665c-rjtnh (<IP_ADDRESS>) -> cilium-test/echo-same-node (echo-same-node.cilium-test:8080)]
[=] Skipping Test [no-policies-from-outside] [3/75] (skipped by condition)
[=] Test [no-policies-extra] [4/75]
........................................................................................................................
[=] Test [allow-all-except-world] [5/75]
........................................................................
[=] Test [client-ingress] [6/75]
......
[=] Test [client-ingress-knp] [7/75]
......
[=] Test [allow-all-with-metrics-check] [8/75]
......
[=] Test [all-ingress-deny] [9/75]
............
[=] Skipping Test [all-ingress-deny-from-outside] [10/75] (skipped by condition)
[=] Test [all-ingress-deny-knp] [11/75]
............
[=] Test [all-egress-deny] [12/75]
........................
[=] Test [all-egress-deny-knp] [13/75]
........................
[=] Test [all-entities-deny] [14/75]
............
[=] Test [cluster-entity] [15/75]
...
[=] Skipping Test [cluster-entity-multi-cluster] [16/75] (skipped by condition)
[=] Test [host-entity-egress] [17/75]
......................................................
[=] Test [host-entity-ingress] [18/75]
..................
[=] Test [echo-ingress] [19/75]
......
[=] Skipping Test [echo-ingress-from-outside] [20/75] (skipped by condition)
[=] Test [echo-ingress-knp] [21/75]
......
[=] Test [client-ingress-icmp] [22/75]
......
[=] Test [client-egress] [23/75]
......
[=] Test [client-egress-knp] [24/75]
......
[=] Test [client-egress-expression] [25/75]
......
[=] Test [client-egress-expression-knp] [26/75]
......
[=] Test [client-with-service-account-egress-to-echo] [27/75]
......
[=] Test [client-egress-to-echo-service-account] [28/75]
......
[=] Test [to-entities-world] [29/75]
.
  ℹ  📜 Applying CiliumNetworkPolicy 'client-egress-to-entities-world' to namespace 'cilium-test'..
  [-] Scenario [to-entities-world/pod-to-world]
  [.] Action [to-entities-world/pod-to-world/http-to-one.one.one.one.-0: cilium-test/client-56f8968958-zk4l9 (<IP_ADDRESS>) -> one.one.one.one.-http (one.one.one.one.:80)]
  ❌ command "curl -w %{local_ip}:%{local_port} -> %{remote_ip}:%{remote_port} = %{response_code} --silent --fail --show-error --output /dev/null --connect-timeout 2 --max-time 10 -H Host: one.one.one.one --retry 3 --retry-all-errors --retry-delay 3 http://one.one.one.one.:80" failed: command failed (pod=cilium-test/client-56f8968958-zk4l9, container=client): "curl: (28) Resolving timed out after 2000 milliseconds\n"
  ℹ  curl output:
  <IP_ADDRESS>:54258 -> <IP_ADDRESS>:80 = 301

  [.] Action [to-entities-world/pod-to-world/https-to-one.one.one.one.-0: cilium-test/client-56f8968958-zk4l9 (<IP_ADDRESS>) -> one.one.one.one.-https (one.one.one.one.:443)]
  [.] Action [to-entities-world/pod-to-world/https-to-one.one.one.one.-index-0: cilium-test/client-56f8968958-zk4l9 (<IP_ADDRESS>) -> one.one.one.one.-https-index (one.one.one.one.:443)]
  [.] Action [to-entities-world/pod-to-world/http-to-one.one.one.one.-1: cilium-test/client2-5668f9f59b-dsv2c (<IP_ADDRESS>) -> one.one.one.one.-http (one.one.one.one.:80)]
  [.] Action [to-entities-world/pod-to-world/https-to-one.one.one.one.-1: cilium-test/client2-5668f9f59b-dsv2c (<IP_ADDRESS>) -> one.one.one.one.-https (one.one.one.one.:443)]
  [.] Action [to-entities-world/pod-to-world/https-to-one.one.one.one.-index-1: cilium-test/client2-5668f9f59b-dsv2c (<IP_ADDRESS>) -> one.one.one.one.-https-index (one.one.one.one.:443)]
  [.] Action [to-entities-world/pod-to-world/http-to-one.one.one.one.-2: cilium-test/client3-7557dd665c-rjtnh (<IP_ADDRESS>) -> one.one.one.one.-http (one.one.one.one.:80)]
  [.] Action [to-entities-world/pod-to-world/https-to-one.one.one.one.-2: cilium-test/client3-7557dd665c-rjtnh (<IP_ADDRESS>) -> one.one.one.one.-https (one.one.one.one.:443)]
  [.] Action [to-entities-world/pod-to-world/https-to-one.one.one.one.-index-2: cilium-test/client3-7557dd665c-rjtnh (<IP_ADDRESS>) -> one.one.one.one.-https-index (one.one.one.one.:443)]
  ℹ  📜 Deleting CiliumNetworkPolicy 'client-egress-to-entities-world' from namespace 'cilium-test'..
[=] Test [to-cidr-external] [30/75]
......
[=] Test [to-cidr-external-knp] [31/75]
......
[=] Skipping Test [from-cidr-host-netns] [32/75] (skipped by condition)
[=] Test [echo-ingress-from-other-client-deny] [33/75]
..........
[=] Test [client-ingress-from-other-client-icmp-deny] [34/75]
............
[=] Test [client-egress-to-echo-deny] [35/75]
............
[=] Test [client-ingress-to-echo-named-port-deny] [36/75]
....
[=] Test [client-egress-to-echo-expression-deny] [37/75]
....
[=] Test [client-with-service-account-egress-to-echo-deny] [38/75]
....
[=] Test [client-egress-to-echo-service-account-deny] [39/75]
..
[=] Test [client-egress-to-cidr-deny] [40/75]
......
[=] Test [client-egress-to-cidr-deny-default] [41/75]
..^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A.^R
^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[.^R
^[[A^[[A.^R
.
[=] Test [health] [42/75]
.........
[=] Skipping Test [north-south-loadbalancing] [43/75] (Feature node-without-cilium is disabled)
[=] Test [pod-to-pod-encryption] [44/75]
.
  [-] Scenario [pod-to-pod-encryption/pod-to-pod-encryption]
  [.] Action [pod-to-pod-encryption/pod-to-pod-encryption/curl-ipv4: cilium-test/client-56f8968958-zk4l9 (<IP_ADDRESS>) -> cilium-test/echo-other-node-bd6cd689f-4snll (<IP_ADDRESS>:8080)]
  🟥 Failed to execute tcpdump on cilium-test/host-netns-6z9pf (<IP_ADDRESS>): failed to wait for tcpdump to terminate
[=] Test [node-to-node-encryption] [45/75]
.
  [-] Scenario [node-to-node-encryption/node-to-node-encryption]
  [.] Action [node-to-node-encryption/node-to-node-encryption/ping-ipv4: cilium-test/client-56f8968958-zk4l9 (<IP_ADDRESS>) -> cilium-test/host-netns-qksw8 (<IP_ADDRESS>:0)]
  🟥 Failed to execute tcpdump on cilium-test/host-netns-6z9pf (<IP_ADDRESS>): failed to wait for tcpdump to terminate
[=] Skipping Test [egress-gateway] [46/75] (skipped by condition)
[=] Skipping Test [egress-gateway-excluded-cidrs] [47/75] (Feature enable-ipv4-egress-gateway is disabled)
[=] Skipping Test [pod-to-node-cidrpolicy] [48/75] (Feature cidr-match-nodes is disabled)
[=] Skipping Test [north-south-loadbalancing-with-l7-policy] [49/75] (Feature node-without-cilium is disabled)
[=] Test [echo-ingress-l7] [50/75]
.......
  ℹ  📜 Applying CiliumNetworkPolicy 'echo-ingress-l7-http' to namespace 'cilium-test'..
  [-] Scenario [echo-ingress-l7/pod-to-pod-with-endpoints]
  [.] Action [echo-ingress-l7/pod-to-pod-with-endpoints/curl-ipv4-0-public: cilium-test/client-56f8968958-zk4l9 (<IP_ADDRESS>) -> curl-ipv4-0-public (<IP_ADDRESS>:8080)]
  [.] Action [echo-ingress-l7/pod-to-pod-with-endpoints/curl-ipv4-0-private: cilium-test/client-56f8968958-zk4l9 (<IP_ADDRESS>) -> curl-ipv4-0-private (<IP_ADDRESS>:8080)]
  [.] Action [echo-ingress-l7/pod-to-pod-with-endpoints/curl-ipv4-0-privatewith-header: cilium-test/client-56f8968958-zk4l9 (<IP_ADDRESS>) -> curl-ipv4-0-privatewith-header (<IP_ADDRESS>:8080)]
  [.] Action [echo-ingress-l7/pod-to-pod-with-endpoints/curl-ipv4-1-public: cilium-test/client-56f8968958-zk4l9 (<IP_ADDRESS>) -> curl-ipv4-1-public (<IP_ADDRESS>:8080)]
  [.] Action [echo-ingress-l7/pod-to-pod-with-endpoints/curl-ipv4-1-private: cilium-test/client-56f8968958-zk4l9 (<IP_ADDRESS>) -> curl-ipv4-1-private (<IP_ADDRESS>:8080)]
  [.] Action [echo-ingress-l7/pod-to-pod-with-endpoints/curl-ipv4-1-privatewith-header: cilium-test/client-56f8968958-zk4l9 (<IP_ADDRESS>) -> curl-ipv4-1-privatewith-header (<IP_ADDRESS>:8080)]
  [.] Action [echo-ingress-l7/pod-to-pod-with-endpoints/curl-ipv4-2-public: cilium-test/client2-5668f9f59b-dsv2c (<IP_ADDRESS>) -> curl-ipv4-2-public (<IP_ADDRESS>:8080)]
  ❌ command "curl -w %{local_ip}:%{local_port} -> %{remote_ip}:%{remote_port} = %{response_code} --silent --fail --show-error --output /dev/null --connect-timeout 2 --max-time 10 http://<IP_ADDRESS>:8080/public" failed: error with exec request (pod=cilium-test/client2-5668f9f59b-dsv2c, container=client2): command terminated with exit code 28
  ℹ  curl output:
  :0 -> :0 = 000

  [.] Action [echo-ingress-l7/pod-to-pod-with-endpoints/curl-ipv4-2-private: cilium-test/client2-5668f9f59b-dsv2c (<IP_ADDRESS>) -> curl-ipv4-2-private (<IP_ADDRESS>:8080)]
  ❌ command "curl -w %{local_ip}:%{local_port} -> %{remote_ip}:%{remote_port} = %{response_code} --silent --fail --show-error --output /dev/null --connect-timeout 2 --max-time 10 http://<IP_ADDRESS>:8080/private" failed with unexpected exit code: error with exec request (pod=cilium-test/client2-5668f9f59b-dsv2c, container=client2): command terminated with exit code 28 (expected 22, found 28)
  [.] Action [echo-ingress-l7/pod-to-pod-with-endpoints/curl-ipv4-2-privatewith-header: cilium-test/client2-5668f9f59b-dsv2c (<IP_ADDRESS>) -> curl-ipv4-2-privatewith-header (<IP_ADDRESS>:8080)]
  ❌ command "curl -w %{local_ip}:%{local_port} -> %{remote_ip}:%{remote_port} = %{response_code} --silent --fail --show-error --output /dev/null --connect-timeout 2 --max-time 10 -H X-Very-Secret-Token: 42 http://<IP_ADDRESS>:8080/private" failed: error with exec request (pod=cilium-test/client2-5668f9f59b-dsv2c, container=client2): command terminated with exit code 28
  ℹ  curl output:
  :0 -> :0 = 000

  [.] Action [echo-ingress-l7/pod-to-pod-with-endpoints/curl-ipv4-3-public: cilium-test/client2-5668f9f59b-dsv2c (<IP_ADDRESS>) -> curl-ipv4-3-public (<IP_ADDRESS>:8080)]
  ❌ command "curl -w %{local_ip}:%{local_port} -> %{remote_ip}:%{remote_port} = %{response_code} --silent --fail --show-error --output /dev/null --connect-timeout 2 --max-time 10 http://<IP_ADDRESS>:8080/public" failed: error with exec request (pod=cilium-test/client2-5668f9f59b-dsv2c, container=client2): command terminated with exit code 28
  ℹ  curl output:
  :0 -> :0 = 000

  [.] Action [echo-ingress-l7/pod-to-pod-with-endpoints/curl-ipv4-3-private: cilium-test/client2-5668f9f59b-dsv2c (<IP_ADDRESS>) -> curl-ipv4-3-private (<IP_ADDRESS>:8080)]
  ❌ command "curl -w %{local_ip}:%{local_port} -> %{remote_ip}:%{remote_port} = %{response_code} --silent --fail --show-error --output /dev/null --connect-timeout 2 --max-time 10 http://<IP_ADDRESS>:8080/private" failed with unexpected exit code: error with exec request (pod=cilium-test/client2-5668f9f59b-dsv2c, container=client2): command terminated with exit code 28 (expected 22, found 28)
  [.] Action [echo-ingress-l7/pod-to-pod-with-endpoints/curl-ipv4-3-privatewith-header: cilium-test/client2-5668f9f59b-dsv2c (<IP_ADDRESS>) -> curl-ipv4-3-privatewith-header (<IP_ADDRESS>:8080)]
  ❌ command "curl -w %{local_ip}:%{local_port} -> %{remote_ip}:%{remote_port} = %{response_code} --silent --fail --show-error --output /dev/null --connect-timeout 2 --max-time 10 -H X-Very-Secret-Token: 42 http://<IP_ADDRESS>:8080/private" failed: error with exec request (pod=cilium-test/client2-5668f9f59b-dsv2c, container=client2): command terminated with exit code 28
  ℹ  curl output:
  :0 -> :0 = 000

  [.] Action [echo-ingress-l7/pod-to-pod-with-endpoints/curl-ipv4-4-public: cilium-test/client3-7557dd665c-rjtnh (<IP_ADDRESS>) -> curl-ipv4-4-public (<IP_ADDRESS>:8080)]
  [.] Action [echo-ingress-l7/pod-to-pod-with-endpoints/curl-ipv4-4-private: cilium-test/client3-7557dd665c-rjtnh (<IP_ADDRESS>) -> curl-ipv4-4-private (<IP_ADDRESS>:8080)]
  [.] Action [echo-ingress-l7/pod-to-pod-with-endpoints/curl-ipv4-4-privatewith-header: cilium-test/client3-7557dd665c-rjtnh (<IP_ADDRESS>) -> curl-ipv4-4-privatewith-header (<IP_ADDRESS>:8080)]
  [.] Action [echo-ingress-l7/pod-to-pod-with-endpoints/curl-ipv4-5-public: cilium-test/client3-7557dd665c-rjtnh (<IP_ADDRESS>) -> curl-ipv4-5-public (<IP_ADDRESS>:8080)]
  [.] Action [echo-ingress-l7/pod-to-pod-with-endpoints/curl-ipv4-5-private: cilium-test/client3-7557dd665c-rjtnh (<IP_ADDRESS>) -> curl-ipv4-5-private (<IP_ADDRESS>:8080)]
  [.] Action [echo-ingress-l7/pod-to-pod-with-endpoints/curl-ipv4-5-privatewith-header: cilium-test/client3-7557dd665c-rjtnh (<IP_ADDRESS>) -> curl-ipv4-5-privatewith-header (<IP_ADDRESS>:8080)]
  ℹ  📜 Deleting CiliumNetworkPolicy 'echo-ingress-l7-http' from namespace 'cilium-test'..
[=] Test [echo-ingress-l7-named-port] [51/75]
.......
  ℹ  📜 Applying CiliumNetworkPolicy 'echo-ingress-l7-http-named-port' to namespace 'cilium-test'..
  [-] Scenario [echo-ingress-l7-named-port/pod-to-pod-with-endpoints]
  [.] Action [echo-ingress-l7-named-port/pod-to-pod-with-endpoints/curl-ipv4-0-public: cilium-test/client-56f8968958-zk4l9 (<IP_ADDRESS>) -> curl-ipv4-0-public (<IP_ADDRESS>:8080)]
  [.] Action [echo-ingress-l7-named-port/pod-to-pod-with-endpoints/curl-ipv4-0-private: cilium-test/client-56f8968958-zk4l9 (<IP_ADDRESS>) -> curl-ipv4-0-private (<IP_ADDRESS>:8080)]
  [.] Action [echo-ingress-l7-named-port/pod-to-pod-with-endpoints/curl-ipv4-0-privatewith-header: cilium-test/client-56f8968958-zk4l9 (<IP_ADDRESS>) -> curl-ipv4-0-privatewith-header (<IP_ADDRESS>:8080)]
  [.] Action [echo-ingress-l7-named-port/pod-to-pod-with-endpoints/curl-ipv4-1-public: cilium-test/client-56f8968958-zk4l9 (<IP_ADDRESS>) -> curl-ipv4-1-public (<IP_ADDRESS>:8080)]
  [.] Action [echo-ingress-l7-named-port/pod-to-pod-with-endpoints/curl-ipv4-1-private: cilium-test/client-56f8968958-zk4l9 (<IP_ADDRESS>) -> curl-ipv4-1-private (<IP_ADDRESS>:8080)]
  [.] Action [echo-ingress-l7-named-port/pod-to-pod-with-endpoints/curl-ipv4-1-privatewith-header: cilium-test/client-56f8968958-zk4l9 (<IP_ADDRESS>) -> curl-ipv4-1-privatewith-header (<IP_ADDRESS>:8080)]
  [.] Action [echo-ingress-l7-named-port/pod-to-pod-with-endpoints/curl-ipv4-2-public: cilium-test/client2-5668f9f59b-dsv2c (<IP_ADDRESS>) -> curl-ipv4-2-public (<IP_ADDRESS>:8080)]
  ❌ command "curl -w %{local_ip}:%{local_port} -> %{remote_ip}:%{remote_port} = %{response_code} --silent --fail --show-error --output /dev/null --connect-timeout 2 --max-time 10 http://<IP_ADDRESS>:8080/public" failed: error with exec request (pod=cilium-test/client2-5668f9f59b-dsv2c, container=client2): command terminated with exit code 28
  ℹ  curl output:
  :0 -> :0 = 000

  [.] Action [echo-ingress-l7-named-port/pod-to-pod-with-endpoints/curl-ipv4-2-private: cilium-test/client2-5668f9f59b-dsv2c (<IP_ADDRESS>) -> curl-ipv4-2-private (<IP_ADDRESS>:8080)]
  ❌ command "curl -w %{local_ip}:%{local_port} -> %{remote_ip}:%{remote_port} = %{response_code} --silent --fail --show-error --output /dev/null --connect-timeout 2 --max-time 10 http://<IP_ADDRESS>:8080/private" failed with unexpected exit code: error with exec request (pod=cilium-test/client2-5668f9f59b-dsv2c, container=client2): command terminated with exit code 28 (expected 22, found 28)
  [.] Action [echo-ingress-l7-named-port/pod-to-pod-with-endpoints/curl-ipv4-2-privatewith-header: cilium-test/client2-5668f9f59b-dsv2c (<IP_ADDRESS>) -> curl-ipv4-2-privatewith-header (<IP_ADDRESS>:8080)]
  ❌ command "curl -w %{local_ip}:%{local_port} -> %{remote_ip}:%{remote_port} = %{response_code} --silent --fail --show-error --output /dev/null --connect-timeout 2 --max-time 10 -H X-Very-Secret-Token: 42 http://<IP_ADDRESS>:8080/private" failed: error with exec request (pod=cilium-test/client2-5668f9f59b-dsv2c, container=client2): command terminated with exit code 28
  ℹ  curl output:
  :0 -> :0 = 000

  [.] Action [echo-ingress-l7-named-port/pod-to-pod-with-endpoints/curl-ipv4-3-public: cilium-test/client2-5668f9f59b-dsv2c (<IP_ADDRESS>) -> curl-ipv4-3-public (<IP_ADDRESS>:8080)]
  ❌ command "curl -w %{local_ip}:%{local_port} -> %{remote_ip}:%{remote_port} = %{response_code} --silent --fail --show-error --output /dev/null --connect-timeout 2 --max-time 10 http://<IP_ADDRESS>:8080/public" failed: error with exec request (pod=cilium-test/client2-5668f9f59b-dsv2c, container=client2): command terminated with exit code 28
  ℹ  curl output:
  :0 -> :0 = 000

  [.] Action [echo-ingress-l7-named-port/pod-to-pod-with-endpoints/curl-ipv4-3-private: cilium-test/client2-5668f9f59b-dsv2c (<IP_ADDRESS>) -> curl-ipv4-3-private (<IP_ADDRESS>:8080)]
  ❌ command "curl -w %{local_ip}:%{local_port} -> %{remote_ip}:%{remote_port} = %{response_code} --silent --fail --show-error --output /dev/null --connect-timeout 2 --max-time 10 http://<IP_ADDRESS>:8080/private" failed with unexpected exit code: error with exec request (pod=cilium-test/client2-5668f9f59b-dsv2c, container=client2): command terminated with exit code 28 (expected 22, found 28)
  [.] Action [echo-ingress-l7-named-port/pod-to-pod-with-endpoints/curl-ipv4-3-privatewith-header: cilium-test/client2-5668f9f59b-dsv2c (<IP_ADDRESS>) -> curl-ipv4-3-privatewith-header (<IP_ADDRESS>:8080)]