Kafka response rewrite to Envoy not work when using SSL

0. Abstract
When I connect Kakfa using SSL, Kafka response rewrite to Envoy not work.
1.  Test kafka
PLAINTEXT
# Producer
[root@ia1zetblab001 ~]# /usr/local/resources/kafka_2.13-2.8.1/bin/kafka-console-producer.sh --topic envot-test --bootstrap-server <IP_ADDRESS>:3092
>OKOKOK
>OKOKOK
>OKOKOK
>

# Consumer
[root@ia1zetllab002 ~]# /usr/local/resources/kafka_2.13-2.8.1/bin/kafka-console-consumer.sh --topic envot-test --from-beginning  --bootstrap-server <IP_ADDRESS>:3092
OKOKOK
OKOKOK
OKOKOK

SSL
# Producer
[root@ia1zetblab001 ~/usr/local/resources/kafka_2.13-2.8.1/bin/kafka-console-producer.sh --topic envot-test --producer.config /root/client-ssl.properties --bootstrap-server <IP_ADDRESS>:3093
>OKOKOK
>OKOKOK
>OKOKOK
>

# Consumer
[root@ia1zetllab002 ~]# /usr/local/resources/kafka_2.13-2.8.1/bin/kafka-console-consumer.sh --topic envot-test --from-beginning --consumer.config /root/client-ssl.properties --bootstrap-server <IP_ADDRESS>:3093
OKOKOK
OKOKOK
OKOKOK

As we can see, kafka cluster test using PLAINTEXT or SSL is no problem
3. Test kafka rewrite to Envoy
envoy.yml
static_resources:
  listeners:
    - address:
        socket_address:
          address: <IP_ADDRESS>
          port_value: 9019
      filter_chains:
      - filters:
        - name: envoy.filters.network.kafka_broker
          typed_config:
            "@type": type.googleapis.com/envoy.extensions.filters.network.kafka_broker.v3.KafkaBroker
            stat_prefix: exampleprefix1
            id_based_broker_address_rewrite_spec: &kafka_rewrite_spec
              rules:
              - id: 1
                # envoy ip
                host: <IP_ADDRESS>
                port: 9019
              - id: 2
                host: <IP_ADDRESS>
                port: 9018
              - id: 3
                host: <IP_ADDRESS>
                port: 9017
        - name: envoy.filters.network.tcp_proxy
          typed_config:
            "@type": type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxy
            stat_prefix: tcp
            cluster: broker_1

    - address:
        socket_address:
          address: <IP_ADDRESS>
          port_value: 9018
      filter_chains:
      - filters:
        - name: envoy.filters.network.kafka_broker
          typed_config:
            "@type": type.googleapis.com/envoy.extensions.filters.network.kafka_broker.v3.KafkaBroker
            stat_prefix: exampleprefix2
            id_based_broker_address_rewrite_spec: *kafka_rewrite_spec
        - name: envoy.filters.network.tcp_proxy
          typed_config:
            "@type": type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxy
            stat_prefix: tcp
            cluster: broker_2

    - address:
        socket_address:
          address: <IP_ADDRESS>
          port_value: 9017
      filter_chains:
      - filters:
        - name: envoy.filters.network.kafka_broker
          typed_config:
            "@type": type.googleapis.com/envoy.extensions.filters.network.kafka_broker.v3.KafkaBroker
            stat_prefix: exampleprefix3
            id_based_broker_address_rewrite_spec: *kafka_rewrite_spec
        - name: envoy.filters.network.tcp_proxy
          typed_config:
            "@type": type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxy
            stat_prefix: tcp
            cluster: broker_3

  clusters:
    - name: broker_1
      connect_timeout: 0.25s
      type: STATIC
      lb_policy: ROUND_ROBIN
      load_assignment:
        cluster_name: kafka_cluster
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    socket_address:
                      address: <IP_ADDRESS>
                      port_value: 3092

    - name: broker_2
      connect_timeout: 0.25s
      type: STATIC
      lb_policy: ROUND_ROBIN
      load_assignment:
        cluster_name: kafka_cluster
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    socket_address:
                      address: <IP_ADDRESS>
                      port_value: 3092

    - name: broker_3
      connect_timeout: 0.25s
      type: STATIC
      lb_policy: ROUND_ROBIN
      load_assignment:
        cluster_name: kafka_cluster
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    socket_address:
                      address: <IP_ADDRESS>
                      port_value: 3092

# Envoy IP/Port is <IP_ADDRESS>:9019

# Producer
[root@ia1zetblab001 ~]# /usr/local/resources/kafka_2.13-2.8.1/bin/kafka-console-producer.sh --topic envot-test --bootstrap-server <IP_ADDRESS>:9019
>QQQ
>QQQ
>QQQ
>

# Consumer
[root@ia1zetllab002 ~]# /usr/local/resources/kafka_2.13-2.8.1/bin/kafka-console-consumer.sh --topic envot-test --from-beginning --bootstrap-server <IP_ADDRESS>:9019
QQQ
QQQ
QQQ

As we can see, kafka rewrite to envoy test success (PLAINTEXT model, without SSL)
4. Test kafka rewrite to Envoy(SSL)
envoy.yml
static_resources:
  listeners:
    - address:
        socket_address:
          address: <IP_ADDRESS>
          port_value: 9019
      filter_chains:
      - filters:
        - name: envoy.filters.network.kafka_broker
          typed_config:
            "@type": type.googleapis.com/envoy.extensions.filters.network.kafka_broker.v3.KafkaBroker
            stat_prefix: exampleprefix1
            id_based_broker_address_rewrite_spec: &kafka_rewrite_spec
              rules:
              - id: 1
                host: <IP_ADDRESS>
                port: 9019
              - id: 2
                host: <IP_ADDRESS>
                port: 9018
              - id: 3
                host: <IP_ADDRESS>
                port: 9017
        - name: envoy.filters.network.tcp_proxy
          typed_config:
            "@type": type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxy
            stat_prefix: tcp
            cluster: broker_1

    - address:
        socket_address:
          address: <IP_ADDRESS>
          port_value: 9018
      filter_chains:
      - filters:
        - name: envoy.filters.network.kafka_broker
          typed_config:
            "@type": type.googleapis.com/envoy.extensions.filters.network.kafka_broker.v3.KafkaBroker
            stat_prefix: exampleprefix2
            id_based_broker_address_rewrite_spec: *kafka_rewrite_spec
        - name: envoy.filters.network.tcp_proxy
          typed_config:
            "@type": type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxy
            stat_prefix: tcp
            cluster: broker_2

    - address:
        socket_address:
          address: <IP_ADDRESS>
          port_value: 9017
      filter_chains:
      - filters:
        - name: envoy.filters.network.kafka_broker
          typed_config:
            "@type": type.googleapis.com/envoy.extensions.filters.network.kafka_broker.v3.KafkaBroker
            stat_prefix: exampleprefix3
            id_based_broker_address_rewrite_spec: *kafka_rewrite_spec
        - name: envoy.filters.network.tcp_proxy
          typed_config:
            "@type": type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxy
            stat_prefix: tcp
            cluster: broker_3

  clusters:
    - name: broker_1
      connect_timeout: 0.25s
      type: STATIC
      lb_policy: ROUND_ROBIN
      load_assignment:
        cluster_name: kafka_cluster
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    socket_address:
                      address: <IP_ADDRESS>
                      port_value: 3093 #SSL Port
      transport_socket:
        name: envoy.transport_sockets.tls
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
          common_tls_context:
            tls_certificates:
              - certificate_chain:
                  filename: /ssl/server-cert.pem
                private_key:
                  filename: /ssl/server-key-no-pass.pem # Cannot have password for envoy

    - name: broker_2
      connect_timeout: 0.25s
      type: STATIC
      lb_policy: ROUND_ROBIN
      load_assignment:
        cluster_name: kafka_cluster
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    socket_address:
                      address: <IP_ADDRESS>
                      port_value: 3093
      transport_socket:
        name: envoy.transport_sockets.tls
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
          common_tls_context:
            tls_certificates:
              - certificate_chain:
                  filename: /ssl/server-cert.pem
                private_key:
                  filename: /ssl/server-key-no-pass.pem # Cannot have password for envoy

    - name: broker_3
      connect_timeout: 0.25s
      type: STATIC
      lb_policy: ROUND_ROBIN
      load_assignment:
        cluster_name: kafka_cluster
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    socket_address:
                      address: <IP_ADDRESS>
                      port_value: 3093
      transport_socket:
        name: envoy.transport_sockets.tls
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
          common_tls_context:
            tls_certificates:
              - certificate_chain:
                  filename: /ssl/server-cert.pem
                private_key:
                  filename: /ssl/server-key-no-pass.pem # Cannot have password for envoy

With client SSL (Failed)
start to produce/consume
/usr/local/resources/kafka_2.13-2.8.1/bin/kafka-console-consumer.sh --topic envot-test --from-beginning --consumer.config /root/client-ssl.properties --bootstrap-server <IP_ADDRESS>:9019

/usr/local/resources/kafka_2.13-2.8.1/bin/kafka-console-producer.sh --topic envot-test --producer.config /root/client-ssl.properties --bootstrap-server <IP_ADDRESS>:9019

envoy log
[2024-08-15 02:28:18.695][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:28:23.695][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:28:28.696][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:28:33.696][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:28:38.697][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:28:43.698][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:28:47.629][18][debug][connection] [source/common/network/connection_impl.cc:788] [Tags: "ConnectionId":"16"] write flush complete
[2024-08-15 02:28:47.630][18][debug][connection] [source/common/network/connection_impl.cc:278] [Tags: "ConnectionId":"16"] closing socket: 1
[2024-08-15 02:28:47.630][18][debug][connection] [source/common/network/connection_impl.cc:146] [Tags: "ConnectionId":"17"] closing data_to_write=0 type=1
[2024-08-15 02:28:47.630][18][debug][connection] [source/common/network/connection_impl.cc:278] [Tags: "ConnectionId":"17"] closing socket: 1
[2024-08-15 02:28:47.630][18][debug][connection] [source/extensions/transport_sockets/tls/ssl_socket.cc:368] [Tags: "ConnectionId":"17"] SSL shutdown: rc=1
[2024-08-15 02:28:47.630][18][debug][pool] [source/common/conn_pool/conn_pool_base.cc:495] [Tags: "ConnectionId":"17"] client disconnected, failure reason:
[2024-08-15 02:28:47.630][18][debug][pool] [source/common/conn_pool/conn_pool_base.cc:463] invoking 1 idle callback(s) - is_draining_for_deletion_=false
[2024-08-15 02:28:47.630][18][debug][pool] [source/common/conn_pool/conn_pool_base.cc:215] [Tags: "ConnectionId":"17"] destroying stream: 0 remaining
[2024-08-15 02:28:47.630][18][debug][pool] [source/common/conn_pool/conn_pool_base.cc:463] invoking 0 idle callback(s) - is_draining_for_deletion_=false
[2024-08-15 02:28:47.630][18][debug][conn_handler] [source/common/listener_manager/active_stream_listener_base.cc:136] [Tags: "ConnectionId":"16"] adding to cleanup list
[2024-08-15 02:28:48.700][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:28:50.515][16][debug][filter] [source/common/tcp_proxy/tcp_proxy.cc:243] [Tags: "ConnectionId":"18"] new tcp proxy session
[2024-08-15 02:28:50.516][16][debug][filter] [source/common/tcp_proxy/tcp_proxy.cc:436] [Tags: "ConnectionId":"18"] Creating connection to cluster broker_1
[2024-08-15 02:28:50.516][16][debug][misc] [source/common/upstream/cluster_manager_impl.cc:2328] Allocating TCP conn pool
[2024-08-15 02:28:50.516][16][debug][pool] [source/common/conn_pool/conn_pool_base.cc:291] trying to create new connection
[2024-08-15 02:28:50.516][16][debug][pool] [source/common/conn_pool/conn_pool_base.cc:145] creating a new connection (connecting=0)
[2024-08-15 02:28:50.516][16][debug][connection] [source/common/network/connection_impl.cc:1009] [Tags: "ConnectionId":"19"] connecting to <IP_ADDRESS>:3093
[2024-08-15 02:28:50.516][16][debug][connection] [source/common/network/connection_impl.cc:1028] [Tags: "ConnectionId":"19"] connection in progress
[2024-08-15 02:28:50.516][16][debug][conn_handler] [source/common/listener_manager/active_tcp_listener.cc:160] [Tags: "ConnectionId":"18"] new connection from <IP_ADDRESS>:51332
[2024-08-15 02:28:50.556][16][debug][connection] [source/common/network/connection_impl.cc:746] [Tags: "ConnectionId":"19"] connected
[2024-08-15 02:28:50.650][16][debug][pool] [source/common/conn_pool/conn_pool_base.cc:328] [Tags: "ConnectionId":"19"] attaching to next stream
[2024-08-15 02:28:50.650][16][debug][pool] [source/common/conn_pool/conn_pool_base.cc:182] [Tags: "ConnectionId":"19"] creating stream
[2024-08-15 02:28:50.650][16][debug][router] [source/common/tcp_proxy/upstream.cc:255] Attached upstream connection [C19] to downstream connection [C18]
[2024-08-15 02:28:50.650][16][debug][filter] [source/common/tcp_proxy/tcp_proxy.cc:830] [Tags: "ConnectionId":"18"] TCP:onUpstreamEvent(), requestedServerName:
[2024-08-15 02:28:50.650][16][debug][kafka] [contrib/kafka/filters/network/source/broker/filter.cc:103] could not process data from Kafka client: invalid length: -18024
[2024-08-15 02:28:53.700][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:28:58.700][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:29:03.701][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:29:08.700][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:29:13.701][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:29:18.702][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:29:23.701][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:29:28.702][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:29:33.703][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:29:38.703][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:29:43.704][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:29:48.704][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:29:53.705][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:29:58.704][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:30:03.705][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:30:08.706][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:30:11.814][22][debug][filter] [source/common/tcp_proxy/tcp_proxy.cc:243] [Tags: "ConnectionId":"20"] new tcp proxy session
[2024-08-15 02:30:11.814][22][debug][filter] [source/common/tcp_proxy/tcp_proxy.cc:436] [Tags: "ConnectionId":"20"] Creating connection to cluster broker_1
[2024-08-15 02:30:11.814][22][debug][misc] [source/common/upstream/cluster_manager_impl.cc:2328] Allocating TCP conn pool
[2024-08-15 02:30:11.814][22][debug][pool] [source/common/conn_pool/conn_pool_base.cc:291] trying to create new connection
[2024-08-15 02:30:11.814][22][debug][pool] [source/common/conn_pool/conn_pool_base.cc:145] creating a new connection (connecting=0)
[2024-08-15 02:30:11.815][22][debug][connection] [source/common/network/connection_impl.cc:1009] [Tags: "ConnectionId":"21"] connecting to <IP_ADDRESS>:3093
[2024-08-15 02:30:11.815][22][debug][connection] [source/common/network/connection_impl.cc:1028] [Tags: "ConnectionId":"21"] connection in progress
[2024-08-15 02:30:11.815][22][debug][conn_handler] [source/common/listener_manager/active_tcp_listener.cc:160] [Tags: "ConnectionId":"20"] new connection from <IP_ADDRESS>:56870
[2024-08-15 02:30:11.855][22][debug][connection] [source/common/network/connection_impl.cc:746] [Tags: "ConnectionId":"21"] connected
[2024-08-15 02:30:11.951][22][debug][pool] [source/common/conn_pool/conn_pool_base.cc:328] [Tags: "ConnectionId":"21"] attaching to next stream
[2024-08-15 02:30:11.951][22][debug][pool] [source/common/conn_pool/conn_pool_base.cc:182] [Tags: "ConnectionId":"21"] creating stream
[2024-08-15 02:30:11.951][22][debug][router] [source/common/tcp_proxy/upstream.cc:255] Attached upstream connection [C21] to downstream connection [C20]
[2024-08-15 02:30:11.951][22][debug][filter] [source/common/tcp_proxy/tcp_proxy.cc:830] [Tags: "ConnectionId":"20"] TCP:onUpstreamEvent(), requestedServerName:
[2024-08-15 02:30:11.951][22][debug][kafka] [contrib/kafka/filters/network/source/broker/filter.cc:103] could not process data from Kafka client: invalid length: -30267
[2024-08-15 02:30:13.706][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:30:18.705][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:30:23.706][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:30:28.705][1][debug][main] [source/server/server.cc:239] flushing stats

produce/consume error
[root@ia1zetblab001 ~]# /usr/local/resources/kafka_2.13-2.8.1/bin/kafka-console-producer.sh --topic envot-test --producer.config /root/client-ssl.properties --bootstrap-server <IP_ADDRESS>:9019
>WWW
[2024-08-15 02:31:15,527] ERROR Error when sending message to topic envot-test with key: null, value: 3 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.TimeoutException: Topic envot-test not present in metadata after 60000 ms.
>[2024-08-15 02:39:13,054] WARN [Producer clientId=console-producer] Bootstrap broker <IP_ADDRESS>:9019 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)

[root@ia1zetllab002 ~]# /usr/local/resources/kafka_2.13-2.8.1/bin/kafka-console-consumer.sh --topic envot-test --from-beginning --consumer.config /root/client-ssl.properties --bootstrap-server <IP_ADDRESS>:9019
[2024-08-15 02:37:50,636] WARN [Consumer clientId=consumer-console-consumer-26575-1, groupId=console-consumer-26575] Bootstrap broker <IP_ADDRESS>:9019 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)

envoy log
[2024-08-15 02:38:28.752][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:38:33.753][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:38:38.754][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:38:43.755][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:38:48.755][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:38:50.792][16][debug][connection] [source/common/network/connection_impl.cc:788] [Tags: "ConnectionId":"18"] write flush complete
[2024-08-15 02:38:50.792][16][debug][connection] [source/common/network/connection_impl.cc:278] [Tags: "ConnectionId":"18"] closing socket: 1
[2024-08-15 02:38:50.792][16][debug][connection] [source/common/network/connection_impl.cc:146] [Tags: "ConnectionId":"19"] closing data_to_write=0 type=1
[2024-08-15 02:38:50.792][16][debug][connection] [source/common/network/connection_impl.cc:278] [Tags: "ConnectionId":"19"] closing socket: 1
[2024-08-15 02:38:50.792][16][debug][connection] [source/extensions/transport_sockets/tls/ssl_socket.cc:368] [Tags: "ConnectionId":"19"] SSL shutdown: rc=1
[2024-08-15 02:38:50.792][16][debug][pool] [source/common/conn_pool/conn_pool_base.cc:495] [Tags: "ConnectionId":"19"] client disconnected, failure reason:
[2024-08-15 02:38:50.793][16][debug][pool] [source/common/conn_pool/conn_pool_base.cc:463] invoking 1 idle callback(s) - is_draining_for_deletion_=false
[2024-08-15 02:38:50.793][16][debug][pool] [source/common/conn_pool/conn_pool_base.cc:215] [Tags: "ConnectionId":"19"] destroying stream: 0 remaining
[2024-08-15 02:38:50.793][16][debug][pool] [source/common/conn_pool/conn_pool_base.cc:463] invoking 0 idle callback(s) - is_draining_for_deletion_=false
[2024-08-15 02:38:50.793][16][debug][conn_handler] [source/common/listener_manager/active_stream_listener_base.cc:136] [Tags: "ConnectionId":"18"] adding to cleanup list
[2024-08-15 02:38:53.756][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:38:58.756][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:39:03.757][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:39:08.757][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:39:13.035][22][debug][kafka] [contrib/kafka/filters/network/source/broker/filter.cc:103] could not process data from Kafka client: invalid length: -30267
[2024-08-15 02:39:13.137][24][debug][filter] [source/common/tcp_proxy/tcp_proxy.cc:243] [Tags: "ConnectionId":"24"] new tcp proxy session
[2024-08-15 02:39:13.137][24][debug][filter] [source/common/tcp_proxy/tcp_proxy.cc:436] [Tags: "ConnectionId":"24"] Creating connection to cluster broker_1
[2024-08-15 02:39:13.137][24][debug][pool] [source/common/conn_pool/conn_pool_base.cc:291] trying to create new connection
[2024-08-15 02:39:13.137][24][debug][pool] [source/common/conn_pool/conn_pool_base.cc:145] creating a new connection (connecting=0)
[2024-08-15 02:39:13.137][24][debug][connection] [source/common/network/connection_impl.cc:1009] [Tags: "ConnectionId":"25"] connecting to <IP_ADDRESS>:3093
[2024-08-15 02:39:13.137][24][debug][connection] [source/common/network/connection_impl.cc:1028] [Tags: "ConnectionId":"25"] connection in progress
[2024-08-15 02:39:13.137][24][debug][conn_handler] [source/common/listener_manager/active_tcp_listener.cc:160] [Tags: "ConnectionId":"24"] new connection from <IP_ADDRESS>:48104
[2024-08-15 02:39:13.177][24][debug][connection] [source/common/network/connection_impl.cc:746] [Tags: "ConnectionId":"25"] connected
[2024-08-15 02:39:13.271][24][debug][pool] [source/common/conn_pool/conn_pool_base.cc:328] [Tags: "ConnectionId":"25"] attaching to next stream
[2024-08-15 02:39:13.271][24][debug][pool] [source/common/conn_pool/conn_pool_base.cc:182] [Tags: "ConnectionId":"25"] creating stream
[2024-08-15 02:39:13.271][24][debug][router] [source/common/tcp_proxy/upstream.cc:255] Attached upstream connection [C25] to downstream connection [C24]
[2024-08-15 02:39:13.271][24][debug][filter] [source/common/tcp_proxy/tcp_proxy.cc:830] [Tags: "ConnectionId":"24"] TCP:onUpstreamEvent(), requestedServerName:
[2024-08-15 02:39:13.271][24][debug][kafka] [contrib/kafka/filters/network/source/broker/filter.cc:103] could not process data from Kafka client: invalid length: -7895
[2024-08-15 02:39:13.757][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:39:18.757][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:39:23.758][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:39:28.759][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:39:33.760][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:39:38.760][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:39:43.761][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:39:48.762][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:39:53.762][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:39:58.762][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:40:03.762][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:40:08.762][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:40:12.076][22][debug][connection] [source/common/network/connection_impl.cc:788] [Tags: "ConnectionId":"20"] write flush complete
[2024-08-15 02:40:12.076][22][debug][connection] [source/common/network/connection_impl.cc:278] [Tags: "ConnectionId":"20"] closing socket: 1
[2024-08-15 02:40:12.076][22][debug][connection] [source/common/network/connection_impl.cc:146] [Tags: "ConnectionId":"21"] closing data_to_write=0 type=1
[2024-08-15 02:40:12.076][22][debug][connection] [source/common/network/connection_impl.cc:278] [Tags: "ConnectionId":"21"] closing socket: 1
[2024-08-15 02:40:12.076][22][debug][connection] [source/extensions/transport_sockets/tls/ssl_socket.cc:368] [Tags: "ConnectionId":"21"] SSL shutdown: rc=1
[2024-08-15 02:40:12.077][22][debug][pool] [source/common/conn_pool/conn_pool_base.cc:495] [Tags: "ConnectionId":"21"] client disconnected, failure reason:
[2024-08-15 02:40:12.077][22][debug][pool] [source/common/conn_pool/conn_pool_base.cc:463] invoking 1 idle callback(s) - is_draining_for_deletion_=false
[2024-08-15 02:40:12.077][22][debug][pool] [source/common/conn_pool/conn_pool_base.cc:215] [Tags: "ConnectionId":"21"] destroying stream: 0 remaining
[2024-08-15 02:40:12.077][22][debug][pool] [source/common/conn_pool/conn_pool_base.cc:463] invoking 0 idle callback(s) - is_draining_for_deletion_=false
[2024-08-15 02:40:12.077][22][debug][conn_handler] [source/common/listener_manager/active_stream_listener_base.cc:136] [Tags: "ConnectionId":"20"] adding to cleanup list
[2024-08-15 02:40:13.762][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:40:18.763][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:40:23.763][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:40:28.764][1][debug][main] [source/server/server.cc:239] flushing stats

Without client SSL (Success)
produce/consume success
[root@ia1zetblab001 ~]# /usr/local/resources/kafka_2.13-2.8.1/bin/kafka-console-producer.sh --topic envot-test --bootstrap-server <IP_ADDRESS>:9019
>EEE
>EEE
>EEE
>

[root@ia1zetllab002 ~]# /usr/local/resources/kafka_2.13-2.8.1/bin/kafka-console-consumer.sh --topic envot-test --from-beginning  --bootstrap-server <IP_ADDRESS>:9019
EEE
EEE
EEE

envoy log
[2024-08-15 02:47:03.798][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:47:08.798][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:47:13.798][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:47:18.799][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:47:23.800][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:47:26.779][16][debug][filter] [source/common/tcp_proxy/tcp_proxy.cc:243] [Tags: "ConnectionId":"30"] new tcp proxy session
[2024-08-15 02:47:26.779][16][debug][filter] [source/common/tcp_proxy/tcp_proxy.cc:436] [Tags: "ConnectionId":"30"] Creating connection to cluster broker_1
[2024-08-15 02:47:26.779][16][debug][misc] [source/common/upstream/cluster_manager_impl.cc:2328] Allocating TCP conn pool
[2024-08-15 02:47:26.779][16][debug][pool] [source/common/conn_pool/conn_pool_base.cc:291] trying to create new connection
[2024-08-15 02:47:26.779][16][debug][pool] [source/common/conn_pool/conn_pool_base.cc:145] creating a new connection (connecting=0)
[2024-08-15 02:47:26.780][16][debug][connection] [source/common/network/connection_impl.cc:1009] [Tags: "ConnectionId":"31"] connecting to <IP_ADDRESS>:3093
[2024-08-15 02:47:26.780][16][debug][connection] [source/common/network/connection_impl.cc:1028] [Tags: "ConnectionId":"31"] connection in progress
[2024-08-15 02:47:26.780][16][debug][conn_handler] [source/common/listener_manager/active_tcp_listener.cc:160] [Tags: "ConnectionId":"30"] new connection from <IP_ADDRESS>:45102
[2024-08-15 02:47:26.820][16][debug][connection] [source/common/network/connection_impl.cc:746] [Tags: "ConnectionId":"31"] connected
[2024-08-15 02:47:26.911][16][debug][pool] [source/common/conn_pool/conn_pool_base.cc:328] [Tags: "ConnectionId":"31"] attaching to next stream
[2024-08-15 02:47:26.911][16][debug][pool] [source/common/conn_pool/conn_pool_base.cc:182] [Tags: "ConnectionId":"31"] creating stream
[2024-08-15 02:47:26.911][16][debug][router] [source/common/tcp_proxy/upstream.cc:255] Attached upstream connection [C31] to downstream connection [C30]
[2024-08-15 02:47:26.911][16][debug][filter] [source/common/tcp_proxy/tcp_proxy.cc:830] [Tags: "ConnectionId":"30"] TCP:onUpstreamEvent(), requestedServerName:
[2024-08-15 02:47:27.067][20][debug][filter] [source/common/tcp_proxy/tcp_proxy.cc:243] [Tags: "ConnectionId":"32"] new tcp proxy session
[2024-08-15 02:47:27.067][20][debug][filter] [source/common/tcp_proxy/tcp_proxy.cc:436] [Tags: "ConnectionId":"32"] Creating connection to cluster broker_3
[2024-08-15 02:47:27.067][20][debug][pool] [source/common/conn_pool/conn_pool_base.cc:291] trying to create new connection
[2024-08-15 02:47:27.067][20][debug][pool] [source/common/conn_pool/conn_pool_base.cc:145] creating a new connection (connecting=0)
[2024-08-15 02:47:27.068][20][debug][connection] [source/common/network/connection_impl.cc:1009] [Tags: "ConnectionId":"33"] connecting to <IP_ADDRESS>:3093
[2024-08-15 02:47:27.068][20][debug][connection] [source/common/network/connection_impl.cc:1028] [Tags: "ConnectionId":"33"] connection in progress
[2024-08-15 02:47:27.068][20][debug][conn_handler] [source/common/listener_manager/active_tcp_listener.cc:160] [Tags: "ConnectionId":"32"] new connection from <IP_ADDRESS>:35608
[2024-08-15 02:47:27.069][20][debug][connection] [source/common/network/connection_impl.cc:746] [Tags: "ConnectionId":"33"] connected
[2024-08-15 02:47:27.088][20][debug][pool] [source/common/conn_pool/conn_pool_base.cc:328] [Tags: "ConnectionId":"33"] attaching to next stream
[2024-08-15 02:47:27.088][20][debug][pool] [source/common/conn_pool/conn_pool_base.cc:182] [Tags: "ConnectionId":"33"] creating stream
[2024-08-15 02:47:27.088][20][debug][router] [source/common/tcp_proxy/upstream.cc:255] Attached upstream connection [C33] to downstream connection [C32]
[2024-08-15 02:47:27.088][20][debug][filter] [source/common/tcp_proxy/tcp_proxy.cc:830] [Tags: "ConnectionId":"32"] TCP:onUpstreamEvent(), requestedServerName:
[2024-08-15 02:47:28.801][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:47:33.801][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:47:38.801][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:47:38.933][26][debug][filter] [source/common/tcp_proxy/tcp_proxy.cc:243] [Tags: "ConnectionId":"34"] new tcp proxy session
[2024-08-15 02:47:38.933][26][debug][filter] [source/common/tcp_proxy/tcp_proxy.cc:436] [Tags: "ConnectionId":"34"] Creating connection to cluster broker_1
[2024-08-15 02:47:38.933][26][debug][misc] [source/common/upstream/cluster_manager_impl.cc:2328] Allocating TCP conn pool
[2024-08-15 02:47:38.933][26][debug][pool] [source/common/conn_pool/conn_pool_base.cc:291] trying to create new connection
[2024-08-15 02:47:38.933][26][debug][pool] [source/common/conn_pool/conn_pool_base.cc:145] creating a new connection (connecting=0)
[2024-08-15 02:47:38.933][26][debug][connection] [source/common/network/connection_impl.cc:1009] [Tags: "ConnectionId":"35"] connecting to <IP_ADDRESS>:3093
[2024-08-15 02:47:38.933][26][debug][connection] [source/common/network/connection_impl.cc:1028] [Tags: "ConnectionId":"35"] connection in progress
[2024-08-15 02:47:38.933][26][debug][conn_handler] [source/common/listener_manager/active_tcp_listener.cc:160] [Tags: "ConnectionId":"34"] new connection from <IP_ADDRESS>:51696
[2024-08-15 02:47:38.976][26][debug][connection] [source/common/network/connection_impl.cc:746] [Tags: "ConnectionId":"35"] connected
[2024-08-15 02:47:39.068][26][debug][pool] [source/common/conn_pool/conn_pool_base.cc:328] [Tags: "ConnectionId":"35"] attaching to next stream
[2024-08-15 02:47:39.068][26][debug][pool] [source/common/conn_pool/conn_pool_base.cc:182] [Tags: "ConnectionId":"35"] creating stream
[2024-08-15 02:47:39.068][26][debug][router] [source/common/tcp_proxy/upstream.cc:255] Attached upstream connection [C35] to downstream connection [C34]
[2024-08-15 02:47:39.068][26][debug][filter] [source/common/tcp_proxy/tcp_proxy.cc:830] [Tags: "ConnectionId":"34"] TCP:onUpstreamEvent(), requestedServerName:
[2024-08-15 02:47:43.802][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:47:45.409][20][debug][connection] [source/common/network/connection_impl.cc:714] [Tags: "ConnectionId":"29"] remote close
[2024-08-15 02:47:45.409][20][debug][connection] [source/common/network/connection_impl.cc:278] [Tags: "ConnectionId":"29"] closing socket: 0
[2024-08-15 02:47:45.409][20][debug][pool] [source/common/conn_pool/conn_pool_base.cc:495] [Tags: "ConnectionId":"29"] client disconnected, failure reason:
[2024-08-15 02:47:45.409][20][debug][pool] [source/common/conn_pool/conn_pool_base.cc:215] [Tags: "ConnectionId":"29"] destroying stream: 0 remaining
[2024-08-15 02:47:45.409][20][debug][connection] [source/common/network/connection_impl.cc:146] [Tags: "ConnectionId":"28"] closing data_to_write=0 type=0
[2024-08-15 02:47:45.409][20][debug][connection] [source/common/network/connection_impl.cc:278] [Tags: "ConnectionId":"28"] closing socket: 1
[2024-08-15 02:47:45.409][20][debug][conn_handler] [source/common/listener_manager/active_stream_listener_base.cc:136] [Tags: "ConnectionId":"28"] adding to cleanup list
[2024-08-15 02:47:48.803][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:47:53.803][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:47:58.804][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:48:02.132][22][debug][filter] [source/common/tcp_proxy/tcp_proxy.cc:243] [Tags: "ConnectionId":"36"] new tcp proxy session
[2024-08-15 02:48:02.132][22][debug][filter] [source/common/tcp_proxy/tcp_proxy.cc:436] [Tags: "ConnectionId":"36"] Creating connection to cluster broker_1
[2024-08-15 02:48:02.132][22][debug][misc] [source/common/upstream/cluster_manager_impl.cc:2328] Allocating TCP conn pool
[2024-08-15 02:48:02.132][22][debug][pool] [source/common/conn_pool/conn_pool_base.cc:291] trying to create new connection
[2024-08-15 02:48:02.132][22][debug][pool] [source/common/conn_pool/conn_pool_base.cc:145] creating a new connection (connecting=0)
[2024-08-15 02:48:02.132][22][debug][connection] [source/common/network/connection_impl.cc:1009] [Tags: "ConnectionId":"37"] connecting to <IP_ADDRESS>:3093
[2024-08-15 02:48:02.132][22][debug][connection] [source/common/network/connection_impl.cc:1028] [Tags: "ConnectionId":"37"] connection in progress
[2024-08-15 02:48:02.132][22][debug][conn_handler] [source/common/listener_manager/active_tcp_listener.cc:160] [Tags: "ConnectionId":"36"] new connection from <IP_ADDRESS>:39220
[2024-08-15 02:48:02.169][22][debug][connection] [source/common/network/connection_impl.cc:746] [Tags: "ConnectionId":"37"] connected
[2024-08-15 02:48:02.254][22][debug][pool] [source/common/conn_pool/conn_pool_base.cc:328] [Tags: "ConnectionId":"37"] attaching to next stream
[2024-08-15 02:48:02.254][22][debug][pool] [source/common/conn_pool/conn_pool_base.cc:182] [Tags: "ConnectionId":"37"] creating stream
[2024-08-15 02:48:02.254][22][debug][router] [source/common/tcp_proxy/upstream.cc:255] Attached upstream connection [C37] to downstream connection [C36]
[2024-08-15 02:48:02.254][22][debug][filter] [source/common/tcp_proxy/tcp_proxy.cc:830] [Tags: "ConnectionId":"36"] TCP:onUpstreamEvent(), requestedServerName:
[2024-08-15 02:48:03.804][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:48:08.805][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:48:13.805][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:48:18.805][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:48:23.807][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:48:28.807][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:48:33.807][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:48:38.807][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:48:43.808][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:48:48.809][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:48:53.809][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:48:58.810][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:49:03.810][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:49:08.810][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:49:13.379][24][debug][connection] [source/common/network/connection_impl.cc:788] [Tags: "ConnectionId":"24"] write flush complete
[2024-08-15 02:49:13.379][24][debug][connection] [source/common/network/connection_impl.cc:278] [Tags: "ConnectionId":"24"] closing socket: 1
[2024-08-15 02:49:13.379][24][debug][connection] [source/common/network/connection_impl.cc:146] [Tags: "ConnectionId":"25"] closing data_to_write=0 type=1
[2024-08-15 02:49:13.379][24][debug][connection] [source/common/network/connection_impl.cc:278] [Tags: "ConnectionId":"25"] closing socket: 1
[2024-08-15 02:49:13.380][24][debug][connection] [source/extensions/transport_sockets/tls/ssl_socket.cc:368] [Tags: "ConnectionId":"25"] SSL shutdown: rc=1
[2024-08-15 02:49:13.380][24][debug][pool] [source/common/conn_pool/conn_pool_base.cc:495] [Tags: "ConnectionId":"25"] client disconnected, failure reason:
[2024-08-15 02:49:13.380][24][debug][pool] [source/common/conn_pool/conn_pool_base.cc:463] invoking 1 idle callback(s) - is_draining_for_deletion_=false
[2024-08-15 02:49:13.380][24][debug][pool] [source/common/conn_pool/conn_pool_base.cc:215] [Tags: "ConnectionId":"25"] destroying stream: 0 remaining
[2024-08-15 02:49:13.380][24][debug][pool] [source/common/conn_pool/conn_pool_base.cc:463] invoking 0 idle callback(s) - is_draining_for_deletion_=false
[2024-08-15 02:49:13.380][24][debug][conn_handler] [source/common/listener_manager/active_stream_listener_base.cc:136] [Tags: "ConnectionId":"24"] adding to cleanup list
[2024-08-15 02:49:13.810][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:49:18.810][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:49:23.811][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:49:27.206][24][debug][filter] [source/common/tcp_proxy/tcp_proxy.cc:243] [Tags: "ConnectionId":"38"] new tcp proxy session
[2024-08-15 02:49:27.206][24][debug][filter] [source/common/tcp_proxy/tcp_proxy.cc:436] [Tags: "ConnectionId":"38"] Creating connection to cluster broker_1
[2024-08-15 02:49:27.206][24][debug][misc] [source/common/upstream/cluster_manager_impl.cc:2328] Allocating TCP conn pool
[2024-08-15 02:49:27.206][24][debug][pool] [source/common/conn_pool/conn_pool_base.cc:291] trying to create new connection
[2024-08-15 02:49:27.206][24][debug][pool] [source/common/conn_pool/conn_pool_base.cc:145] creating a new connection (connecting=0)
[2024-08-15 02:49:27.206][24][debug][connection] [source/common/network/connection_impl.cc:1009] [Tags: "ConnectionId":"39"] connecting to <IP_ADDRESS>:3093
[2024-08-15 02:49:27.206][24][debug][connection] [source/common/network/connection_impl.cc:1028] [Tags: "ConnectionId":"39"] connection in progress
[2024-08-15 02:49:27.206][24][debug][conn_handler] [source/common/listener_manager/active_tcp_listener.cc:160] [Tags: "ConnectionId":"38"] new connection from <IP_ADDRESS>:33618
[2024-08-15 02:49:27.244][24][debug][connection] [source/common/network/connection_impl.cc:746] [Tags: "ConnectionId":"39"] connected
[2024-08-15 02:49:27.329][24][debug][pool] [source/common/conn_pool/conn_pool_base.cc:328] [Tags: "ConnectionId":"39"] attaching to next stream
[2024-08-15 02:49:27.329][24][debug][pool] [source/common/conn_pool/conn_pool_base.cc:182] [Tags: "ConnectionId":"39"] creating stream
[2024-08-15 02:49:27.329][24][debug][router] [source/common/tcp_proxy/upstream.cc:255] Attached upstream connection [C39] to downstream connection [C38]
[2024-08-15 02:49:27.329][24][debug][filter] [source/common/tcp_proxy/tcp_proxy.cc:830] [Tags: "ConnectionId":"38"] TCP:onUpstreamEvent(), requestedServerName:
[2024-08-15 02:49:28.811][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:49:33.811][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-08-15 02:49:38.812][1][debug][main] [source/server/server.cc:239] flushing stats

Another case

Envoy config Upstream and Downstream SSL, use SSL port 3093
Error, No subject alternative names matching IP address <IP_ADDRESS>(Envoy IP) found

Envoy config without SSL, use SSL port 3093
producer/consumer with SSL no work, without SSL have error

@adamkotwasinski Hi Adam. Could you help to give some suggestion? Thanks a lot

cc @mattklein123 @adamkotwasinski as codeowners

Could you also check if it works with Envoy and SSL enabled but without Kafka filter?
Get rid of envoy.filters.network.kafka_broker and configure the Kafka cluster accordingly (advertised.listeners etc.).
I want to eliminate the SSL side of this problem.
Otherwise, you'd need to debug the client side (for Java, reduce the log level and see what exceptions/failures are happening).

@LinG955 Great that you got some more info into that!
In general, Kafka broker filter does not know (and does not care) about the transport - if the payloads can be deserialized into Kafka objects, then they are going to be. Otherwise the data is just going to be ignored. If you didn't get anything from that log line it'd mean it's upper filters that are failing.
Unfortunately for SSL issues/configs I'm not the best person to ask (and that's why I initially asked to remove broker filter from the config). The comments about SAN config etc. sound reasonable as after all we're making the upstream clients connect somewhere else, so there could be something missing.


add gnome-keyring to store passwords and keys

This PR adds Gnome Keyring according to this discussion.
This is unable to manage ssh-keys yet, see: https://github.com/NixOS/nixpkgs/pull/310978

@lilyinstarlight Does that look good to you?


Fix pagination bootstrap compatibility.

Fix #189 .
Before (current flat-ui):

...

After (bootstrap default):

...

@designmodo do you prefer that I open a Pull Request without 'dist/' changes?

@StefanYohansson No, it's ok! Thank you, again, for your participation!


Failed to convert object detection model to tensorflow lite

Please go to Stack Overflow for help and support:
https://stackoverflow.com/questions/tagged/tensorflow
If you open a GitHub issue, here is our policy:

It must be a bug, a feature request, or a significant problem with documentation (for small docs fixes please send a PR instead).
The form below must be filled out.
It shouldn't be a TensorBoard issue. Those go here.

Here's why we have that policy: TensorFlow developers respond to issues. We want to focus on work that benefits the whole community, e.g., fixing bugs and adding features. Support only helps individuals. GitHub also notifies thousands of people when issues are filed. We want them to see you communicating an interesting problem, rather than being redirected to Stack Overflow.

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow):
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): 18.04
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device:
TensorFlow installed from (source or binary): from binary
TensorFlow version (use command below): 2.0
Python version: 3.69
Bazel version (if compiling from source):
GCC/Compiler version (if compiling from source):7.4.0
CUDA/cuDNN version: 10.2
GPU model and memory:
Exact command to reproduce:
frozen_model_path = '/home/tflite/tflite_graph.pb'
input_arrays=["image_tensor"]
output_arrays=["detection_boxes","detection_scores","num_detections","detection_classes"]
input_tensor={"image_tensor":[1,300,300,3]}
converter = tf.compat.v1.lite.TFLiteConverter.from_frozen_graph(frozen_model_path, input_arrays, output_arrays, input_tensor)
tflite_model = converter.convert()

You can collect some of this information using our environment capture script:
https://github.com/tensorflow/tensorflow/tree/master/tools/tf_env_collect.sh
You can obtain the TensorFlow version with:
python -c "import tensorflow as tf; print(tf.version.GIT_VERSION, tf.version.VERSION)"

Describe the problem
I am using object detection API. I have a pretrained ssd_mobilenet_v2_coco model and I want to convert it to tensorflow lite(and do float16 quantization). In the folder of the model, I have checkpoint, frozen_inference_graph.pb, model.ckpt.data-00000-of-00001, model.ckpt.index, model.ckpt.meta, pipeline.config, and saved_model folder.
I used the following command to get the tflite.pb
python3 export_tflite_ssd_graph.py --output_directory=/home/tflite --pipeline_config_path=/home/exportedLP1/pipeline.config --trained_checkpoint_prefix=/home/exportedLP1/model.ckpt
Then I used the method posted online to convert tflite.pb. The last step  tflite = converter.convert() caused "abort(core dumped)'. According to posts I read online, I should be able to convertt to tflite without any issue.
Source code / logs
Current thread 0x00007efbff6f1740 (most recent call first):
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/lite/toco/python/toco_from_protos.py", line 56 in execute
File "/usr/local/lib/python3.6/dist-packages/absl/app.py", line 250 in _run_main
File "/usr/local/lib/python3.6/dist-packages/absl/app.py", line 299 in run
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/platform/app.py", line 40 in run
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/lite/toco/python/toco_from_protos.py", line 93 in main
File "/usr/local/bin/toco_from_protos", line 8 in
Aborted (core dumped)

@valorusa
please share simple stand alone code for us to replicate the issue faced by you, with complete error logs.

Hi, all the codes are here:
frozen_model_path = '/home/tflite/tflite_graph.pb'
input_arrays=["image_tensor"]
output_arrays=["detection_boxes","detection_scores","num_detections","detection_classes"]
input_tensor={"image_tensor":[1,300,300,3]}
converter = tf.compat.v1.lite.TFLiteConverter.from_frozen_graph(frozen_model_path, input_arrays, output_arrays, input_tensor)
tflite_model = converter.convert()

@valorusa
i have replicated the code shared by you and face this error, as requested please share simple stand alone code such that we can replicate the error faced by you to analyse it.

Hi Thanks for the response. I attached the file /home/tflite/tflite_graph.pb here. Just unzip it to /home/tflite/
tflite_graph.zip

Hi Thanks for the response. I tried to attached the file /home/tflite/tflite_graph.pb here. But it is too large to attach it here. So I add the description here:
1.go to tensorflow object detection model zoo to download ssd_mobilenet_v2_coco. unzip it under current directory you will get home/your_usrname/ssd_mobilenet_v2_coco_2018_03_29/, and open the  pipeline.config file and comment out this line batch_norm_trainable: true
2.make a new directory under your user name: /home/your_usrname/tflite,
3.open the object detection api directory, go to ./models/research/object_detection
4.then in the terminal, run  python3 export_tflite_ssd_graph.py  --output_directory=/home/your_usrname/tflite --pipeline_config_path=/home/your_usrname/ssd_mobilenet_v2_coco_2018_03_29/pipeline.config --trained_checkpoint_prefix=/home/your_username/ssd_mobilenet_v2_coco_2018_03_29/model.ckpt
5.now you should be able to find tflite_graph.pb and tflite_graph.pbtxt in home/your_usrname/tflite/
6.in python, run the following codes:
frozen_model_path = '/home/your_usrname/tflite/tflite_graph.pb'
input_arrays=["image_tensor"]
output_arrays=["detection_boxes","detection_scores","num_detections","detection_classes"]
input_tensor={"image_tensor":[1,300,300,3]}
converter = tf.compat.v1.lite.TFLiteConverter.from_frozen_graph(frozen_model_path, input_arrays, output_arrays, input_tensor)
tflite_model = converter.convert()

Hello @valorusa I found your description very useful. Just one question:
Where did you find information about input_arrays and output_arrays?
I successfully converted ssd_mobilenet_v2_coco_2018_03_29 but I am not able to convert a custom model because I don't know input and output arrays.
Thanks

Hi @Chris1hub I use the tensor flow object detection api to do the object detection and recognition. The input arrays are the names of the input tensors and the output arrays are the names of the output tensors. They are used in the below  inference routine:
with session.graph.as_default():
image_tensor = detect_graph.get_tensor_by_name('image_tensor:0')
    # Each box represents a part of the image where a particular object was detected.
    boxes = detect_graph.get_tensor_by_name('detection_boxes:0')

    # Each score represent how level of confidence for each of the objects.
    # Score is shown on the result image, together with the class label.
    scores = detect_graph.get_tensor_by_name('detection_scores:0')
    classes = detect_graph.get_tensor_by_name('detection_classes:0')
    num_detections = detect_graph.get_tensor_by_name('num_detections:0')

    # Actual detection.
    (boxes, scores, classes, num_detections) = session.run([boxes, scores, classes, num_detections],feed_dict={image_tensor: image_np_expanded})
    return (boxes, scores, classes, num_detections)

@valorusa Could you please try on latest stable version of tf 2.5 or 2.4.1 and let us know if this is still an issue.Thanks!


Stage configuration arguments are parsed as Quantity

When a configuration file of a pipeline is parsed, stage initialization arguments not meant for optimization are parsed as quantities with units, even when only a simple number is given. When a config file contains seed = 42, it currently gets Quantity(42.0, dimensionless) as an argument into its __init__ function, when it should get the number 42 as an integer.

Fixed with PR #609 / commit 8b6d88627e46519b1431effe4626eb0ac0085678


[type:feat]Add dynamically scale admin module

Make sure that:

[x] You have read the contribution guidelines.
[x] You submit test cases (unit or integration tests) that back your changes.
[x] Your local test passed ./mvnw clean install -Dmaven.javadoc.skip=true.

not delete icon

not delete icon

done.

you should add sql in upgrade script, for example, 2.6.1-2.7.0-mysql.sql, 2.6.1-2.7.0-pg.sql, and so on...

you should add sql in upgrade script, for example, 2.6.1-2.7.0-mysql.sql, 2.6.1-2.7.0-pg.sql, and so on...

done.

@Terminator111 check ci pls


DCMAW-8964: Adding logger to STS mock

DCMAW-8964
What changed
Added logger to STS mock
Why did it change
Previously the STS mock was importing logger from backend-api and causing build failures
Evidence of stack now deploying successfully
Gathering...
Checklists

[ ] There is a ticket raised for this PR that is present in the branch name
[ ] No PII data logged. See guidance here
[ ] Demo to a BA, TA, and the team.
[x] Update README with any new instructions or tasks

Did we want to also apply these changes in the same PR? Was showing as an error previously.
https://github.com/govuk-one-login/mobile-id-check-async/pull/108/files

Good catch - I approved based on this being deployed and tested locally but that's still outstanding. @jamesamooney ^^


Suppress list of definitions in inline code

Hi, Thanks for the awesome stylesheet!
When using doxygen-awesome.css, I get a list of all definitions of variables used in a function in inline code (those which are already hyperlinked in the code fragment). I don't get this list with the default output. Setting SOURCE_BROWSER to NO doesn't suppress it. How can I suppress it?
See the issue after line 15 within the code fragment.

Specifying the style sheet in HTML_EXTRA_STYLESHEET rather than HTML_STYLESHEET solved the problem!


The difference of implementation

Hi. Thanks for sharing codes. Your work is excellent and significant for inspiring others.
However, I tried to reproduce the performance of your method, Pari-Conv,  and only got 92.0% twice on MoldelNet40.
And the codes of Pari-conv seem to be different from that provided in the paper. The dynamic kernel shall be a kCC matrix and operated with matrix multiplication. But in the model.py, it's k*C and used with Hadamard product.
Is it the reason for low performance？ Please check Lines 107-130 in model.py.

Thanks for liking our work! Your words inspire us too!
Let me clarify the implementation first. It is actually the same as the description in the paper. But we do a little trick for simplicity.
Our dynamic kernel is factorized into a diagonal matrix and a basis matrix. We implementat the basis matrix as a linear layer self.basis_matrix(), and the diagonal matrix as a vector self.dynamic_kernel(APPF).
Note that A matrix multipilcation with a diagonal matrix equals to the Hadamard product with the diagonal vector. So they are actually the same.
For the performance, the current version has some differeces with the initial version in the pre-processing steps. I'll check the code and update it in a few days.


Search by Role should be exact match

Role should be exact match rather than partial match

Handled since UserRole is mapped to role column


RFC: repo status per build step

Summary
This is a proposal to add the ability for every step in a Drone pipeline to be able to create a repo status. Currently only 1 "global" status is created for the entire pipeline (e.g. all steps together).
I feel this could be extremely valuable combined with new ability to allow for a single step not to fail the pipeline (#2071).
Goals
Here are some goals I envision.  There are details in each goal that could be tweaked, for example naming of things.

each status would update immediately after a step finishes (not waiting for an entire group to finish for example)
each status would link directly to the logs of that step inside the build
would be "opt-in", meaning a step would not create a status by default but can be configured to do so from .drone.yml
allow to customize context for a step, defaults to continuous-integration/drone/<step-name>

Example Pipeline
pipeline:
  # Does not create a status
  restore-cache:
    image: plugins/drone-s3-cache

  # Creates a status with default context
  lint:
    image: node
    group: test
    status: true
    commands:
      - lint command

  # Creates a status with default context
  test:
    image: node
    group: test
    status: true
    commands:
      - test command

  # Does not create a status
  rebuild-cache:
    image: plugins/drone-s3-cache

  # Create a status with custom context
  build:
    image: plugins/docker
    status: continuous-delivery/docker

  # Create a status with custom context
  deploy-test:
    group: deploy
    status: continuous-delivery/k8s/test

  # Create a status with custom context
  deploy-master:
    group: deploy
    status: continuous-delivery/k8s/master

Now there are two of this plugins, you should also submit your plugin to the registry ;)

I also wonder how this feature relates to the Checks API. If we publish every step to the checks API would this still be necessary? I feel like there might be some feature overlap, although I'm not sure yet :)

If the checks API is supported, several status against the same commit will likely become an obsolete request/need/requirement.
But until checks support is missing, status it is :)

I feel like if there was a plugins/status that this issue would solve itself. We could think of it like the git plugin which is implicitly added. If you want to control how status gets reported then you'd just put it at different parts of the pipeline. Or have a status per pipeline.
Honestly in general it would be nice if there was a way to have drone/scm run as a service that pipelines can then interact with. This would open up a lot of different things like checks. So you could make a plugin that would run a linter and then pick up the results and report that back.

Would support for GitHub Checks supersede this request?  With GitHub checks you can report the individual step status directly in the check details section. Example pictured below:

I believe checks would supersede this.


Update continuous-delivery-faqs.md

Added 3 Faq
Thanks for contributing to the Harness Developer Hub! Our code owners will review your submission.
Description

Please describe your changes: \3 Faq Added

PR lifecycle
We aim to merge PRs within one week or less, but delays happen sometimes.
If your PR is open longer than two weeks without any human activity, please tag a code owner in a comment.
PRs must meet these requirements to be merged:

[ ] Successful preview build.
[ ] Code owner review.
[ ] No merge conflicts.
[ ] Release notes/new features docs: Feature/version released to at least one prod environment.

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://663cf3d5b3295f10c8f4054e--harness-developer.netlify.app


Hyperlink to file throws exception

When setting a hyperlink to filename an exception is thrown "Hyperlink is invalid.".
Powerpoint should be able to link to other presentations.
Code below throws the exception.
using var sourcePresentation = SCPresentation.Open("SourceTemplate.pptx");
if (File.Exists("NewPresentation.pptx"))
{
File.Delete("NewPresentation.pptx");
}
sourcePresentation.SaveAs("NewPresentation.pptx");
//create four shapes with a long text in each
var slide = sourcePresentation.Slides[0];
for (int i = 0; i < 4; i++)
{
var shape = slide.Shapes.AddRectangle(150, 100 + 100 * i, 500, 50);
shape.TextFrame.Text = "This is a long text #" + i;
shape.TextFrame.Paragraphs[0].Portions[0].Hyperlink = "SourceTemplate.pptx";

}
sourcePresentation.Save();

Thank you for the bug report. Please check this fix v0.45.3-beta.1


Add support for legacy compat runtime for vm:true

Uses gcr.io/google-appengine/java-compat:latest for compat deployments when the user specifies vm:true in their app.yaml file.

Closing. We'll hold off on adding support for vm:true, as it's deprecated and will be decommissioned soon anyway. See also https://cloud.google.com/appengine/docs/flexible/java/upgrading


Suggestion: Ctrl + F4 to Close Window

I'm very used to this shortcut in Visual Studio. I've made the customization myself, but I would guess it would be nice to have it as a default for this extension.

This issue was moved to Microsoft/vscode-vs-keybindings#1


Use correct name for per-uri policy

Seems like the name for the policy should be different, otherwise it overwrites the per-site policy.

@vhorne
Can you review the proposed changes?
NOTE: the listed 'author' value seems to be incorrect. can you update it?
Important: When the changes are ready for publication, adding a #sign-off comment is the best way to signal that the PR is ready for the review team to merge.
#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team

#sign-off

@Jak-MS go ahead and merge


enhanced Runtime.wrapReflectFunc(return Promise for js)

I have used goja deeply, and the biggest benefit it gives me is that Runtime.wrapReflectFunc can easily call go code (for example, calling the go standard library with js has brought me great convenience).
However, wrapReflectFunc cannot be customized and does not provide too many options, which caused me to encounter a lot of embarrassment when calling go code with js, mainly the following three problems:

When the last return value error of go is not nil, goja will throw an exception for js. This prevents me from calling functions like errors.New etc. In addition, when the error of the io.Writer interface is not nil, the first return value n int may return the number of bytes written, and the exception thrown by goja prevents me from getting n.
The int64/uint64 returned by some go functions exceeds the maximum integer of js and will lose precision, which makes me unable to use some go functions normally. Of course, you can make additional adaptations for these special functions, but this is obviously not as convenient as using wrapReflectFunc, especially Adapting individual member functions of struct is awkward.
wrapReflectFunc will block the call, which causes the world of js to pause when calling time-consuming go functions, and since goja has built-in support for Promise, it should be a matter of course for wrapReflectFunc to support starting a new goroutine execution function and return Promise for js.

About two years ago, I submitted a piece of code. At that time, the plan was to allow hook wrapReflectFunc so that users could customize some wrapReflectFunc behaviors but it was not merged. Now this code is a new solution I thought of, which is simpler than before and will not cause much change to the existing code of goja, which will not affect efficiency.
When js calls the go function through wrapReflectFunc, an optional tag can be passed in. If no tag is passed in, goja will work according to the previous code. If it is passed in, it will work according to the tag and the parameter position of the go function will be moved backward in turn.
This will hardly cause any compatibility issues, nor will it have any impact on efficiency, but it can help people who need to solve the above three problems. I hope you can consider merging this code, even if you don't, please consider other official solutions to the above problems.
This code mainly adds three tags:

GoRaw Set this flag wrapReflectFunc will not throw an exception because of the error return value but will return error as the return value to js.
// will throw errr
errors.New("test")

// will get return error, and not throw any
const e = errors.New(GoRaw , "test")

GoNumber Setting this flag wrapReflectFunc will convert the js return value int64->goja.Int64 uint64->goja.Uint64 []int64->[]goja.Int64 []uint64->[]goja.Uint64 , This guarantees no loss of precision for int64 uint64 (also defines common math functions for Int64 Uint64 ).
// Returning number may lose precision
strconv.ParseInt(s, 10, 64)

// Returning object, the implementation is a goja.Int64 without loss of precision
strconv.ParseInt(GoNumber, s, 10, 64)

GoAsync Set this flag to execute the function in a new goroutine, js will immediately return a Promise. (You need to call the Runtime.SetRunOnLoop function to set the RunOnLoop function provided by the event loop to the Runtime, because Promise must rely on the event loop)
// Blocking calls, all code in js will stop
time.Sleep(time.Second)

// This will be called asynchronously and return a Promise immediately without blocking, and will be notified by the Promise when the function call completes
time.Sleep(time.Second).then(()=>{
  // go function call completed
})

In addition, several tags are defined for js, which are combinations of the above three tags, the full markup is as follows:

GoRaw
GoNumber
GoRawNumber
GoAsync
GoAsyncRaw
GoAsyncNumber
GoAsyncRawNumber

@zuiwuchang hi, any news about this pr ? pretty interested about it


Tests not passing?

Shouldn't the tests pass?
OSX Chrome
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_2) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.120 Safari/535.2
Tests completed in 46 milliseconds.
5 tests of 12 passed, 7 failed.

I obtain the same results on Linux Chrome x64 15.0.874.121

This is due to Chrome not setting cookie while path is not set.
This can be fixed by changing jquery.cookie.js at line 31:
            options.path ? '; path=' + options.path : '',

by
            '; path=' + (options.path ? options.path : ''),

You can change test.js accordingly:
at Line 46:
    equals($.cookie('c', 'v'), 'c=v', 'should return written cookie string');

by
    equals($.cookie('c', 'v'), 'c=v; path=', 'should return written cookie string');

at line 51:
        ' v', 'should not encode');

by
        ' v; path', 'should not encode');

Did you run the tests off a server? For me it all works fine (Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_2) AppleWebKit/535.7 (KHTML, like Gecko) Chrome/16.0.912.75 Safari/535.7).

I confirm this is now working on Mozilla/5.0 (X11; Linux i686) AppleWebKit/535.7 (KHTML, like Gecko) Chrome/16.0.912.75 Safari/535.7
It might be a bug that has been fixed in Google Chrome.


Consider to use EU vocabularities for version 1.0 standard

Map all references to EU vocabularities https://op.europa.eu/en/web/eu-vocabularies/home
It should affect:

countries
languages
owner_type
owner_link
topics

And probably: content_type too

Done for topics


Deployment progress bar not in sync with resources list

When a new service instance is created, the deployment progress bar shows the message not resources available yet. When resources become available, as can be seen from the resources tab, the deployment progress bar keeps showing the no resources available yet message instead of showing the actual progress.
bug-deployment-progress-bar.webm

This is an issue in the backend not the frontend. See https://github.com/inmanta/inmanta-core/issues/4864


fixed feedback component

Fixes issue 1303

This partially fixes issue 1303.
It fixes it for all pages except the main page.


testing hound config.

@mcritchlow Great! Do we want to try .jshintrc configuration? Hound uses JSHint to review JavaScript code.

Changes Unknown when pulling 989f039a3daad082f6415cb6bbb4a149236b4049 on feature/code_review_tools into ** on develop**.

Changes Unknown when pulling f4a36b610f997d4bc03a71e5558dbb13ff35cb72 on feature/code_review_tools into ** on develop**.


unauthorized error

[x] It is better to jump to the login page automatically instead of popping up an unauthorized error

Browser: Chrome Version 69.0.3497.100 (Official Build) (64-bit)
OS: MacOS 10.13.5

When api get unauthorized error, page will redirect to login.


[XPU][OptEW] Allow use-def graphs of elementwise optimizable operations

Allow operands being used by other optimizable operations to enable
elementwise operations graph optimizations.
Depends on https://github.com/intel/intel-xpu-backend-for-triton/pull/2670. Do not merge.
Review only 2a4ce1843bbf1f179ae7a2781c50b9199ec2530f

@victor-eds this is now ready for rebasing.

Nor merging. This pass can be dropped as previous passes should be propagating optimal layouts to EW operations.


NAT-Gateway sku tier: Regional

Describe the noise
When running what-if on defined infrastructure, the NAT Gateway wants to change the SKU Tier to regional.
This is not possible to set as a property on the resource.
Resource type (i.e. Microsoft.Storage/storageAccounts)
Microsoft.Network/natGateways/
apiVersion (i.e. 2019-04-01)
2021-05-01
Client (PowerShell, Azure CLI, or API)
Bicep
Relevant ARM Template code (we only need the resource object for the above resourceType and apiVersion, but if it's easier you can include the entire template
resource natGateway 'Microsoft.Network/natGateways@2021-05-01' = {   name: 'ng-${projectName}-${environment}'   location: location   tags: tags   sku: {     name: 'Standard'   }   properties: {     publicIpAddresses: [       {         id: natPIP.id       }     ]   } }
Expected response (i.e. "I expected no noise since the template has not been modified since the resources were deployed)
I expected no noise since the template has not been modified since the resources were deployed
Current (noisy) response (either include a screenshot of the what-if output, or copy/paste the text)
~ Microsoft.Network/natGateways/ng-xxxx-prod [2021-05-01]
- sku.tier: "Regional"
Additional context
Add any other context about the problem here.

Still present in API verison 2022-07-01. Any updates? @alex-frankel

Also present in API version 2022-11-01

This is a duplicate of #262 @alex-frankel


Bug: Connecting to Coinbase Wallet is going to Safari

Describe the bug
https://www.loom.com/share/c99358a4a6cd4d0f92996eae9b2c7c15
Expected behavior
No response
Steps to reproduce the bug
Click to connect Coinbase Wallet Account
Attempt to connect in app
Observe converse.xyz website is opened instead

This is fixed


Apple Mach-O Linker (Id) Error Linker command failed with exit code 1

Ld /Users/---/Library/Developer/Xcode/DerivedData/---/Build/Intermediates/---.build/Release-iphoneos/snowflakeTests.build/Objects-normal/arm64/snowflakeTests normal arm64
cd /Volumes/LaCie/---/snowflake/ios
export IPHONEOS_DEPLOYMENT_TARGET=8.2
export PATH=---
ld: file not found: /Users/---/Library/Developer/Xcode/DerivedData/---/Build/Products/Release-iphoneos/snowflake.app/snowflake
clang: error: linker command failed with exit code 1 (use -v to see invocation)
I have gone through like 10 different build fail errors and fixed all but this one. How do I build snowflake from xcode successfully to run on a connected iphone? Seems like maybe something simple I'm missing but I have no idea.

You might want to consider doing this through the Apple process supported by Fastlane which is working in Snowflake.
You would have to install and setup Fastlane for your Mac which would get your Apple ID and Bundler ID.  I assume you have a Developers Account w/ Apple.
Then you would use the Match to establish your Certs on a private GitHub repository.
Look at the fastlane/Fastfile and you will see the following section which is the fastlane ios beta:
lane :beta do
    match(type: "appstore") # more information: https://codesigning.guide
    gym(scheme: "snowflake",
        project: "./ios/snowflake.xcodeproj"
        ) # Build your app - more options available
    pilot

    # sh "your_script.sh"
    # You can also use other beta testing services here (run `fastlane actions`)
  end

When you run fastlane ios beta the above code will pull in your certs from your private repo w/ match, build the app with gym and deploy it to Testflight via pilot.
I had a lot of trouble trying to do all this manually and when I converted it to Fastlane it went smooth as butter.
You will then use iTunesConnect and then within My Apps you will see your app ready for install either for internal or external users.
I realize it's a little overwhelming, at least for me, to start using the Apple eco system for installing an app but believe me, once you get the Fastlane script working, it's a fantastic time saver.

I will give it a shot thanks @bartonhammond


Fix for vulnerability https://security.snyk.io/vuln/SNYK-JAVA-COMSQUAREUPOKHTTP3-2958044

The link is broken, due to uppercase letter.

This fix https://security.snyk.io/vuln/SNYK-JAVA-COMSQUAREUPOKHTTP3-2958044


Add support for HTTPS proxies

The net/http package in Go supports the use of the HTTPS_PROXY environment variable to specify the URL of the proxy server. e.g. https://proxy.mydomain.com:8080. This exposes an otel.https_proxy_url value that users can use to set the environment variable.

We should really make it so that manifest.yaml just doesn't have checksums. That entire mechanism of deploying is going away, and it's tedious and unnecessary to have to update them.

I'll squash merge and clean up the commit message. Where is the :shame: emoji.


Purge use of old _lazy_array functionality.

Closes #2706

If you say "Fixes #2706" somewhere in the PR description, I believe it automatically closes that issue when this PR get merged.
ALthough, maybe "closes #2706" works too?

Looking good @bjlittle !

@lbdreyer See https://help.github.com/articles/closing-issues-using-keywords/

Although it may need to be in a comment rather than the description ...


Update domains-faq.yml

The inheritance of the domain in example is incorrect, subdomains www.contoso.com, www.partners.contoso.com till here the inheritance is correct, but for this domain its grand parent domain www.partners.marketing.contoso.com, so it should be www.marketing.partners.contoso.com

@JohanFreelancer9 - kindly do the copy editor review for this PR. Thanks


Swapping locales feature

added LocaleButton to intro and menu
tested and fixed switching on the fly for wonder views, intro, timeline, artifacts, and menu
added LocaleLogic.isEnglish helper, swapped to use in _results_grid and _scrolling_content

https://user-images.githubusercontent.com/932757/188753700-82d4b52d-1210-4f23-902f-cf42770f7530.MP4

~stubbed in placeholder button for testing locale switching (see demo below)~
~TODO:~

~needs design polish and direction on whether the button is available at all times or just in menu~
~there is a cast exception on startup, probably due to the strings being used in the button in app scaffold (before the bootstrap has chance to populate $strings property).  🤔 Quick fix could be to just use "简体中文" and "English" in the Text? Might be a non-issue if the swap button is moved to the menu only (instead of being globally available in app scaffold)~
~https://user-images.githubusercontent.com/932757/188741589-66c99de8-969c-47cc-abb5-2fe49d71dd16.png~
~https://user-images.githubusercontent.com/932757/188741600-208e65f6-849a-431b-b0cc-a0ead0dd221b.png~

~Demo:~
~https://user-images.githubusercontent.com/932757/188741671-54bf1d47-93f6-4212-a30f-9de1c843301e.mp4~

Why change the locale with the button, and if you need to add 2-3 more languages, will need to constantly switch?) Why not depend on the phone's locale?

It does depend on the users locale, but this would give them the option to switch it (override).

okay, but in this case, we need to have the opportunity to return to the locale language. If I speak Spanish and accidentally press the button I cant return from English to Spanish; yes, know the application doesnt have Spanish locale, but I want to add it soon.

For example - add manual language selection in the application settings (add a corresponding item in the menu on the main screen) with the selection through some picker or list, rather than switching between two languages.

Yes, if we ended up having more than 2 languages we would move to some sort of pop-up list. Since we have only 2 now, we're just going with a simple toggle.


[MINOR][INFRA] Upgrade Jira client to 2.0.0

What changes were proposed in this pull request?
SPARK-10498 added the initial Jira client requirement with 1.0.3 five year ago (2016 January). As of today, it causes dev/merge_spark_pr.py failure with Python 3.9.4 due to this old dependency. This PR aims to upgrade it to the latest version, 2.0.0. The latest version is also a little old (2018 July).

https://pypi.org/project/jira/#history

Why are the changes needed?
Jira==2.0.0 works well with both Python 3.8/3.9 while Jira==1.0.3 fails with Python 3.9.
BEFORE
$ pyenv global 3.9.4
$ pip freeze | grep jira
jira==1.0.3
$ dev/merge_spark_pr.py
Traceback (most recent call last):
  File "/Users/dongjoon/APACHE/spark-merge/dev/merge_spark_pr.py", line 39, in <module>
    import jira.client
  File "/Users/dongjoon/.pyenv/versions/3.9.4/lib/python3.9/site-packages/jira/__init__.py", line 5, in <module>
    from .config import get_jira
  File "/Users/dongjoon/.pyenv/versions/3.9.4/lib/python3.9/site-packages/jira/config.py", line 17, in <module>
    from .client import JIRA
  File "/Users/dongjoon/.pyenv/versions/3.9.4/lib/python3.9/site-packages/jira/client.py", line 165
    validate=False, get_server_info=True, async=False, logging=True, max_retries=3):
                                          ^
SyntaxError: invalid syntax

AFTER
$ pip install jira==2.0.0
$ dev/merge_spark_pr.py
git rev-parse --abbrev-ref HEAD
Which pull request would you like to merge? (e.g. 34):

Does this PR introduce any user-facing change?
No. This is a committer-only script.
How was this patch tested?
Manually.

Test build #137513 has started for PR 32215 at commit 13d5c8a.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42087/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42087/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42087/

Test build #137513 has finished for PR 32215 at commit 13d5c8a.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137513/

Merged to master

Thank you, @HyukjinKwon !


Patch to support UNECE Buy Ship Pay  and UNECE Recommendation 21 Contexts

Patch to include support of UNECE Buy Ship Pay  and UNECE Recommendation 21 Contexts
https://vocabulary.uncefact.org/unece-context.jsonld
https://vocabulary.uncefact.org/rec21.jsonld
0001-Added-support-for-UNECE-BSP-and-UNECE-REC21-contexts.patch
0002-Missed-out-updates-to-context-update-scrript.patch

Apologies for the delay. I am going to close this issue and would recommend using a ContextLoader to add additional context files with the with_context_map_from method.


Calendar dates jump around on changing date in Safari

Safari 10.0.2, MacOS 10.12.2

This is specifically related to the .c-datepicker__day--selected::after styles.
Removing the content stops the jumping around. The .u-hover-ball-effect::before style, which is similar, doesn't have this effect. 🤔

Also related - the rome <td> elements have smaller height in Safari than in other browsers (31 vs 35px).

I can no longer replicate in 10.0.3.


DateValidator should check if date is already in DateValidator::$timestampAttributeFormat

I have form field in custom date format that is converted to mysql format:
['some_date', 'date', 'format' => 'dd.MM.yyyy', 'timestampAttribute' => 'some_date', 'timestampAttributeFormat' => 'php:Y-m-d']

Saving with form works great, but if I need to update some other field without using form and try to validate, it gives me an invalid format error, because some_date is not converted to dd.MM.yyyy format and is still in php:Y-m-d. So attributes in DateValidator::$timestampAttributeFormat format should also be valid.

Such request is not implemenatble: if DateValidator will fill timestampAttribute only if it is not set, it will be unable to update its value anyhow.
You should either use a scenarios to separate validation of user input and internal one or create a separated field some_date_source, which should be used in user form and care the DateValidator.

Maybe I have completely missed the point of timestampAttributeFormat, but I see it as a way to ensure date fields are always in valid format for db (php:Y-m-d). So if user submits form with date in dd.MM.yyyy format, it gets converted and all is good. If date is loaded from db and is already in valid php:Y-m-d format, then it should not give validation errors. So it seems logical that DateValidator uses both format and timestampAttributeFormat for validation.

This issue may be also fixed if validation is triggered only on dirty attributes #8610

I suppose this could be implemented.

Thanks :thumbsup:

Resolved by commit 779b1e90cea13ca3a05768a8e82b96b022d1f710


Create Polkadot-Api-Cli-Gui.md

Project Abstract
Developing user-friendly graphical interface for the Polkadot Api CLI
Grant level

[ ] Level 1:  Up to $10,000, 2 approvals
[X] Level 2:  Up to $30,000, 3 approvals
[ ] Level 3:  Unlimited, 5 approvals (for >$100k: Web3 Foundation Council approval)

Application Checklist

[x] The application template has been copied and aptly renamed (project_name.md).
[X] I have read the application guidelines.
[X] Payment details have been provided (bank details via email or Polkadot (USDC & USDT) address in the application).
[X] I am aware that, in order to receive a grant, I (and the entity I represent) have to successfully complete a KYC/KYB check.
[X] The software delivered for this grant will be released under an open-source license specified in the application.
[X] The initial PR contains only one commit (squash and force-push if needed).
[X] The grant will only be announced once the first milestone has been accepted (see the announcement guidelines).
[ ] I prefer the discussion of this application to take place in a private Element/Matrix channel. My username is: @_______:matrix.org (change the homeserver if you use a different one)

Thanks for your thorough answers @tolgayayci though I like the concept, I also have some concerns:

For this project, we request that this GUI application's link will be added to the relevant pages such as api-cli and tools in Polkadot wiki etc. Cli users will want to use it when they see such an easy-to-use alternative. We also tried to use cli, but as we mentioned in the proposal, the learning curve is high if you do not have deep experience with it before.

The foundation tries to stay objective when it comes to projects, so unfortunately, we wouldn't be able to add your tools to our web pages. We also can't provide guidance to teams to add themselves to the site. The grants program is here to bootstrap projects, but we always hope to see that projects can eventually develop a community organically. The grants program also shouldn't be the sole source of income for the project.

Grant applications are required to write only development, so we did not talk much about maintenance, but we guarantee to fix the project if there is any breaking change for a year. Because if the polkadot runtime changes, the API may also change, so some modules and methods may not work.

I would disagree with this statement, because, as our guidelines state, successful applicants are required to write documentation as well. And while maintenance may not in the scope of the grant itself, we still ask teams to somehow demonstrate that the project will be maintained after the completion of the grant. I would have to agree that it doesn't appear this is the case with Awesome Polka or PolkaFlow.

Thanks @tolgayayci for your reply. While I agree that the PolkaFlow site seems to be more active, the dashboard doesn't seem to be open-source, as there is nothing in the repo. This makes it a lot harder for us to judge the value of the project, and also personally makes me worry that you might not keep this cli tool published either.
Generally we look at things like commit activity, number of contributors, community forks, stars, and insights such as code frequency. We also look at external factors, such as your team's activity in forum discussions, and community engagement with the product.
To answer your question, a great example of a level 1 grant that has continued to be maintained afterwards might be ParaSpell. By taking a look at their code frequency I can see the activity from when the grant started. They had a couple of level 1 grants, and then subsequently went for Treasury funding once the community started using the project.
We understand that not every project will necessarily be successful, but we try to do our due diligence in ensuring that it can become successful. This is why maintenance is important. At this point, I'm not yet convinced about this project and therefore am not going to approve it. But I appreciate the time and effort you've put into this application.

Thanks @justmert for your response. I did go to https://flowana.app/polkadot/projects and when you scroll down and click on "GitHub" that's the repo it links to. This is why I thought it was the official repo. Apologies for the mistake. In that case though, I would ask why the website is hosted in your personal repos instead of the official one for YK Labs.
I checked out both the repos above but the back-end repo has no .readme file and the front-end repo .readme is still the Next.js template text. I appreciate that they are active but I'm still not seeing much evidence of user adoption or dev engagement here. Without more information, I'm afraid this doesn't really change my stance.

@tolgayayci, sorry for the long wait. Based on the price and scope of this proposal as well as previous deliveries, I am not going to support this application. I am also skeptical about the rest of the committee being convinced, but in any case I will bring your application to their attention on Wednesday. If there are follow-up questions then, we can discuss them here or schedule a call.

@tolgayayci thanks for all your hard work and efforts regarding the application. Unfortunately, it wasn't able to garner enough approvals from the committee. Therefore I'm going to go ahead and close it, but we thank you for your interest in the W3F grants program, and wish you the best of luck in finding funding.


Use LIST instead of MLSD as command with FTPES security

My server doesn't allow MSLD command is there a way to use standard LIST command?

yes, you can change it to LIST as is mentioned in the test file.
  _ftpConnect.listCommand = ListCommand.LIST;


Allow using local file:// locations with find_links

Pip lets you use a local file:// url in --find-links, e.g.:
$ .venv/bin/pip download six
Collecting six
  Downloading six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB)
Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Saved ./six-1.16.0-py2.py3-none-any.whl
Successfully downloaded six
$ .venv/bin/pip install --no-index --find-links file:///tmp/six-1.16.0-py2.py3-none-any.whl six
Looking in links: file:///tmp/six-1.16.0-py2.py3-none-any.whl
Processing /tmp/six-1.16.0-py2.py3-none-any.whl
Installing collected packages: six
Successfully installed six-1.16.0

But if you try that in unearth, it's not allowed:
>>> from unearth import PackageFinder
>>> finder = PackageFinder(index_urls=[], find_links=["file:///tmp/six-1.16.0-py2.py3-none-any.whl"])
>>> finder.find_matches("six")
<unearth.utils.LazySequence at 0x105aab950>
>>> list(_)
Failed to collect links from file:///tmp/six-1.16.0-py2.py3-none-any.whl: NotHTTP: the file looks like an archive but its content-type cannot be checked by a HEAD request.
[]

I think we shouldn't need to check local files by HEAD request, we can just check if the file exists. Thoughts?

unearth is to find package links, if you already have the link, why you need to use unearth then?

A few reasons:

To supplement index data with some additional local files (e.g. patched versions), it is non-trivial to find the best match for a requirement from a union of index and files.
Disabling access to PyPI index during tests, but whitelisting a subset of packages using local files.
The landing page says “Unearth serves as a stable replacement for pip's PackageFinder API”. With only a subset of features it can not be a drop-in replacement for pip subprocesses.

My use case was investigating replacing pip subprocesses scraping (slow, fragile) in https://github.com/wimglenn/johnnydep with unearth package finders.


bootstrap config transparently switches between v2 and v3 leading to unexpected rejected configs

Certain changes to a bootstrap file seem to trigger Envoy to process other parts of the bootstrap differently (v2 vs v3).
Starting with this config file:
envoy-sds.yaml.txt
Validation passes
Adding a bit of config:
dynamic_resources:
  lds_config:
    api_type: GRPC
    grpc_services:
      envoy_grpc:
        cluster_name: foo

Causes an unrelated part of the config to fail:
[2020-05-08 20:35:17.380][1][critical][main] [source/server/server.cc:98] error initializing configuration '/config/envoy-sds-lds.yaml': Protobuf message (type envoy.config.bootstrap.v3.Bootstrap reason INVALID_ARGUMENT:(static_resources.clusters[1]) hosts: Cannot find field.) has unknown fields
[2020-05-08 20:35:17.380][1][info][main] [source/server/server.cc:673] exiting
Protobuf message (type envoy.config.bootstrap.v3.Bootstrap reason INVALID_ARGUMENT:(static_resources.clusters[1]) hosts: Cannot find field.) has unknown fields

Full broken config file:
envoy-sds-lds.yaml.txt
Configs are run with docker run -v $HOME/kube/local:/config -p 15000:15000 envoyproxy/envoy-dev -c /config/envoy-sds-lds.yaml
envoy  version: 49efb9841a58ebdc43a666f55c445911c8e4181c/1.15.0-dev/Clean/RELEASE/BoringSSL

Just adding
dynamic_resources:
  lds_config:

is fine so must be something in ConfigSource triggering it here. I have seen this before though, so it is not specific to this field as far as I know

I'm pretty sure this is a parsing error and the issue is the config has been upgraded to v3 before the error happens. I will defer to @htuch though.

After playing with this a bit more I realized:
dynamic_resources:
  lds_config:
    api_type: GRPC

is not actually valid config in either version. So it seems like (without any understanding of the code) an error in the config causes upgrade to v3, which then gets a different error maybe?

Yeah, Envoy does this transparent upgrade by default, see https://www.envoyproxy.io/docs/envoy/latest/operations/cli#cmdoption-bootstrap-version for how to set this explicitly.
Does this resolve the issue?

Yeah the flag is useful. The UX here isn't great because I am randomly presented with a new error that was previously valid because I made a mistake in the config. However if we did something like displaying the v2 and v3 errors, then for v3 config users they would always see v2 errors probably, so it wouldn't make too much sense I think. Seems like the flag is the only way to get the user intent and there probably isn't too much we can determine automatically beyond that. Thanks!


asp.net core  upgrade to 2.0 error

[ ] I read and understood how to enable logging

Issue / Steps to reproduce the problem

  $exception	{System.MissingMethodException: Method not found: 'Microsoft.Extensions.DependencyInjection.IServiceCollection Microsoft.Extensions.DependencyInjection.AuthenticationServiceCollectionExtensions.AddAuthentication(Microsoft.Extensions.DependencyInjection.IServiceCollection)'.

at Microsoft.Extensions.DependencyInjection.IdentityServerBuilderExtensionsCore.AddRequiredPlatformServices(IIdentityServerBuilder builder)
at Microsoft.Extensions.DependencyInjection.IdentityServerServiceCollectionExtensions.AddIdentityServer(IServiceCollection services)

Relevant parts of the log file
<log goes here>

$exception	{System.MissingMethodException: Method not found: 'Microsoft.Extensions.DependencyInjection.IServiceCollection Microsoft.Extensions.DependencyInjection.AuthenticationServiceCollectionExtensions.AddAuthentication(Microsoft.Extensions.DependencyInjection.IServiceCollection)'.
at Microsoft.Extensions.DependencyInjection.IdentityServerBuilderExtensionsCore.AddRequiredPlatformServices(IIdentityServerBuilder builder)
at Microsoft.Extensions.DependencyInjection.IdentityServerServiceCollectionExtensions.AddIdentityServer(IServiceCollection services)

invalid.


VERY LOW SPEED

Visa can do 65k transactions per second whereas DOGE can do only 70 transactions per second . We want doge that it should do 700k transactions  per second

Visa can do 65k transactions per second whereas DOGE can do only 70 transactions per second . We want doge that it should do 700k transactions per second

@keshav1612  Visa can ban your account at anytime. Anyway, if you use in platform transfer like binance pay, you get the same speed as visa

@keshav1612
You should read this recent article: https://vitalik.ca/general/2021/05/23/scaling.html
@yuanoook

if you use in platform transfer like binance pay

Binance can also ban your account at any time.  The answer is to move off-chain.

I was sure we had a scaling issue open already, but can anyone find it?

I was sure we had a scaling issue open already, but can anyone find it?

The authoritarian dictator in me may have slipped and left one open, perhaps.
Closing this one though.


Icon(s) Request: Food

Icon Request

Icon name: Food
Tags of this icon: burger, drink, alcohol etc.
Use case:  For food related apps

Screenshots of some from Fontawesome:

Even if its not all of these, we could have a few i.e. meat, burger, cheese, wheat etc. to differentiate between allergies etc.

Thank you for your advice. We'll deal with some of them in the feature versions

@itsezc super request. @xiaochunjimmy it's really needed. Thanks in advance.

@xiaochunjimmy I'd also request the following additional "Food" icons
Chef:

Spatula:

Hey,
Is this being worked on?

@nimabrownlee Yes, as it is listed as icon request. I am sure @xiaochunjimmy will get to them asap.

Would also like to see more in the food category for things like a takeaway icon (box with chopsticks) or restaurant

That would be fantastic to have! It would certainly help if there was at least some general icon for the food category (e.g. an apple)

2023, any update on food related icon?


Indicate VSCode Version on download page.

Currently there is no way to find out, which version is offered for download on the website. It would be great if the download page would indicate that in some way (or at lease the downloaded file would have the version in its name). Especially since versions seem out of sync quite regularly for a short period after release, which is annoying because the only way of finding out is to download and install.

Good suggestion. Added to vscode-website backlog.

Closed and tracked in vscode-website backlog.


imxrt:serial Ensure the cache is updated if the DMA has updated again

Summary
Solves https://github.com/apache/nuttx/issues/10912
The DMA can bring in more rx data, than the number of
DMA completions call backs. The call backs happen on
idle, 1/2 and full events. But in between these events
the DMA can write more data to the buffers memory that
need to be brought in to the cache. (invalidate)
We do the invalidate on the reads from the fifo memory
if the the DMA as commited since the last read.
Impact
Without this change data was dropped at high baud rates.
Testing
imxrt1170 px4 nxp_fmurt1170-v1

excellent findings david, im sure this took a lot of hardcore debugging! thank you!

Under "Impact" where it says "There is dropped data at high baud rates" do you mean that this change fixes the dropped data at high baud rates, or do you mean that data is continues to be dropped at high baud rates despite the fix? (Not saying that the latter should block merging; in fact, it's probably better to merge a change that improves things, even if not perfect yet.) Thanks!

Let's merge this change and continue improvements in the next PRs

@davids5 I am working on imxrt117x where I also have found some problems with edma. I am using edma with Flexio driver to get ADC samples in. My driver is using the same logic as imxrt_serial rxdma logic and SOMETIMES I am getting corrupted data. Especially when the system is stressed. What  I have found out so far is that DONE flag is sometimes not set when it should be., although interrupt is triggered. Because I am relying on DONE flag passed to my callback function, I get wrong context in. If the system is not stressed, DONE flag is set every done-interrupt, half-interrupt has DONE flag set to zero. Looking further with the debugger, I've noticed that when DONE flag was not set, destination address was a few words ahead. This lead me to conclusion that my interrupt routine servicing is delayed a little bit and eDMA ran a little bit further, activating new major loop. From the documentation:
Channel Done
This field indicates whether the eDMA has completed the major loop. The eDMA engine sets the value of
this field to 1 when the CITER count reaches zero. The value of this field is reset to 0 by the hardware
(when the channel is activated) or by software.
one can see that DONE flag can be reset by hardware. Because imxrt_edma is checking DONE status quite a few times and is also influencing on imxrt_dmach_getcount which is then used in imxrt_serial, maybe this could also be a reason to get missing characters. What do you think? I think imxrt_edma driver should be revised here.

@jturnsek https://github.com/apache/nuttx/commit/6ad7b82cd67e4a2c392419d3e21398ae21495471 changed this behavior.
See the addition of imxrt_dmach_idle arch/arm/src/imxrt/imxrt_edma.c  9bb1666cbd0ef06990d9a5c015a040ce652b8549 and Ensuring the DMA Stop (called off a DMA completion) did not invoke the DMA completion callback again  9bb1666cbd0ef06990d9a5c015a040ce652b8549

@jturnsek @davids5 To handle high load, maybe this driver needs a DMA page-flipping implementation?

@jturnsek 6ad7b82 changed this behavior. See the addition of imxrt_dmach_idle arch/arm/src/imxrt/imxrt_edma.c 9bb1666 and Ensuring the DMA Stop (called off a DMA completion) did not invoke the DMA completion callback again 9bb1666

@davids5 Fix was imxrt_serial related, yes. But my point was that we have an imxrt_edma driver calling callback function and passing DONE status to it, which my not be passed every time. If eDMA is set as it is in imxrt_serial driver (and my driver), API is not working or one could not rely on done argument.
@hartmannathan Of course I agree with you. We should look how to simplify use of adriver in case of high load situations, becuase this is the main reason for DMA to exist.

@jturnsek How would you like to change it?

@jturnsek How would you like to change it?

I think it would be better to use different method for detection of DONE, in case of LOOPBACK destination enabled. Maybe somehow with CITER checking or similar? I do not have a viable solution yet.


Added spreadable overload for sample

This change adds a spreadable overload for sample.
This allows an arbitrary number of streams (more than the 2-5 currently typed) to be sampled, which matches the implementation.
If you have an array of streams with a common type, you can also spread that into the call and expect typing to flow through.

I've messed things up with that second commit.
Let me delete this and try again.