sudden disappearance of CHI pods

After running the clickhouse for couple of months, the CHI pod has disappeared on its own.
@hodgesrm had advised on adding stop: "no" for fix. However, I had already resolved by changing shard count and reverting, to trigger re-creation of CHI pod (by changing state of CHI out of completed to force pod re-creation).
Nevertheless, we had been unable to find the root cause of the issue. So, this issue could re-appear in the future.
CHI definition:
apiVersion: clickhouse.altinity.com/v1
kind: ClickHouseInstallation
metadata:
  name: my-release-clickhouse
  namespace: default
  labels:
    helm.sh/chart: clickhouse-23.12.2
    app.kubernetes.io/name: clickhouse
    app.kubernetes.io/instance: my-release
    app.kubernetes.io/component: clickhouse
    app.kubernetes.io/version: "22.8.8"
    app.kubernetes.io/managed-by: Helm
spec:
  defaults:
    templates:
      dataVolumeClaimTemplate: data-volumeclaim-template
      serviceTemplate: service-template

  configuration:
    users:
      admin/password: redacted
      admin/networks/ip:
        - "<IP_ADDRESS>/8"
        - "<IP_ADDRESS>/10"
        - "<IP_ADDRESS>/12"
        - "<IP_ADDRESS>/24"
        - "<IP_ADDRESS>/15"
        - "<IP_ADDRESS>/16"
      admin/profile: default
      admin/quota: default

    profiles:
      default/allow_experimental_window_functions: "1"
      default/allow_nondeterministic_mutations: "1"

    clusters:
      - name: "cluster"
        templates:
          podTemplate: pod-template
        layout:
          replicasCount: 1
          shardsCount: 1

    settings:
      format_schema_path: /etc/clickhouse-server/config.d/
      prometheus/endpoint: /metrics
      prometheus/port: 9363
      user_defined_executable_functions_config: /etc/clickhouse-server/functions/custom-functions.xml
      user_scripts_path: /var/lib/clickhouse/user_scripts/

    files:
      events.proto: |
        syntax = "proto3";
        message Event {
          string uuid = 1;
          string event = 2;
          string properties = 3;
          string timestamp = 4;
          uint64 team_id = 5;
          string distinct_id = 6;
          string created_at = 7;
          string elements_chain = 8;
        }
    zookeeper:
      nodes:
        - host: my-release-zookeeper-0.my-release-zookeeper-headless
          port: 2181

  templates:
    podTemplates:
      - name: pod-template
        metadata:
          labels:
            helm.sh/chart: clickhouse-23.12.2
            app.kubernetes.io/name: clickhouse
            app.kubernetes.io/instance: my-release
            app.kubernetes.io/component: clickhouse
            app.kubernetes.io/version: "22.8.8"
            app.kubernetes.io/managed-by: Helm
          annotations: {}
        podDistribution:
          - topologyKey: kubernetes.io/hostname
            type: ReplicaAntiAffinity
        spec:
          serviceAccountName: my-release-clickhouse
          priorityClassName: ""
          securityContext:
            fsGroup: 101
            runAsGroup: 101
            runAsUser: 101

          volumes:
            - name: shared-binary-volume
              emptyDir: {}
            - name: custom-functions-volume
              configMap:
                name: my-release-clickhouse-custom-functions
          initContainers: []
          containers:
            - name: clickhouse
              image: docker.io/clickhouse/clickhouse-server:22.8.8-alpine
              imagePullPolicy: IfNotPresent
              command:
                - /bin/bash
                - -c
                - /usr/bin/clickhouse-server --config-file=/etc/clickhouse-server/config.xml

              ports:
                - name: http
                  containerPort: 8123
                - name: client
                  containerPort: 9000
                - name: interserver
                  containerPort: 9009

              volumeMounts:
                - name: data-volumeclaim-template
                  mountPath: /var/lib/clickhouse
                - name: shared-binary-volume
                  mountPath: /var/lib/clickhouse/user_scripts
                - name: custom-functions-volume
                  mountPath: /etc/clickhouse-server/functions
              resources:
                requests:
                  cpu: 100m
                  memory: 200Mi

    serviceTemplates:
      - name: service-template
        generateName: my-release-clickhouse
        metadata:
          labels:
            helm.sh/chart: clickhouse-23.12.2
            app.kubernetes.io/name: clickhouse
            app.kubernetes.io/instance: my-release
            app.kubernetes.io/component: clickhouse
            app.kubernetes.io/version: "22.8.8"
            app.kubernetes.io/managed-by: Helm
        spec:
          type: ClusterIP
          ports:
            - name: http
              port: 8123
              nodePort: null
            - name: tcp
              port: 9000
              nodePort: null
    volumeClaimTemplates:
      - name: data-volumeclaim-template
        spec:
          accessModes:
            - ReadWriteOnce
          resources:
            requests:
              storage: 100Gi
          storageClassName: standard

Context: Slack Message

Is your kind: ClickHouseInstallation managed by the same Helm chart with clickhouse-operator?
Could you share the following commands?
kubectl get deploy --all-namespaces -l app=clickhouse-operator
kubectl get pod --all-namespaces -l clickhouse.altinity.com/chi

@Slach Yes, it is managed by the same chart with clickhouse-operator.
kubectl get deploy --all-namespaces app.kubernetes.io/component=operator

NAMESPACE     NAME                                        READY   UP-TO-DATE   AVAILABLE   AGE
default       my-release-clickhouse-operator              1/1     1            1           60d
test          my-release-clickhouse-operator              1/1     1            1           38d

kubectl get pod --all-namespaces -l clickhouse.altinity.com/chi

NAMESPACE     NAME                                                 READY   STATUS    RESTARTS        AGE
default       chi-my-release-clickhouse-cluster-0-0-0              1/1     Running   0               3d21h
test          chi-my-release-clickhouse-cluster-0-0-0              1/1     Running   0               4d19h

could you share operator logs from test namespace?
kubectl logs -n test -l app.kubernetes.io/component=operator --since=7d

I was getting the error below:
error: invalid argument "7d" for "--since" flag: time: unknown unit "d" in duration "7d"

Hence, replaced 7d with 168h:
$ kubectl logs -n test -l app.kubernetes.io/component=operator --since=168h

Defaulted container "my-release-clickhouse-operator" out of: my-release-clickhouse-operator, my-release-clickhouse-metrics-exporter
E0822 10:51:33.116468       1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: Get "https://<IP_ADDRESS>:443/api/v1/namespaces/test/services?resourceVersion=101836934": dial tcp <IP_ADDRESS>:443: i/o timeout
I0822 10:51:38.125884       1 trace.go:205] Trace[301511361]: "Reflector ListAndWatch" name:k8s.io/client-go/informers/factory.go:134 (22-Aug-2023 10:51:08.124) (total time: 30001ms):
Trace[301511361]: [30.00114905s] [30.00114905s] END
E0822 10:51:38.125902       1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.StatefulSet: failed to list *v1.StatefulSet: Get "https://<IP_ADDRESS>:443/apis/apps/v1/namespaces/test/statefulsets?resourceVersion=101836529": dial tcp <IP_ADDRESS>:443: i/o timeout
I0822 10:51:38.355293       1 trace.go:205] Trace[2060365486]: "Reflector ListAndWatch" name:k8s.io/client-go/informers/factory.go:134 (22-Aug-2023 10:51:08.354) (total time: 30001ms):
Trace[2060365486]: [30.001212232s] [30.001212232s] END
E0822 10:51:38.355312       1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: Get "https://<IP_ADDRESS>:443/api/v1/namespaces/test/configmaps?resourceVersion=101837009": dial tcp <IP_ADDRESS>:443: i/o timeout
I0822 10:51:42.173930       1 trace.go:205] Trace[250496038]: "Reflector ListAndWatch" name:pkg/client/informers/externalversions/factory.go:117 (22-Aug-2023 10:51:12.172) (total time: 30001ms):
Trace[250496038]: [30.001189387s] [30.001189387s] END
E0822 10:51:42.173949       1 reflector.go:138] pkg/client/informers/externalversions/factory.go:117: Failed to watch *v1.ClickHouseOperatorConfiguration: failed to list *v1.ClickHouseOperatorConfiguration: Get "https://<IP_ADDRESS>:443/apis/clickhouse.altinity.com/v1/namespaces/test/clickhouseoperatorconfigurations?resourceVersion=101836611": dial tcp <IP_ADDRESS>:443: i/o timeout

$ kubectl logs -n default -l app.kubernetes.io/component=operator --since=168h

Defaulted container "my-release-clickhouse-operator" out of: my-release-clickhouse-operator, my-release-clickhouse-metrics-exporter
E0822 10:51:38.591455       1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: Get "https://<IP_ADDRESS>:443/api/v1/namespaces/default/services?resourceVersion=101836934": dial tcp <IP_ADDRESS>:443: i/o timeout
I0822 10:51:41.314893       1 trace.go:205] Trace[1051692860]: "Reflector ListAndWatch" name:pkg/client/informers/externalversions/factory.go:117 (22-Aug-2023 10:51:11.313) (total time: 30001ms):
Trace[1051692860]: [30.001164853s] [30.001164853s] END
E0822 10:51:41.314912       1 reflector.go:138] pkg/client/informers/externalversions/factory.go:117: Failed to watch *v1.ClickHouseOperatorConfiguration: failed to list *v1.ClickHouseOperatorConfiguration: Get "https://<IP_ADDRESS>:443/apis/clickhouse.altinity.com/v1/namespaces/default/clickhouseoperatorconfigurations?resourceVersion=101836881": dial tcp <IP_ADDRESS>:443: i/o timeout
I0822 10:51:41.768145       1 trace.go:205] Trace[523315261]: "Reflector ListAndWatch" name:k8s.io/client-go/informers/factory.go:134 (22-Aug-2023 10:51:11.767) (total time: 30000ms):
Trace[523315261]: [30.000906333s] [30.000906333s] END
E0822 10:51:41.768163       1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: Get "https://<IP_ADDRESS>:443/api/v1/namespaces/default/configmaps?resourceVersion=101837009": dial tcp <IP_ADDRESS>:443: i/o timeout
I0822 10:51:42.946368       1 trace.go:205] Trace[838394852]: "Reflector ListAndWatch" name:k8s.io/client-go/informers/factory.go:134 (22-Aug-2023 10:51:12.945) (total time: 30000ms):
Trace[838394852]: [30.000528326s] [30.000528326s] END
E0822 10:51:42.946386       1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Pod: failed to list *v1.Pod: Get "https://<IP_ADDRESS>:443/api/v1/namespaces/default/pods?resourceVersion=101836916": dial tcp <IP_ADDRESS>:443: i/o timeout

looks like nothing happens 22 Aug
Could you share?
kubectl logs -n default -l app.kubernetes.io/component=operator -c clickhouse-operator
kubectl logs -n test -l app.kubernetes.io/component=operator -c clickhouse-operator

Sure, here are the logs:
$ kubectl logs -n default -l app.kubernetes.io/component=operator -c my-release-clickhouse-operator

I0823 18:03:06.328611       1 worker-reconciler.go:111] worker-reconciler.go:52:reconcileCHI():end:default/my-release-clickhouse
I0823 18:03:06.328632       1 worker.go:414] worker.go:380:updateCHI():end:default/my-release-clickhouse
E0823 18:03:49.676857       1 connection.go:190] Exec():FAILED Exec(http://clickhouse_operator:***@chi-my-release-clickhouse-cluster-0-0.default.svc.cluster.local:8123/) context deadline exceeded for SQL: SYSTEM DROP DNS CACHE
W0823 18:03:49.712800       1 retry.go:52] exec():chi-my-release-clickhouse-cluster-0-0.default.svc.cluster.local:FAILED single try. No retries will be made for Applying sqls
I0823 18:03:49.801384       1 worker.go:339] default/my-release-clickhouse/5d89fdfb-a136-430a-a8a5-310c07edef5b:IPs of the CHI-1 [<IP_ADDRESS>]
I0823 18:03:49.812084       1 worker.go:343] default/my-release-clickhouse/13b958a0-3cfd-46a6-a3ce-368a46cc97df:Update users IPS-1
I0823 18:03:49.821067       1 worker.go:1089] updateConfigMap():default/my-release-clickhouse/13b958a0-3cfd-46a6-a3ce-368a46cc97df:Update ConfigMap default/chi-my-release-clickhouse-common-usersd
I0823 18:03:49.903031       1 worker.go:339] default/my-release-clickhouse/88d992b4-a76a-4a2b-af19-2cc81d64548a:IPs of the CHI-1 [<IP_ADDRESS>]
I0823 18:03:49.910434       1 worker.go:343] default/my-release-clickhouse/5892dd12-16a8-4e6f-baf6-e5ec8776277d:Update users IPS-1
I0823 18:03:49.914824       1 worker.go:1089] updateConfigMap():default/my-release-clickhouse/5892dd12-16a8-4e6f-baf6-e5ec8776277d:Update ConfigMap default/chi-my-release-clickhouse-common-usersd

$ kubectl logs -n test -l app.kubernetes.io/component=operator -c my-release-clickhouse-operator

I0823 17:13:58.953558       1 worker-deleter.go:64] worker-deleter.go:64:dropReplicas():start:test/my-release-clickhouse/f3f569fa-3d23-4b03-a914-71c7bf07cf7d:drop replicas based on AP
I0823 17:13:58.953588       1 worker-deleter.go:81] worker-deleter.go:81:dropReplicas():end:test/my-release-clickhouse/f3f569fa-3d23-4b03-a914-71c7bf07cf7d:processed replicas: 0
I0823 17:13:58.953612       1 worker.go:573] includeStopped():test/my-release-clickhouse/f3f569fa-3d23-4b03-a914-71c7bf07cf7d:add CHI to monitoring
I0823 17:13:59.355282       1 controller.go:609] OK update watch (test/my-release-clickhouse): {"namespace":"test","name":"my-release-clickhouse","clusters":[{"name":"cluster","hosts":[{"name":"0-0","hostname":"chi-my-release-clickhouse-cluster-0-0.test.svc.cluster.local","tcpPort":9000,"httpPort":8123}]}]}
I0823 17:13:59.360534       1 worker.go:540] test/my-release-clickhouse:all IP addresses are in place
I0823 17:13:59.526393       1 worker.go:611] test/my-release-clickhouse/7de65ad0-12b1-4823-8657-ca1e4984bb8f:IPs of the CHI-2 [<IP_ADDRESS>]
I0823 17:13:59.534938       1 worker.go:615] test/my-release-clickhouse/789d4f97-3cee-499d-9e78-631399e02bea:Update users IPS-2
I0823 17:13:59.748918       1 worker.go:636] finalizeReconcileAndMarkCompleted():test/my-release-clickhouse/f3f569fa-3d23-4b03-a914-71c7bf07cf7d:reconcile completed successfully, task id: f3f569fa-3d23-4b03-a914-71c7bf07cf7d
I0823 17:14:00.146065       1 worker-reconciler.go:111] worker-reconciler.go:52:reconcileCHI():end:test/my-release-clickhouse
I0823 17:14:00.146085       1 worker.go:414] worker.go:380:updateCHI():end:test/my-release-clickhouse

=( this is not a full logs, this is only logs for 23 Aug

@Slach This is all that there is. :(

Closing it as it is not reproducible. Could be something related to Helm

@alex-zaitsev This is happening more often in recent time when trying to increase the ClickHouse PVC size in ClickHouse installation. Not able to pin point the root cause of the issue.
Operator logs:
I0729 03:49:26.917146       1 worker.go:1089] updateConfigMap():my-release/my-release-clickhouse/58053693-2b8a-4d1d-b5ed-e0432448d655:Update ConfigMap my-release/chi-my-release-clickhouse-deploy-confd-cluster-0-0
I0729 03:49:27.928385       1 cluster.go:84] Run query on: chi-my-release-clickhouse-cluster-0-0.my-release.svc.cluster.local of [chi-my-release-clickhouse-cluster-0-0.my-release.svc.cluster.local]
I0729 03:49:28.033592       1 worker-reconciler.go:292] reconcileHostStatefulSet():Reconcile host 0-0. ClickHouse version: <IP_ADDRESS>
I0729 03:49:28.033769       1 worker.go:134] shouldForceRestartHost():Force restart is not required. Host: 0-0
I0729 03:49:28.033802       1 worker-reconciler.go:304] reconcileHostStatefulSet():Reconcile host 0-0. Reconcile StatefulSet
I0729 03:49:28.033959       1 creator.go:589] getPodTemplate():my-release/my-release-clickhouse/58053693-2b8a-4d1d-b5ed-e0432448d655:statefulSet chi-my-release-clickhouse-cluster-0-0 use custom template: pod-template
W0729 03:49:28.034289       1 creator.go:994] containerAppendVolumeMount():my-release/my-release-clickhouse/58053693-2b8a-4d1d-b5ed-e0432448d655:container.Name:clickhouse volumeMount.Name:data-volumeclaim-template already used
I0729 03:49:28.036205       1 worker.go:1290] getStatefulSetStatus():my-release/chi-my-release-clickhouse-cluster-0-0:cur and new StatefulSets ARE DIFFERENT based on labels. StatefulSet reconcile is required for: my-release/chi-my-release-clickhouse-cluster-0-0
I0729 03:49:28.036374       1 worker.go:1431] updateStatefulSet():Update StatefulSet(my-release/chi-my-release-clickhouse-cluster-0-0) - started
I0729 03:49:28.348502       1 worker.go:1401] waitConfigMapPropagation():Wait for ConfigMap propagation for 9.003953254s 996.046746ms/10s
E0729 03:49:37.360361       1 creator.go:76] updateStatefulSet():StatefulSet update failed. err: StatefulSet.apps "chi-my-release-clickhouse-cluster-0-0" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbidden
E0729 03:49:37.361923       1 creator.go:100] updateStatefulSet():NOT EQUAL: AP item start -------------------------
modified spec items: 24
ap item path [0]:'.Template.Spec.Volumes[1].VolumeSource.ConfigMap.DefaultMode'
ap item value[0]:'nil'
ap item path [1]:'.Template.Spec.Containers[0].LivenessProbe.SuccessThreshold'
ap item value[1]:'0'
ap item path [2]:'.Template.Spec.Containers[0].ReadinessProbe.Handler.HTTPGet.Scheme'
ap item value[2]:'""'
ap item path [3]:'.Template.Spec.SchedulerName'
ap item value[3]:'""'
ap item path [4]:'.Template.Spec.Containers[0].LivenessProbe.Handler.HTTPGet.Scheme'
--
I0729 03:49:42.476651       1 poller.go:213] pollStatefulSet():my-release/chi-my-release-clickhouse-cluster-0-0:OK  :ObservedGeneration:2 Replicas:1 ReadyReplicas:1 CurrentReplicas:0 UpdatedReplicas:0 CurrentRevision:chi-my-release-clickhouse-cluster-0-0-8f9b7b45b UpdateRevision:chi-my-release-clickhouse-cluster-0-0-8f9b7b45b
W0729 03:50:27.487009       1 reflector.go:436] pkg/client/informers/externalversions/factory.go:117: watch of *v1.ClickHouseInstallation ended with: an error on the server ("unable to decode an event from the watch stream: http2: client connection lost") has prevented the request from succeeding
W0729 03:50:27.487049       1 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.StatefulSet ended with: an error on the server ("unable to decode an event from the watch stream: http2: client connection lost") has prevented the request from succeeding
W0729 03:50:27.487052       1 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.Endpoints ended with: an error on the server ("unable to decode an event from the watch stream: http2: client connection lost") has prevented the request from succeeding
W0729 03:50:27.487012       1 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.Service ended with: an error on the server ("unable to decode an event from the watch stream: http2: client connection lost") has prevented the request from succeeding
W0729 03:50:27.487096       1 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.ConfigMap ended with: an error on the server ("unable to decode an event from the watch stream: http2: client connection lost") has prevented the request from succeeding
W0729 03:50:27.487127       1 reflector.go:436] pkg/client/informers/externalversions/factory.go:117: watch of *v1.ClickHouseInstallationTemplate ended with: an error on the server ("unable to decode an event from the watch stream: http2: client connection lost") has prevented the request from succeeding
W0729 03:50:27.487145       1 reflector.go:436] k8s.io/client-go/informers/factory.go:134: watch of *v1.Pod ended with: an error on the server ("unable to decode an event from the watch stream: http2: client connection lost") has prevented the request from succeeding
W0729 03:50:27.487294       1 reflector.go:436] pkg/client/informers/externalversions/factory.go:117: watch of *v1.ClickHouseOperatorConfiguration ended with: an error on the server ("unable to decode an event from the watch stream: http2: client connection lost") has prevented the request from succeeding
I0729 03:50:27.488331       1 labeler.go:292] deleteLabelReadyPod():FAIL get pod for host 'my-release/0-0' err: Get "https://<IP_ADDRESS>:443/api/v1/namespaces/my-release/pods/chi-my-release-clickhouse-cluster-0-0-0": http2: client connection lost
E0729 03:50:57.489774       1 poller.go:237] pollStatefulSet():my-release/chi-my-release-clickhouse-cluster-0-0:my-release/chi-my-release-clickhouse-cluster-0-0 Get() FAILED
I0729 03:50:58.323884       1 trace.go:205] Trace[1632088119]: "Reflector ListAndWatch" name:k8s.io/client-go/informers/factory.go:134 (29-Jul-2024 03:50:28.323) (total time: 30000ms):
Trace[1632088119]: [30.000791007s] [30.000791007s] END
E0729 03:50:58.323928       1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Pod: failed to list *v1.Pod: Get "https://<IP_ADDRESS>:443/api/v1/namespaces/my-release/pods?resourceVersion=409134742": dial tcp <IP_ADDRESS>:443: i/o timeout
I0729 03:50:58.387047       1 trace.go:205] Trace[1562350316]: "Reflector ListAndWatch" name:k8s.io/client-go/informers/factory.go:134 (29-Jul-2024 03:50:28.386) (total time: 30000ms):
Trace[1562350316]: [30.000679957s] [30.000679957s] END
E0729 03:50:58.387074       1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.StatefulSet: failed to list *v1.StatefulSet: Get "https://<IP_ADDRESS>:443/apis/apps/v1/namespaces/my-release/statefulsets?resourceVersion=409134694": dial tcp <IP_ADDRESS>:443: i/o timeout
I0729 03:50:58.482514       1 trace.go:205] Trace[1523333257]: "Reflector ListAndWatch" name:k8s.io/client-go/informers/factory.go:134 (29-Jul-2024 03:50:28.481) (total time: 30000ms):
Trace[1523333257]: [30.000761918s] [30.000761918s] END
E0729 03:50:58.482587       1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: Get "https://<IP_ADDRESS>:443/api/v1/namespaces/my-release/configmaps?resourceVersion=409134621": dial tcp <IP_ADDRESS>:443: i/o timeout
I0729 03:50:58.627969       1 trace.go:205] Trace[328066567]: "Reflector ListAndWatch" name:pkg/client/informers/externalversions/factory.go:117 (29-Jul-2024 03:50:28.626) (total time: 30001ms):
Trace[328066567]: [30.001690277s] [30.001690277s] END
E0729 03:50:58.627999       1 reflector.go:138] pkg/client/informers/externalversions/factory.go:117: Failed to watch *v1.ClickHouseOperatorConfiguration: failed to list *v1.ClickHouseOperatorConfiguration: Get "https://<IP_ADDRESS>:443/apis/clickhouse.altinity.com/v1/namespaces/my-release/clickhouseoperatorconfigurations?resourceVersion=409134300": dial tcp <IP_ADDRESS>:443: i/o timeout
I0729 03:50:58.680316       1 trace.go:205] Trace[1867447287]: "Reflector ListAndWatch" name:k8s.io/client-go/informers/factory.go:134 (29-Jul-2024 03:50:28.679) (total time: 30000ms):
Trace[1867447287]: [30.000683297s] [30.000683297s] END
E0729 03:50:58.680433       1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: Get "https://<IP_ADDRESS>:443/api/v1/namespaces/my-release/services?resourceVersion=409134696": dial tcp <IP_ADDRESS>:443: i/o timeout
I0729 03:50:58.870657       1 trace.go:205] Trace[321750039]: "Reflector ListAndWatch" name:pkg/client/informers/externalversions/factory.go:117 (29-Jul-2024 03:50:28.869) (total time: 30000ms):
Trace[321750039]: [30.000749817s] [30.000749817s] END
E0729 03:50:58.870684       1 reflector.go:138] pkg/client/informers/externalversions/factory.go:117: Failed to watch *v1.ClickHouseInstallation: failed to list *v1.ClickHouseInstallation: Get "https://<IP_ADDRESS>:443/apis/clickhouse.altinity.com/v1/namespaces/my-release/clickhouseinstallations?resourceVersion=409134690": dial tcp <IP_ADDRESS>:443: i/o timeout
I0729 03:50:59.038154       1 trace.go:205] Trace[345492881]: "Reflector ListAndWatch" name:pkg/client/informers/externalversions/factory.go:117 (29-Jul-2024 03:50:29.037) (total time: 30000ms):
Trace[345492881]: [30.000824067s] [30.000824067s] END
E0729 03:50:59.038264       1 reflector.go:138] pkg/client/informers/externalversions/factory.go:117: Failed to watch *v1.ClickHouseInstallationTemplate: failed to list *v1.ClickHouseInstallationTemplate: Get "https://<IP_ADDRESS>:443/apis/clickhouse.altinity.com/v1/namespaces/my-release/clickhouseinstallationtemplates?resourceVersion=409134495": dial tcp <IP_ADDRESS>:443: i/o timeout
I0729 03:50:59.057219       1 trace.go:205] Trace[1113844881]: "Reflector ListAndWatch" name:k8s.io/client-go/informers/factory.go:134 (29-Jul-2024 03:50:29.056) (total time: 30000ms):
Trace[1113844881]: [30.000738938s] [30.000738938s] END
E0729 03:50:59.057261       1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Endpoints: failed to list *v1.Endpoints: Get "https://<IP_ADDRESS>:443/api/v1/namespaces/my-release/endpoints?resourceVersion=409134706": dial tcp <IP_ADDRESS>:443: i/o timeout

@alex-zaitsev @Slach can you please help here?

Failed to watch *v1.Endpoints: failed to list *v1.Endpoints: Get "https://<IP_ADDRESS>:443/api/v1/namespaces/my-release/endpoints?resourceVersion=409134706": dial tcp <IP_ADDRESS>:443: i/o timeout
means something wrong with your kubernetes API server


L1 Transfer Completed list is missing the success identifier

Describe the bug
L1 Transfer Completed list is missing the success identifier
Problem Pr: https://github.com/godwokenrises/light-godwoken/pull/224
Steps to reproduce the behavior:
1.See the following figure for details

Can you describe the details of this issue?

Can you provide more details of this issue? Thanks.

{"txHash":"0x6116445d9a98ca6ce79d66417bddc0c0949771007a4c12f8a304c430992011db","type":"transfer","status":"fail","recipient":"ckb1qz0n46hjl3pe2jwtepcvv5ehf9p6l94qvk9addgmarvfsvvrum6j7qgpfqmgcxwrszjv0alunk79pxe5pntf0ctwqq67z7yk","capacity":"0","amount":"25540000","token":{"type":{"code_hash":"0x5e7a36a77e68eecc013dfa2fe6a23f3b6c344b04005808694ae6dd45eea4cfd5","hash_type":"type","args":"0xe5451c05231e1df43e4b199b5d12dbed820dfbea2769943bb593f874526eeb55"},"name":"dCKB","symbol":"dCKB","uan":"dCKB.ckb","decimals":8,"tokenURI":"https://cryptologos.cc/logos/nervos-network-ckb-logo.svg?v=002","balance":"0x2f5d098c"},"symbol":"dCKB","decimals":8,"date":"2022-11-08 20:50:12"}]


👌 IMPROVE: Center align badges

Aligns the badges to the center along the center aligned headings/icon.
Related #385
P.S. Silly me. Should have handled in the #385 — I thought it'd be aligned but GitHub parses it with a <p> tag.
Peace! ✌

@ahmadawais thanks, it looks great!

Cheers!


portmod: 2.0.3 -> 2.1.0

Motivation for this change
A new version was released
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

[ ] NixOS test(s) (look inside nixos/tests)
[ ] and/or package tests
[ ] or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
21.11 Release Notes (or backporting 21.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Tried to update the graphics-overhaul mod set. As a result, added openmw to the binary dependancies for bsatool.
This mod set contain other bugs that I'll upstream later. Portmod comes with it's own updating method for mod repository.
(Ooh... didn't knew this package tests stuff...)

@ofborg eval

Why does the bot complain about a "picoscope" program that isn't even in my branch ? Anyway, that doesn't seems to be related to my changes.

the eval was broken for a bit so all pr's made during that time got the eval error
i've restarted the basic eval checks


[BrokenLinksH2] Fixed broken links

Updated anaplan urls.

Docs Build status updates of commit 4dfc96b:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

powerquery-docs/Connectors/Anaplan.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel

Docs Build status updates of commit e9150e4:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

powerquery-docs/Connectors/Anaplan.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel

Docs Build status updates of commit e9150e4:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

powerquery-docs/Connectors/Anaplan.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel


The log edit embed fails if you edit a message > 1024 characters

We need to chunk the edit and send it that way

closed in a direct commit


Require HTTPS for Weatherdrop

OP, thank you for securing the interwebs. We'd be compromised without you. 💯


hello,I'm interseted in your article ，so  when will you release your code?

Hello. Thanks for you interest! I am planning to release the code in the next 2-3 weeks.

Hi, I uploaded the code for the Image-Net based experiments. Soon, I will also upload the code for the MiniImageNet based experiments.


Create LICENSE

@all-contributors please add @Roozenlz for infrastructure, tests and code

@all-contributors please add @chengzhongxue for infrastructure, tests and code

@all-contributors please add @chengzhongxue for infrastructure, tests and code

@all-contributors please remove @chengzhongxue for infrastructure, tests and code


array no longer works on freebsd/openbsd

The sed expressions only work with gnu sed.
These work on both freebsd/openbsd (and continue to work on linux)
array_element_encode() {
  sed 's/%/%25/g' | sed -e :a -e '$!N; s/\n/%0A/; ta'
}

array_element_decode() {
  sed -e 's/%0[aA]/\
/g' -e 's/%25/%/g'
}

I am not really sure if that case-sensitivity in _decode is needed?

Omitting :a, $!N, and ta will break with multi-line elements.

nvm, I didn't read it properly

Care to make it a PR?

Ok, added.


fix url encoding

变更内容

修复：URL中特殊字符的编码问题

Coverage decreased (-0.03%) to 90.719% when pulling 1d3faad351f75d607d03bbbfc4eb192bfc961608 on fix-sign-url into 2d4691ac1af4fa0197178f31d8f9ca23798aa8e3 on master.

Coverage decreased (-0.02%) to 90.731% when pulling f6736426da8da17cd4f8e1779d205a7ae44970f6 on fix-sign-url into 2d4691ac1af4fa0197178f31d8f9ca23798aa8e3 on master.

Coverage increased (+0.2%) to 90.952% when pulling 5ffa6d205578530a5b888a26d3fe4351bfa66d5f on fix-sign-url into 2d4691ac1af4fa0197178f31d8f9ca23798aa8e3 on master.

Coverage increased (+0.2%) to 90.952% when pulling 5ffa6d205578530a5b888a26d3fe4351bfa66d5f on fix-sign-url into 2d4691ac1af4fa0197178f31d8f9ca23798aa8e3 on master.

Coverage increased (+0.2%) to 90.952% when pulling 5ffa6d205578530a5b888a26d3fe4351bfa66d5f on fix-sign-url into 2d4691ac1af4fa0197178f31d8f9ca23798aa8e3 on master.

Coverage increased (+0.2%) to 90.952% when pulling 857f13b5e62865d2cf6f26eb9fe90a8ae8309a22 on fix-sign-url into 2d4691ac1af4fa0197178f31d8f9ca23798aa8e3 on master.


Converting saved checkpoints to pb file

I'm trying to convert the saved cktp files into a pb file.
I'm using the following to do so:
import tensorflow as tf

saver = tf.train.import_meta_graph('./checkpoints/pre_norm2pro/Epoch_(209)_(1100of2100).ckpt.meta', clear_devices=True)
graph = tf.get_default_graph()
input_graph_def = graph.as_graph_def()
sess = tf.Session()
saver.restore(sess, "./checkpoints/pre_norm2pro/Epoch_(209)_(1100of2100).ckpt.data")

The errors I'm getting are:
2018-06-04 16:23:27.608892: W tensorflow/core/util/tensor_slice_reader.cc:95] Could not open ./checkpoints/pre_norm2pro/Epoch_(209)_(1100of2100).ckpt.data-00000-of-00001: Data loss: not an sstable (bad magic number): perhaps your file is in a different file format and you need to use a different restore operator? 2018-06-04 16:23:27.608914: W tensorflow/core/framework/op_kernel.cc:1192] Data loss: Unable to open table file ./checkpoints/pre_norm2pro/Epoch_(209)_(1100of2100).ckpt.data-00000-of-00001: Data loss: not an sstable (bad magic number): perhaps your file is in a different file format and you need to use a different restore operator?
The model was trained successfully and runs inference using test.py
A complete log file is attached
log.txt
Edit: Also I assume the names of the output nodes are 'pred' and 'h4'?

@andrewginns I am sorry I haven't got an experience for converting .ckpt to .pb. I think you can try
saver.restore(sess, "./checkpoints/pre_norm2pro/Epoch_(209)_(1100of2100).ckpt") without .data, that's how I restore the variables.

@LynnHo I figured it out. Created a program to create it from the .ckpt files. Useful for conversion to other model formats like Intel Movidius graphs.
import os, argparse

import tensorflow as tf

# The original freeze_graph function
# from tensorflow.python.tools.freeze_graph import freeze_graph

dir = os.path.dirname(os.path.realpath(__file__))

def freeze_graph(model_dir, output_node_names):
    """Extract the sub graph defined by the output nodes and convert
    all its variables into constant

    Args:
        model_dir: the root folder containing the checkpoint state file
        output_node_names: a string, containing all the output node's names,
                            comma separated
    """
    if not tf.gfile.Exists(model_dir):
        raise AssertionError(
            "Export directory doesn't exists. Please specify an export "
            "directory: %s" % model_dir)

    if not output_node_names:
        print("You need to supply the name of a node to --output_node_names.")
        return -1

    # We retrieve our checkpoint fullpath
    checkpoint = tf.train.get_checkpoint_state(model_dir)
    input_checkpoint = checkpoint.model_checkpoint_path

    # We precise the file fullname of our freezed graph
    absolute_model_dir = "/".join(input_checkpoint.split('/')[:-1])
    output_graph = absolute_model_dir + "/frozen_model.pb"

    # We clear devices to allow TensorFlow to control on which device it will load operations
    clear_devices = True

    # We start a session using a temporary fresh Graph
    with tf.Session(graph=tf.Graph()) as sess:
        # We import the meta graph in the current default Graph
        saver = tf.train.import_meta_graph(input_checkpoint + '.meta', clear_devices=clear_devices)

        # We restore the weights
        saver.restore(sess, input_checkpoint)

        # We use a built-in TF helper to export variables to constants
        output_graph_def = tf.graph_util.convert_variables_to_constants(
            sess, # The session is used to retrieve the weights
            tf.get_default_graph().as_graph_def(), # The graph_def is used to retrieve the nodes
            output_node_names.split(",") # The output node names are used to select the usefull nodes
        )

	# fix nodes
        for node in output_graph_def.node:
          if node.op == 'RefSwitch':
            node.op = 'Switch'
            for index in xrange(len(node.input)):
              if 'moving_' in node.input[index]:
                node.input[index] = node.input[index] + '/read'
          elif node.op == 'AssignSub':
            node.op = 'Sub'
            if 'use_locking' in node.attr: del node.attr['use_locking']

        # Finally we serialize and dump the output graph to the filesystem
        with tf.gfile.GFile(output_graph, "wb") as f:
            f.write(output_graph_def.SerializeToString())
        print("%d ops in the final graph." % len(output_graph_def.node))

    return output_graph_def

if __name__ == '__main__':
    parser = argparse.ArgumentParser()
    parser.add_argument("--model_dir", type=str, default="", help="Model folder to export")
    parser.add_argument("--output_node_names", type=str, default="", help="The name of the output nodes, comma separated.")
    args = parser.parse_args()

    freeze_graph(args.model_dir, args.output_node_names)

This also accounts for strange node renaming due to the use of a moving_mean. Done through the #fix nodes section

hi  Mr
i use this code for convert checkpoint ( .meta, .data and .index ) model files into one graph.pb file... but  error from output_node_names .... please Explain for me about this parametr in deep neural network.


Add support for getting usage tokens for openaiAPI

fix #814

@YanjiuHe I believe the 20e4b5609a35c204b0e93ac6fcd38ae821a86d1d resolves https://github.com/spring-projects/spring-ai/issues/814 ?

I'm going to be reviewing the classes for ChatResponse and metadata, I think it was a mistake to inherit from HashMap, instead a hashmap should be used as a field to capture any non-standard metadata.


docs: update homepage

Description
Currently, the default access address of the official website has been adjusted to vite.dev. Many places in the code still use vitejs.dev. I don't know whether they all need to be adjusted.

Yeah I think we'd want to replace all vitejs.dev to vite.dev at one point. I find and replace all should work well.


Mention struct tags

Struct tags are widely used by Go standard library, but not mentioned in tour or Effective Go at all.

+1

I don't think is an essential part of the golang, I would put it in a more advanced category.
Structs already have I think 6 pages dedicated to it.


nix-folder2channel: init

add nix-folder2channel script to make the nixpkgs into a channel in developing nix packages.
Motivation for this change
Some developers and I are confusing about make the source code into a channel when developing nix packages.
It is trival, but friendly for a new developer to contribute on the nix.
I lack of deep knowledge about nix, and am willing to get any better suggestion and take time to work on this.
If you have any comment or suggestion on this script, I am appreciated.
Fix #14698
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[X] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[X] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[X] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[X] Fits CONTRIBUTING.md.

The PR is reopened because I had packed an input method package, and the package needs to change some configure in nixos and rebuild OS to make sure everything goes well, such that this script is still helpful when developing a package need to change the configuration in NixOS.
I temporally closed the PR, because I forgot to check whether this is solved or not in NixUI.md, which is mentioned in the issue, and make sure that this tool is needed for NixOS.  And the command nix install -d seems not to cover the scenario aforementioned. I am sorry for the inconvenience.
Thanks.

@yanganto Are you still interested in trying to land this and is it still meaningful? Did a 2nd review just fall through the cracks?

Sorry I forgot this one, I will check this tomorrow and I will close it or fix possible conflict tomorrow. :pray:

move to #323596

Let me update it to the latest master first.

@abathur I think this PR is before flake.  Now we have flake it will be easier to customize things, so we do not need to turn a customized nixpkgs into a channel now.  I will close these.


Larger socket connection limit for memory node and routing node

Codecov Report

Merging #42 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #42   +/-   ##
=======================================
  Coverage   66.64%   66.64%
=======================================
  Files          45       45
  Lines        1745     1745
=======================================
  Hits         1163     1163
  Misses        582      582

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c4589f6...4e40f05. Read the comment docs.


Whitespace handling

At present, all the generated code uses spaces. I would like if the generator had a flag to switch to tabs (or autodetected from index.js), but at the very least, it would be nice if the generator auto-detected when it was modifying files (such as tests, services/index.js, etc) so that the added lines matched the file they were being added to.

The easiest way right now would probably be to change the whitespace setting in the .eslintrc.json and then run it with --fix which should update all files accordingly.
I'd like to avoid adding a new prompt for every style choice that could potentially be made. What I'd recommend to customize the generator is to fork it, make the changes, publish it into your own namespace and then use it just like any other Yeoman generator. We do always take pull requests with improvement suggestions of course.

Thanks again for the suggestion. Our goal with the generator is to pick sane defaults and keep the number of prompts as low as possible, especially when it is not relevant to other generators. Every new prompt will have to be implemented, tested, documented and released and also requires all tutorials  that are showing and/or explaining the prompts (including screenshots) to be updated so we do have to carefully any new options we are going to add.
Since I think there is a fairly straightforward workaround I am going to close this issue.

Sorry I didn't reply sooner, I was in Chicago all week. Totally understand why you wouldn't want to add a new prompt/option.


Initialize Interactor during configuration

Jira issue:
https://glia.atlassian.net/browse/MOB-3436
What was solved?
To allow requesting reactive omnibrowse engagement, the Interactor instance should be initialized on the configuring stage. Current start engagement and Configuration methods were deprecated to support the new approach
Release notes:
Integrators need to start using the latest configuration and startEngagement methods

[ ] Feature
[ ] Ignore
[ ] Release notes (Is it clear from the description here?)
[X] Migration guide (If changes are needed for integrator already using the SDK - what needs to be communicated? Add underneath please)

Additional info:

[ ] Is the feature sufficiently tested? All tests fixed? Necessary unit, acceptance, snapshots added? Check that at least new public classes & methods are covered with unit tests
[ ] Did you add logging beneficial for troubleshooting of customer issues?
[ ] Did you add new logging? We would like the logging between platforms to be similar. Refer to Logging from iOS SDKs → Things to consider for newly added logs in Confluence for more information.

Screenshots:

!squash

Test added e8c7c5a

!squash


DuckDB benchmark with 1.1B Taxi Data?

It might be useful to add a DuckDB benchmark for the 1.1 Billion Taxi Rides datasets that Mark Litwintschik has done for a number of databases/warehouses.   It is not a formal benchmark, but it is a fairly well known dataset with a collection of queries for analytics that has been used to compare and contrast various systems.
A DuckDB analysis/blog post might go a long way to get people excited in comparison to sqlite or even real db systems such as Postgres.
https://tech.marksblogg.com/billion-nyc-taxi-rides-sqlite-parquet-hdfs.html

Yeah I tried to reproduce that dataset at some point did not work out so well...

As a starting point, I remember Uwe did a run through of the taxi data (although I don't think it was the entire dataset) comparing it against sqlite.
Taking DuckDB for a spin | Uwe’s Blog
https://uwekorn.com/2019/10/19/taking-duckdb-for-a-spin.html

Looking more closely at the Uwe's blog post, I see he only did 2016-01 data.  In doing some other searches, I found that a new version of the embedded MonetDB (now named MonetDBe) was released (nice!), and I see that there is a blog post doing the same 2016-01 dataset...
Taking MonetDB/e for a taxi ride | MonetDB
https://www.monetdb.org/blog/taking-MonetDBe-for-a-taxi-ride
I might dig into this a little deeper and do some testing on my own.  If I do, I will be sure to post back some results...

I'm trying to create the dataset again right now. Will post for download somewhere so this has an end.

has there been any update on this? curious to see if duckdb will scale to this dataset with multiple threads

@tobymao I am happy to confirm that DuckDB has no issues running queries on this dataset with multiple threads etc. We were waiting for a hosting agreement so we can make the dataset available with benchmark results as well.

I did a simplified version of what Mark Litwintschik did here: https://tech.marksblogg.com/benchmarks.html
Key differences:  loaded 72 months of data from yellow cab only (Mark includes Uber, Green, etc), and did not transform to include weather, borough, cab_type columns within the fact table.

18 columns instead of 45
1.03 billion instead of 1.1 billion
3 of the 4 queries Mark uses still apply to this simplified data load.
1 query against cab type changed to look at a vendor_name (and vendor_id)

-Used AWS 5d.8xlarge instance 16 cores (32 HT), 256GB memory.
'Taxi_Q1_variant'|1|"select vendor_name, count(*) from trip group by 1 "
'Taxi_Q2'|2|"select passenger_count, avg(total_amt) from trip group by 1 "
'Taxi_Q3'|3|"SELECT passenger_count, EXTRACT(year from trip_pickup_datetime), count(*) FROM trip GROUP BY 1,2"
'Taxi_Q4'|4|"SELECT passenger_count, EXTRACT(year from trip_pickup_datetime) as year, round(trip_distance), count(*) trips FROM trip GROUP BY 1,2,3 order by year, trips desc;"

'Taxi_Q1_variant2'|5|"select vendor_id, count(*) from trip group by 1 "

wget https://s3.amazonaws.com/nyc-tlc/trip+data/yellow_tripdata_2009-{01..12}.csv
...
wget https://s3.amazonaws.com/nyc-tlc/trip+data/yellow_tripdata_2014-{01..12}.csv
cleaned up a few rows with extra columns in 2 months of 2010 per Mark's suggestions.

Run as in-memory through python.
let read_csv_auto determine structure:
create table trip as select * from read_csv_auto('/data1/yellow_tripdata_2009-01.csv',header=true)

Raw timings with 1 through 32 threads (pragma threads=N)
                         |    1    |    2    |    3    |    4    |    6    |    8    |   12    |   16    |   24    |   32
Taxi_Q1_variant 1'       | 33.181 | 16.628 | 11.075 | 8.308 | 5.544 | 4.156 | 2.774 | 2.084 | 1.801 | 1.585
'Taxi_Q2'                | 6.506 | 3.297 | 2.2 | 1.652 | 1.105 | 0.828 | 0.556 | 0.419 | 0.349 | 0.299
'Taxi_Q3'                | 12.318 | 6.233 | 4.14 | 3.103 | 2.075 | 1.557 | 1.04 | 0.782 | 0.688 | 0.617
'Taxi_Q4'                | 44.666 | 21.517 | 14.559 | 11.073 | 7.445 | 5.905 | 4.055 | 3.604 | 3.397 | 4.103
Taxi_Q1_variant 2'       | 4.492 | 2.366 | 1.559 | 1.169 | 0.775 | 0.586 | 0.391 | 0.293 | 0.253 | 0.225

Key Learning:

Extremely strong linear scaling: -a validation of their vector-volcano model for these use cases.
Example:  Q3 is 0.78 seconds at 16 threads.  Perfect linear scaling extrapolated from 1 thread is 0.77 seconds.   (beyond 16 threads linear scaling is not expected )

Additional Learnings:

Group by vendor_name (string) is about 7x slower than group by vendor_id (integer) (2.084 vs 0.293 seconds at 16 threads).  There is a future optimization Mark and Hannes have planned around dictionary handling that should allow string performance here to approach integer (with few distinct strings yielding a tight dictionary).
Q4 is about 6x slower than Q3, likely because it  use a less optimized hash aggregate strategy due to higher number of group by items (doesn't use perfect hash afaik).
Decrease "perfect linear scaling" beyond 16 threads is a by-product of ht architecture.  Vectorization and streaming data input minimize L1 data and L1 instruction cache misses to the point where the CPU is productive "most" of the time, leaving fewer opportunities to run a second thread transparently.  There could be artifacts from a virtual environment at play as well.
It took about 3 minutes to add the vendor_id column and populate 1.03 billion rows via a case statement.  I don't have a direct comparison, but was both fast and easy to do this.

Wow awesome stuff. Are operations on floats (total_amt) parallel? I read in #854 that floating point summation was not parallel https://github.com/cwida/duckdb/issues/854#issuecomment-677455468.
Looks like duckdb can stand up to these other db's without having to install anything external!

Great benchmark! Fantastic to see DuckDB compete with these other DBs.
How long did the data import step take?
Thanks!

Something like 3 hours to load 72 months, I didn't get the exact timings.
On Tue, Feb 23, 2021, 1:44 PM Alex-Monahan<EMAIL_ADDRESS>wrote:

Great benchmark! Fantastic to see DuckDB compete with these other DBs.
How long did the data import step take?
Thanks!
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/cwida/duckdb/issues/1121#issuecomment-784533700, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ACEBER5GLRVFEEASTFJUIEDTAQOUDANCNFSM4TW2EXDA
.

Wow awesome stuff. Are operations on floats (total_amt) parallel? I read in #854 that floating point summation was not parallel #854 (comment).

Be default, the read_csv_auto defined fare_amount as a double which scales nicely.


Fix grammar

Changed "There are no chaos infrastructure in your project." to "There are no chaos infrastructures in your project."
Signed-off-by: Smriti S<EMAIL_ADDRESS>

Preview environment: https://ci.pr.harness.io/pr-603


Feature request: HTTP resolver middleware should have access to path parameters

Use case
HTTP resolver middlewares should have access to path parameters to help with use cases like resource-specific authnz, logging enrichment, etc. Access is already available in an undocumented manner that may be considered "private" according to PEP 8 (see "User Experience" below).
Solution/User Experience
Current:
def middleware(app: APIGatewayRestResolver, next_middleware: NextMiddleware) -> Response:
    path_params = app.context.get("_route_args")
    logger.append_keys(path_params=path_params)
    return next_middleware(app)

New:
def middleware(app: APIGatewayRestResolver, next_middleware: NextMiddleware) -> Response:
    path_params = app.context.path_parameters
    logger.append_keys(path_params=path_params)
    return next_middleware(app)

Alternative solutions
Perhaps it would be ideal to have the parsed path parameters available via app.current_event instead (as with API Gateway resolvers), but using app.context is certainly easier and more in line with the current state. Providing data in the powertools app.current_event that is not present in a typical Lambda handler's event could also lead to confusion.
Acknowledgment

[X] This feature request meets Powertools for AWS Lambda (Python) Tenets
[X] Should this be considered in other Powertools for AWS Lambda languages? i.e. Java, TypeScript, and .NET

Here's one solution for implementation (I can make a PR if this is deemed a valuable suggestion, tips for how to document it welcome):
diff --git a/aws_lambda_powertools/event_handler/api_gateway.py b/aws_lambda_powertools/event_handler/api_gateway.py
index 7c4d6769..4cbd2bd8 100644
--- a/aws_lambda_powertools/event_handler/api_gateway.py
+++ b/aws_lambda_powertools/event_handler/api_gateway.py
@@ -423,7 +423,7 @@ class Route:
             print("=================")

         # Add Route Arguments to app context
-        app.append_context(_route_args=route_arguments)
+        app.context["_route_args"] = route_arguments

         # Call the Middleware Wrapped _call_stack function handler with the app
         return self._middleware_stack(app)
@@ -898,10 +898,17 @@ class ResponseBuilder(Generic[ResponseEventT]):
         }

+class RouterContext(dict):
+    @property
+    def path_parameters(self) -> Dict[str, Any]:
+        """Get path parameters for the current request."""
+        return self.get("_route_args", {})
+
+
 class BaseRouter(ABC):
     current_event: BaseProxyEvent
     lambda_context: LambdaContext
-    context: dict
+    context: RouterContext
     _router_middlewares: List[Callable] = []
     processed_stack_frames: List[str] = []

@@ -1317,6 +1324,11 @@ class BaseRouter(ABC):

     def append_context(self, **additional_context):
         """Append key=value data as routing context"""
+        if "_route_args" in additional_context:
+            warnings.warn(
+                "The '_route_args' context key is used internally by powertools and should not be overwrit
ten",
+                stacklevel=2,
+            )
         self.context.update(**additional_context)

     def clear_context(self):
@@ -1421,7 +1433,7 @@ def _registered_api_adapter(
         The API Response Object

     """
-    route_args: Dict = app.context.get("_route_args", {})
+    route_args: Dict = app.context.path_parameters
     logger.debug(f"Calling API Route Handler: {route_args}")
     return app._to_response(next_middleware(**route_args))

@@ -1494,7 +1506,7 @@ class ApiGatewayResolver(BaseRouter):
         self._debug = self._has_debug(debug)
         self._enable_validation = enable_validation
         self._strip_prefixes = strip_prefixes
-        self.context: Dict = {}  # early init as customers might add context before event resolution
+        self.context = RouterContext()  # early init as customers might add context before event resolutio
n
         self.processed_stack_frames = []
         self._response_builder_class = ResponseBuilder[BaseProxyEvent]

@@ -2453,7 +2465,7 @@ class Router(BaseRouter):
         self._routes: Dict[tuple, Callable] = {}
         self._routes_with_middleware: Dict[tuple, List[Callable]] = {}
         self.api_resolver: Optional[BaseRouter] = None
-        self.context = {}  # early init as customers might add context before event resolution
+        self.context = RouterContext()  # early init as customers might add context before event resolutio
n
         self._exception_handlers: Dict[Type, Callable] = {}

     def route(
diff --git a/tests/functional/event_handler/required_dependencies/test_router.py b/tests/functional/event_h
andler/required_dependencies/test_router.py
index d96f5035..b6111017 100644
--- a/tests/functional/event_handler/required_dependencies/test_router.py
+++ b/tests/functional/event_handler/required_dependencies/test_router.py
@@ -1,16 +1,23 @@
+import pytest
+
 from aws_lambda_powertools.event_handler import (
     ALBResolver,
     APIGatewayHttpResolver,
+    ApiGatewayResolver,
     APIGatewayRestResolver,
     LambdaFunctionUrlResolver,
     Response,
 )
+from aws_lambda_powertools.event_handler.middlewares import NextMiddleware
+from aws_lambda_powertools.event_handler.openapi.params import Path
 from aws_lambda_powertools.event_handler.router import (
     ALBRouter,
     APIGatewayHttpRouter,
     APIGatewayRouter,
     LambdaFunctionUrlRouter,
+    Router,
 )
+from aws_lambda_powertools.shared.types import Annotated
 from aws_lambda_powertools.utilities.data_classes import (
     ALBEvent,
     APIGatewayProxyEvent,
@@ -74,3 +81,71 @@ def test_lambda_function_url_router_event_type():
     app.include_router(router)
     result = app(load_event("lambdaFunctionUrlEvent.json"), {})
     assert result["body"] == "routed"
+
+
<EMAIL_ADDRESS>+    "router,resolver,event_file",
+    [
+        (ALBRouter, ALBResolver, "albEvent.json"),
+        (APIGatewayRouter, APIGatewayRestResolver, "apiGatewayProxyEvent.json"),
+        (APIGatewayHttpRouter, APIGatewayHttpResolver, "apiGatewayProxyV2Event_GET.json"),
+        (LambdaFunctionUrlRouter, LambdaFunctionUrlResolver, "lambdaFunctionUrlEvent.json"),
+    ],
+)
+def test_path_parameters_in_context(
+    router: Router,
+    resolver: ApiGatewayResolver,
+    event_file: str,
+) -> None:
+    app = resolver(enable_validation=True)
+    router = router()
+    path_params = {
+        "str_param": "str_value",
+        "int_param": 3,
+    }
+
+    def bar(app: APIGatewayRestResolver, next_middleware: NextMiddleware) -> Response[str]:
+        assert app.context.path_parameters == path_params
+        return next_middleware(app)
+
+    @router.route(rule="/<str_param>/<int_param>", method=["GET"], middlewares=[bar])
+    def foo(str_param: Annotated[str, Path()], int_param: Annotated[int, Path()]) -> Response[str]:
+        return Response(status_code=200, body="routed")
+
+    app.include_router(router)
+    event = load_event(event_file)
+    event["path"] = event["rawPath"] = f"/{path_params['str_param']}/{path_params['int_param']}"
+    result = app(event, {})
+    assert result["body"] == "routed"
+
+
<EMAIL_ADDRESS>+    "router,resolver,event_file",
+    [
+        (ALBRouter, ALBResolver, "albEvent.json"),
+        (APIGatewayRouter, APIGatewayRestResolver, "apiGatewayProxyEvent.json"),
+        (APIGatewayHttpRouter, APIGatewayHttpResolver, "apiGatewayProxyV2Event_GET.json"),
+        (LambdaFunctionUrlRouter, LambdaFunctionUrlResolver, "lambdaFunctionUrlEvent.json"),
+    ],
+)
+def test_path_parameters_static_path(
+    router: Router,
+    resolver: ApiGatewayResolver,
+    event_file: str,
+) -> None:
+    app = resolver(enable_validation=True)
+    router = router()
+
+    def bar(app: APIGatewayRestResolver, next_middleware: NextMiddleware) -> Response[str]:
+        assert app.context.path_parameters == {}
+        return next_middleware(app)
+
+    @router.route(rule="/static", method=["GET"], middlewares=[bar])
+    def foo() -> Response[str]:
+        return Response(status_code=200, body="routed")
+
+    app.include_router(router)
+    event = load_event(event_file)
+    event["path"] = event["rawPath"] = "/static"
+    result = app(event, {})
+    assert result["body"] == "routed"

Hey Kim, did you face any issues accessing via the app.current_event within
the middleware?
The app instance is the same for both middlewares and routes by design.
PS: I’m reading on my phone but will check tomorrow on my laptop more
attentively in case I missed anything.
On Sun, 23 Jun 2024 at 12:35, Kim Säppi @.***> wrote:

Here's one solution for implementation (I can make a PR if this is deemed
a valuable suggestion, tips for how to document it welcome):
diff --git a/aws_lambda_powertools/event_handler/api_gateway.py b/aws_lambda_powertools/event_handler/api_gateway.py
index 7c4d6769..4cbd2bd8 100644--- a/aws_lambda_powertools/event_handler/api_gateway.py+++ b/aws_lambda_powertools/event_handler/api_gateway.py@@ -423,7 +423,7 @@ class Route:
print("=================")
     # Add Route Arguments to app context-        app.append_context(_route_args=route_arguments)+        app.context["_route_args"] = route_arguments

     # Call the Middleware Wrapped _call_stack function handler with the app
     return self._middleware_stack(app)@@ -898,10 +898,17 @@ class ResponseBuilder(Generic[ResponseEventT]):
     }

+class RouterContext(dict):+    @property+    def path_parameters(self) -> Dict[str, Any]:+        """Get path parameters for the current request."""+        return self.get("_route_args", {})++
class BaseRouter(ABC):
current_event: BaseProxyEvent
lambda_context: LambdaContext-    context: dict+    context: RouterContext
_router_middlewares: List[Callable] = []
processed_stack_frames: List[str] = []
@@ -1317,6 +1324,11 @@ class BaseRouter(ABC):
 def append_context(self, **additional_context):
     """Append key=value data as routing context"""+        if "_route_args" in additional_context:+            warnings.warn(+                "The '_route_args' context key is used internally by powertools and should not be overwrit

ten",+                stacklevel=2,+            )
self.context.update(**additional_context)
 def clear_context(self):@@ -1421,7 +1433,7 @@ def _registered_api_adapter(
     The API Response Object

 """-    route_args: Dict = app.context.get("_route_args", {})+    route_args: Dict = app.context.path_parameters
 logger.debug(f"Calling API Route Handler: {route_args}")
 return app._to_response(next_middleware(**route_args))

@@ -1494,7 +1506,7 @@ class ApiGatewayResolver(BaseRouter):
self._debug = self._has_debug(debug)
self._enable_validation = enable_validation
self._strip_prefixes = strip_prefixes-        self.context: Dict = {}  # early init as customers might add context before event resolution+        self.context = RouterContext()  # early init as customers might add context before event resolutio
n
self.processed_stack_frames = []
self._response_builder_class = ResponseBuilder[BaseProxyEvent]
@@ -2453,7 +2465,7 @@ class Router(BaseRouter):
self._routes: Dict[tuple, Callable] = {}
self._routes_with_middleware: Dict[tuple, List[Callable]] = {}
self.api_resolver: Optional[BaseRouter] = None-        self.context = {}  # early init as customers might add context before event resolution+        self.context = RouterContext()  # early init as customers might add context before event resolutio
n
self._exception_handlers: Dict[Type, Callable] = {}
 def route(diff --git a/tests/functional/event_handler/required_dependencies/test_router.py b/tests/functional/event_h

andler/required_dependencies/test_router.py
index d96f5035..b6111017 100644--- a/tests/functional/event_handler/required_dependencies/test_router.py+++ b/tests/functional/event_handler/required_dependencies/test_router.py@@ -1,16 +1,23 @@+import pytest+
from aws_lambda_powertools.event_handler import (
ALBResolver,
APIGatewayHttpResolver,+    ApiGatewayResolver,
APIGatewayRestResolver,
LambdaFunctionUrlResolver,
Response,
)+from aws_lambda_powertools.event_handler.middlewares import NextMiddleware+from aws_lambda_powertools.event_handler.openapi.params import Path
from aws_lambda_powertools.event_handler.router import (
ALBRouter,
APIGatewayHttpRouter,
APIGatewayRouter,
LambdaFunctionUrlRouter,+    Router,
)+from aws_lambda_powertools.shared.types import Annotated
from aws_lambda_powertools.utilities.data_classes import (
ALBEvent,
APIGatewayProxyEvent,@@ -74,3 +81,71 @@ def test_lambda_function_url_router_event_type():
app.include_router(router)
result = app(load_event("lambdaFunctionUrlEvent.json"), {})
assert result["body"] == @.(+    "router,resolver,event_file",+    [+        (ALBRouter, ALBResolver, "albEvent.json"),+        (APIGatewayRouter, APIGatewayRestResolver, "apiGatewayProxyEvent.json"),+        (APIGatewayHttpRouter, APIGatewayHttpResolver, "apiGatewayProxyV2Event_GET.json"),+        (LambdaFunctionUrlRouter, LambdaFunctionUrlResolver, "lambdaFunctionUrlEvent.json"),+    ],+)+def test_path_parameters_in_context(+    router: Router,+    resolver: ApiGatewayResolver,+    event_file: str,+) -> None:+    app = resolver(enable_validation=True)+    router = router()+    path_params = {+        "str_param": "str_value",+        "int_param": 3,+    }++    def bar(app: APIGatewayRestResolver, next_middleware: NextMiddleware) -> Response[str]:+        assert app.context.path_parameters == path_params+        return next_middleware(app)++    @router.route(rule="/<str_param>/<int_param>", method=["GET"], middlewares=[bar])+    def foo(str_param: Annotated[str, Path()], int_param: Annotated[int, Path()]) -> Response[str]:+        return Response(status_code=200, body="routed")++    app.include_router(router)+    event = load_event(event_file)+    event["path"] = event["rawPath"] = f"/{path_params['str_param']}/{path_params['int_param']}"+    result = app(event, {})+    assert result["body"] == @.(+    "router,resolver,event_file",+    [+        (ALBRouter, ALBResolver, "albEvent.json"),+        (APIGatewayRouter, APIGatewayRestResolver, "apiGatewayProxyEvent.json"),+        (APIGatewayHttpRouter, APIGatewayHttpResolver, "apiGatewayProxyV2Event_GET.json"),+        (LambdaFunctionUrlRouter, LambdaFunctionUrlResolver, "lambdaFunctionUrlEvent.json"),+    ],+)+def test_path_parameters_static_path(+    router: Router,+    resolver: ApiGatewayResolver,+    event_file: str,+) -> None:+    app = resolver(enable_validation=True)+    router = router()++    def bar(app: APIGatewayRestResolver, next_middleware: NextMiddleware) -> Response[str]:+        assert app.context.path_parameters == {}+        return next_middleware(app)++    @router.route(rule="/static", method=["GET"], middlewares=[bar])+    def foo() -> Response[str]:+        return Response(status_code=200, body="routed")++    app.include_router(router)+    event = load_event(event_file)+    event["path"] = event["rawPath"] = "/static"+    result = app(event, {})+    assert result["body"] == "routed"
—
Reply to this email directly, view it on GitHub
https://github.com/aws-powertools/powertools-lambda-python/issues/4609#issuecomment-2184937310,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAZPQBCPW6VPWQM7EXGAOILZI2QGJAVCNFSM6AAAAABJYH5IB6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBUHEZTOMZRGA
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***
.com>

Hey Kim, did you face any issues accessing via the app.current_event within the middleware? The app instance is the same for both middlewares and routes by design.

No, there is no issue with that. The problem is that I would like to have access to the parsed path parameters in middlewares. This is available in app.current_event for API Gateway, where it's a native feature of API Gateway and part of the Lambda event ("pathParameters"): https://docs.aws.amazon.com/lambda/latest/dg/services-apigateway.html#apigateway-example-event
However, other resolvers like ALB don't have this luxury since routing is handled exclusively with ALBResolver and path parameters aren't part of the Lambda event: https://docs.aws.amazon.com/lambda/latest/dg/services-alb.html
In a route, path parameters can be accessed like this:
@router.route(rule="/<str_param>/<int_param>", method=["GET"], middlewares=[bar])
def foo(str_param: Annotated[str, Path()], int_param: Annotated[int, Path()]) -> Response[str]:
    return Response(status_code=200, body=f"{str_param}/{int_param}")

But the middleware bar would, as far as I'm aware, need to parse them from the raw path or use the undocumented app.context["_route_args"] (unless using an API Gateway resolver).

Gotcha! It sounds like a stable albeit longer solution would be to create a
standardised Request object (I remember there’s an issue for that).
Similar to Starlette, this would also help rely less on the event structure
and do it the legwork for customers.
We’re making a release this Thursday but if you’d like to help speed this
up in July perhaps, we’d love help drafting a Request object to expose
path, query string (and multi-value), headers (and multi-value), cookies,
etc.
On Sun, 23 Jun 2024 at 18:58, Kim Säppi @.***> wrote:

Hey Kim, did you face any issues accessing via the app.current_event
within the middleware? The app instance is the same for both middlewares
and routes by design.
No, there is no issue with that. The problem is that I would like to have
access to the parsed path parameters in middlewares. This is available in
app.current_event for API Gateway, where it's a native feature of API
Gateway and part of the Lambda event ("pathParameters"):
https://docs.aws.amazon.com/lambda/latest/dg/services-apigateway.html#apigateway-example-event
However, other resolvers like ALB don't have this luxury since routing is
handled exclusively with ALBResolver and path parameters aren't part of
the Lambda event:
https://docs.aws.amazon.com/lambda/latest/dg/services-alb.html
In a route, path parameters can be accessed like this:
@router.route(rule="/<str_param>/<int_param>", method=["GET"], middlewares=[bar])def foo(str_param: Annotated[str, Path()], int_param: Annotated[int, Path()]) -> Response[str]:
return Response(status_code=200, body=f"{str_param}/{int_param}")
But the middleware bar would, as far as I'm aware, need to parse them
from the raw path or use the undocumented app.context["_route_args"]
(unless using an API Gateway resolver).
—
Reply to this email directly, view it on GitHub
https://github.com/aws-powertools/powertools-lambda-python/issues/4609#issuecomment-2185153117,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAZPQBHM2RZP6JSEJJGC3H3ZI35ERAVCNFSM6AAAAABJYH5IB6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBVGE2TGMJRG4
.
You are receiving this because you commented.Message ID:
@.***
com>

I think that sounds very good. I wasn't able to find the issue right now, but I'll keep an eye out. Perhaps it doesn't make sense to implement a temporary solution if a better solution is being planned. Maybe it's best to close this ticket?

Turns out the issue I mentioned was the precursor to have our new Data Validation feature (#1955) - we inject request Payloads, Headers, Query String, Path parameters, etc. inferring from type annotation.
We should keep this issue altogether. Unless @leandrodamascena disagrees, I think we should add this feature into the data validation middleware only to not confuse customers with the DX. If we see a Request model, we inject it.
If we stick by WSGI "standard", we can keep it simple like Starlette while handling multi-value query string and multi-value headers.

We can discuss this when you get back from the PTO @heitorlessa!


Tracking algorithm in WebXRImageTracking

Hi,
Wondering what the algorithm used in WebXRImageTracking. For example, how to detect or register an image, how to get the perspective/distance?

@TinaLiuArcher  was this discussed on the forum ?
cc @RaananW

WebXRImageTracking is using WebXR's native Marker Tracking feature (https://immersive-web.github.io/marker-tracking/). It depends on the underlying system. The algoruthm was not implemented by us.


Question about Queries when using SSR in client components

So i have a question when using NextJS client side components with apollo client SSR mode and where the queries are stored.
in this blog post: https://www.apollographql.com/blog/how-to-use-apollo-client-with-next-js-13
we have this following code snippet at the bottom
"use client";

import { useSuspenseQuery } from "@apollo/experimental-nextjs-app-support/ssr";

import { gql } from "@apollo/client";

const query = gql`query Now {
    now(id: "1")
}`;

export default function Page() {
  const { data } = useSuspenseQuery(query);

  return <main>{data.now}</main>;
}

So my question when we execute a query in the client side component, is the graphql query sent to the server then executed on the server and result sent back to the client component, or is the query stored server side and executed server side and results only sent to the client side? (basically writing a server side code but in client components)
If the query is stored on the client and sent to the server and executed as is, i feel like there could be some security problems if an user changes the query on the client side before executing it on the server using the SSR since even if the query is executed on the server it might not be what the original developer intended if modified by a third party

Despite their name, in Next.js Client Components execute on the server for the first page load.
These is nothing transported from client to server. Your component executes on the server, and we transport the result over so you don't make the same network request a second time.

Okay i see, so even if it looks like we are writing it on the client etc, its actually all executed on the server even if the graphql query itself is in a client component so the client never knows about the query itself or can change manipulate it in any way. Only the query result are transported to the client
Thanks for your clarification it really helped me decide on how to structure my project security wise in the simplest way possible


Add option to show pn.json with plchostname

Current option --sp shows pn.json with IP address as EndpointUrl. New option --sph shows plchostname as EndpointUrl.

Add functionality to generate publisher config with hostnames and minor changes.


PHP error and Customize is not working

Issue description:
by enabling Kirki plugin customizer appearance is not working, I get a critical error on-site health, an active PHP session was detected. "A PHP session was created by a session_start() function call. This interferes with REST API and loopback requests. The session should be closed by session_write_close() before making any HTTP requests.”
Version used:
4.0.21
Using theme_mods or options?
theme_mods
PHP error messages that might be related

JS error messages that might be related

Code to reproduce the issue (config + field(s))

Hi @mukhtar-88,

what theme are you using?
have you notified the original theme developer about the issue?
can you please try to temporarily disable all other plugins to see if that makes any difference?

If that didn't work for you, can you please temporarily enable debug + logging as explained here (https://wp-pagebuilderframework.com/docs/debugging/) and recreate the issue? After that, can you please paste what's in the debug log file?
Thank you.

Is there a fix for this?

Hey @mukhtar-88 , can you please update to the latest version (v4.0.22)?

Is there a fix for this?

Hi @fudojahic , the latest version (v4.0.22) brought some fixes. Can you try to update that version?
Thanks!

Is there a fix for this?

Hi @fudojahic , the latest version (v4.0.22) brought some fixes. Can you try to update that version? Thanks!

I did and it didin't fix it I am also running  5.9.1 version wordpress. I had to revert KIRKI to 3.1.9 to make it work!

Everything is up to date and Customizer is not working when I enable Kirki Customizer Framework plugin, so I have to keep it disabled to use customizer. When the plugin is enabled I get the below error when I open the customizer from dashboard Apearance > Customize. I did disable all other plugins the issue is with Kirki only others are all working fine.
There has been a critical error on this website. Please check your site admin email inbox for instructions.
Learn more about troubleshooting WordPress.
and I am using eLearni theme Version: 2.2
By the DesignThemes team

Hi @mukhtar-88,
Thank you for the additional details.
Just real quick,

have you notified the original theme developer about the issue?
can you please try to temporarily disable all other plugins to see if that makes any difference?

If that didn't work for you, can you please temporarily enable debug + logging as explained here (https://wp-pagebuilderframework.com/docs/debugging/) and recreate the issue? After that, can you please paste what's in the debug log file?

I have the same problem reported by our friend.
realize it has to do with updating to the latest versions of PHP
A PHP session was created by calling the session_start() function. This interferes with REST API and loopback requests. The session must be closed by session_write_close() before making any HTTP requests.
The REST API is a way for WordPress, and other applications, to communicate with the server. An example is the block editor canvas, which relies on the API to display and save your posts and pages.
The REST API request failed due to the following error.
Error: cURL error 28: Operation timed out after 10003 milliseconds with 0 bytes received (http_request_failed)
PHP Compatibility Checker plugin diagnosed the following conflicts
FILE: /home1/danielw10/public_html/wp-content/plugins/kirki/packages/kirki-framework/util/src/Helper.php
FOUND 0 ERRORS AND 1 WARNING AFFECTING 1 LINE
48 | WARNING | Since PHP 7.0, functions inspecting arguments, like func_get_args(), no longer report the original value as passed to a parameter, but will instead provide the current value. The parameter "$array" was used, and possibly changed (by reference), on line 36.

I resolved the situation by installing version 4.0.21 and then upgrading to the latest version 4.0.22 of the Kirki Customizer Framework directly from WordPress.
At least it didn't show up but the error message in wordpress diagnostics.

Hi @Danielw10,
thank you for sharing these additional details. We are going to look into this.

Hey @Danielw10,
quick question - what exact PHP version are you running?
Thank you.

I upgraded to PHP 7.3 but had this problem with kirki, I believe it's an incompatibility issue. And the last alternative was to go back to PHP 7.2 where in a way there was a certain stability in terms of response speed with the server, but the problem persists since PHP 7.0 version as reported in the wordpress diagnostics and in the PHP Compatibility Checker plugin.
I'm using the latest version of kirk-framework : 4.0.22

?


MqttOpenTest.testConnectRetrySub AssertionFailedError

The test output file is from Windows.
TEST-quarks.test.connectors.mqtt.MqttOpenTest.xml.txt
Here's an excerpt:

junit.framework.AssertionFailedError: 11bd1462_bb66_44ff_a1c1_4045e01339ff_testConnectRetrySub_sub contents:[]
at quarks.test.topology.TopologyAbstractTest.completeAndValidate(TopologyAbstractTest.java:85)
at quarks.test.connectors.common.ConnectorTestBase.completeAndValidate(ConnectorTestBase.java:90)
at quarks.test.connectors.common.ConnectorTestBase.completeAndValidate(ConnectorTestBase.java:79)
at quarks.test.connectors.mqtt.MqttStreamsTestManual.testConnectRetrySub(MqttStreamsTestManual.java:916)

@mnwone pls close this.  I know Kathey was getting clean runs on Windows recently.  A new Apache Quarks Jira can be created if this happens again.  Thanks.


cable causing client/server crash

Issue description:
Cable is causing a server crash
What happens:
Server crashes on log in or single player client crashes on block break
What you expected to happen:
Not that ^
Steps to reproduce:
¯_(ツ)_/¯
Version (Make sure you are on the latest version before reporting):

Minecraft: 1.10.2
Forge: FTBserver-1.10.2-<IP_ADDRESS>5-universal
Refined Storage: refinedstorage-1.2.18

Does this issue occur on a server? [yes/no]
yes
If a (crash)log is relevant for this issue, link it here:
crash-2017-01-25_23.32.00-server.txt

Why are you saying that you are on Refined Storage 1.2.18 while the crashlog says you're on 1.2.17?
UCHIJAAAA	refinedstorage{1.2.17} [Refined Storage] (refinedstorage-1.2.17.jar)
Update to 1.2.18, it fixed the issue.

Sorry the crash report is from Refined Storage 1.2.17 but it is also occurring on 1.2.18... this bug is still very much valid.

crash-2017-01-26_23.26.37-server.txt

That is another issue.

Nope, it occurs in another place. So it's an other issue.

It seem to have something to do with a cable BEFORE a cable connected to crafters, but I'd take the reports word for it not mine ;)

Even though they look identical (same error) they are ocurring at another place in the code. That's why it's important to absolutely include the crashlog from the latest version, not an older one.

yeah I'm not sure why I attached the 1.2.17 one, that's my bad... hopefully you can get it sorted, I was a hardcore AE fan and not immediately sold on RS, but tbh I love it man. Keep up the good work.

Thank you, shouldn't be hard to fix. Expect a release this weekend.


[UI] Fix build warnings for Meshery UI

Current Behavior

Ensure that all the build warnings for Meshery UI are completely fixed.

Screenshots/Logs

Environment

Browser: Chrome Safari Firefox
Host OS: Mac Linux Windows
Meshery Server Version: stable-v
Meshery Client Version: stable-v
Platform: Docker Kubernetes

Contributor Guides and Handbook

🎨 Wireframes and designs for Meshery UI in Figma (open invite)
🖥 Contributing to Meshery UI
🙋🏾🙋🏼 Questions: Discussion Forum and Community Slack

@Pushkarm029 up for grab?

@theBeginner86 Hey!, I solved this issue, can I give a Pull request?

is this issue resolved or can i work on it ?

I am working on it.

@theBeginner86  I think this issue may be already resolved, or am I not able to reproduce it.

Yes, looks like all are fixed now.
Thank you @Pushkarm029 for confirming 👍


Login scopes and responsive issues on mobile.

Check the following Screenrecord, tested with iPhone 12 mini, iOS 17.3.1:
https://github.com/medplum/medplum/assets/28322469/30cffc18-8af9-48b8-a43b-b2b1933f401d
What's happening
When login on Medplum App from a mobile:

The login form doesn't fit well in the screen, the user has to scroll horizontally to get to the Next and Sign in buttons.
The login shows a last step screen after user password, asking to Choose scope and a list of pre-checked checkboxes.

What I expect to happen

The login form should fit the mobile screen so the user doesn't have to scroll horizontally to get to the buttons.
Scopes should not be asked to confirm if they are already set on Authentication request, like this request:

GET https://api.medplum.com/oauth2/authorize?
    response_type=code&
    client_id=XXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXX&
    redirect_uri=https://my.app.com/redirect&
    state=XXXXXXXXXXXX&
    scope=offline_access openid email phone profile

Confirmed, we'll get you a fix soon.

Hi @codyebberson thanks for the feedback.

Regarding the Scopes UI, understood, I think it would be good having that setting on ClientApplication so there is no need to make any changes on auth request.
Regarding the responsiveness on the login UI it has improved but still has some unexpected behavior. When the user taps on the InputField there is a ZoomIn that causes some parts of the form to be cropped so the "Next" button looks cropped and again the user has to scroll horizontally, the same happens in the Password screen if the user wants to tap on the Show/Hide password, it's mandatory to scroll horizontally. Check the following Screenrecord.

https://github.com/medplum/medplum/assets/28322469/59391431-883d-4467-b8c1-5a68266b6bd9

@codyebberson you're right, changing from current maximum-scale=5 to maximum-scale=1 does the trick, I just tested it, so with the meta tag you proposed is enough:
<meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1, minimum-scale=1" />

by the way just with this would be enough:
<meta name="viewport" content="user-scalable=no,initial-scale=1,maximum-scale=1">

Hi @codyebberson any progress on this? Last time we agreed the responsive behavior only misses that minor change in the viewport, but what about the login scopes, is it going to be a queryParam or a conf in the ClientApplication?

Hi @codyebberson is this going to be fixed any time soon?

The responsive behavior in mobile screens still misses the fix we agree above:

<meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1, minimum-scale=1" />

And regarding the login scopes? You mentioned above some option to override the requirement like a query parameter in the url or a configuration setting on the ClientApplication, any of those approaches works good for me (preferably the ClientApplication setting to let the authorization request unchanged), another way could be to keep the scopes confirmation screen for the user to grant it as SMART-on-FHIR mandates but at least showing each scope with a user friendly text like Google and other IDP does with their users for instance:

:white_check_mark: Allow to see my profile, like name, birthdate and picture (profile)
:white_check_mark: Allow to see my email (email)
:white_check_mark: Allow to see my phone (phone)
:white_check_mark: Remember me (offline_access)
Instead of what currently shows up to the user which not necessarily understands those terms:


Installation via PIP

I had some issues installing via pip as it didn't pull dependencies, and could not import properly after installation.
(tmp) ➜  /tmp pip install meraki
Collecting meraki
Installing collected packages: meraki
Successfully installed meraki-0.34
(tmp) ➜  /tmp python
Python 2.7.10 (default, Feb 22 2018, 22:42:27)
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-9<IP_ADDRESS>)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from meraki import meraki
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named meraki
>>>

I added a setup.py and restructured slightly for what I think would be the required updates, feel free to PR if you like
https://github.com/meraki/dashboard-api-python/compare/master...kecorbin:master

You're using python (2.7.10) when python3 (3.6) is required for this project.

Regardless of 2.7/3.6, my issue was that pip install meraki did not drag the dependencies with it.
Python 3.6.5 (default, Mar 30 2018, 06:41:53)
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-9<IP_ADDRESS>)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from meraki import meraki
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/kecorbin/VirtualEnvs/meraki/lib/python3.6/site-packages/meraki/meraki.py", line 17, in <module>
    import requests
ModuleNotFoundError: No module named 'requests'

I was simply suggesting an improvement to what is a pretty cool project.  Also the lib worked fine for me in 2.7 with the proposed changes.


"Waiting for" phase not working depending on the resource version

The "waiting for" phase is working only for Deployment declared as v1beta1 and StatefulSet declared as v1beta1 versions.
Spray should also wait Deployment for versions v1 and v1beta2, and StatefulSet for version v1.

Closed by https://github.com/gemalto/helm-spray/pull/30


Added overlay

Issue: #83 Add overlay when sidebar opens on small devices
I added a div as an overlay to darken everything but the sidebar on smaller devices.

Done!

Oops, fixed that!

Amazing work! Thank you.
@all-contributors please add @JLambertazzo for code


testing that linter fails cancels tests

Chart tests were skipped because we failed linter. Proving the change works. Closing ticket.


Support top level async

@thdxr mentioned in slack:

Using top level await requires use of esm which means setting type: "module" to true.  This makes it so require statements cannot be used.
As part of the deploy process a script gets generated at .build/run.js which does use require statements and is unable to run.  Possibly can fix it by renaming it to .build/run.cjs

Link to Slack discussion: https://serverless-stack.slack.com/archives/C01JG3B20RY/p1623769542372000
Link to AWS announcement of top level async: https://aws.amazon.com/about-aws/whats-new/2021/02/aws-lambda-now-supports-node-js-14/

This paves the way for creating an easy way to load SSM/Secrets Manager parameters at startup


Home Assistant loses TOD sensors state after restart.

Home Assistant release with the issue:

Home Assistant 0.94.3
Last working Home Assistant release (if known):
Unknown
Operating environment (Hass.io/Docker/Windows/etc.):

Raspbian Stretch
Component/platform:

Time of day (TOD)
Description of problem:
All TOD sensors lose their state and become 'off' after each HA restart and don't update until next appropriate event (may be several hours).
Expected: keep the state which TOD had before HA restart.
Problem-relevant configuration.yaml entries and (fill out even if it seems unimportant):
binary_sensor:
  - platform: tod
    name: tod_evening
    after: '20:00'
    before: '23:00'
  - platform: tod
    name: tod_full_day
    after: sunrise
    before: sunset

Traceback (if applicable):

Additional information:

I still wait for the update, since the platform can't be used now in case HASS needs to be restarted during the day.

Can't reproduce

  - platform: tod
    name: No Sun Time
    after: sunset
    before: sunrise

Try to restart HA after midnight.

@Erelen-Laiquendi there is no difference between after: sunset and before: sunrise. Try to use only one condition :)

https://www.home-assistant.io/components/tod/#considerations

I can confirm. In 0.96.5 in case you restart HA after sunset and before sunrise all TODs have 'off' value, either it uses sunset/sunrise or time interval. So, it still doesn't work as should be.

So many versions no any update of so serious error. Such way there is no use of TOD component.

Still waiting for the fix. 0.99.3

Not updated yet...100.3

Same problem on version 0.101.3

it's 103.3 but still not fixed - once you get your instance rebooted over night, you TOD night sensor (21:00 - 07:00) not working any more...homeassistant.update_entity do not help. Only manual change via Developer Tools-State works...
Guy please fix it...i think it's and\or issue - i mean it's checking current time to be between from to...and its not working for over night. ie if right now is 1am and my TOD specified from 21 to 07, component checks if
(now>start_time and now<end_time)
with actual values it will be
(01>21 and 01<07) == FALSE

I promised to come back and give a final feedback. And yes, with 0.105 this issue is fully resolved. Thanks to all that participated in its resolution


How to get access to data from a store?

I have read how to combine stores to exchange data between them
https://gist.github.com/gaearon/d77ca812015c0356654f
but I still wonder, how do I get access to what the state managed by a store programmatically? Maybe from a store, maybe from an action creator.
Example: You want to store an invoice to a server, but you need to get information about a user (id perhaps) from another store to do that.
I read on twitter
https://twitter.com/dan_abramov/status/621679403756601344
that you have to pass that information (id of user) along with the action. Does that mean the component will already get that information from a store an add it when it call the action creator?
I know this got a bit involved, I hope it still makes sense :)

Suppose, you already have the user details in the users store but not on the rendered component (invoice), to pass user_id with the save action (button click). One thing you can do is access the store's current state through connected component's this.context.store.getState()

Given the 1.0.0-rc version, in redux there is usually only one store, which is the instance you get from createStore().
The "flux stores" in redux are just reduce function, they don't hold any state.
If you return a function in the action creators, the arguments you get are dispatch and getState. The first one is used to dispatch your action object after an async operation, the second one is used to get some extra data from the store.
For example https://github.com/gaearon/redux/blob/breaking-changes-1.0/examples/counter/actions/CounterActions.js#L16
In your case it could look something like this:
function saveInvoice (data) {
  return (dispatch, getState) => {
    // get the user state
    const { user } = getState()

    fetch(invoiceUrl, {
      method: 'post',
      body: JSON.stringify({
        userId: user.id,
        invoice: data
      })
    })
    .then(res => res.json())
    .then(res => dispatch({
      type: 'INVOICE_SAVE',
      payload: res
    }))
  }
}

@emmenko if promise middleware is used, shouldn't you return the fetch request?

I'm not using the promise middleware here, just the thunk. You may want to have a look here for promise support.

Thanks to @tundrax and @emmenko, great answers.
To wrap up: I can

EITHER compose all the information I need in a controller component and pass it along as payload to the action
OR add the missing information in an action creator using getState that I get passed in from redux

Correct? If so: Do you have any preference which way yo use?

Both are OK and it depends on your specific use case.
For example if you have 10 controllers that look up identical information from the same state slice and pass it to action creator, it's easier to do this inside the action creator itself.
Finally, you can always have a “vanilla” action creator that just returns an action and accepts a parameter, and a “smarter” action creator that uses getState and calls dispatch(vanillaActionCreator(someParam, getState().something)).

It seems obvious to me that thunk callbacks should accept a single argument: the store.
The existing signature is easily changed to:
function myActionCreator() {
  return ({ dispatch, getState }) => {
    // ... do stuff ...
  };
}

But for plugins like redux-persist, we're able to access the store via actionCreators, e.g.
function purgePersistedState() {
  return (store) => {
    store.persistor.purge();
  };
}

How else are we supposed to expose store-related extensions to actions creators?

@jchook : not sure I understand your use case.  What do you mean by "expose store-related extensions" ?
Also, note that redux-thunk is simply one possible middleware.  You can certainly write your own custom middleware that passes in a different API to  functions, although middleware don't actually get the "real" store reference - they get a wrapped/subset of the store API.