containers stuck terminating / creating - many runc init processes

Description
Hi, I raised an issue https://github.com/opencontainers/runc/issues/3448 in the runc repo, but as you can see the issue appears to be that containerd (1.6.2) is not started the created containers.

The issue affected basically every node as soon as they came up
Have other clusters running the same setup which makes me think it is triggered by load or a certain type of workload
Kubernetes 1.22.8
Most pods, after failing to start were then met with OutOfcpu making me think that the CPU requests were being accounted for even if nothing was running (all nodes basically idle).
Switching to docker CRI "fixed"
Nodes are set up to have both docker and containerd present, can switch to either runtime with a configuration change to point kubelet to either.
Restarting containerd service fixes.
Running containerd-stress density will even fail to start a container and is stuck in the same way.

Steps to reproduce the issue

Scale up a workload so the nodes are fully committed (e.g. cluster-autoscaler will start creating nodes)
Wait for node to start accumulating runc init processes. Containerd logs will report report errors creating containers/sandboxes etc.

Describe the results you received and expected
We have a way to switch container runtime from containerd -> docker. We expected containerd to work in the same way as docker.
What version of containerd are you using?
1.6.2
Any other relevant information
runc version, CRI configuration, OS/Kernel version, etc.
runc
VERSION:
   1.0.3
commit: v1.0.3-0-gf46b6ba
spec: 1.0.2-dev
go: go1.16.15
libseccomp: 2.5.1

(I also tried on 1.1.1 with same results which makes me think the issue could be with the shim?)
{
  "status": {
    "conditions": [
      {
        "type": "RuntimeReady",
        "status": true,
        "reason": "",
        "message": ""
      },
      {
        "type": "NetworkReady",
        "status": true,
        "reason": "",
        "message": ""
      }
    ]
  },
  "cniconfig": {
    "PluginDirs": [
      "/opt/cni/bin"
    ],
    "PluginConfDir": "/etc/cni/net.d",
    "PluginMaxConfNum": 1,
    "Prefix": "eth",
    "Networks": [
      {
        "Config": {
          "Name": "cni-loopback",
          "CNIVersion": "0.3.1",
          "Plugins": [
            {
              "Network": {
                "type": "loopback",
                "ipam": {},
                "dns": {}
              },
              "Source": "{\"type\":\"loopback\"}"
            }
          ],
          "Source": "{\n\"cniVersion\": \"0.3.1\",\n\"name\": \"cni-loopback\",\n\"plugins\": [{\n  \"type\": \"loopback\"\n}]\n}"
        },
        "IFName": "lo"
      },
      {
        "Config": {
          "Name": "aws-cni",
          "CNIVersion": "0.4.0",
          "Plugins": [
            {
              "Network": {
                "name": "aws-cni",
                "type": "aws-cni",
                "ipam": {},
                "dns": {}
              },
              "Source": "{\"mtu\":\"9001\",\"name\":\"aws-cni\",\"pluginLogFile\":\"/var/log/aws-routed-eni/plugin.log\",\"pluginLogLevel\":\"DEBUG\",\"type\":\"aws-cni\",\"vethPrefix\":\"eni\"}"
            },
            {
              "Network": {
                "name": "egress-v4-cni",
                "type": "egress-v4-cni",
                "ipam": {
                  "type": "host-local"
                },
                "dns": {}
              },
              "Source": "{\"enabled\":\"false\",\"ipam\":{\"dataDir\":\"/run/cni/v6pd/egress-v4-ipam\",\"ranges\":[[{\"subnet\":\"<IP_ADDRESS>/22\"}]],\"routes\":[{\"dst\":\"<IP_ADDRESS>/0\"}],\"type\":\"host-local\"},\"mtu\":9001,\"name\":\"egress-v4-cni\",\"nodeIP\":\"<IP_ADDRESS>\",\"pluginLogFile\":\"/var/log/aws-routed-eni/egress-v4-plugin.log\",\"pluginLogLevel\":\"DEBUG\",\"type\":\"egress-v4-cni\"}"
            },
            {
              "Network": {
                "type": "portmap",
                "capabilities": {
                  "portMappings": true
                },
                "ipam": {},
                "dns": {}
              },
              "Source": "{\"capabilities\":{\"portMappings\":true},\"snat\":true,\"type\":\"portmap\"}"
            }
          ],
          "Source": "{\n  \"cniVersion\": \"0.4.0\",\n  \"name\": \"aws-cni\",\n  \"disableCheck\": true,\n  \"plugins\": [\n    {\n      \"name\": \"aws-cni\",\n      \"type\": \"aws-cni\",\n      \"vethPrefix\": \"eni\",\n      \"mtu\": \"9001\",\n      \"pluginLogFile\": \"/var/log/aws-routed-eni/plugin.log\",\n      \"pluginLogLevel\": \"DEBUG\"\n    },\n    {\n      \"name\": \"egress-v4-cni\",\n      \"type\": \"egress-v4-cni\",\n      \"mtu\": 9001,\n      \"enabled\": \"false\",\n      \"nodeIP\": \"<IP_ADDRESS>\",\n      \"ipam\": {\n         \"type\": \"host-local\",\n         \"ranges\": [[{\"subnet\": \"<IP_ADDRESS>/22\"}]],\n         \"routes\": [{\"dst\": \"<IP_ADDRESS>/0\"}],\n         \"dataDir\": \"/run/cni/v6pd/egress-v4-ipam\"\n      },\n      \"pluginLogFile\": \"/var/log/aws-routed-eni/egress-v4-plugin.log\",\n      \"pluginLogLevel\": \"DEBUG\"\n    },\n    {\n      \"type\": \"portmap\",\n      \"capabilities\": {\"portMappings\": true},\n      \"snat\": true\n    }\n  ]\n}"
        },
        "IFName": "eth0"
      }
    ]
  },
  "config": {
    "containerd": {
      "snapshotter": "overlayfs",
      "defaultRuntimeName": "runc",
      "defaultRuntime": {
        "runtimeType": "",
        "runtimePath": "",
        "runtimeEngine": "",
        "PodAnnotations": null,
        "ContainerAnnotations": null,
        "runtimeRoot": "",
        "options": null,
        "privileged_without_host_devices": false,
        "baseRuntimeSpec": "",
        "cniConfDir": "",
        "cniMaxConfNum": 0
      },
      "untrustedWorkloadRuntime": {
        "runtimeType": "",
        "runtimePath": "",
        "runtimeEngine": "",
        "PodAnnotations": null,
        "ContainerAnnotations": null,
        "runtimeRoot": "",
        "options": null,
        "privileged_without_host_devices": false,
        "baseRuntimeSpec": "",
        "cniConfDir": "",
        "cniMaxConfNum": 0
      },
      "runtimes": {
        "runc": {
          "runtimeType": "io.containerd.runc.v2",
          "runtimePath": "",
          "runtimeEngine": "",
          "PodAnnotations": null,
          "ContainerAnnotations": null,
          "runtimeRoot": "",
          "options": {
            "BinaryName": "",
            "CriuImagePath": "",
            "CriuPath": "",
            "CriuWorkPath": "",
            "IoGid": 0,
            "IoUid": 0,
            "NoNewKeyring": false,
            "NoPivotRoot": false,
            "Root": "",
            "ShimCgroup": "",
            "SystemdCgroup": false
          },
          "privileged_without_host_devices": false,
          "baseRuntimeSpec": "",
          "cniConfDir": "",
          "cniMaxConfNum": 0
        }
      },
      "noPivot": false,
      "disableSnapshotAnnotations": true,
      "discardUnpackedLayers": false,
      "ignoreRdtNotEnabledErrors": false
    },
    "cni": {
      "binDir": "/opt/cni/bin",
      "confDir": "/etc/cni/net.d",
      "maxConfNum": 1,
      "confTemplate": "",
      "ipPref": ""
    },
    "registry": {
      "configPath": "",
      "mirrors": null,
      "configs": null,
      "auths": null,
      "headers": null
    },
    "imageDecryption": {
      "keyModel": "node"
    },
    "disableTCPService": true,
    "streamServerAddress": "<IP_ADDRESS>",
    "streamServerPort": "0",
    "streamIdleTimeout": "4h0m0s",
    "enableSelinux": false,
    "selinuxCategoryRange": 1024,
    "sandboxImage": "k8s.gcr.io/pause:3.6",
    "statsCollectPeriod": 10,
    "systemdCgroup": false,
    "enableTLSStreaming": false,
    "x509KeyPairStreaming": {
      "tlsCertFile": "",
      "tlsKeyFile": ""
    },
    "maxContainerLogSize": 16384,
    "disableCgroup": false,
    "disableApparmor": false,
    "restrictOOMScoreAdj": false,
    "maxConcurrentDownloads": 3,
    "disableProcMount": false,
    "unsetSeccompProfile": "",
    "tolerateMissingHugetlbController": true,
    "disableHugetlbController": true,
    "device_ownership_from_security_context": false,
    "ignoreImageDefinedVolumes": false,
    "netnsMountsUnderStateDir": false,
    "enableUnprivilegedPorts": false,
    "enableUnprivilegedICMP": false,
    "containerdRootDir": "/var/lib/containerd",
    "containerdEndpoint": "/run/containerd/containerd.sock",
    "rootDir": "/var/lib/containerd/io.containerd.grpc.v1.cri",
    "stateDir": "/run/containerd/io.containerd.grpc.v1.cri"
  },
  "golang": "go1.17.2",
  "lastCNILoadStatus": "OK",
  "lastCNILoadStatus.default": "OK"
}

Linux $IP 5.4.0-1071-aws #76~18.04.1-Ubuntu SMP Mon Mar 28 17:49:57 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

#   Copyright 2018-2022 Docker Inc.

#   Licensed under the Apache License, Version 2.0 (the "License");
#   you may not use this file except in compliance with the License.
#   You may obtain a copy of the License at

#       http://www.apache.org/licenses/LICENSE-2.0

#   Unless required by applicable law or agreed to in writing, software
#   distributed under the License is distributed on an "AS IS" BASIS,
#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#   See the License for the specific language governing permissions and
#   limitations under the License.

disabled_plugins = []

[metrics]
  address = "<IP_ADDRESS>:1338"

#root = "/var/lib/containerd"
#state = "/run/containerd"
#subreaper = true
#oom_score = 0

#[grpc]
#  address = "/run/containerd/containerd.sock"
#  uid = 0
#  gid = 0

[debug]
#  address = "/run/containerd/debug.sock"
#  uid = 0
#  gid = 0
# supported values [trace, debug, info, warn, error, fatal, panic]
# https://github.com/containerd/containerd/blob/v1.6.1/cmd/containerd/command/main.go#L89
  level = "error"
# https://github.com/containerd/containerd/blob/v1.6.1/log/context.go#L45-L49
  format = "json"

wanted to node, am running both docker and containerd at the same time, both are configured to use the same socket path which I understand could cause contention with cgroup management.
Would this possibly cause the stated issue? Will reconfigure them regardless.

Okay, I think I've found a way to reproduce this which is simply create a large number of pods at once. Did a  k scale deployment $some_largish_workload --replicas 1000, and after about 5 minutes, scale back down.
Lots of pods stuck in Terminating.
containerd logs are filled with
Apr 05 01:51:51 containerd[4743]: {"error":"rpc error: code = DeadlineExceeded desc = failed to stop sandbox container \"88956ab6c466107f184d53b63f9a6f7622aaadd6f2ef5e7967af41e1ef1f11e2\" in \"SANDBOX_READY\" state: wait sandbox container \"88956ab6c466107f184d53b63f9a6f7622aaadd6f2ef5e7967af41e1ef1f11e2\": context deadline exceeded","level":"error","msg":"StopPodSandbox for \"88956ab6c466107f184d53b63f9a6f7622aaadd6f2ef5e7967af41e1ef1f11e2\" failed","time":"2022-04-05T01:51:51.759136525Z"}
Apr 05 01:51:51 containerd[4743]: {"error":"rpc error: code = DeadlineExceeded desc = failed to stop sandbox container \"e1f9a29f5642a2f7893b29ab1f6e391ed3482b0c6e5c0bc0758a2e0f5e634785\" in \"SANDBOX_READY\" state: wait sandbox container \"e1f9a29f5642a2f7893b29ab1f6e391ed3482b0c6e5c0bc0758a2e0f5e634785\": context deadline exceeded","level":"error","msg":"StopPodSandbox for \"e1f9a29f5642a2f7893b29ab1f6e391ed3482b0c6e5c0bc0758a2e0f5e634785\" failed","time":"2022-04-05T01:51:51.759132456Z"}
Apr 05 01:51:51 containerd[4743]: {"error":"rpc error: code = DeadlineExceeded desc = failed to stop sandbox container \"3344542a24a2a1a565175ba41f2cc59bc9edcee972f41e66159b1cc7511bc2bd\" in \"SANDBOX_READY\" state: wait sandbox container \"3344542a24a2a1a565175ba41f2cc59bc9edcee972f41e66159b1cc7511bc2bd\": context deadline exceeded","level":"error","msg":"StopPodSandbox for \"3344542a24a2a1a565175ba41f2cc59bc9edcee972f41e66159b1cc7511bc2bd\" failed","time":"2022-04-05T01:51:51.759147676Z"}

and
Apr 05 01:51:46 containerd[4743]: {"error":"failed to reserve sandbox name \"node-exporter-qk5h5_kube-system_1b750210-d162-4df4-8414-965ebb017395_0\": name \"node-exporter-qk5h5_kube-system_1b750210-d162-4df4-8414-965ebb017395_0\" is reserved for \"1e6cd2a72fcf198817387e4a662af875274ed8ec5bf889211b9cfb81f115b7c3\"","level":"error","msg":"RunPodSandbox for \u0026PodSandboxMetadata{Name:node-exporter-qk5h5,Uid:1b750210-d162-4df4-8414-965ebb017395,Namespace:kube-system,Attempt:0,} failed, error","time":"2022-04-05T01:51:46.457243498Z"}

 ps aux | grep 'runc init' | wc -l
63

sudo lsof -p $(pidof containerd) | wc -l

barely any load
$ vmstat 1
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 1  0      0 184559104<PHONE_NUMBER>808    0    0    14   282  173  374  1  0 98  0  0
 0  0      0 184556992<PHONE_NUMBER>836    0    0     0  1692 3820 8323  0  0 100  0  0
 0  0      0 184555584<PHONE_NUMBER>840    0    0     0     0 2928 7495  0  0 99  0  0
 7  0      0 184553856<PHONE_NUMBER>844    0    0     0     0 3510 9174  1  0 99  0  0
 0  0      0 184548608<PHONE_NUMBER>896    0    0     0   520 7330 18170  1  0 99  0  0
 1  0      0 184550368<PHONE_NUMBER>940    0    0     0     0 6056 14737  0  0 99  0  0
 7  0      0 184549376<PHONE_NUMBER>932    0    0     0     0 6444 18897  1  0 99  0  0

Could you mind to fill the issue with the template, like containerd version, cri config etc.? Thanks

Could you mind to fill the issue with the template, like containerd version, cri config etc.? Thanks

Have updated the OP

So, did a little more digging after I reproduced again.
crictl pods listing pods as ready
POD ID              CREATED             STATE               NAME                               NAMESPACE            ATTEMPT             RUNTIME
e61e65ad28451       9 minutes ago       Ready               kube2iam-9dv4k                     kube-system          0                   (default)
258739eb027c3       9 minutes ago       Ready               node-exporter-kt8mm                kube-system          0                   (default)
4edb5e2af14a2       9 minutes ago       Ready               sensor-d8577f7bc-7bbkp             stackrox             0                   (default)
98e621bf432c1       9 minutes ago       Ready               sensor-d8577f7bc-7bvnq             stackrox             0                   (default)
dd4e78f2295a8       9 minutes ago       Ready               sensor-d8577f7bc-w9lfr

kubectl is listing these all as Terminating.
All the pods are (critctl inspectp ...)
    "state": "SANDBOX_READY",

Trying to start one
DEBU[0000] get runtime connection
DEBU[0000] connect using endpoint 'unix:///run/containerd/containerd.sock' with '2s' timeout
DEBU[0000] connected successfully using endpoint: unix:///run/containerd/containerd.sock
DEBU[0000] StartContainerRequest: &StartContainerRequest{ContainerId:c2d99d797f8b2,}
DEBU[0000] StartContainerResponse: nil
FATA[0000] starting the container "c2d99d797f8b2": rpc error: code = Unknown desc = failed to set starting state for container "c2d99d797f8b2ce049de65996a11a1f69264994ee3e98a49762208400a3636be": container is already in starting state

Stop (hangs which probably explains everything stuck in terminating state)
DEBU[0000] get runtime connection
DEBU[0000] connect using endpoint 'unix:///run/containerd/containerd.sock' with '2s' timeout
DEBU[0000] connected successfully using endpoint: unix:///run/containerd/containerd.sock
DEBU[0000] StopPodSandboxRequest: &StopPodSandboxRequest{PodSandboxId:cb7c349224ef6,}

crictl ps -a
8836dee036e7f       c665473c2d1a8       19 minutes ago       Created             sensor                  0                   8a7deac1b642b
a59b45d544f76       c665473c2d1a8       19 minutes ago       Created             sensor                  0                   160cc44eb7c73
db79b04a35a7b       c665473c2d1a8       19 minutes ago       Created             sensor                  0                   60cd9518b2f7b
7a2e19dc692a8       c665473c2d1a8       19 minutes ago       Created             sensor                  0                   e61a1f8611d35
ca2b96e347096       c665473c2d1a8       19 minutes ago       Created             sensor                  0                   cb7c349224ef6
70618e5bcee8f       c665473c2d1a8       19 minutes ago       Created             sensor                  0                   3c6d3a8271d71
671d994a87f9e       c665473c2d1a8       19 minutes ago       Created             sensor                  0                   08db027c453c7
2c3286678513e       c665473c2d1a8       19 minutes ago       Created             sensor                  0                   63ddb9ad380de
c2d99d797f8b2       c665473c2d1a8       19 minutes ago       Created             sensor                  0                   748fd9b0028f1
07e165f398e36       c665473c2d1a8       19 minutes ago       Created             sensor                  0                   163358db1e87d
b43cb402e379c       c665473c2d1a8       19 minutes ago       Created             sensor                  0                   1360a745aecc4
9f3e59bf3afbf       c665473c2d1a8       19 minutes ago       Created             sensor                  0                   f92c831f7f95d

{
  "status": {
    "id": "c2d99d797f8b2ce049de65996a11a1f69264994ee3e98a49762208400a3636be",
    "metadata": {
      "attempt": 0,
      "name": "sensor"
    },
    "state": "CONTAINER_CREATED",
    "createdAt": "2022-04-06T01:08:44.498852593Z",
    "startedAt": "0001-01-01T00:00:00Z",
    "finishedAt": "0001-01-01T00:00:00Z",
    "exitCode": 0,
    "image": {

using runc

runc --root /run/containerd/runc/k8s.io/ state d30e0c0d500fabbeb7c3fa645b7f5e3178d015c87e0e58123a12d8f31de85eb6
{
  "ociVersion": "1.0.2-dev",
  "id": "d30e0c0d500fabbeb7c3fa645b7f5e3178d015c87e0e58123a12d8f31de85eb6",
  "pid": 46995,
  "status": "created",
  "bundle": "/run/containerd/io.containerd.runtime.v2.task/k8s.io/d30e0c0d500fabbeb7c3fa645b7f5e3178d015c87e0e58123a12d8f31de85eb6",
  "rootfs": "/run/containerd/io.containerd.runtime.v2.task/k8s.io/d30e0c0d500fabbeb7c3fa645b7f5e3178d015c87e0e58123a12d8f31de85eb6/rootfs",
  "created": "2022-04-06T01:30:27.819165909Z",
  "annotations": {
    "io.kubernetes.cri.container-name": "kube2iam",
    "io.kubernetes.cri.container-type": "container",
    "io.kubernetes.cri.image-name": "foo",
    "io.kubernetes.cri.sandbox-id": "8f84b0fce446f14ae493a2bb6612e08c4a33c2f86eacf3bb0e8f19935c861787",
    "io.kubernetes.cri.sandbox-name": "foo",
    "io.kubernetes.cri.sandbox-namespace": "kube-system"
  },
  "owner": ""

@alam0rt is there too many  runc init processes?  Thanks.

@alam0rt is there too many runc init processes?  Thanks.

i will do this once i can trigger the issue again - thanks

containerd-goroutines.gz
shims-goroutine-dump.gz (contains every shim goroutine output so be warned, most are "stuck" so lots of examples :) )

thanks. It is caused by dead lock in metrics so that there is no task.Start to kick runC-init. Seeking the solution to fix it.

@alam0rt  Did the issue resolve for you?
I tried with containerd 1.6.4 but I am still facing this issue.


Build fails

Build fails on conda env create -f environment.yml with the error:
Solving environment: failed
ResolvePackageNotFound:

xz==5.2.2=1
libpng==1.6.27=0
readline==6.2=2
mkl==2017.0.3=0
bzip2==1.0.6=3
numpy==1.13.0=py35_0
tk==8.5.18=0

Am on Ubuntu and am also experiencing this issue, it seems like those have been pulled out from their repo


Should the optional WavelengthBands be replaced by a mechanism like IofQxy vs IofQ?

In the SANS workflow, we currently have an optional WavelengthBands parameter which if left as None will produce a one-dimensional $I(Q)$ result.
However, if we define it, the workflow will produce $I(Q, {\rm band})$.
This strongly resembles the issue we had with $I(Q)$ vs $I(Qx, Qy)$.
See also in the powder diffraction workflow where we request a result binned in d-spacing and 2theta (as opposed to just d-spacing).
So instead of just setting the WavelengthBands and magically getting something different at the end, we could also require the user to compute e.g. NormalizedIofQinBands?

Wouldn't that mean you need 4 different types? I feel that might be too confusing?


User Interface Design

Story

As a developer,
I should have a wireframe of possible screens to develop against

Acceptance Criteria

Digital drawings submitted in a PR that reflect and label different screens for this application
Should include all possible screens that you can conceive: Splash, Login, Profile, etc
You can use any tool you want to create them, but the output submitted should be a png or jpeg

@vincent12322 @alextadt @rouge888

Do you want the files/images posted here or brought to class?

Do each of us need to submit a copy if we work as a group?
Or would you rather us each create one individually for a variety of choices to pick from?

@alextadt place your images in the repo at docs/prototypes/<yourfiles>

@alextadt Each file should be in the following convention:
-.

MVP Screens After Discussion

Login
Sign up
Real Time Map
Create Spot
Check In


feat: async image validation

Connaisseur now validates all images in side a single request in parallel.
fixes #226

Codecov Report

Merging #334 (7cd25ac) into develop (e45fb1a) will decrease coverage by 0.15%.
The diff coverage is 87.17%.

@@             Coverage Diff             @@
##           develop     #334      +/-   ##
===========================================
- Coverage    96.65%   96.49%   -0.16%
===========================================
  Files           22       22
  Lines         1077     1085       +8
===========================================
+ Hits          1041     1047       +6
- Misses          36       38       +2

Impacted Files
Coverage Δ

connaisseur/flask_server.py
92.92% <84.84%> (-1.58%)
:arrow_down:

connaisseur/validators/cosign/cosign_validator.py
98.61% <100.00%> (ø)

connaisseur/validators/interface.py
100.00% <100.00%> (ø)

...naisseur/validators/notaryv1/notaryv1_validator.py
95.83% <100.00%> (ø)

...naisseur/validators/notaryv2/notaryv2_validator.py
80.00% <100.00%> (ø)

connaisseur/validators/static/static_validator.py
93.75% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e45fb1a...7cd25ac. Read the comment docs.


Adding Tushar

Adding Tushar to the Project

Added and ensured he was able to view the project, access it and modify issues.


API docs don't mention the fact that there needs to be a v1-alpha in the URL

I followed these instructions on getting fleet to listen on port 49153 for REST api calls.
Initially I tried:
curl localhost:49153/machines

which returned a 404.  After some trial and error and googling, I managed to figure it out:
curl localhost:49153/v1-alpha/machines

which returned valid JSON.
I think the docs should be updated to mention that all REST api calls should be prefixed with /v1-alpha/
I'm on CoreOS (stable) 522.6.0.

The docs are tracking 0.9, which is the release where the API was finalized. The current stable channel hasn't been updated with this version, which is why you're seeing /v1-alpha/ work. Sorry for the confusion.
You can find the release notes for CoreOS here: https://coreos.com/releases/

@tleyden are you asking for a blurb be added to that doc telling a user where the API will be hosted once the socket is deployed?

The current stable channel hasn't been updated with this version, which is why you're seeing /v1-alpha/ work

Ah ok.  So on 0.9, you would simply use curl localhost:49153/machines?

are you asking for a blurb be added to that doc telling a user where the API will be hosted once the socket is deployed?

I was just confused about the endpoint URL's, which I think @robszumski cleared up.

@tleyden I don't think he did. The API lives at $IP:$PORT/fleet/v1/... as of v0.9, with backwards compatibility for the .../v1-alpha/... path that we started with. .../machines is not hosted at the root resource in pre-v0.9 or v0.9+

Ah ok.  Yeah I think the docs should be enhanced to mention that the endpoint URL's should be of the form $IP:$PORT/fleet/v1/... as of v0.9.

@tleyden See https://github.com/coreos/fleet/pull/1116

Thanks


Fix semver vulnerability

What's this PR do?

Update the version of semver
Remove unused nodejs module
Add CHANGELOG file

What are the issue related?

semver vulnerable to Regular Expression Denial of Service

Where should the reviewer start?

package.json
index.js
CHANGELOG.md

Hello @cutsin, I hope you're doing well. Could you please take a look at this? Let me now how I can help you.


Schema Generators Completion

Prerequisites:

[x] Refactor to use decoration pattern so we can inherit more functionality per field type generating schema.
[ ] Add self-proclaiming schema generation to core so fields and streams can describe themselves.

Core Schema Responsibilities:

[ ] Individual schema definitions (streams, field types, internal schema proclamations).
[ ] Self-proclaiming schema definitions (field types describing themselves).
[ ] Validate stream JSON definitions.

API Schema Responsibilities:

[ ] Use all other schema utilities to assemble an API schema specifically.
[ ] Only provide API-specific schema generation.

Dev Schema Responsibilities:

[ ] Generate d.ts files from the other utilities.
[ ] Only provide dev-oriented schema generation.

Having self-valildating, code completing json files (most editors understand this)
{
  "$schema": "path/to/stream.schema.json",
  "config":{}
  "fields": {}
}

And it would be great to validate the json streams files against their schemas at runtime too

justinrainbow/json-schema
opis/json-schema

Opis seems like a good one.

It also seems to me like a good idea to add HTTP tests. Ones that have the openapi spec ingrained with like league/openapi-psr7-validator. Something like a trait that allows validating requests inside a test. Should be super simple but very effective.

Got fields/types refactored which is also a precursor to this.

Having self-valildating, code completing json files (most editors understand this)
{
  "$schema": "path/to/stream.schema.json",
  "config":{}
  "fields": {}
}

And it would be great to validate the json streams files against their schemas at runtime too

justinrainbow/json-schema
opis/json-schema

Opis seems like a good one.
Also league/openapi-psr7-validator

I would hesitate to validate at runtime just because of performance. Perhaps a streams:validate command.

API command is all that is left.

This is done.


the command for  the connection fields of every V1 units

Hi, I would like to ask what is the command for the receptive fields of v1 neurons. By simplification, this
model uses the connection fields from LGN to V1 as the receptive fields of v1 units, right ? I found the
command like
c = Collector(), data=c(time=times), c.CFs.LGNOnAfferent=c.collect(gcal.projections.V1.LGNOnAfferent,grid=True)
does this command can give receptive fields of every single units? Because I plot the grid, it is 11*11,
I am not sure the connection fields are actually the ones of 11*11 neurons by simplification, rather than
every unit's.

The projection plotting is designed to be independent of the size of each sheet, which means that it will give an 11x11 grid of CFs by default, selecting every n-th unit in x and y.  (If your sheet happened to be smaller than 11x11, which would presumably be very rare, it would even duplicate units to get it up to 11x11!)  You can set the value 11 to the actual size of your target sheet (V1 in this case), but that might be a huge number of plots.

I don't understand the 11*11 grid of CFs well. Does it mean, every unit whose coordinates locate in the same grid have the same connection field ?

No.  Let's say you have a 9x9 V1 and you ask for a 3x3 grid of CFs.  You'll get CFs from units in V1 that are marked with parentheses:
(0,0) 0,1  0,2 (0,3) 0,4  0,5 (0,6) 0,7  0,8 (0,9)
 1,0  1,1  1,2  1,3  1,4  1,5  1,6  1,7  1,8  1,9
 2,0  2,1  2,2  2,3  2,4  2,5  2,6  2,7  2,8  2,9
(3,0) 3,1  3,2 (3,3) 3,4  3,5 (3,6) 3,7  3,8 (3,9)
 4,0  4,1  4,2  4,3  4,4  4,5  4,6  4,7  4,8  4,9
 5,0  5,1  5,2  5,3  5,4  5,5  5,6  5,7  5,8  5,9
(6,0) 6,1  6,2 (6,3) 6,4  6,5 (6,6) 6,7  6,8 (6,9)
 7,0  7,1  7,2  7,3  7,4  7,5  7,6  7,7  7,8  7,9
 8,0  8,1  8,2  8,3  8,4  8,5  8,6  8,7  8,8  8,9
(9,0) 9,1  9,2 (9,3) 9,4  9,5 (9,6) 9,7  9,8 (9,9)

I.e., it regularly samples the V1 array of neurons, selecting a 3x3 set of them to show CFs for.

update_connectionfields will plot all connection fields for any specified unit.

Could you show an example for using update_connectionfileds ? I am not sure how to give the projection name to this command.

It returns the connection fields for that unit for all projections.

For example, I run
from topo.analysis.command import update_connectionfields update_connectionfields[0,0]
show nothing to me.

It won't actually open any plots, just define the data.  I don't have the commands handy to see the syntax for then plotting what was defined, but there should be some in the examples.


Cannot decompile APK File using APKTool- Help Needed

Information

Apktool Version (apktool -version) - APK Tool 2.1.1
Operating System (Mac, Linux, Windows) - Mac OSX El Capitan
APK From? (Playstore, ROM, Other) - Yotaphone2 System Apps

Can somebody please help me understand why I cannot decompile the following system APK which I have taken from my android YotaPhone 2. I am having a nightmare with it.
YD_YotaphoneSDK.apk.zip
https://1drv.ms/u/s!As3ob0Phb3hmgRP5ZMwFY2XrjNuM
I was able to perfectly decompile 2 other APK's taken from the same location however when attempting to do so with this one using Mac's terminal the following error occured:
Users-Mac-mini:bin user$ ./apktool d YD_yotaphonesdk.apk
I: Using Apktool 2.1.1 on YD_yotaphonesdk.apk
I: Loading resource table...
I: Decoding AndroidManifest.xml with resources...
I: Loading resource table from file: /Users/user/Library/apktool/framework/1.apk
I: Regular manifest package...
I: Decoding file-resources...
I: Decoding values */* XMLs...
Exception in thread "main" brut.androlib.err.UndefinedResObject: resource spec: 0x01000000
	at brut.androlib.res.data.ResPackage.getResSpec(ResPackage.java:59)
	at brut.androlib.res.data.ResTable.getResSpec(ResTable.java:66)
	at brut.androlib.res.data.ResTable.getResSpec(ResTable.java:62)
	at brut.androlib.res.data.value.ResReferenceValue.getReferent(ResReferenceValue.java:57)
	at brut.androlib.res.data.value.ResStyleValue.serializeToResValuesXml(ResStyleValue.java:54)
	at brut.androlib.res.AndrolibResources.generateValuesFile(AndrolibResources.java:513)
	at brut.androlib.res.AndrolibResources.decode(AndrolibResources.java:264)
	at brut.androlib.Androlib.decodeResourcesFull(Androlib.java:128)
	at brut.androlib.ApkDecoder.decode(ApkDecoder.java:108)
	at brut.apktool.Main.cmdDecode(Main.java:163)
	at brut.apktool.Main.main(Main.java:81)

I have searched for a solution on the internet and have tried deleting the framework file that came with APKTool but that didnt seem to work.
Any advice and guidance will be very helpfull.
Regards

Generally that error is fixed deleting the framework installed (as its usually out of date). However, you said you already tried that.
I'll take a look when I get free.

thanks, yeah i have tried that, nothing i seem to try is working at the moment. Or if its possible to decompile it yourself and send it to me that would be excellent as I dont need to do it more than once.

everytime i delete it and re-run the process, the 1.apk file returns to the framework folder

is there anyway to delete it and ensure it doesnt return?

is there anyway to delete it and ensure it doesnt return?

No it requires it.
I need the framework files from this OEM. You pulled the APK from the Yodaphone system, which means you need the Yodaphone framework files.

thanks, I rooted my phone and installed the framework file using the 'if' function and I am getting the same result. How can i be sure that it has definitely installed the new frameork file? I have attached the pulled framework apk here. Regards
framework-res.apk.zip

I have also used one opf the older framework files found on the net to no avail

Not sure. I've seen this error before in #1065
I don't think that resource ID is valid, as the type pair is 00.

what is your suggestion for overocming this issue?

Research a solution or wait.

After I change file that "C:\Users\WindowAccount\apktool\framework\1.apk", I resolve this issue.


RucioInjector component can become extremely slow

Impact of the bug
WMAgent
Describe the bug
Every now and then we see performance issues with the RucioInjector components, where each interaction with the Rucio server - via HTTP REST APIs - can take a few secs (sometimes a few 10s of seconds).
This causes production workflows to sit in the "file mismatch" Unified category, until agents can successfully go through that backlog and insert data into Rucio.
This morning we could spot an elapsed time of 24 seconds to insert 1 file into a block, basically each iteration of this block of code:
https://github.com/dmwm/WMCore/blob/master/src/python/WMComponent/RucioInjector/RucioInjectorPoller.py#L291-L306
so we need to rule out any possible delays coming from self._updateLFNState(), which is the agent oracle row update.
How to reproduce it
Not clear (and right now it only affects WMAgents at CERN, not at FNAL)
Expected behavior
The first objective with this issue is to identify the root cause of this slowness, then based on that, there might be many different options on how to tackle it, for instance:

communicate with the Rucio/DM team to work on improvements on the Rucio server
adopt some concurrent mechanism in RucioInjector (multi-threading, multi-processing, etc)
make database schema changes to track container and block injection status in Rucio.

Additional context and error message
Right now, vocms0252 and vocms0253 are having a hard time to complete a cycle. For the record, last completed cycle in vocms0252 was at (and this is the total time it took to complete the cycle):
2022-10-25 02:48:55,043:140231764350720:INFO:BaseWorkerThread:RucioInjectorPoller took 334480.844 secs to execute

From this morning when we were watching the RucioInjector log (note the constant ~24secs per file injection):
2022-10-31 14:54:34,673:140231764350720:INFO:RucioInjectorPoller:Successfully inserted 1 files on block /NMSSM_XToYHTo2W2BTo2Q1L1Nu2B_MX-800_MY-300_TuneCP5_13TeV-madgraph-pythia8/RunIISummer20UL18RECO-106X_upgrade2018_realistic_v11_L1v1-v2/AODSIM#a6e93581-f87a-45d4-b17a-63c972328fb4
2022-10-31 14:54:58,714:140231764350720:INFO:RucioInjectorPoller:Successfully inserted 1 files on block /NMSSM_XToYHTo2Tau2B_MX-2400_MY-70_TuneCP5_13TeV-madgraph-pythia8/RunIISummer20UL16MiniAODAPVv2-106X_mcRun2_asymptotic_preVFP_v11-v2/MINIAODSIM#9670c832-93d9-409c-8b11-b5aad6972a6d
2022-10-31 14:55:22,774:140231764350720:INFO:RucioInjectorPoller:Successfully inserted 1 files on block /NMSSM_XToYHTo2B2Tau_MX-2600_MY-60_TuneCP5_13TeV-madgraph-pythia8/RunIISummer20UL16MiniAODAPVv2-106X_mcRun2_asymptotic_preVFP_v11-v2/MINIAODSIM#c32fe6f8-b4e9-4c57-8415-201e3c41a559
2022-10-31 14:55:49,811:140231764350720:INFO:RucioInjectorPoller:Successfully inserted 1 files on block /ZH_HToZZTo2Nu2X_2LFilter_4LVeto_M600_TuneCP5_13TeV_powheg2-minlo-HZJ_JHUGenV735_pythia8/RunIISummer20UL16MiniAODAPVv2-106X_mcRun2_asymptotic_preVFP_v11-v2/MINIAODSIM#2f1e0493-fb69-4270-9956-909f17fc7bb0
2022-10-31 14:56:13,935:140231764350720:INFO:RucioInjectorPoller:Successfully inserted 1 files on block /NMSSM_XToYHTo2B2Tau_MX-700_MY-500_TuneCP5_13TeV-madgraph-pythia8/RunIISummer20UL16RECOAPV-106X_mcRun2_asymptotic_preVFP_v8-v2/AODSIM#fca3e101-2d70-46d9-939f-64c68b239f94
2022-10-31 14:56:37,885:140231764350720:INFO:RucioInjectorPoller:Successfully inserted 1 files on block /NMSSM_XToYHTo2W2BTo2Q1L1Nu2B_MX-2800_MY-500_TuneCP5_13TeV-madgraph-pythia8/RunIISummer20UL16MiniAODAPVv2-106X_mcRun2_asymptotic_preVFP_v11-v2/MINIAODSIM#3969eba0-206e-4015-b6ce-581a220d1218
2022-10-31 14:57:01,949:140231764350720:INFO:RucioInjectorPoller:Successfully inserted 1 files on block /NMSSM_XToYHTo2W2BTo2Q1L1Nu2B_MX-2400_MY-1600_TuneCP5_13TeV-madgraph-pythia8/RunIISummer20UL17RECO-106X_mc2017_realistic_v6-v2/AODSIM#f05b78f6-929f-46e5-af61-f247c961a93d
2022-10-31 14:57:26,014:140231764350720:INFO:RucioInjectorPoller:Successfully inserted 1 files on block /NMSSM_XToYHTo2B2Tau_MX-4000_MY-125_TuneCP5_13TeV-madgraph-pythia8/RunIISummer20UL16MiniAODAPVv2-106X_mcRun2_asymptotic_preVFP_v11-v2/MINIAODSIM#7242ea99-7e95-429c-8ef2-aae30d041888

I am debugging the file insertion step in vocms0252 and I should have some more information on this in the coming hour or so.

@ericvaandering , it would be useful to have your input on this issue. In particular, I'm interesting in the following information:

the throughput stats from Rucio REST APIs, e.g. how many request per second it can sustain
any average stats, e.g. mean/median, Rucio APIs take to process any given block/file
timber like dashboard, e.g. see CMSWEB timber, which can provide current state of Rucio server load and provide breakdown of current clients based on API, IPs, DNs, etc. such that we may correlate activity of WMAgents and Rucio instances.

PS.: I updated the initial description with a log snippet from this morning (timestamp is CERN time).

@ericvaandering , it would be useful to have your input on this issue. In particular, I'm interesting in the following information:

the throughput stats from Rucio REST APIs, e.g. how many request per second it can sustain

We've never tried to measure this and of course it very much depends on what the request is.

any average stats, e.g. mean/median, Rucio APIs take to process any given block/file

I thought we had metrics which were on the level of the URI and the GET/PUT/DELETE method. But I'm not finding them at the moment. Maybe they were lost in the transition to flask. I've asked.

timber like dashboard, e.g. see CMSWEB timber, which can provide current state of Rucio server load and provide breakdown of current clients based on API, IPs, DNs, etc. such that we may correlate activity of WMAgents and Rucio instances.

We run fluentd on the Rucio system, so I presume the logs are somewhere and someone from the monitoring group could easily build the parsers to make such a dashboard.
To answer a question in your mail, we also run kube-eagle so you should be able to see if there is a correlation between what you see and the pod or node loading. The pods in question are server-rucio-server-[hash]-[hash] and the nodes begin with cmsrucioprod1

Eric, the kube-eagle plots do not give any useful information about individual APIs or requests. They only show the state of the pod/node that's it. The pod may be fine or not, while API time/CPU/RAM usage can deviate based on a load. The fluentd logs is again is something you control and anyone outside Rucio have no idea about their format. Therefore, it should be you/Rucio team which should process them and push appropriate metrics to MONIT/ES. For instance, that what we do with CMSWEB, i.e. we process cmsweb logs and push these metrics to MONIT. Then, Monitoring team can build out of them appropriate timber dashboard. I suggest that you take CMSWEB timber dashboard and try to extract similar information from your Rucio logs (fluentd) and push them to MONIT. @mrceyhun can provide current CMSWEB schema of metrics we extract from CMSWEB. Since Rucio is not part of CMSWEB we do not have these metrics by default.
It would be desired if you'll open such request and provide necessary metrics per API. For instance, since your service process the request you can easily dump into log or prometheus each request time along with CPU/RAM snapshots. This can be done using psutil python module. In particular, I provided
ProcessStats for WMcore services which can capture on thread level useful stats. I hope you can investigate and request from Rucio to provide such monitoring metrics. As you can see it will tremendously help to understand the current state of Rucio servers and will be especially useful during debug process. I understand that it may take time to implement this but we need to have clear idea what each service is doing at any given point of time to pin-point its issues.

The rucio server logs are standard Apache logs or something very similar so it’s not efficient for me to restart this from the ground up when the parser already exists.
And if this problem happens with CERN agents and not FNAL, the problem is likely in the agent or oracle, not the common element Rucio.
So I can put in a request for better monitoring from the rucio team but nothing is likely to happen until we supply the resources to do it.
Sent from a mobile device. Please excuse my brevity or transcription errors.
On Oct 31, 2022, at 12:33 PM, Valentin Kuznetsov @.***> wrote:

Eric, the kube-eagle plots do not give any useful information about individual APIs or requests. They only show the state of the pod/node that's it. The pod may be fine or not, while API time/CPU/RAM usage can deviate based on a load. The fluentd logs is again is something you control and anyone outside Rucio have no idea about their format. Therefore, it should be you/Rucio team which should process them and push appropriate metrics to MONIT/ES. For instance, that what we do with CMSWEB, i.e. we process cmsweb logs and push these metrics to MONIT. Then, Monitoring team can build out of them appropriate timber dashboard. I suggest that you take CMSWEB timber dashboard and try to extract similar information from your Rucio logs (fluentd) and push them to MONIT. @mrceyhunhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_mrceyhun&d=DwMFaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=EHaoB-POFWGrYFvPXoj1bQ&m=JpyuhSIUSQEy6eICQWAU6Za7-SOUuI41Ac6HmTvIR0Eiw_S5LyY8BQlGBg5jXnrP&s=mvKDGczL7OGXP2SMltjYLBrVGH_SU5vKt9dBrCkA-8I&e= can provide current CMSWEB schema of metrics we extract from CMSWEB. Since Rucio is not part of CMSWEB we do not have these metrics by default.
It would be desired if you'll open such request and provide necessary metrics per API. For instance, since your service process the request you can easily dump into log or prometheus each request time along with CPU/RAM snapshots. This can be done using psutilhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_giampaolo_psutil&d=DwMFaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=EHaoB-POFWGrYFvPXoj1bQ&m=JpyuhSIUSQEy6eICQWAU6Za7-SOUuI41Ac6HmTvIR0Eiw_S5LyY8BQlGBg5jXnrP&s=jvCw8KhxITELRexDEXTgteC9bMDC9htXqZxHy05rMxY&e= python module. In particular, I provided
ProcessStatshttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_dmwm_WMCore_blob_master_src_python_Utils_ProcessStats.py&d=DwMFaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=EHaoB-POFWGrYFvPXoj1bQ&m=JpyuhSIUSQEy6eICQWAU6Za7-SOUuI41Ac6HmTvIR0Eiw_S5LyY8BQlGBg5jXnrP&s=IpQiOK4zjyIdWyXQNH-JXfmiRdNKV6rBIffUiOVNudw&e= for WMcore services which can capture on thread level useful stats. I hope you can investigate and request from Rucio to provide such monitoring metrics. As you can see it will tremendously help to understand the current state of Rucio servers and will be especially useful during debug process. I understand that it may take time to implement this but we need to have clear idea what each service is doing at any given point of time to pin-point its issues.
—
Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_dmwm_WMCore_issues_11356-23issuecomment-2D1297435268&d=DwMFaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=EHaoB-POFWGrYFvPXoj1bQ&m=JpyuhSIUSQEy6eICQWAU6Za7-SOUuI41Ac6HmTvIR0Eiw_S5LyY8BQlGBg5jXnrP&s=U6RAFAoHHAvHZ1sV10StrmJA7AQDv0s5MDK5kq-onAI&e=, or unsubscribehttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AAMYJLSHOROGB7R2SSSPBJTWF77H5ANCNFSM6AAAAAARTIOK4U&d=DwMFaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=EHaoB-POFWGrYFvPXoj1bQ&m=JpyuhSIUSQEy6eICQWAU6Za7-SOUuI41Ac6HmTvIR0Eiw_S5LyY8BQlGBg5jXnrP&s=Vz9Ax8Q-JPx0OlwWobSaaq1eAjOr_QdbKOjUzeKEg1g&e=.
You are receiving this because you were mentioned.Message ID: @.***>

@amaltaro , I doubt I have enough information to reach out Kate. I'm not sure I understand clearly the problem. If you think it is ORACLE, we must provide Kate specific pointers which should include:

account name
database name
query
node information where query initiated (vocms0252 and vocms0253 I presume)
I do not posses these information. So far I only know that RucioInjector is taking long time, but it does not mean I have all of the above information to discuss with Kate. Please provide additional information to proceed.

Hi all, as a reply to Valentin's comment above, you can see the schema of cmsweb frontend services in here .
As you know, cmsweb frontend services write their logs to files and filebeat fetch logs from these logs and send to Logstash. Logstash parses logs, apply logics and fits to the defined schema and then send to MONIT timber.
In short, pipeline is: cmsweb services -> /.../logs/frontend/access_log_* -> Filebeat -> Logstash -> MONIT timber.
As Valentin said, we don't have to follow this pipeline if there are better ways to provide logs. There are so many solutions to propagate data to MONIT with or without Logstash. If there is any help needed on monitoring side, feel free to reach us.

Just another update on this.
I made an in-place update to the RucioInjector code with a couple of debugging lines, and after another restart of the component in vocms0252, I see that the whole cycle completed in a much shorter time (timestamp local to CERN):
2022-11-01 22:30:50,073:140359888467712:INFO:BaseWorkerThread:RucioInjectorPoller took 11237.486 secs to execute

and since then, cycles have been pretty quick (magically):
2022-11-01 22:37:45,828:140359888467712:INFO:BaseWorkerThread:RucioInjectorPoller took 115.395 secs to execute
2022-11-01 22:42:46,836:140359888467712:INFO:BaseWorkerThread:RucioInjectorPoller took 0.656 secs to execute
2022-11-01 22:47:47,824:140359888467712:INFO:BaseWorkerThread:RucioInjectorPoller took 0.634 secs to execute

About vocms0253, it's yet to complete the cycle started yesterday.

@amaltaro can you please paste a snippet or a patch of the lines you've changed. Thanks.

Sure, here it is:
cmst1@vocms0252:/data/srv/wmagent/current $ diff -rup apps/wmagentpy3/lib/python3.8/site-packages/WMComponent/RucioInjector/RucioInjectorPoller.py{.bkp,}
--- apps/wmagentpy3/lib/python3.8/site-packages/WMComponent/RucioInjector/RucioInjectorPoller.py.bkp	2022-10-31 16:14:47.450548013 +0100
+++ apps/wmagentpy3/lib/python3.8/site-packages/WMComponent/RucioInjector/RucioInjectorPoller.py	2022-11-02 02:07:09.237512294 +0100
@@ -301,6 +301,7 @@ class RucioInjectorPoller(BaseWorkerThre
                                                bytes=fileInfo['size'], state="A",
                                                adler32=fileInfo['checksum']['adler32']))

+                    logging.info("AMR inserting %d files on block %s", len(listLfns), block)
                     if self.rucio.createReplicas(rse=rseName, files=injectData, block=block):
                         logging.info("Successfully inserted %d files on block %s", len(listLfns), block)
                         self._updateLFNState(listLfns)
@@ -346,6 +347,7 @@ class RucioInjectorPoller(BaseWorkerThre
                 for block in migratedBlocks[location][container]:
                     logging.info("Closing block: %s", block)
                     if self.rucio.closeBlockContainer(block):
+                        logging.info("AMR setting block status as 'Closed' in dbsbuffer for %s", block)
                         self.setBlockClosed.execute(block)
                     else:
                         logging.error("Failed to close block: %s. Will retry again later.", block)

the component is behaving though, so I still cannot isolate potential problems.

Thanks @amaltaro!
Oh it was actually just including two log messages. I was left with the impression you have found the magic spell that actually could solve the problem. Sorry for my misunderstanding.
Strange indeed the lag is gone after such a minor intervention. Shouldn't be related.

And now with a better log in place (vocms0252), we can clearly see that the the oracle operation takes only a few miliseconds, while the rucio python client close API takes almost 10seconds to complete:
2022-11-07 19:27:30,950:140198058891008:INFO:RucioInjectorPoller:Closing block: /WplusH_HTo2LongLivedTo4G_MH-125_MFF-50_ctau-100cm_TuneCP5_13TeV-powheg-pythia8/RunIISummer20UL17MiniAODv2-106X_mc2017_realistic_v9-v2/MINIAODSIM#381532ad-6af7-4bcd-964c-04f4162c621c
2022-11-07 19:27:39,963:140198058891008:INFO:RucioInjectorPoller:AMR setting block status as 'Closed' in dbsbuffer for /WplusH_HTo2LongLivedTo4G_MH-125_MFF-50_ctau-100cm_TuneCP5_13TeV-powheg-pythia8/RunIISummer20UL17MiniAODv2-106X_mc2017_realistic_v9-v2/MINIAODSIM#381532ad-6af7-4bcd-964c-04f4162c621c
2022-11-07 19:27:39,966:140198058891008:INFO:RucioInjectorPoller:Closing block: /WplusH_HTo2LongLivedTo4G_MH-125_MFF-50_ctau-10cm_TuneCP5_13TeV-powheg-pythia8/RunIISummer20UL18MiniAODv2-106X_upgrade2018_realistic_v16_L1v1-v2/MINIAODSIM#104a187a-fa8b-44ef-bfc4-d1322e3faec7
2022-11-07 19:27:48,967:140198058891008:INFO:RucioInjectorPoller:AMR setting block status as 'Closed' in dbsbuffer for /WplusH_HTo2LongLivedTo4G_MH-125_MFF-50_ctau-10cm_TuneCP5_13TeV-powheg-pythia8/RunIISummer20UL18MiniAODv2-106X_upgrade2018_realistic_v16_L1v1-v2/MINIAODSIM#104a187a-fa8b-44ef-bfc4-d1322e3faec7
2022-11-07 19:27:48,970:140198058891008:INFO:RucioInjectorPoller:Closing block: /WminusH_HTo2LongLivedTo4G_MH-125_MFF-7_ctau-20cm_TuneCP5_13TeV-powheg-pythia8/RunIISummer20UL17MiniAODv2-106X_mc2017_realistic_v9-v2/MINIAODSIM#6d1e0b8a-f904-470c-95d0-cd026b227754

this is the actual call:
https://github.com/dmwm/WMCore/blob/master/src/python/WMCore/Services/Rucio/Rucio.py#L430
@ericvaandering would you be able to pull any useful information from the server logs for these queries? From the rucio-clients perspective, these are a few examples:
client.close("cms", "/RunIISummer20UL18MiniAODv2-106X_upgrade2018_realistic_v16_L1v1-v2/MINIAODSIM#104a187a-fa8b-44ef-bfc4-d1322e3faec7")
or
client.close("cms", "/RunIISummer20UL17MiniAODv2-106X_mc2017_realistic_v9-v2/MINIAODSIM#381532ad-6af7-4bcd-964c-04f4162c621c")

timestamps above are CERN localtime.
If you prefer to discuss this over slack, just ping me over there.

Hi @ericvaandering  @amaltaro do you have any updates on this issue? In order to provide some context from the ops perspective, I'm attaching a screenshot which shows the number of workflows whose announcements are delayed due to this slowness over the last 6 months. As you can see, it got really worse in the last 2 months.

@ericvaandering we ran a few tests in the past week and AFAIR you managed to spot some of those slow queries in the log, right? Would you have any other suggestion on how we can investigate it?
We upgraded WMAgents at the beginning of this week and I have already spotted a few RucioInjector heartbeat timeouts in the new agents. None in the FNAL agents though, still puzzled!

Sorry, I don't have any new insights on this. You are using the Rucio client for this, right? All I can think of is that there is something going on with the certificates, but even that makes limited amounts of sense because the request to the Rucio server is not authenticated by certificate, just the rucio token.

After tweaking the nproc kernel limit (max user processes) on both vocms0281 and vocms0282, RucioInjector seems to be behaving properly. This would also explain why we see it only at the CERN agents, currently set to 4096 soft nproc, while FNAL agents have 16384.
JIRA ticket to puppetize these changes has been created: https://its.cern.ch/jira/browse/CMSVOC-413

That is not good! Thanks for the heads up Todor. Let me be optimistic here and assume that this happen while the VoC was puppetizing these changes (which apparently had issues during that process). I just restarted all the components to ensure that the expected limits are in place.
Please let me know if you spot further issues with this node in the coming days.

Changes to the nproc limit have been applied both at FNAL nodes (set to 65k, soft) and at CERN (set to 52k, soft).
RucioInjector has been behaving fine since then.
I am closing this issue out, but if it pops up again in the coming days/weeks, we should reopen it.

I am sorry to say it, but I need to reopen this issue yet again:
vocms0281
2023-01-20 05:30:17,329:140193328658176:INFO:BaseWorkerThread:RucioInjectorPoller took 30949.271 secs to execute

While at  the same time the respective component polling cycle latency for the same component at a FNAL agent is:
2023-01-20 08:54:29,179:140090104010496:INFO:BaseWorkerThread:RucioInjectorPoller took 237.586 secs to execute

Thanks Todor. I don't plan to work on it in the coming days, so I just removed myself from the assignee such that it's free to whoever has a free slot.

Today I was looking at the system, debugging some different issues, and I had to look at a fairly new agent:  vocms0256.  And since this was a fresh one, I managed to notice in the RucioInjector's logs a gradual decline of the component performance, strongly dependent on the number of write operations the component performs to Rucio. A single container or block level rules creation varies between 3-6 seconds. Here are  few examples:
2023-01-24 00:39:26,740:140216297350912:INFO:RucioInjectorPoller:Creating container rule for /NMSSM_XToYHTo6B_MX-700_MY-400_TuneCP5_13TeV-madgraph-pythia8/RunIISummer20UL17MiniAODv2-106X_mc2017_realistic_v9-v2/MINIAODSIM against RSE (tier=2|tier=1)&cms_type=real&rse_type=DISK
2023-01-24 00:39:30,117:140216297350912:INFO:RucioInjectorPoller:Container rule created for /NMSSM_XToYHTo6B_MX-700_MY-400_TuneCP5_13TeV-madgraph-pythia8/RunIISummer20UL17MiniAODv2-106X_mc2017_realistic_v9-v2/MINIAODSIM under rule id: ['88e642e6e77641eca0d4f0aab6c1c302']

In contrast to the FNAL agents, where those operations take much less than a second:
2022-11-16 01:56:14,189:139649909565184:INFO:RucioInjectorPoller:Creating container rule for /NMSSM_XToYHTo2B2WTo2B4Q_MX-2800_MY-450_TuneCP5_13TeV-madgraph-pythia8/RunIISummer20UL16NanoAODv9-106X_mcRun2_asymptotic_v17-v1/NANOAODSIM against RSE (tier=2|tier=1)&cms_type=real&rse_type=DISK
2022-11-16 01:56:14,570:139649909565184:INFO:RucioInjectorPoller:Container rule created for /NMSSM_XToYHTo2B2WTo2B4Q_MX-2800_MY-450_TuneCP5_13TeV-madgraph-pythia8/RunIISummer20UL16NanoAODv9-106X_mcRun2_asymptotic_v17-v1/NANOAODSIM under rule id: ['638401dc2f6e46a89ac9f2e81981b5cd']

And I have not tried to plot the dependency, but at a first glance it seems to be leaner to the number of write operations we have piled up in the component. And the more the component lags the more of those pileup.... we are in a kind of spiral trend here.
FYI @amaltaro @vkuznet @khurtado

Todor, it may be a good observation but it certainly does not provide further insight about the issue. I rather prefer to see a manual call to Rucio to inject some info and see how long it takes. In other words, is it possible to write stand-along script or small python code which will make call to rucio in a similar way as RucioInjectorPoller does. Doing this way we can measure time spend on Rucio call and run it on different nodes, e.g. at CERN VM or FNAL one.

Hi @vkuznet I'll get back to it soon. We are currently debugging a separate  problem related  to the lack of pressure in the system.
Indeed, a manual call to Rucio was my next step planed as well.

The previous in-place change that I had made in vocms0256 got reverted by Puppet. So here goes a request:
https://its.cern.ch/jira/browse/CMSVOC-430
to temporarily disable it in the CERN agents such that we can finally confirm whether limits.d settings have any effect on this or not.

Now that we got vocms0255 back functional (after rebuilding the machine from scratch), and disabling puppet in that node, here it goes my first change to the /etc/security/limits.d/20-nproc.conf file:
From
cmst1      soft    nproc     52000

to:
cmst1      soft    nproc     unlimited

Due to the inability to reproduce the RucioInjector heartbeat timeouts over the last 2 or 3 weeks (all the agents are working just fine!), I am going to close this issue out once again.
I do not have any explanation as of why the heartbeat timeouts stopped though. If it reoccurs, we should reopen this ticket.

After seeing CERN-agent slowness once again starting last week (June 19), we see that the duty cycle of RucioInjector increases again. It takes a couple of days to start hitting the heartbeat timeout. Example from vocms0281:
2023-06-25 14:44:14,198:139813330302720:INFO:BaseWorkerThread:RucioInjectorPoller took 900.566 secs to execute
2023-06-25 15:00:02,999:139813330302720:INFO:BaseWorkerThread:RucioInjectorPoller took 648.447 secs to execute
2023-06-25 15:13:39,720:139813330302720:INFO:BaseWorkerThread:RucioInjectorPoller took 516.358 secs to execute
2023-06-25 15:25:18,454:139813330302720:INFO:BaseWorkerThread:RucioInjectorPoller took 398.350 secs to execute
2023-06-25 15:42:13,180:139813330302720:INFO:BaseWorkerThread:RucioInjectorPoller took 714.368 secs to execute
2023-06-25 16:00:42,023:139813330302720:INFO:BaseWorkerThread:RucioInjectorPoller took 808.490 secs to execute
2023-06-25 16:21:16,035:139813330302720:INFO:BaseWorkerThread:RucioInjectorPoller took 933.625 secs to execute
2023-06-25 17:01:54,546:139813330302720:INFO:BaseWorkerThread:RucioInjectorPoller took 2138.160 secs to execute
2023-06-25 17:24:22,508:139813330302720:INFO:BaseWorkerThread:RucioInjectorPoller took 1047.598 secs to execute
2023-06-25 18:07:57,840:139813330302720:INFO:BaseWorkerThread:RucioInjectorPoller took 2314.973 secs to execute
2023-06-25 18:55:00,909:139813330302720:INFO:BaseWorkerThread:RucioInjectorPoller took 2522.699 secs to execute
2023-06-25 19:38:25,979:139813330302720:INFO:BaseWorkerThread:RucioInjectorPoller took 2304.673 secs to execute
2023-06-25 20:41:58,750:139813330302720:INFO:BaseWorkerThread:RucioInjectorPoller took 3512.416 secs to execute
2023-06-25 22:16:22,430:139813330302720:INFO:BaseWorkerThread:RucioInjectorPoller took 5363.300 secs to execute
2023-06-26 01:30:49,941:139813330302720:INFO:BaseWorkerThread:RucioInjectorPoller took 11366.883 secs to execute
2023-06-26 07:50:12,184:139813330302720:INFO:BaseWorkerThread:RucioInjectorPoller took 22461.849 secs to execute

Note that FNAL agents are again unaffected and their duty cycle is normal and within the hearbeat timeout.
I am reopening this issue such that we can reconsider it for Q3 and hopefully identify and resolve this issue for good.


Implement checks for unchecked_* operations

Requested feature: RMC should detect failures when dealing with unchecked arithmetic operations.
Use case: Unchecked arithmetic operations assume that the operands respect some invariant. If they don't, the result has an undefined behavior. We should add checks to verify that all the invariants are respected.
Link to relevant documentation (Rust reference, Nomicon, RFC):
https://doc.rust-lang.org/std/primitive.u32.html
https://doc.rust-lang.org/std/primitive.i32.html
https://doc.rust-lang.org/std/primitive.f32.html#method.to_int_unchecked
Is this a breaking change? No

Removing soundness since we don't claim support to all UBs related to arithmetic operations.


DraggableGroupingMixin Sort problem

Hi,
When I use Draggable Grouping Mixin, on third column in sort is number, and when I try to sort desc, nothing happens.
First two columns are string (and default), third is number.
groupingMixin.plugin.setDroppedGroups(["NazivPartnera", "NazivPj"]);

Please help,
Regards,
Nikola (Salvus)

Closed due inactivity
Also check the issue guidelines.
If you still need help, you can ask this on stackoverflow with our new tag.
https://stackoverflow.com/tags/serenity-platform


Availability of eShopSupport Project from Steve Sanderson's Presentation

Hello,
I realize this might not be the ideal place to ask, but I'm curious if there's any timeframe for when the eShopSupport project, featured in the excellent presentation by @SteveSandersonMS, will be publicly available.
The talk was a few months ago, and I'm eager to explore the project further. In the video at ~56:20, he shared the link: https://github.com/dotnet/eShopSupport
The project and the presentation were fantastic, and I'm eager to explore the intricacies of the solutions you came up with.
Thank you!

We've scheduled an internal review of it for early next week, and after that hope to be in a position to publish it within a further week or two (assuming no major issues are found). Hope that's OK!


RPP Test Suite Upgrade - CSV to Binary Conversion

merge golden outputs as a single binary file instead off multiple CSV files

@sampath1117 Please do a round of review and ensure tests passing

@sampath1117 Please do a round of review and ensure tests passing

@r-abishek I have tested the code from my end
All tests pass on HOST and HIP for both image and voxel functions


UI_BUTTON_ON_HOVERED Event not in Constants

Expected behaviour
The new hover events are available, as listed in the docs (on_hover, on_unhover) are not in the version on PyPI.
Platform and software (please complete the following information):

OS: Windows 10 64 bit
Pygame GUI version: 0.5.1
Pygame version: 2.0.0dev6

Additional context
I am told that installing directly from the repo works so I wonder if PyPI is on a different version.

This is just because I've already documented the new event in the main
branch.
You can look at versioned docs on read the docs to see the events docs for
0.5.0.
On Tue, 7 Apr 2020, 14:37 Snayff<EMAIL_ADDRESS>wrote:

Describe the bug
A clear and concise description of what the bug is.
Expected behaviour
The new hover events are listed in the docs (on_hover, on_unhover) are not
in the version on PyPI.
Platform and software (please complete the following information):

OS: Windows 10 64 bit
Pygame GUI version: 0.5.1
Pygame version: 2.0.0dev6

Additional context
I am told that installing directly from the repo works so I wonder if PyPI
is on a different version.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/MyreMylar/pygame_gui/issues/99, or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADGDGGXRVQL2ZVTRAXKWL5TRLMT77ANCNFSM4MDD2HBA
.

My bad, I assumed the latest docs were for the latest release.
Unrelated, but as there's nowhere to give feedback; v0.5 is excellent. I've been using it quite a lot in the last few days and it's working well. Easier to use, more feature rich. Great work, mate.


add imagenet benchmarks for theano and lasagne

Hello,
I am interested in comparing performance of theano against tensorflow and torch. I have ported tensorflow implementation in theano/lasagne. Mostly it is line to line correspondance. I am slightly confused by stat calculations of time measurements, so I replaced them by numpy stats.

Sorry, did I miss a point why theano benchmarks are irrelevant compare to torch and tensorflow?

Hi Konstantin, no you did not, it is plainly my fault for missing the notification -- terribly sorry for the delay.

What are the numbers like on your machine?

I didn't run them on GPU, only on CPU to be sure everything works. Hardware setup I have access to is quite different from yours to have comparable numbers.

okay, are there best practices wrt Theano flags and command line arguments I have to use to make sure I get maximum performance? Does theano enable cudnn by default or do I need any flags?

It should detect cudnn automatically and be reasonably fast by default. Although I suggest following ~/.theanorc config:
[global]
floatX = float32
mode=FAST_RUN
device=gpu0
force_device=True

[dnn]
enabled=True

[cuda]
root=/path/to/cuda

[lib]
cnmem=0.9

[nvcc]
fastmath=True

It makes theano fail if it can't attach to gpu or find cudnn.

Which cudnn algo is used by other framework? They can be changed by a few
Theano flags. If you want Theano to time each (via cudnn) and pick the
fastest, you can use this Theano flags:
dnn.conv.algo_fwd=time_once
dnn.conv.algo_bwd_filter=time_once
dnn.conv.algo_bwd_data=time_once
(add that in your .theanorc)
[dnn.conv]
algo_fwd=time_once
algo_bwd_filter=time_once
algo_bwd_data=time_once
That flag also allow to use swap the default algo used by cudnn to other
values.
On Wed, Apr 13, 2016 at 2:32 PM, Konstantin Shmelkov <
<EMAIL_ADDRESS>wrote:

It should detect cudnn automatically and be reasonably fast by default.
Although I suggest following ~/.theanorc config:
[global]
floatX = float32
mode=FAST_RUN
device=gpu0
force_device=True
[dnn]
enabled=True
[cuda]
root=/path/to/cuda
[lib]
cnmem=0.9
[nvcc]
fastmath=True
It makes theano fail if it can't attach to gpu or find cudnn.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
https://github.com/soumith/convnet-benchmarks/pull/94#issuecomment-209585222

I managed to get Titan X today. Here are numbers I got with suggested time_once option:
$ python benchmark_imagenet.py -a alexnet -B 128
Using gpu device 0: GeForce GTX TITAN X (CNMeM is enabled with initial size: 90.0% of memory, cuDNN 4007)
Forward across 100 steps, 0.054 +/- 0.001 sec / batch
Forward-Backward across 100 steps, 0.115 +/- 0.001 sec / batch

$ python benchmark_imagenet.py -a overfeat -B 128
Using gpu device 0: GeForce GTX TITAN X (CNMeM is enabled with initial size: 90.0% of memory, cuDNN 4007)
Forward across 100 steps, 0.137 +/- 0.001 sec / batch
Forward-Backward across 100 steps, 0.400 +/- 0.003 sec / batch

$ python benchmark_imagenet.py -a googlenet -B 128
Using gpu device 0: GeForce GTX TITAN X (CNMeM is enabled with initial size: 90.0% of memory, cuDNN 4007)
Forward across 100 steps, 0.170 +/- 0.001 sec / batch
Forward-Backward across 100 steps, 0.536 +/- 0.003 sec / batch

$ python benchmark_imagenet.py -a vgg -B 64
Using gpu device 0: GeForce GTX TITAN X (CNMeM is enabled with initial size: 90.0% of memory, cuDNN 4007)
Forward across 100 steps, 0.212 +/- 0.001 sec / batch
Forward-Backward across 100 steps, 0.684 +/- 0.003 sec / batch


Extension isn't working

I can't get this extension to work in VS Code v1.80.1 (the latest release as of today). I've tried every troubleshooting step I can think of, including:

Enabling the extension
Restarting VS Code
Checking extension settings
Checking for conflicting settings in settings.json
Disabling all other extensions
Confirming that the language modes are correctly set
Reinstalling the extension
Testing TODO highlighting in several different .js and .html files

Please check if your file format is the Todohighlist Include list. For example if you are using a *.vue file, it maybe not on the default list.

Yep--I tried it on .js, .css, .html, and .py files.

This minimal setting file does work for me. Obviously without further customisation it will only highlight TODO: and FIXME: strings, which are the only built-in ones. What are you trying to highlight?

Ah--I figured it out. I was writing TODO (no colon), but the default keyword matching only works for TODO: and FIXME: (with colons). Thanks for the help!


silently add user permissions

Description
Add the ability to flag -s on other commands such as user permissions rather than outputting - running LP Perms
Proposed Behaviour
Aid with keeping players immersed and not identifying that a LP command is being run to a general player
Extra Details
No response

normal players should not be running luckperms commands to get an output beyond just the version number

I think it's a good idea to add this feature. Could be used to prevent administrators from receiving certains log messages from luckperms on a big server

there is already a command that players with permission to see the log can use to stop seeing log messages in game.

normal players should not be running luckperms commands to get an output beyond just the version number

In my instance I am adding a click event that runs a luckperms command, normal player 'accepts' the dialogue, and shortly after is greeted with a beautifully big bright LP message to indicate it is running - I feel it is unecessary for normal players to have this info and removes from the immersive experience.  If the normal player is not running a version check, I feel they shouldn't be informed about the background process of a click event.
command attached to the click event is
/lp user @p permission set quest_accepted.heimerdinger.a true

Command is correctly run, and permission is correctly added as expected, its the message that is irrelevant for the normal user

This sounds more like an issue with your setup. Are you making the event run the command as the player? You shouldn't be doing this, make the command run from console instead if possible.

This sounds more like an issue with your setup. Are you making the event run the command as the player? You shouldn't be doing this, make the command run from console instead if possible.

Hmm well, the player right clicks the Confirm, with all luck perm permissions set to false, and then it initiates the command through Mythic Mobs, populates the trigger.name as the player who triggered the events name, Trigger being a right click on a mob and seems to get the info message
/lp user <trigger.name> permission set quest_accepted.heimerdinger.a true
I'm not very java savvy or anything like that but:
This seems to be where the Running version message is generated, sitting under: Args2<LuckPermsPlugin, Map<Component, Component>> INFO = (plugin, storageMeta) -> join(newline()
And the user promote/demote has examples of a flag to silence the message like so:
silence example in user demote
conditional for silence message
was thinking I could add something similar in here
just not really sure if this is where the message is being generated or if that is being consumed by PermissionSet.java

This sounds more like an issue with your setup. Are you making the event run the command as the player? You shouldn't be doing this, make the command run from console instead if possible.

Exactly this - the command triggers should be executed as the console, not as the player.
Seems like mythicmobs supports this: https://git.mythiccraft.io/mythiccraft/MythicMobs/-/wikis/skills/mechanics/command
if you configure this correctly, the player shouldn't get the "running luckperms version aaa" feedback in their chat.

This sounds more like an issue with your setup. Are you making the event run the command as the player? You shouldn't be doing this, make the command run from console instead if possible.

Exactly this - the command triggers should be executed as the console, not as the player.
Seems like mythicmobs supports this: https://git.mythiccraft.io/mythiccraft/MythicMobs/-/wikis/skills/mechanics/command
if you configure this correctly, the player shouldn't get the "running luckperms version aaa" feedback in their chat.

actually already tried setting up the command mechanic to work with the tellraw command, running with the asOp functionality:
it works in the exact same way as the jsonmessage, I feel the only difference is that jsonmessage allows you to nicely view your message.  I have infact even tried using the asCaster. please see below.
According to MythicMob docs if setup with asOp The player that Triggers the command with right click actually executes the command with all permissions
I've tried running the command in several ways now, all with the same result:
  Skills:
  - command{c="tellraw <trigger.name> [<&dq><&dq>,<&dq>\n<&dq>,<&dq>\n<&dq>,<&lc><&dq>text<&dq>:<&dq>Would you like to start Heimerdinger's quest? <&dq>,<&dq>color<&dq>:<&dq>dark_aqua<&dq><&rc>,<&lc><&dq>text<&dq>:<&dq>[Confirm]<&dq>,<&dq>color<&dq>:<&dq>dark_green<&dq>,<&dq>clickEvent<&dq>:<&lc><&dq>action<&dq>:<&dq>run_command<&dq>,<&dq>value<&dq>:<&dq>/lp user <trigger.name> permission set quest_accepted.heimerdinger.a true<&dq><&rc>,<&dq>hoverEvent<&dq>:<&lc><&dq>action<&dq>:<&dq>show_text<&dq>,<&dq>contents<&dq>:[<&lc><&dq>text<&dq>:<&dq>Click to Confirm<&dq>,<&dq>color<&dq>:<&dq>dark_green<&dq><&rc>]<&rc><&rc>,<&dq>\n<&dq>,<&dq>\n<&dq>]";} @trigger ~onInteract

  Skills:
  - command{c="tellraw <trigger.name> [<&dq><&dq>,<&dq>\n<&dq>,<&dq>\n<&dq>,<&lc><&dq>text<&dq>:<&dq>Would you like to start Heimerdinger's quest? <&dq>,<&dq>color<&dq>:<&dq>dark_aqua<&dq><&rc>,<&lc><&dq>text<&dq>:<&dq>[Confirm]<&dq>,<&dq>color<&dq>:<&dq>dark_green<&dq>,<&dq>clickEvent<&dq>:<&lc><&dq>action<&dq>:<&dq>run_command<&dq>,<&dq>value<&dq>:<&dq>/lp user <trigger.name> permission set quest_accepted.heimerdinger.a true<&dq><&rc>,<&dq>hoverEvent<&dq>:<&lc><&dq>action<&dq>:<&dq>show_text<&dq>,<&dq>contents<&dq>:[<&lc><&dq>text<&dq>:<&dq>Click to Confirm<&dq>,<&dq>color<&dq>:<&dq>dark_green<&dq><&rc>]<&rc><&rc>,<&dq>\n<&dq>,<&dq>\n<&dq>]";asOp=true} @trigger ~onInteract

  Skills:
  - command{c="tellraw @p [<&dq><&dq>,<&dq>\n<&dq>,<&dq>\n<&dq>,<&lc><&dq>text<&dq>:<&dq>Would you like to start Heimerdinger's quest? <&dq>,<&dq>color<&dq>:<&dq>dark_aqua<&dq><&rc>,<&lc><&dq>text<&dq>:<&dq>[Confirm]<&dq>,<&dq>color<&dq>:<&dq>dark_green<&dq>,<&dq>clickEvent<&dq>:<&lc><&dq>action<&dq>:<&dq>run_command<&dq>,<&dq>value<&dq>:<&dq>/lp user @p permission set quest_accepted.heimerdinger.a true<&dq><&rc>,<&dq>hoverEvent<&dq>:<&lc><&dq>action<&dq>:<&dq>show_text<&dq>,<&dq>contents<&dq>:[<&lc><&dq>text<&dq>:<&dq>Click to Confirm<&dq>,<&dq>color<&dq>:<&dq>dark_green<&dq><&rc>]<&rc><&rc>,<&dq>\n<&dq>,<&dq>\n<&dq>]";asOp=true} @None ~onInteract

  Skills:
  - jsonmessage{m="['','\n','\n',{'text':'Would you like to start Heimerdinger\'s quest? ','color':'dark_aqua'},{'text':'[Confirm]','color':'dark_green','clickEvent':{'action':'run_command','value':'/lp user <trigger.name> permission set quest_accepted.heimerdinger.a true -s'},'hoverEvent':{'action':'show_text','contents':[{'text':'Click to start Heimerdinger\'s quest.','color':'dark_aqua'}]}},'\n','\n']"} @trigger ~onInteract

All results gave the same:


Will vill set to 1 if vl is set to 0?

Based on spec, it is valid to set vl to 0 using configuration setting instructions. But based on application of vector instructions, setting vl to 0 makes no sense since zero element will be processed. So I'm wondering if vill in vtype register will be set if vl is set to 0? Or what are the expected next-steps if vl is set to 0?

No, setting vl to 0 does not cause vill to be set to 1.  There is a good reason for this: it eliminates a loop-head branch when vectorizing for loops.  The natural vector loop does the right thing even if the trip count is zero.

Right, I understand. Also, does setting vl to 0 mean that all the dst elements are tail elements? So all of the dst elements should be written based on ta/tu setting.

See section 5.4. Most instructions have no effect at all when vstart >= vl (which includes the case of vstart = 0), and so agnostic elements are not modified.

I got it. Thanks for the explanation.


[Fancy Zones] Add additional hotkey to expand current application on neigbouring zone

Summary of the new feature/enhancement

I have eight equally sized zones but if I only have three open windows it would be better to connect to neighbouring zones and enlarge my windows.
Proposed technical implementation details (optional)

By pressing the Shift+CTRL key and holding the window between two zones it should snap over both zones.

Duplicate of https://github.com/microsoft/PowerToys/issues/1972


Provide an ENV checker utility

So... it turns out that Ruby gets extremely confused if chefdk is in the path both via rbenv and directly, especially if it's invoked via bundle install.
I suspect that a "check I did it right" utility (checkenv.sh or similar) would go a long way to help n00bs from messing up.
I think it needs to check for GEM_PATH, GEM_ROOT, and then the presence of rbenv shims in PATH alongside chefdk anything else that could be checked.
Consider it an anti pattern checker... would have helped with #2 as well.

I added a script and a Troubleshooting section in the README that should fix this.  You can run $SHELL "$(rbenv root)/plugins/rbenv-chefdk/sanity-check.sh" to see if it is sane or not.


TypeError: argument of type 'int' is not iterable

Hello,
Trying "download-pod-logs", git cloned it "now"  and getting the below issue
[cloud-user@rext03-0009 viya4-ark]$ python3 viya-ark.py download-pod-logs --namespace lab --tail 1000
Downloading pod logs..............................DONE
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "/usr/lib64/python3.6/multiprocessing/pool.py", line 119, in worker
result = (True, func(*args, **kwds))
File "/home/cloud-user/viya4-ark/download_pod_logs/model.py", line 233, in _write_log
file_content = SASStructuredLoggingParser.parse_log(log)
File "/home/cloud-user/viya4-ark/viya_ark_library/structured_logging/parser.py", line 40, in parse_log
unstructured_log.append(SASStructuredLoggingParser.parse_log_entry(log_entry))
File "/home/cloud-user/viya4-ark/viya_ark_library/structured_logging/parser.py", line 65, in parse_log_entry
if required_key not in structured_log_entry:
TypeError: argument of type 'int' is not iterable
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "viya-ark.py", line 139, in
main(sys.argv[1:])
File "viya-ark.py", line 77, in main
command.run(argv[1:])
File "/home/cloud-user/viya4-ark/download_pod_logs/download_pod_logs.py", line 47, in run
main(argv)
File "/home/cloud-user/viya4-ark/download_pod_logs/download_pod_logs.py", line 146, in main
selected_components=args.selected_components, tail=args.tail, noparse=args.noparse)
File "/home/cloud-user/viya4-ark/download_pod_logs/model.py", line 150, in download_logs
err_info: List[Tuple[Optional[Text], Text]] = process.get_process().get(timeout=self._wait)
File "/usr/lib64/python3.6/multiprocessing/pool.py", line 644, in get
raise self._value
TypeError: argument of type 'int' is not iterable
Am I missing something ?
Thanks !
Younes

(on master branch)

(this was on BYO K8S made by GEL)

tried downloading the 1.1.1 release in case but same issue

@sasjowood I think this is the same thing being fixed in #81. Please confirm and dup this one if so.

@frayos and @erharb, that is correct, this issue is currently being tracked in issue #81. Please watch issue #81 for updates on the status of this fix. I'm closing this as a duplicate.


Timestamp support?

I see the message in the README about timestamps being ignored. I’m looking to add time synchronization and receive scheduled OSC messages on a dozen ESP32s. I’m happy to submit a PR to add timestamp support, but wanted to check:

Is this something you’d be interested in merging?
Where should I look in the codebase to add timestamp support?

Hi, @jhurliman
Thank you very much for considering your contribution. Yes, I'm interested in it. For the current status,

Decoding timestamp in the bundle probably works (refer here)
The timestamp using when encoding bundle without your own TimeTag is dummy value (refer here)

You can pass your own TimeTag when making the bundle (begin_bundle(your_timetag))

...Huh? You could use it if you wanted to... maybe!

If you can confirm you can use TimeTag, we should fix the comment in the README

I don't see an example of how to send a manually constructed OscMessage, could you point me in the right direction?
OscMessage msg_out;
OscEncoder wr;

// Get the current time (ESP32)
const uint64_t now = esp_timer_get_time();

wr.init().begin_bundle(TimeTag(now));
wr.encode(msg_out.init(OSC_TIME_RESPONSE).push(now));
wr.end_bundle();
// no matching function for call to 'arduino::osc::Manager<WiFiUDP>::send(const String&, const uint16_t&, OscMessage&)'
OscWiFi.send(msg_in.remoteIP(), OSC_SEND_PORT, msg_out);

Supported (but not tested) sending with timetag in main branch https://github.com/hideakitai/ArduinoOSC/pull/57
Please try
    const uint64_t now = esp_timer_get_time();
    const auto tt = OscTimeTag(now);
    OscWiFi.send(host, send_port, tt, "/msg", 1, 2.2f, "hello");

Ah sorry, it was not a bundle

The bundle is now supported in the main branch. Can you try this example? @jhurliman
https://github.com/hideakitai/ArduinoOSC/blob/main/examples/arduino/OscWiFiBundle/OscWiFiBundle.ino

If you still need help, please reopen this issue.


Incorrect margins on dialog actions

#54

:tada: This issue has been resolved in version 4.2.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Automatic Fetch Option Ignored

Configure Repository
Automatic Actions: Fetch  - disable / deselect
Fails, with application asking for authentication details every 10 minutes.
Deselecting Fetch every and or setting minutes to zero is ignored after submit OK. Opening Configure Repository again and the option is still selected.
Editing the remote repository configuration files,
[autofetch]
	enable = false
	minutes = 0

also does not work.
Gittyup versions affected: 1.1.0, 1.1.1
Operating System: openSUSE Tumbleweed 20220205
KDE Plasma Version: 5.23.5
KDE Frameworks Version: 5.90.0
Qt Version: 5.15.2
Kernel Version: 5.16.4-1-default (64-bit)
Graphics Platform: X11
Processors: 8 × Intel® Core™ i7-2670QM CPU @ 2.20GHz
Memory: 15.6 GiB of RAM
Graphics Processor: Quadro 3000M/PCIe/SSE2

@leoplaw thanks fpr reporting. I will have a look into it

Further information:
I have two remotes configured for the local repo.

GitHub - uses GitHub authentication
private server which uses SSH for authentication

GittyUp is always asking every 10 minutes for the private server SSH authentication.
As outlined above, no matter what I do, I cannot seem to disable this.

If you have saving credentials enabled, SSH passwords should be saved.
Is the package git-credential-libsecret or git-credential-gnome-keyring installed on your system?

The issue appears to be fixed in latest version:  v1.1.2 - 2022-08-12


bootstrap.sh and activate.sh do essentially the same thing

Problem
//scripts/bootstrap.sh and //scripts/activate.sh do essentially the same thing, but their roles aren't well documented.  Both eventually run their namesakes in pigweed, but those namesakes are actually the same file
Proposed Solution
document the difference between them or delete one of them

I definitely agree with merging them, filed https://bugs.chromium.org/p/pigweed/issues/detail?id=229 for this


Refactor the compose build command flagging process

There is no ideal about GlobalFlag(like namespace, address). I just refactor the compose build command.
Signed-off-by: Laitron<EMAIL_ADDRESS>

Maybe you can update this pr after #1792 has been merged. I introduce a new global flag process helper function in #1792. it may help you

Maybe you can update this pr after #1792 has been merged. I introduce a new global flag process helper function in #1792. it may help you

After merge pr #1793, Should we refactor
func NewClient(ctx context.Context, namespace, address string, opts ...containerd.ClientOpt) (*containerd.Client, context.Context, context.CancelFunc, error)

to
func NewClient(commonOpts types.CommonOptions, GOptions * types.GlobalCommandOptions, opts ...containerd.ClientOpt) (*containerd.Client, context.Context, context.CancelFunc, error)

first? Composer subcommand use this func many times, and maybe we refactor them one more time if not refactor it firstly.

func NewClient(commonOpts types.CommonOptions, GOptions * types.GlobalCommandOptions, opts ...containerd.ClientOpt) (*containerd.Client, context.Context, context.CancelFunc, error)

What is commonOpts types.CommonOptions?
If NewClient only needs 2 global flags namespace, address, I think it's fine passing the two instead of a whole GOptions * types.GlobalCommandOptions.

Meanwhile, for compose (sub)commands, a good start point is the getComposer command:
https://github.com/containerd/nerdctl/blob/3e0fa4f7b1cb8e98be202c820c88572cac55308a/cmd/nerdctl/compose.go#L83
It can be splitted into 2 sub functions:

getComposeOptions(cmd *cobra.Command) composer.Options that on the cmd side and construct the composer.Options from cmd. (basically, the first part of original function).
getComposer(opt composer.Options, client *containerd.Client, stdout, stderr io.Writer) (*composer.Composer, error) that on the pkg side.

What is commonOpts types.CommonOptions?

If NewClient only needs 2 global flags namespace, address, I think it's fine passing the two instead of a whole GOptions * types.GlobalCommandOptions.

As you see, commonOpts types.CommonOptions come from PR #1792 which is outdated after you reviewed.
SGTM

Meanwhile, for compose (sub)commands, a good start point is the getComposer function:
https://github.com/containerd/nerdctl/blob/3e0fa4f7b1cb8e98be202c820c88572cac55308a/cmd/nerdctl/compose.go#L83
It can be splitted into 2 sub functions:

getComposeOptions(cmd *cobra.Command) composer.Options that on the cmd side and construct the composer.Options from cmd. (basically, the first part of original function).

getComposer(opt composer.Options, client *containerd.Client, stdout, stderr io.Writer) (*composer.Composer, error) that on the pkg side and accept a compose option and stdout/stderr from cmd and construct the composer (the rest of original function).

SGTM, I will finish this.


refactor(border-component_test): DRY up component wrapping

Built on top of #14

This is outdated now with the arrival of TestBed


[Autocomplete][Joy] Incorrect typescript type onChange value

Duplicates

[X] I have searched the existing issues

Latest version

[X] I have tested the latest version

Steps to reproduce 🕹
No response
Current behavior 😯
The Autocomplete onChange value type is string[] but is returning a single string. This causes an error that have to be disabled with // @ts-ignore

Expected behavior 🤔
The type of the value should be string unless is multiple.
Context 🔦
No response
Your environment 🌎
No response

@marcpachecog Thanks for reporting the issue. Could you please attach a CodeSandbox that reproduces the issue?

By the way, can you try adding defaultValue or value to the Autocomplete?


feat(Topology): Define PerfectSpace

Introduces a new class, PerfectSpace, which requires that Set.univ is a perfect set. This is equivalent to say that no points are isolated (perfectSpace_iff_forall_not_isolated).

This PR is part of my effort to formalize Rubin's theorem into Mathlib.
See https://leanprover.zulipchat.com/#narrow/stream/287929-mathlib4/topic/Defining.20perfect.20spaces for the zulip discussion on this subject.

[ ] depends on: #10272

This PR has been stripped down to only the changes that I critically need to progress in my formalization of Rubin's theorem, since I want to keep the PR chain for it as small as possible.

I would recommend making a preliminary PR to do just the file split (+ import updates) and no other changes. This should make it easier to review the rest of your changes

Quick rebase and commit cleanup :)

Thanks!
maintainer merge

Thanks!
bors merge


Negative marking even when there is a unattempt question

At the start it works fine but once a question is attempted then after it negates the points even when the question is unattempted.
i.e. It even does negative marking for the unattempted questions.

Hey @abhijitnathwani ,
Is this issue still open ?

@saurabh259
@guvi007 added a commit that states that a popup has been added. However, when I checked out the code, I found that the question has all the options selected by default, and we can now choose any one of the options. There is no way to proceed without any option selected.
@guvi007, can you look at the popup, I'm not able to generate and review it, hence haven't added merged the commit.


Fix item names in crafting menu being colored wrong

Related Issues
Close #933.
Summary
Fixes the text color of entries in the crafting menu.

LGTM


add: implemented redis FLUSH commands

Implemented:

FlushAll
FLushDB
FLushAllAsync
FLushDBAsync

For more detail see:  redis docs
Thanks!

@gokure

Thanks! But we're not going to support FLUSH command in go-zero.

@kevwan. Ok, Thank you.

We can close this PR then)

Thanks for contribution! Closing.


Capture data for builds running on BuildKite

https://buildkite.com/

I would like to close this until we see high demand for this CI server amongest the users of the CCUD extension.


fix start failed issue

Coverage increased (+0.05%) to 17.583% when pulling 1f2076aa50e784e7e7c8c9b1fe69b1492dc157f9 on ascrutae:zhangxin/fix/start-failed into cd1d3a3bcc4dd86f2f0c72bac0bcffabe8ec82d4 on wu-sheng:dev/3.2.1.


fix(dropdown-option): mac chrome visibility issue

Summary
Hopefully resolves #37 @guptaashray
The issue seemed to be the .options height changing, which is unnecessary with visibility property.
I still could not reproduce the initial selection text not showing, or the multi-select tags text not showing.

:tada: This PR is included in version 1.1.2 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


He created 13 wallet addresses, and some of these  wallet addresses were airdropped!!

Related Safe Addresses
0xc76c8e9bd5a2341dd7cf5ad17ed2ccf647e102b4
0x56c2366b0f7a64d49a4ff7b04c28ddb813edc588
0x62eeefab55104bc0d1ae2ad181e2b67476e5ef47
0x38b074e474afda6967868e2b12b829dc815bf3fd
0x633b7218644b83d57d90e7299039ebab19698e9c
0xc102d2544a7029f7ba04beb133deadaa57fdf6b4
0x67575860eefc18b98c6ff2cd50f7c7684fa85676
0x386f1721fce8df5c50a6c4dfb92fad894c4eb51a
0x74cb633d1cf1a4a73f621c630a3b1b003450e5a3
0xc1d512d5e1550b196be47c5234ec566cee7c8d3
0x4e2f98c96e2d595a83afa35888c4af58ac343e44
0x44c21334524ed8ade10d18a2e3b1636c60f0289b
0x47ba9203e1d8cd5b25a063329ae88a6755a6e0bc
0xc7768a53234a72b53a980c977eb07c8c8f5e76a1
0x9626b7fa440e8c7001b72aac31a9e47150e58d8c
0xe452420eede6a8075470a5decd40ef3e1f6d2493
0xfa002b29130bdb1f464473e7c524c1c287f1c969
Reasoning
他的 ETH 来自 FTX。
他拥有许多 ERC-20 代币，以及许多从 gate.io 交易的代币。他还创建了 sudoswap 池。最重要的是普通用户持有的地址。空投成名者。
https://etherscan.io/address/0xD81152389720712E74FaBb14C12a4A14BdB6869d
Methodology
with target as (
select *
from url('https://raw.githubusercontent.com/safe-global/safe-user-allocation-reports/main/safe_user_allocations_reworked.csv', CSVWithNames)
), transactions as (
select t1.*
from ethereum.transactions t1
join target t2
on t1.to_address = t2.safe_address
)
select distinct to_address
from transactions
where from_address = lower('0xD81152389720712E74FaBb14C12a4A14BdB6869d')
Safe Address
eth:0xab31c9A6504b82ec0B1021a8b294eBE04D3bFB4e

Creating multiple Safes with the same account is not airdrop farming per se. Could you please elaborate on why you think this is in fact someone airdrop farming?

how dare u copy my reason??

We haven't received any further information on this ticket and therefore deemed this submission invalid.
Unfortunately just the pure fact that Safes...

were created by the same address
have the same owner
conducted the same transactions
received assets from the same address

... is not enough reasoning for a valid submission, as there are legit user behaviour leading to such as well (relayer, scripts, power users, exchanges)
This submission didn't provide sufficient indicators to convince the team that there is some targeted airdrop farming behind the provided addresses. As per the rules, when in doubt we will default in favor of the airdrop recipients in order to not risk any false positives when sorting out airdrop hunters.


Revert change introduced in #1654

Is it worth making this configurable? I can see reasons why the previous behaviour might be wanted, too.

@gatecat yap, we are already changing that


Typing "desc" often results in sqlDropStoredProc code snippet

Issue Type: Bug
Often at the end of queries, when I do an order by desc, as I type "desc" and hit enter to go to the next line, I end up triggering the sqlDropStoredProc function code snippet:

Based on the highlighting above, it seems that it's treating each letter (D-E-S-C) as a separate "keyword" in the code snippet lookup, as not only do they not appear together as "desc" in the phrase "sqlDropStoredProc", but they don't even appear in the same order!
How can I stop this from happening?
Azure Data Studio version: Azure Data Studio 1.8.0 (96e214593190071f0b520439929b218f9fa673de, 2019-06-06T01:37:31.184Z)
OS version: Darwin x64 18.6.0

System Info

Item
Value

CPUs
Intel(R) Core(TM) i7-7920HQ CPU @ 3.10GHz (8 x 3100)

GPU Status
2d_canvas: enabledchecker_imaging: disabled_offflash_3d: enabledflash_stage3d: enabledflash_stage3d_baseline: enabledgpu_compositing: enabledmultiple_raster_threads: enabled_onnative_gpu_memory_buffers: enabledrasterization: enabledsurface_synchronization: enabled_onvideo_decode: enabledwebgl: enabledwebgl2: enabled

Load (avg)
2, 2, 2

Memory (System)
16.00GB (0.21GB free)

Process Argv

Screen Reader
no

VM
22%

Extensions (9)

Extension
Author (truncated)
Version

admin-pack
Mic
0.0.2

agent
Mic
0.40.0

dacpac
Mic
0.3.0

import
Mic
0.8.0

profiler
Mic
0.8.0

schema-compare
Mic
0.3.0

server-report
Mic
0.1.5

whoisactive
Mic
0.1.1

sql-search
Red
0.3.3

@winzig if you don't use snippets at all you can disable with the below setting.  There is also an option to sort snippets at the bottom of the list, which may help.

I may need to do that, but it seems weird that:
(a) It’s triggering at the end of a sql statement (none of these snippets make sense to trigger at that point)
(b) It’s matching each individual letter of the word I typed, and in the wrong order


Fixed links in usage examples

nix/_test.nix -> nix/services/_test.nix

I should’ve used permalinks, but any changes in the API would mean updating the permalink, so this should be fine.
Thanks for noticing and fixing it.


Fig barfs when using 'centos6.6' image tag, but not '6.6'

Fig.yml
nginx:
  build: ./nginx
  links:
    - phpfpm
  ports: ['8080:8080']
  volumes:
    - logs:/var/log
phpfpm:
  build: ./phpfpm-symfony
  ports: ['9000:9000']
  volumes:
    - logs/phpfpm:/var/log/php-fpm
  volumes_from:
    - data
data:
  image: centos6.6
  volumes :
    - ../vapor:/srv/http

$ fig up
Recreating vapor_data_1...
Pulling image centos6.6...
Traceback (most recent call last):
  File "/usr/local/Cellar/fig/1.0.1/libexec/bin/fig", line 8, in <module>
    load_entry_point('fig==1.0.1', 'console_scripts', 'fig')()
  File "/usr/local/Cellar/fig/1.0.1/libexec/lib/python2.7/site-packages/fig/cli/main.py", line 31, in main
    command.sys_dispatch()
  File "/usr/local/Cellar/fig/1.0.1/libexec/lib/python2.7/site-packages/fig/cli/docopt_command.py", line 21, in sys_dispatch
    self.dispatch(sys.argv[1:], None)
  File "/usr/local/Cellar/fig/1.0.1/libexec/lib/python2.7/site-packages/fig/cli/command.py", line 28, in dispatch
    super(Command, self).dispatch(*args, **kwargs)
  File "/usr/local/Cellar/fig/1.0.1/libexec/lib/python2.7/site-packages/fig/cli/docopt_command.py", line 24, in dispatch
    self.perform_command(*self.parse(argv, global_options))
  File "/usr/local/Cellar/fig/1.0.1/libexec/lib/python2.7/site-packages/fig/cli/command.py", line 56, in perform_command
    handler(project, command_options)
  File "/usr/local/Cellar/fig/1.0.1/libexec/lib/python2.7/site-packages/fig/cli/main.py", line 427, in up
    insecure_registry=insecure_registry,
  File "/usr/local/Cellar/fig/1.0.1/libexec/lib/python2.7/site-packages/fig/project.py", line 174, in up
    for (_, container) in service.recreate_containers(insecure_registry=insecure_registry):
  File "/usr/local/Cellar/fig/1.0.1/libexec/lib/python2.7/site-packages/fig/service.py", line 207, in recreate_containers
    tuples.append(self.recreate_container(c, insecure_registry=insecure_registry, **override_options))
  File "/usr/local/Cellar/fig/1.0.1/libexec/lib/python2.7/site-packages/fig/service.py", line 237, in recreate_container
    new_container = self.create_container(**options)
  File "/usr/local/Cellar/fig/1.0.1/libexec/lib/python2.7/site-packages/fig/service.py", line 185, in create_container
    insecure_registry=insecure_registry
  File "/usr/local/Cellar/fig/1.0.1/libexec/vendor/lib/python2.7/site-packages/docker/client.py", line 733, in pull
    repository, insecure=insecure_registry
  File "/usr/local/Cellar/fig/1.0.1/libexec/vendor/lib/python2.7/site-packages/docker/auth/auth.py", line 62, in resolve_repository_name
    'Invalid repository name ({0})'.format(repo_name))
docker.errors.InvalidRepository: Invalid repository name (centos6.6)

$ docker images
REPOSITORY             TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
vapor_nginx            latest              fdabf5218329        40 seconds ago      477.6 MB
vapor_phpfpm           latest              31ad262a2c04        2 minutes ago       653.7 MB
<none>                 <none>              d622617e184a        About an hour ago   628.3 MB
<none>                 <none>              79f1ba22cc1d        2 hours ago         674.3 MB
jprjr/centos-php-fpm   latest              196721f9544a        6 days ago          361 MB
jenkins                latest              41001f44325b        4 weeks ago         661.5 MB
centos                 6.6                 eeed74b237f9        4 weeks ago         202.6 MB
centos                 centos6.6           eeed74b237f9        4 weeks ago         202.6 MB
mysql                  latest              cb4d7ccf2eea        4 weeks ago         282.6 MB

If I use image: 6.6 in my fig.yml, everything works as expected.

there is no centos6.6 image so this is expected, you want centos:centos6.6 or centos:6.6.

ugh, yeah stupid, thanks @dnephin


Upgrading h3-js to 3.6.2 breaks h3-hexagon-layer-low-zoom test

Description

Updating h3-js to<EMAIL_ADDRESS>breaks deck.gl's testing suite.
Repro Steps

Updating modules/geo-layers/package.json to a later version of h3-js causes the issue.
Run yarn run test browser afterwards.
The test produces a blank screenshot, vs the previous artifact below:

Environment (please complete the following information):

Framework Version: [e.g. deck.gl 6.3.0]: 7.4.0-alpha.2
Browser Version: [e.g. Chrome 71.0] Chrome 78.0
OS: [e.g. Mac OS X 10.4] Mac OS X 10.14.6

Logs

# h3-hexagon-layer-low-zoom
not ok 5432 match: 96.85%
  ---
    operator: fail
    stack: |-
      Error: match: 96.85%
          at Test.assert [as _assert] (http://localhost:5000/test-browser.js:260968:54)
          at Test.bound [as _assert] (http://localhost:5000/test-browser.js:260820:32)
          at Test../node_modules/tape/lib/test.js.Test.fail (http://localhost:5000/test-browser.js:261061:10)
          at Test.bound [as fail] (http://localhost:5000/test-browser.js:260820:32)
          at Object.onTestFail (http://localhost:5000/test-browser.js:288152:43)
          at SnapshotTestRunner._fail (http://localhost:5000/test-browser.js:38410:22)
          at http://localhost:5000/test-browser.js:38246:14

The breakage is in h3-js, created issue here: https://github.com/uber/h3-js/issues/76