Multinode dashboard extension

Hi, great project, mate, thanks!
I don't seem to be able to get data over multiple nodes, though.
In our company we have an on-premise cluster of PCs with GPUs, on top of which there is a k8s cluster and we are looking for a proper monitoring solution.
I use 3 nodes in my test setup -> 1 master and 2 nodes. All of them are to pick up GPU workloads.
i used Helm chart to deploy the exporters
helm install ozdemir utkuozdemir/nvidia-gpu-exporter -f nvidia-gpu-utku-ozdemir-values.yml

with the following values to allow deploying to master nodes as well
tolerations:
  - key: node-role.kubernetes.io/master
    effect: NoSchedule
    operator: Exists

service:
  type: NodePort
  nodePort: 30699

NOTE: By the way, as it turns out it's not possible to set a nodePort in Helm values, are you planning on adding this feature by any chance? would be very convenient.
exporters seem to be deployed successfully
vvcServiceAccount@k8s-master-node0:~$ k get pod -o wide
NAME                                                      READY   STATUS    RESTARTS   AGE   IP          NODE               NOMINATED NODE   READINESS GATES
utku-ozdemir-nvidia-gpu-exporter-6gt5r                         1/1     Running   1          19d   <IP_ADDRESS>   k8s-worker-node1   <none>           <none>
utku-ozdemir-nvidia-gpu-exporter-7jqgs                         1/1     Running   1          19d   <IP_ADDRESS>   k8s-master-node0   <none>           <none>
utku-ozdemir-nvidia-gpu-exporter-m9h72                         1/1     Running   0          19d   <IP_ADDRESS>   k8s-worker-node2   <none>           <none>

logs from all 3 exporters are the following
level=info ts=2021-08-13T15:08:03.434Z caller=main.go:65 msg="Listening on address" address=:9835
ts=2021-08-13T15:08:03.435Z caller=log.go:124 level=info msg="TLS is disabled." http2=false

prometheus specs for scraping
prometheus:
  prometheusSpec:
    additionalScrapeConfigs:
    - job_name: nvidia_gpu_exporter
      static_configs:
        - targets: [ '<IP_ADDRESS>:31585' ]

with 31585 being the random nodePort assigned to the sevice and <IP_ADDRESS> being the ip address of the master node
NAME                               TYPE       CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
utku-ozdemir-nvidia-gpu-exporter   NodePort   <IP_ADDRESS>   <none>        9835:31585/TCP   19d

i used this grafana dashboard to display the collected data: https://grafana.com/grafana/dashboards/14574
the results are good:

but there is only one node in the dropdown (one of the worker nodes).
am i missing something in my setup or is it not possible to have all nodes in the dropdown list to have an overview per node?
ideally, it would be awesome to also have a summary dashboard with averaged metrics to monitor the whole cluster at once.
do you by any chance plan on developing something like that or maybe know some dashboards that already do that?
i tried the official one (nvidia/gpu-operator with https://grafana.com/grafana/dashboards/12239 as a dashboard) but it is not nearly as impressive as yours and also it had a bunch of empty charts and "no data" type of situations.
so, to sum up

would be cool to add nodePort option to Helm chart values
is it possible to have an overview of multiple nodes?

thanks!

i can surely provide any extra logs or setup details if needed

Hi, glad that you liked the project, and thanks for the detailed description!
I didn't have a chance to test it on a multi-node cluster with GPUs so I was expecting to face this kind of issues.
I'll add NodePort support to the chart, no problem.
In the meantime: Is <IP_ADDRESS> in your scrapeConfig target the IP of the Service? If so, it might be the problem. Prometheus has to scrape each and every DaemonSet pod every N seconds (whatever period you have configured) to get all the metrics from all GPUs. When it only scrapes a service every N seconds, it will hit one of the pods randomly (maybe round-robin, maybe with some other LB algorithm).
I guess you don't use Prometheus Operator, since I see that you don't use the ServiceMonitor custom resource in the chart. In any case, it wouldn't work correctly either - it would cause the same problem you are facing.
For the moment, you can try to add each pod IP of the DS as the target to prometheus, and then check the metrics/Grafana. That way we can verify that the issue is actually this one I described. After that, we can look into ways to solve it (Pod IPs is not a real solution because they change all the time).

Seems I have to add this into the chart instead of ServiceMonitor: https://stackoverflow.com/a/66482645/1005102
If you cannot use Prometheus Operator, NodePorts can be useful for you since nodes have stable IPs that can be used for scraping.

thanks for the quick response and the information!
i actually do use prometheus-operator, i just excluded not relevant info in my opinion, that's the full prometheus section of the values file for prometheus-operator:
prometheus:
  service:
    type: NodePort
    nodePort: 30658
  serviceMonitorSelectorNilUsesHelmValues: false
  prometheusSpec:
    additionalScrapeConfigs:
    - job_name: nvidia_gpu_exporter
      static_configs:
        - targets: [ '<IP_ADDRESS>:31585' ]

i will try adding each pod IP as a target for prometheus and will get back to you soon with the results.

<IP_ADDRESS> is the ip address of the PC where master node is deployed by the way

you identified the problem correctly indeed!
after adding each pod IP as target for prometheus, i can see all 3 nodes in the dropdown.

the config i used
prometheus:
  service:
    type: NodePort
    nodePort: 30658
  serviceMonitorSelectorNilUsesHelmValues: false
  prometheusSpec:
    additionalScrapeConfigs:
    - job_name: nvidia_gpu_exporter
      static_configs:
        - targets: [ '<IP_ADDRESS>:9835', '<IP_ADDRESS>:9835', '<IP_ADDRESS>:9835' ]

I released chart version 0.3.0 with support for hostPort and nodePort options. Should be visible here in ~30mins https://artifacthub.io/packages/helm/utkuozdemir/nvidia-gpu-exporter
(About ServiceMonitor - turns out there was no problem with that - its behavior is to scrape all matching pods already)
In your use-case I recommend using a hostPort and putting <NODE_IP>:<HOST_PORT> for each node to your static scrape config targets.
Using nodePort will not work properly because it will end as the same behavior - Service will still load-balance to the pods.

thanks very much, @utkuozdemir, works perfectly!
this issue if fully covered, awesome support.
that's the settings i used:
# ozdemir-nvidia-gpu-exporter Helm values.yml

hostPort:
  enabled: true
  port: 31585

# prometheus operator values

prometheus:
  ...
  prometheusSpec:
    additionalScrapeConfigs:
    - job_name: nvidia_gpu_exporter
      static_configs:
        - targets: [ '<IP_ADDRESS>:31585', '<IP_ADDRESS>:31585', '<IP_ADDRESS>:31585' ]

where <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS> are ip addresses of my nodes.

Glad that I could help, thanks for reporting and all the information.


[cli] Loading rke config doesn't work

What kind of request is this (question/bug/enhancement/feature request):
We try to create cluster using rancher cli. When we load a rke configuration with cloud_provider vsphere the virtual_center is not set.
Steps to reproduce (least amount of steps as possible):
Generate RKE configuration with rke binary
Create a cluster using rancher cli with --rke-config option
# rke config --name cluster.yml
# rancher cluster create test --rke-config cluster.yml

- cluster.yml

# If you intened to deploy Kubernetes in an air-gapped environment,
# please consult the documentation on how to configure custom RKE images.
nodes:
- address: "node1"
  port: "22"
  internal_address: ""
  role:
  - controlplane
  - etcd
  hostname_override: master
  user: ubuntu
  #docker_socket: /var/run/docker.sock
  ssh_key: ""
  ssh_key_path: ~/.ssh/id_rsa
  ssh_cert: ""
  ssh_cert_path: ""
  labels: {}
  taints: []
- address: "node2"
  port: "22"
  internal_address: ""
  role:
  - worker
  hostname_override: worker-1
  user: ubuntu
  #docker_socket: /var/run/docker.sock
  ssh_key: ""
  ssh_key_path: ~/.ssh/id_rsa
  ssh_cert: ""
  ssh_cert_path: ""
  labels: {}
  taints: []
- address: "node3"
  port: "22"
  internal_address: ""
  role:
  - worker
  hostname_override: worker-2
  user: '`'
  #docker_socket: /var/run/docker.sock
  ssh_key: ""
  ssh_key_path: ~/.ssh/id_rsa
  ssh_cert: ""
  ssh_cert_path: ""
  labels: {}
  taints: []
- address: "node4"
  port: "22"
  internal_address: ""
  role:
  - worker
  hostname_override: worker-3
  user: ubuntu
  #docker_socket: /var/run/docker.sock
  ssh_key: ""
  ssh_key_path: ~/.ssh/id_rsa
  ssh_cert: ""
  ssh_cert_path: ""
  labels: {}
  taints: []
services:
  etcd:
    image: ""
    extra_args: {}
    extra_binds: []
    extra_env: []
    external_urls: []
    ca_cert: ""
    cert: ""
    key: ""
    path: ""
    uid: 0
    gid: 0
    snapshot: null
    retention: ""
    creation: ""
    backup_config: null
  kube-api:
    image: ""
    extra_args: {}
    extra_binds: []
    extra_env: []
    service_cluster_ip_range: <IP_ADDRESS>/16
    service_node_port_range: ""
    pod_security_policy: false
    always_pull_images: false
    secrets_encryption_config: null
    audit_log: null
    admission_configuration: null
    event_rate_limit: null
  kube-controller:
    image: ""
    extra_args: {}
    extra_binds: []
    extra_env: []
    cluster_cidr: <IP_ADDRESS>/16
    service_cluster_ip_range: <IP_ADDRESS>/16
  scheduler:
    image: ""
    extra_args: {}
    extra_binds: []
    extra_env: []
  kubelet:
    image: ""
    extra_args: {}
    extra_binds: []
    extra_env: []
    cluster_domain: test-cluster
    infra_container_image: ""
    cluster_dns_server: <IP_ADDRESS>
    fail_swap_on: false
    generate_serving_certificate: false
  kubeproxy:
    image: ""
    extra_args: {}
    extra_binds: []
    extra_env: []
network:
  plugin: flannel
  options:
    flannel_backend_type: vxlan
  mtu: 0
  node_selector: {}
authentication:
  strategy: x509
  sans: []
  webhook: null
addons: ""
addons_include: []
system_images:
  etcd: rancher/coreos-etcd:v3.4.3-rancher1
  alpine: rancher/rke-tools:v0.1.52
  nginx_proxy: rancher/rke-tools:v0.1.52
  cert_downloader: rancher/rke-tools:v0.1.52
  kubernetes_services_sidecar: rancher/rke-tools:v0.1.52
  kubedns: rancher/k8s-dns-kube-dns:1.15.0
  dnsmasq: rancher/k8s-dns-dnsmasq-nanny:1.15.0
  kubedns_sidecar: rancher/k8s-dns-sidecar:1.15.0
  kubedns_autoscaler: rancher/cluster-proportional-autoscaler:1.7.1
  coredns: rancher/coredns-coredns:1.6.5
  coredns_autoscaler: rancher/cluster-proportional-autoscaler:1.7.1
  kubernetes: rancher/hyperkube:v1.17.2-rancher1
  flannel: rancher/coreos-flannel:v0.11.0-rancher1
  flannel_cni: rancher/flannel-cni:v0.3.0-rancher5
  calico_node: rancher/calico-node:v3.10.2
  calico_cni: rancher/calico-cni:v3.10.2
  calico_controllers: rancher/calico-kube-controllers:v3.10.2
  calico_ctl: rancher/calico-ctl:v2.0.0
  calico_flexvol: rancher/calico-pod2daemon-flexvol:v3.10.2
  canal_node: rancher/calico-node:v3.10.2
  canal_cni: rancher/calico-cni:v3.10.2
  canal_flannel: rancher/coreos-flannel:v0.11.0
  canal_flexvol: rancher/calico-pod2daemon-flexvol:v3.10.2
  weave_node: weaveworks/weave-kube:2.5.2
  weave_cni: weaveworks/weave-npc:2.5.2
  pod_infra_container: rancher/pause:3.1
  ingress: rancher/nginx-ingress-controller:nginx-0.25.1-rancher1
  ingress_backend: rancher/nginx-ingress-controller-defaultbackend:1.5-rancher1
  metrics_server: rancher/metrics-server:v0.3.6
  windows_pod_infra_container: rancher/kubelet-pause:v0.1.3
ssh_key_path: ~/.ssh/id_rsa
ssh_cert_path: ""
ssh_agent_auth: false
authorization:
  mode: rbac
  options: {}
ignore_docker_version: false
kubernetes_version: "v1.17.2-rancher1-2"
private_registries: []
ingress:
  provider: "nginx"
  options: {}
  node_selector: {}
  extra_args: {}
  dns_policy: ""
  extra_envs: []
  extra_volumes: []
  extra_volume_mounts: []
cluster_name: ""
cloud_provider:
  name: vsphere
  vsphereCloudProvider:
    global:
      insecure-flag: true
    virtual_center:
      IP_VCENTER:
        user: user
        password: password
        datacenters: LOCAL
        port: 443
    workspace:
      server: IP_VCENTER
      datacenter: LOCAL
      folder: kube
      default-datastore: Kubernetes
    disk:
      scsicontrollertype: pvscsi
prefix_path: ""
addon_job_timeout: 0
bastion_host:
  address: ""
  port: ""
  user: ""
  ssh_key: ""
  ssh_key_path: ""
  ssh_cert: ""
  ssh_cert_path: ""
monitoring:
  provider: metrics-server
  options: {}
  node_selector: {}
restore:
  restore: false
  snapshot_name: ""
dns: null

```

**Result:**
Cluster is created but cloud_provider doesn't contain virtual_center configuration.

```
Version: v3
clusters:
  test:
    answers: {}
    dockerRootDir: /var/lib/docker
    enableNetworkPolicy: false
    localClusterAuthEndpoint: {}
    rancherKubernetesEngineConfig:
      addonJobTimeout: 30
      authentication:
        strategy: x509
      authorization:
        mode: rbac
      bastionHost: {}
      cloudProvider:
        name: vsphere
        vsphereCloudProvider:
          disk:
            scsicontrollertype: pvscsi
          global:
            insecure-flag: true
          network: {}
          workspace:
            datacenter: LOCAL
            default-datastore: Kubernetes
            folder: kube
            server: IP_VCENTER
      ignoreDockerVersion: true
      ingress:
        provider: nginx
      kubernetesVersion: v1.17.2-rancher1-2
      monitoring:
        provider: metrics-server
      network:
        plugin: canal
      nodes:
      - address: node1
        port: "22"
        role:
        - controlplane
        - etcd
        user: ubuntu
      - address: node2
        port: "22"
        role:
        - worker
        user: ubuntu
      - address: node3
        port: "22"
        role:
        - worker
        user: '`'
      - address: node4
        port: "22"
        role:
        - worker
        user: ubuntu
      restore: {}
      services:
        etcd:
          backupConfig:
            enabled: true
            intervalHours: 12
            retention: 6
          creation: 12h
          extraArgs:
            election-timeout: "5000"
            heartbeat-interval: "500"
          retention: 72h
          snapshot: false
        kubeApi: {}
        kubeController: {}
        kubelet: {}
        kubeproxy: {}
        scheduler: {}
```
**Other details that may be helpful:**

Rancher Log
```
2020/02/14 10:01:07 http: TLS handshake error from <IP_ADDRESS>:51690: EOF
2020/02/14 10:01:13 [INFO] Deleting cluster [c-ddtfm]
2020/02/14 10:01:18 [INFO] kontainerdriver rancherkubernetesengine listening on address <IP_ADDRESS>:37387
2020/02/14 10:01:18 [INFO] kontainerdriver rancherkubernetesengine stopped
2020/02/14 10:01:18 [INFO] Deleted cluster [c-ddtfm]
2020/02/14 10:01:18 [INFO] [mgmt-cluster-rbac-remove] Deleting namespace c-ddtfm
2020/02/14 10:01:18 [ERROR] ClusterController c-ddtfm [mgmt-cluster-rbac-remove] failed with : clusters.management.cattle.io "c-ddtfm" not found
2020/02/14 10:01:23 [INFO] [mgmt-project-rbac-remove] Deleting namespace p-2t8th
2020/02/14 10:01:23 [INFO] [mgmt-project-rbac-remove] Deleting namespace p-j7pxl
E0214 10:01:24.045505      58 tokens_controller.go:261] error synchronizing serviceaccount c-ddtfm/default: secrets "default-token-pzlw5" is forbidden: unable to create new content in namespace c-ddtfm because it is being terminated
E0214 10:01:24.055308      58 tokens_controller.go:261] error synchronizing serviceaccount c-ddtfm/default: secrets "default-token-s6cd7" is forbidden: unable to create new content in namespace c-ddtfm because it is being terminated
2020/02/14 10:01:24 [INFO] [mgmt-auth-crtb-controller] Deleting roleBinding clusterrolebinding-86sdz
E0214 10:01:24.070128      58 tokens_controller.go:261] error synchronizing serviceaccount c-ddtfm/default: secrets "default-token-hb65d" is forbidden: unable to create new content in namespace c-ddtfm because it is being terminated
E0214 10:01:24.094973      58 tokens_controller.go:261] error synchronizing serviceaccount c-ddtfm/default: secrets "default-token-pbfmz" is forbidden: unable to create new content in namespace c-ddtfm because it is being terminated
2020/02/14 10:01:26 [INFO] [mgmt-cluster-rbac-delete] Creating namespace c-55mkz
2020/02/14 10:01:26 [INFO] [mgmt-cluster-rbac-delete] Creating Default project for cluster c-55mkz
2020/02/14 10:01:26 [INFO] [mgmt-project-rbac-create] Creating namespace p-c4zqs
2020/02/14 10:01:26 [INFO] [mgmt-cluster-rbac-delete] Creating System project for cluster c-55mkz
2020/02/14 10:01:26 [INFO] [mgmt-project-rbac-create] Creating creator projectRoleTemplateBinding for user user-xmjsf for project p-c4zqs
2020/02/14 10:01:26 [INFO] [mgmt-project-rbac-create] Creating namespace p-djx6b
2020/02/14 10:01:26 [INFO] [mgmt-cluster-rbac-delete] Updating cluster c-55mkz
2020/02/14 10:01:26 [INFO] [mgmt-project-rbac-create] Creating creator projectRoleTemplateBinding for user user-xmjsf for project p-djx6b
2020/02/14 10:01:26 [INFO] [mgmt-auth-crtb-controller] Setting InitialRolesPopulated condition on project p-c4zqs
2020/02/14 10:01:26 [INFO] [mgmt-project-rbac-create] Creating creator clusterRoleTemplateBinding for user user-xmjsf for cluster c-55mkz
2020/02/14 10:01:26 [INFO] [mgmt-project-rbac-create] Updating project p-c4zqs
2020/02/14 10:01:26 [INFO] [mgmt-auth-prtb-controller] Creating clusterRole p-c4zqs-projectowner
2020/02/14 10:01:26 [INFO] [mgmt-auth-crtb-controller] Setting InitialRolesPopulated condition on project p-djx6b
2020/02/14 10:01:26 [INFO] [mgmt-auth-prtb-controller] Creating clusterRole p-djx6b-projectowner
2020/02/14 10:01:26 [INFO] [mgmt-auth-crtb-controller] Creating clusterRole c-55mkz-clusterowner
2020/02/14 10:01:26 [INFO] [mgmt-project-rbac-create] Updating project p-djx6b
2020/02/14 10:01:26 [INFO] [mgmt-auth-prtb-controller] Creating roleBinding for membership in project p-c4zqs for subject user-xmjsf
2020/02/14 10:01:26 [INFO] [mgmt-auth-crtb-controller] Creating clusterRoleBinding for membership in cluster c-55mkz for subject user-xmjsf
2020/02/14 10:01:26 [INFO] [mgmt-auth-prtb-controller] Creating clusterRole c-55mkz-clustermember
2020/02/14 10:01:26 [INFO] [mgmt-project-rbac-create] Updating project p-c4zqs
2020/02/14 10:01:26 [INFO] [mgmt-auth-prtb-controller] Creating roleBinding for membership in project p-djx6b for subject user-xmjsf
2020/02/14 10:01:26 [INFO] [mgmt-auth-crtb-controller] Setting InitialRolesPopulated condition on cluster
2020/02/14 10:01:26 [INFO] [mgmt-auth-crtb-controller] Creating role cluster-owner in namespace c-55mkz
2020/02/14 10:01:26 [INFO] [mgmt-cluster-rbac-delete] Updating cluster c-55mkz
2020/02/14 10:01:26 [INFO] [mgmt-auth-prtb-controller] Creating clusterRoleBinding for membership in cluster c-55mkz for subject user-xmjsf
2020/02/14 10:01:26 [INFO] [mgmt-auth-prtb-controller] Creating clusterRole c-55mkz-clustermember
2020/02/14 10:01:26 [INFO] [mgmt-auth-crtb-controller] Creating roleBinding for subject user-xmjsf with role cluster-owner in namespace
2020/02/14 10:01:26 [INFO] [mgmt-project-rbac-create] Updating project p-djx6b
2020/02/14 10:01:26 [INFO] [mgmt-auth-prtb-controller] Creating role project-owner in namespace c-55mkz
2020/02/14 10:01:26 [INFO] [mgmt-auth-crtb-controller] Creating role cluster-owner in namespace p-djx6b
2020/02/14 10:01:26 [INFO] [mgmt-auth-prtb-controller] Creating roleBinding for subject user-xmjsf with role project-owner in namespace
2020/02/14 10:01:26 [INFO] [mgmt-auth-crtb-controller] Creating roleBinding for subject user-xmjsf with role cluster-owner in namespace
2020/02/14 10:01:26 [INFO] [mgmt-auth-prtb-controller] Creating role project-owner in namespace p-c4zqs
2020/02/14 10:01:26 [INFO] [mgmt-auth-crtb-controller] Creating role cluster-owner in namespace p-c4zqs
2020/02/14 10:01:26 [ERROR] ProjectRoleTemplateBindingController p-djx6b/creator-project-owner [mgmt-auth-prtb-controller] failed with : clusterroles.rbac.authorization.k8s.io "c-55mkz-clustermember" already exists
2020/02/14 10:01:26 [INFO] [mgmt-auth-prtb-controller] Updating clusterRoleBinding clusterrolebinding-hr5hn for cluster membership in cluster c-55mkz for subject user-xmjsf
2020/02/14 10:01:26 [INFO] [mgmt-auth-prtb-controller] Creating role admin in namespace p-c4zqs
2020/02/14 10:01:26 [INFO] [mgmt-auth-prtb-controller] Creating roleBinding for subject user-xmjsf with role project-owner in namespace
2020/02/14 10:01:26 [INFO] [mgmt-auth-crtb-controller] Creating roleBinding for subject user-xmjsf with role cluster-owner in namespace
2020/02/14 10:01:26 [INFO] [mgmt-auth-prtb-controller] Creating roleBinding for subject user-xmjsf with role project-owner in namespace
2020/02/14 10:01:26 [INFO] [mgmt-auth-prtb-controller] Creating role project-owner in namespace p-djx6b
2020/02/14 10:01:26 [INFO] [mgmt-auth-prtb-controller] Creating role admin in namespace p-djx6b
2020/02/14 10:01:26 [INFO] [mgmt-auth-prtb-controller] Creating roleBinding for subject user-xmjsf with role admin in namespace
2020/02/14 10:01:26 [INFO] [mgmt-auth-prtb-controller] Creating roleBinding for subject user-xmjsf with role project-owner in namespace
2020/02/14 10:01:26 [INFO] [mgmt-auth-prtb-controller] Creating roleBinding for subject user-xmjsf with role admin in namespace
2020/02/14 10:01:26 [INFO] [mgmt-cluster-rbac-delete] Updating cluster c-55mkz
2020/02/14 10:01:28 http: TLS handshake error from <IP_ADDRESS>:51750: EOF
E0214 10:01:29.067006      58 tokens_controller.go:261] error synchronizing serviceaccount p-j7pxl/default: secrets "default-token-29m9d" is forbidden: unable to create new content in namespace p-j7pxl because it is being terminated
2020/02/14 10:01:29 [INFO] [mgmt-auth-prtb-controller] Updating owner label for roleBinding clusterrolebinding-kt7vb
E0214 10:01:29.077595      58 tokens_controller.go:261] error synchronizing serviceaccount p-j7pxl/default: secrets "default-token-lz2wg" is forbidden: unable to create new content in namespace p-j7pxl because it is being terminated
2020/02/14 10:01:29 [INFO] [mgmt-auth-prtb-controller] Deleting roleBinding clusterrolebinding-kt7vb
E0214 10:01:29.092664      58 tokens_controller.go:261] error synchronizing serviceaccount p-j7pxl/default: secrets "default-token-kngk5" is forbidden: unable to create new content in namespace p-j7pxl because it is being terminated
E0214 10:01:29.117356      58 tokens_controller.go:261] error synchronizing serviceaccount p-j7pxl/default: secrets "default-token-m6mn4" is forbidden: unable to create new content in namespace p-j7pxl because it is being terminated
2020/02/14 10:01:31 http: TLS handshake error from <IP_ADDRESS>:51760: EOF
2020/02/14 10:01:39 http: TLS handshake error from <IP_ADDRESS>:51848: EOF
2020/02/14 10:01:39 http: TLS handshake error from <IP_ADDRESS>:51852: EOF
2020/02/14 10:01:39 http: TLS handshake error from <IP_ADDRESS>:51850: EOF
2020/02/14 10:01:39 http: TLS handshake error from <IP_ADDRESS>:51854: EOF
2020/02/14 10:01:39 http: TLS handshake error from <IP_ADDRESS>:51842: EOF
2020/02/14 10:01:56 http: TLS handshake error from <IP_ADDRESS>:51862: EOF
2020/02/14 10:01:56 http: TLS handshake error from <IP_ADDRESS>:51864: EOF

```

**Environment information**
- Rancher version (`rancher/rancher`/`rancher/server` image tag or shown bottom left in the UI): rancher  v2.3.5
User Interface	v2.3.36
Helm	.14.3-rancher1
Machine	v0.15.0-rancher29

<!--
If the reported issue is regarding a created cluster, please provide requested info below
-->

**Cluster information**
- Cluster type (Hosted/Infrastructure Provider/Custom/Imported): vSphere
- Machine type (cloud/VM/metal) and specifications (CPU/memory): VM

gz#15447
SURE-2820

Confirmed in Rancher 2.4.3 with Rancher CLI 2.4.3-rc1

Duplicate of https://github.com/rancher/rancher/issues/27982

The Workaround is not helping, some keys are not working at all.
If you try to change the network plugin to calico
network:
  plugin: calico

It never changes and stays by default.

@zaggash did you tried:
--network-provider value    Network provider for the cluster (flannel, canal, calico) (default: "canal")

Reproduced with CLI v2.4.13. This is definitely a CLI issue and has to do with how we parse top-level keys in the RKE config file.
I don't yet know why (will inquire), but we convert all top-level keys to JSON format. For example, rancher_kubernetes_engine_config becomes rancherKubernetesEngineConfig. As a result, the original key, due to the modification, is not found by the CLI. It's as if this value is not initialized, and the CLI uses default values for it. The same is true for other top-level objects whose key are modified by the CLI before actually being send to Rancher server.

I may have to ascertain the exact reason for this behavior, but looks like, indeed, it has to do with how we parse and marshal the given config file.

Also, for all keys inside rancher_kubernetes_engine_config or any other object at any depth, like .services.etcd.backup_config, the CLI expects a key to be in camelCase, not snake_case. So it wants backupConfig. This is configured in rancher/rancher, the CLI just uses it. But rancher_kubernetes_engine_config itself should be in camelCase first to be unmarshalled at all. Basically, all keys should be in camelCase.

Opened a PR for the change, but a draft currently, in light of the 2.6.3 release. Will mark as ready for review once 2.6.3 is out, targeting 2.6.4.

Test Environment:
Rancher version: v2.6-head e55a04c
Rancher cluster type: HA
Docker version: 20.10
Downstream cluster type: rancher cli

Testing:
Tested this issue with the following steps:

Provision a Rancher cluster
Get an API Key and use it with the Rancher CLI to login to the cluster.
Get the following example RKE config for a new cluster: (Used cluster.yml provided in previous comment)
Use the older version of the Rancher CLI without the fix. Run rancher cluster create test-bad --rke-config cluster.yml.
Now inspect the YAML in the browser for the new pending cluster. Notice that none of the settings is present in the config. Because deserialization failed, there are default keys and values. For example, addon_job_timeout is set to 45 and not 30, as in the config. Note also that services are missing settings for all components: kube-controller, kube-api and others.
Now use the version of the CLI that has the fix: rancher cluster create test-fixed --rke-config cluster.yml.
Inspect the YAML and notice that most, although not all, settings have been applied. Notice how the services have their settings applied.

Result - Low Pass
After creating the cluster, the YAML was inspected and it seems only backupConfig service had the YAML settings applied. The other settings do not seem to have the settings applied as mentioned in the testing steps. The following settings were not set with the values from the cluster.yml file.
dockerRootDir: /var/lib/docker
localClusterAuthEndpoint:
    enabled: false
rancherKubernetesEngineConfig:
    addonJobTimeout: 45
ingress: {}
kubernetesVersion: v1.22.6-rancher1-2
network:
    mtu: 0
    options:
      flannel_backend_type: vxlan
kubeApi: {}
kubeController: {}
kubelet: {}
kubeproxy: {}
scheduler: {}
node_drain_input:
      delete_local_data: false
      force: false
      grace_period: -1
      ignore_daemon_sets: true
      timeout: 120

Below is the full YAML file after creating cluster using rancher v2.6.4-rc1 cli with the configurations given in the previous comment.
apiVersion: management.cattle.io/v3
kind: Cluster
metadata:
  annotations:
    authz.management.cattle.io/creator-role-bindings: '{"created":["cluster-owner"],"required":["cluster-owner"]}'
    field.cattle.io/creatorId: u-gndx8
    lifecycle.cattle.io/create.cluster-agent-controller-cleanup: "true"
    lifecycle.cattle.io/create.cluster-scoped-gc: "true"
    lifecycle.cattle.io/create.mgmt-cluster-rbac-remove: "true"
  creationTimestamp: "2022-02-23T19:52:28Z"
  finalizers:
  - wrangler.cattle.io/mgmt-cluster-remove
  - controller.cattle.io/cluster-agent-controller-cleanup
  - controller.cattle.io/cluster-scoped-gc
  - controller.cattle.io/cluster-provisioner-controller
  - controller.cattle.io/mgmt-cluster-rbac-remove
  generateName: c-
  generation: 13
  labels:
    cattle.io/creator: norman
  managedFields:
  - apiVersion: management.cattle.io/v3
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .: {}
          f:field.cattle.io/creatorId: {}
        f:generateName: {}
        f:labels:
          .: {}
          f:cattle.io/creator: {}
    manager: Go-http-client
    operation: Update
    time: "2022-02-23T19:52:28Z"
  - apiVersion: management.cattle.io/v3
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          f:authz.management.cattle.io/creator-role-bindings: {}
          f:lifecycle.cattle.io/create.cluster-agent-controller-cleanup: {}
          f:lifecycle.cattle.io/create.cluster-scoped-gc: {}
          f:lifecycle.cattle.io/create.mgmt-cluster-rbac-remove: {}
        f:finalizers:
          .: {}
          v:"controller.cattle.io/cluster-agent-controller-cleanup": {}
          v:"controller.cattle.io/cluster-provisioner-controller": {}
          v:"controller.cattle.io/cluster-scoped-gc": {}
          v:"controller.cattle.io/mgmt-cluster-rbac-remove": {}
          v:"wrangler.cattle.io/mgmt-cluster-remove": {}
      f:spec: {}
      f:status: {}
    manager: rancher
    operation: Update
    time: "2022-02-23T19:52:29Z"
  name: c-gbdgg
  resourceVersion: "385297"
  uid: 4f3c326a-4786-4e15-9d62-f2d8291c1619
spec:
  agentImageOverride: ""
  answers: {}
  description: ""
  desiredAgentImage: ""
  desiredAuthImage: ""
  displayName: test-fixed
  dockerRootDir: /var/lib/docker
  enableClusterAlerting: false
  enableClusterMonitoring: false
  enableNetworkPolicy: false
  fleetWorkspaceName: fleet-default
  internal: false
  localClusterAuthEndpoint:
    enabled: false
  rancherKubernetesEngineConfig:
    addonJobTimeout: 45
    authentication:
      strategy: x509
    authorization: {}
    bastionHost: {}
    cloudProvider: {}
    enableCriDockerd: false
    ignoreDockerVersion: true
    ingress: {}
    kubernetesVersion: v1.22.6-rancher1-2
    monitoring: {}
    network:
      plugin: canal
    restore: {}
    rotateEncryptionKey: false
    services:
      etcd:
        backupConfig:
          enabled: true
          intervalHours: 12
          retention: 6
          s3BackupConfig: null
      kubeApi: {}
      kubeController: {}
      kubelet: {}
      kubeproxy: {}
      scheduler: {}
    sshAgentAuth: false
    systemImages: {}
    upgradeStrategy:
      drain: false
      maxUnavailableControlplane: "1"
      maxUnavailableWorker: 10%
  windowsPreferedCluster: false
status:
  agentImage: ""
  aksStatus:
    privateRequiresTunnel: null
    rbacEnabled: null
    upstreamSpec: null
  allocatable:
    cpu: "0"
    memory: "0"
    pods: "0"
  appliedEnableNetworkPolicy: false
  appliedPodSecurityPolicyTemplateId: ""
  appliedSpec:
    agentImageOverride: ""
    answers: {}
    description: ""
    desiredAgentImage: ""
    desiredAuthImage: ""
    displayName: ""
    enableClusterAlerting: false
    enableClusterMonitoring: false
    enableNetworkPolicy: null
    internal: false
    localClusterAuthEndpoint:
      enabled: false
    windowsPreferedCluster: false
  authImage: ""
  capabilities:
    ingressCapabilities:
    - {}
    loadBalancerCapabilities: {}
    nodePortRange: 30000-32767
  capacity:
    cpu: "0"
    memory: "0"
    pods: "0"
  conditions:
  - status: "True"
    type: Pending
  - lastUpdateTime: "2022-02-23T19:52:29Z"
    message: waiting for etcd, controlplane and worker nodes to be registered
    reason: Provisioning
    status: Unknown
    type: Provisioned
  - lastUpdateTime: "2022-02-23T19:52:29Z"
    message: Waiting for API to be available
    status: Unknown
    type: Waiting
  - lastUpdateTime: "2022-02-23T19:52:28Z"
    status: "True"
    type: BackingNamespaceCreated
  - lastUpdateTime: "2022-02-23T19:52:28Z"
    status: "True"
    type: DefaultProjectCreated
  - lastUpdateTime: "2022-02-23T19:52:28Z"
    status: "True"
    type: SystemProjectCreated
  - lastUpdateTime: "2022-02-23T19:52:28Z"
    status: "True"
    type: InitialRolesPopulated
  - lastUpdateTime: "2022-02-23T19:52:28Z"
    status: "True"
    type: CreatorMadeOwner
  - lastUpdateTime: "2022-02-23T19:52:29Z"
    status: "True"
    type: NoDiskPressure
  - lastUpdateTime: "2022-02-23T19:52:29Z"
    status: "True"
    type: NoMemoryPressure
  - lastUpdateTime: "2022-02-23T19:52:29Z"
    status: "True"
    type: SecretsMigrated
  - lastUpdateTime: "2022-02-23T19:52:39Z"
    status: "False"
    type: Connected
  driver: rancherKubernetesEngine
  eksStatus:
    managedLaunchTemplateID: ""
    managedLaunchTemplateVersions: null
    privateRequiresTunnel: null
    securityGroups: null
    subnets: null
    upstreamSpec: null
    virtualNetwork: ""
  gkeStatus:
    privateRequiresTunnel: null
    upstreamSpec: null
  limits:
    cpu: "0"
    memory: "0"
    pods: "0"
  provider: ""
  requested:
    cpu: "0"
    memory: "0"
    pods: "0"

@timhaneunsoo, just to make sure - for the config file, did you use the example from my comment or from the issue description?
I just tried on v2.6-head using my example, and the services got deserialized properly. And I tried the example in the description and got the same behavior as you - only backupConfig in etcd got deserialized.

After review, the fix was found in rancher cli v2.6.4-rc2. Upon testing with the updated cli version, the fix is confirmed and testing is Pass.
Here is the correct YAML below:
apiVersion: management.cattle.io/v3
kind: Cluster
metadata:
  annotations:
    authz.management.cattle.io/creator-role-bindings: '{"created":["cluster-owner"],"required":["cluster-owner"]}'
    field.cattle.io/creatorId: u-8fsf6
    lifecycle.cattle.io/create.cluster-agent-controller-cleanup: "true"
    lifecycle.cattle.io/create.cluster-scoped-gc: "true"
    lifecycle.cattle.io/create.mgmt-cluster-rbac-remove: "true"
  creationTimestamp: "2022-02-23T23:00:37Z"
  finalizers:
  - wrangler.cattle.io/mgmt-cluster-remove
  - controller.cattle.io/cluster-agent-controller-cleanup
  - controller.cattle.io/cluster-scoped-gc
  - controller.cattle.io/cluster-provisioner-controller
  - controller.cattle.io/mgmt-cluster-rbac-remove
  generateName: c-
  generation: 14
  labels:
    cattle.io/creator: norman
  managedFields:
  - apiVersion: management.cattle.io/v3
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .: {}
          f:field.cattle.io/creatorId: {}
        f:generateName: {}
        f:labels:
          .: {}
          f:cattle.io/creator: {}
    manager: Go-http-client
    operation: Update
    time: "2022-02-23T23:00:37Z"
  - apiVersion: management.cattle.io/v3
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          f:authz.management.cattle.io/creator-role-bindings: {}
          f:lifecycle.cattle.io/create.cluster-agent-controller-cleanup: {}
          f:lifecycle.cattle.io/create.cluster-scoped-gc: {}
          f:lifecycle.cattle.io/create.mgmt-cluster-rbac-remove: {}
        f:finalizers:
          .: {}
          v:"controller.cattle.io/cluster-agent-controller-cleanup": {}
          v:"controller.cattle.io/cluster-provisioner-controller": {}
          v:"controller.cattle.io/cluster-scoped-gc": {}
          v:"controller.cattle.io/mgmt-cluster-rbac-remove": {}
          v:"wrangler.cattle.io/mgmt-cluster-remove": {}
      f:spec: {}
      f:status: {}
    manager: rancher
    operation: Update
    time: "2022-02-23T23:00:38Z"
  name: c-w664p
  resourceVersion: "6888"
  uid: 7a6b076f-6e0c-4fda-8397-eb58d3da55c9
spec:
  agentImageOverride: ""
  answers: {}
  defaultPodSecurityPolicyTemplateName: restricted
  description: ""
  desiredAgentImage: ""
  desiredAuthImage: ""
  displayName: test-fixed
  dockerRootDir: /apps/docker
  enableClusterAlerting: false
  enableClusterMonitoring: false
  enableNetworkPolicy: false
  fleetWorkspaceName: fleet-default
  internal: false
  localClusterAuthEndpoint:
    enabled: true
  rancherKubernetesEngineConfig:
    addonJobTimeout: 30
    authentication:
      strategy: x509|webhook
    authorization: {}
    bastionHost: {}
    cloudProvider: {}
    dns:
      nodelocal:
        updateStrategy: {}
    enableCriDockerd: false
    ignoreDockerVersion: true
    ingress:
      defaultBackend: true
      defaultIngressClass: true
      provider: nginx
    kubernetesVersion: v1.17.5-rancher1-1
    monitoring:
      provider: metrics-server
      replicas: 1
    network:
      plugin: canal
    privateRegistries:
    - url: harbor.davcor.co
    restore: {}
    rotateEncryptionKey: false
    services:
      etcd:
        backupConfig:
          enabled: true
          intervalHours: 12
          retention: 6
          s3BackupConfig: null
          timeout: 300
        creation: 12h
        extraArgs:
          election-timeout: "5000"
          heartbeat-interval: "500"
        retention: 72h
        snapshot: false
      kubeApi:
        auditLog:
          enabled: true
        eventRateLimit:
          enabled: true
        podSecurityPolicy: true
        secretsEncryptionConfig:
          enabled: true
        serviceNodePortRange: 30000-32767
      kubeController:
        extraArgs:
          address: <IP_ADDRESS>
          feature-gates: RotateKubeletServerCertificate=true
          profiling: "false"
      kubelet:
        extraArgs:
          anonymous-auth: "false"
          authorization-mode: Webhook
          event-qps: "0"
          feature-gates: RotateKubeletServerCertificate=true
          make-iptables-util-chains: "true"
          protect-kernel-defaults: "true"
          streaming-connection-idle-timeout: 1800s
          tls-cipher-suites: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
      kubeproxy: {}
      scheduler:
        extraArgs:
          address: <IP_ADDRESS>
          profiling: "false"
    sshAgentAuth: false
    systemImages: {}
    upgradeStrategy:
      drain: false
      maxUnavailableControlplane: "1"
      maxUnavailableWorker: 10%
      nodeDrainInput:
        gracePeriod: -1
        ignoreDaemonSets: true
        timeout: 120
  windowsPreferedCluster: false
status:
  agentImage: ""
  aksStatus:
    privateRequiresTunnel: null
    rbacEnabled: null
    upstreamSpec: null
  allocatable:
    cpu: "0"
    memory: "0"
    pods: "0"
  appliedEnableNetworkPolicy: false
  appliedPodSecurityPolicyTemplateId: ""
  appliedSpec:
    agentImageOverride: ""
    answers: {}
    description: ""
    desiredAgentImage: ""
    desiredAuthImage: ""
    displayName: ""
    enableClusterAlerting: false
    enableClusterMonitoring: false
    enableNetworkPolicy: null
    internal: false
    localClusterAuthEndpoint:
      enabled: false
    windowsPreferedCluster: false
  authImage: ""
  capabilities:
    ingressCapabilities:
    - customDefaultBackend: false
      ingressProvider: nginx
    loadBalancerCapabilities: {}
    nodePortRange: 30000-32767
    pspEnabled: true
  capacity:
    cpu: "0"
    memory: "0"
    pods: "0"
  conditions:
  - status: "True"
    type: Pending
  - lastUpdateTime: "2022-02-23T23:00:38Z"
    message: waiting for etcd, controlplane and worker nodes to be registered
    reason: Provisioning
    status: Unknown
    type: Provisioned
  - lastUpdateTime: "2022-02-23T23:00:38Z"
    message: Waiting for API to be available
    status: Unknown
    type: Waiting
  - lastUpdateTime: "2022-02-23T23:00:37Z"
    status: "True"
    type: BackingNamespaceCreated
  - lastUpdateTime: "2022-02-23T23:00:37Z"
    status: "True"
    type: DefaultProjectCreated
  - lastUpdateTime: "2022-02-23T23:00:37Z"
    status: "True"
    type: SystemProjectCreated
  - lastUpdateTime: "2022-02-23T23:00:37Z"
    status: "True"
    type: InitialRolesPopulated
  - lastUpdateTime: "2022-02-23T23:00:37Z"
    status: "True"
    type: CreatorMadeOwner
  - lastUpdateTime: "2022-02-23T23:00:38Z"
    status: "True"
    type: NoDiskPressure
  - lastUpdateTime: "2022-02-23T23:00:38Z"
    status: "True"
    type: NoMemoryPressure
  - lastUpdateTime: "2022-02-23T23:00:38Z"
    status: "True"
    type: SecretsMigrated
  - lastUpdateTime: "2022-02-23T23:00:38Z"
    status: "False"
    type: Connected
  driver: rancherKubernetesEngine
  eksStatus:
    managedLaunchTemplateID: ""
    managedLaunchTemplateVersions: null
    privateRequiresTunnel: null
    securityGroups: null
    subnets: null
    upstreamSpec: null
    virtualNetwork: ""
  gkeStatus:
    privateRequiresTunnel: null
    upstreamSpec: null
  limits:
    cpu: "0"
    memory: "0"
    pods: "0"
  provider: ""
  requested:
    cpu: "0"
    memory: "0"
    pods: "0"

Currently the CLI download found in the Rancher UI v2.6-head is v2.6.4-rc1 so I ran into the error while testing. The CLI download will be updated to the latest version before the release.


Flaky Test

When running the tests, one will often fail, but not systematically:
gleam test
  Compiling blah
   Compiled in 0.56s
    Running blah_test.main
..F.....................
Failures:

  1) internet_test.uri_test
     Failure: ?assertMatch({ ok , _ }, A)
       expected: = { ok , _ }
            got: {error,nil}
     output:

Finished in 0.079 seconds
24 tests, 1 failures
(devbox) pascal@air-de-pascal-1 blah % gleam test
   Compiled in 0.03s
    Running blah_test.main
........................
Finished in 0.080 seconds
24 tests, 0 failures
(devbox) pascal@air-de-pascal-1 blah % gleam test
   Compiled in 0.02s
    Running blah_test.main
........................
Finished in 0.078 seconds
24 tests, 0 failures
(devbox) pascal@air-de-pascal-1 blah % gleam test
   Compiled in 0.02s
    Running blah_test.main
..F.....................
Failures:

  1) internet_test.uri_test
     Failure: ?assertMatch({ ok , _ }, A)
       expected: = { ok , _ }
            got: {error,nil}
     output:

Finished in 0.079 seconds
24 tests, 1 failures

Weird, I can't reproduce this!


Raspberry pi 4 support for otaclient.

Introduction
This PR introduces raspberry pi 4 support for otaclient, along with some refinements and refactor for boot_control module.
NOTE:
The refinement to the boot_control._common module(the new OTAStatusFilesControl and SlotMountHelper) is only applied to rpi_boot module, it will be applied to cboot and grub module and finally replace the old Mixin helpers in future PR.
Changes

rpi_boot module for rpi4 boot control support,
boot_control.common:  new OTAStatusFilesControl and SlotMountHelper classes, which are prepared for refined boot_control implementation framework in an effort of reducing duplicated code and unify implementation. Now ota_status files control logic and slot mounting logic are abstracted and split from the detailed platform specific implementation, become standalone shared classes.
common: implement replace_atomic function.
test files for new boot_control.common.
test files for rpi_boot module.
other minor related changes.

Ticket
T4PB-23180
TODO

[x] boot control test
[ ] boot control test with firmware update
[ ] full OTA update test
[x] test files implementation

@obi-t4 I add feature of updating firmware at the first reboot during switching boot at c13428f. This commit will use the flash-kernel system script to install the new firmware comes with new ota-image.
I will test it now and let you know if it works as expected.

In c13428f, when in the first reboot during switching boot, otaclient will call flash-kernel external script to copy new dtb files, bootloader binaries, along with kernel and initrd.img file from the new slot's rootfs to /boot/firmware(system-boot partition).
Actually we only need new dtb files and bootloader binaries, but since kernel and initrd.img also being copied to /boot/firmware, to save disk space, we do the rename(vmlinuz->vmlinuz_<current_slot>, initrd.img->initrd.img_) again.
Directly remove the redundant vmlinuz and initrd.img is OK but I think it is strange so I choose rename.

@obi-t4 firmware update implementation and test are finished!
2-stage reboot mechanism is introduced to support firmware update.
In first stage reboot, a.k.a first reboot after local ota update is applied, otaclient will finalize the switching boot and install the new firmware to system-boot partition, and then reboot(note that the ota_status is still UPDATING in the first stage reboot);
In the second reboot, otaclient will finally finish the ota update and set ota_status to SUCCESS.
With the above mechanism, raspberry pi will always use the latest firmware comes with the new ota-image.

I will adjust the test files to support new rpi_boot with firmware update feature.

@obi-t4 test files update is also finished!
Please focus review on _rpi_boot._RPIBootControl class, especially the _RPIBootControl.finalize_switch_boot method, which relates to the 2-stage reboot mechanism for updating/applying new firmware, thank you!

@Bodong-Yang

There is a design fault in rpi_boot standby slot detection/preparation mechanism,

OK. Can you describe shortly?

@Bodong-Yang

There is a design fault in rpi_boot standby slot detection/preparation mechanism,

OK. Can you describe shortly?

Previously I detect standby slot by fslabel, but there might be edge condition that the standby slot lost its fslabel when the formatting is interrupted(or in any unexpected status).
In such case, if the otaclient restarts, it will fail to detect the standby slot if the standby slot is unavailable and fslabel is not set.
So I refactor how the standby slot being detected by using the similar logic as grub controller that detect standby by finding the sibling slot of the active slot.
Also when during update, I will always set the standby slot's fslabel to standby slot_id.

The refactor is finished, I am testing it right now.

by finding the sibling slot of the active slot.

OK, thanks for the explanation!

@obi-t4 refactoring is finished, I am testing it on rpi4 device right now.

@obi-t4 Full ota update with latest commit confirmed working!
But due to rebuild mode still copy files from active slot, the update takes around 1.2hour to finish.
I think the in-place update mode that directly update standby slot is more suitable for raspberry pi.


Autocomplete Custom Properties

Is it possible in anyway, or as a future feature, for this plugin to support autocompleting custom properties/css variables? Possibly based on a variable file passed through a setting or something?

+1

Closing as this package has been deprecated in favor of the official styled-components/typescript-styled-plugin fork
After updating to that fork, follow up in the styled-components repo if this is still an issue


[Feature] - Support IAM Instance Profile Credentials

IAM Instance profile credentials use three pieces of auth: access key, secret key, AND token. It would be nice to be able to optionally include a token to authenticate.

@ckelner - I'll have a PR for your review shortly.

@ryansydnor awesome :+1:

Check it out! https://github.com/ckelner/AWSResco/pull/39


Return tests on CI and coveralls checks

Return tests on CI and coveralls checks were removed when we were moving the cardano-launcher repo, and it would be time to get it back and to get tests checks back.

Current progress on branch https://github.com/input-output-hk/cardano-shell/tree/ksaric/GH-280


Fix crash when target(parameter of Value.preserveToString) is undefined in WSH

Fix crash when target(parameter of Value.preserveToString) is undefined in WSH https://github.com/paulmillr/es6-shim/issues/467

Codecov Report

Merging #470 (d00ef8f) into master (8d44bc1) will decrease coverage by 30.52%.
The diff coverage is 100.00%.

@@             Coverage Diff             @@
##           master     #470       +/-   ##
===========================================
- Coverage   86.05%   55.52%   -30.53%
===========================================
  Files           2        2
  Lines        2352     2352
  Branches      607      607
===========================================
- Hits         2024     1306      -718
- Misses        328     1046      +718

Impacted Files
Coverage Δ

es6-shim.js
56.16% <100.00%> (-31.41%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8d44bc1...d00ef8f. Read the comment docs.

* typeof(globals.Reflect): object
* typeof(ReflectShims.defineProperty): undefined
overrideNative(globals.Reflect, 'defineProperty', ReflectShims.defineProperty); <-- error

This means that globals.Reflect variable is going to be undefined.
How can I fix a problem that ReflectShims.defineProperty is undefined?

Ah - if Reflect exists but Reflect.defineProperty does not, then that block of code should do branch, and call defineProperty versus overrideNative.

Found a better solution.

@gnh1201 what better solution? Is #467 still an issue?


Bump go version to 1.10.2 for travis

The travis build fails with undefined: strings.Builder. This is due to an old go version (1.8.1 :scream: )
Also, see a similar fix for publishing bot for more details: https://github.com/kubernetes/publishing-bot/pull/74.
This blocks merging any PRs sent directly to this repo: https://github.com/kubernetes/client-go/pull/466.
/assign @sttts

Should this be 1.10.4 now: https://github.com/kubernetes/kubernetes/pull/68802?
That PR hasn't merged (needs milestone and priority labels).

Or we could bump it to 1.10.4 as a follow-up. 1.10.2 should be enough to unblock us.

Travis is green :tada:

/lgtm
/approve


Is it possible to rotate the video?

We have a stream coming from an RTSP camera, but it is fitted upside down due to design decisions. So, now I want to rotate the stream 180 degrees before displaying it. Is it possible with your library?

Use videoRotation parameter.
See https://github.com/alexeyvasilyev/rtsp-client-android/blob/master/app/src/main/java/com/alexvas/rtsp/demo/live/LiveFragment.kt


How to report standard errors in parentheses when using linearmodels.iv.results.compare()?

``

I think you are missing some details about what you are tying to do and what you are getting.

You are right.
I am using linearmodels.iv.results.compare() to compare multiple instances of linearmodels.IV2SLS estimated over different samples. When printing the linearmodels.iv.results.IVModelComparison object returned by compare(), coefficients are reported with t-stats in parentheses. I'd rather have standard errors reported in parentheses. It would also be useful to report significance following star convention.
Thank you!
results = OrderedDict()
hs = [1*10**6*h for h in [1,0.5,0.25,0.1,0.05,0.01]]

for h in hs:
    df_tmp = df_est[
        (df_est['Y_i_19']>=t-h)
        & (df_est['Y_i_19']<=t+h)
    ]

    result = IV2SLS(
        dependent=df_tmp['log_Y_i_20'],
        exog=df_tmp[['const','log_Y_i_19']],
        endog=df_tmp['D_i'],
        instruments=df_tmp['Z_i']
    ).fit()

    results[f'h={str(h/(1*10**6))}M'] = result

print(compare(results))

                                               Model Comparison
==============================================================================================================
                             h=1.0M         h=0.5M        h=0.25M         h=0.1M        h=0.05M        h=0.01M
--------------------------------------------------------------------------------------------------------------
Dep. Variable            log_Y_i_20     log_Y_i_20     log_Y_i_20     log_Y_i_20     log_Y_i_20     log_Y_i_20
Estimator                   IV-2SLS        IV-2SLS        IV-2SLS        IV-2SLS        IV-2SLS        IV-2SLS
No. Observations             151967         122903          93102          35606          17322           3330
Cov. Est.                    robust         robust         robust         robust         robust         robust
R-squared                   -1.5846         0.0851        -0.2010        -0.2142        -0.3362        -2.1844
Adj. R-squared              -1.5846         0.0851        -0.2010        -0.2143        -0.3364        -2.1863
F-statistic               1.483e+04      1.565e+04         6396.8         311.58         38.126         20.954
P-value (F-stat)             0.0000         0.0000         0.0000         0.0000       5.26e-09      2.818e-05
==================     ============   ============   ============   ============   ============   ============
const                        4.0536         1.9123         0.0268         0.1623        -1.6381        -70.091
                           (139.97)       (35.518)       (0.2547)       (0.3685)      (-1.3129)      (-3.2516)
log_Y_i_19                   0.3436         0.6923         1.0036         0.9816         1.2813         12.687
                           (72.458)       (77.990)       (57.560)       (13.420)       (6.1731)       (3.5327)
D_i                         -1.8016        -0.2069         0.5407         0.4457         0.5566         1.3686
                          (-43.181)      (-6.3586)       (12.759)       (6.2278)       (5.4522)       (4.5442)
==================== ============== ============== ============== ============== ============== ==============
Instruments                     Z_i            Z_i            Z_i            Z_i            Z_i            Z_i
--------------------------------------------------------------------------------------------------------------

T-stats reported in parentheses

See the precison option here.  There is no support for stars.  I am under the impression that stars aren't considered best practice anymore.

Is someone wanted to add an optional stars argument, I'd be ok with that.
On Fri, Feb 21, 2020, 20:11 Simón Ramírez Amaya<EMAIL_ADDRESS>wrote:

Closed #248 https://github.com/bashtage/linearmodels/issues/248.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/bashtage/linearmodels/issues/248?email_source=notifications&email_token=ABKTSRNI5EMZDH5KOPJAPETREAYPZA5CNFSM4KZGSGRKYY3PNVWWK3TUL52HS4DFWZEXG43VMVCXMZLOORHG65DJMZUWGYLUNFXW5KTDN5WW2ZLOORPWSZGOWZ7PR6Q#event-3061774586,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABKTSRIRJCMWHTOPH2FHB2LREAYPZANCNFSM4KZGSGRA
.

Hi, I would love the stars enhancement. Just seconding the demand for it :)


Background image doesn't work

Hi, I have the Background Image option selected, with the default image, but it doesn't set any background image.

Hi @cbdavide, thanks for reporting. I could be wrong, but your version of Atom seems to be behind a number of key releases. Could you check that? Also, I'd be interested to know if there is any information in the console.

Ohh! men, I'm sorry, it was the version of Atom, everything works fine!
Sorry for the inconvenient.

Ohh! men, I'm sorry, it was the version of Atom, everything works fine!
Sorry for the inconvenient.


Add my vim-astro plugin

There has been no official support for a long time and many developers have requested it. I think it would be a great contribution to your wonderful project.

What kind of changes does this PR include?

New or updated content

Description

What does this PR change? Give us a brief description.

Adds link and description about using Vim or Neovim to develop with Astro!

Oh thank you so much Sarah, that was fast! 🚀
It is an honor to be able to contribute something to Astro and now be part of Team Docs. 🥳


Could it be used in React as well?

This is a bug tracker. What you're submitting here is not, by any stretch, a bug.


Spinner during local compilation consumes ~17% CPU

Describe the bug
During compilation on the local device, the spinner "./qt/spinner" consumes 17% of the CPU on the device. Considering the already constrained processor, it may be worth investigating on how to decrease the utilization.
How to reproduce or log data

Open an SSH connection to a device.
rm -rf /data/openpilot
git clone --recurse-submodules https://github.com/commaai/openpilot.git /data/openpilot
Reboot the device
Let the device start up and display the spinner on screen while compiling (a progress bar should appear at the bottom)
Reconnect your SSH session to the device
htop

Device/Version information (please complete the following information):

Device: comma two
Dongle ID: 40ca690add6c053f
Version: d40891739fa72e0ed658c2beca15f303def08681

Additional context
Although not important for an end user, for a developer this could result in a decrease in time for developer to work on the project.

@deanlee interested in this one?

Perhaps no better way to do this, 30fps requires 8% cpu even without any drwaing. the quicker solution is to set it to 20fps

Perhaps no better way to do this, 30fps requires 8% cpu even without any drwaing. the quicker solution is to use OpenGL 2d painting ,and set fps to 20

Reduced a bit in #20618. Further improvements are welcome, but at its current CPU usage, this isn't really of concern as it doesn't affect the experience for release2 users.


BUG: complex256 type created by entering complex128 into linalg.sqrtm causes errors.

Describe your issue.
When scipy.linalg operates on a complex array/matrix it returns a complex array with double the precision of the input array. As a result if given a complex128 type array it will return a complex256 array.
This type of array is not supported by numpy.linalg functions and will result in errors. And  scipi.linalg functions it will automatically downcast any complex256 type to a complex128 type.
This causes many problems in large code bases with combine scipy and numpy.
I have attached a sample code which shows the behavior of both scipy and numpy when acting on this datatype.
Reproducing Code Example
import numpy as np
import scipy as sp

# create a numpy array of type complex128
a = np.array([[1.0, 3.0], [1.0, 4.0]],dtype=np.complex128)
print(type(np.square(a[0][0])))

# sqrtm returns an array of complex256
b = sp.linalg.sqrtm(a)
print(type(np.square(b[0][0])))

# arrays of type complex128 will run fine for both
# scipy and numpy functions
print(type(np.linalg.eig(a)[0][0]))
print(type(sp.linalg.eig(a)[0][0]))

# arrays of type complex256 will run for
# scipy functions, but not for the exact same numpy function
# it would seem that scipy functions reduce the precision of
# the complex256 inputs while numpy functions throw a TypeError
print(type(sp.linalg.eig(b)[0][0]))
print(type(np.linalg.eig(b)[0][0]))

Error message
<class 'numpy.complex128'>
<class 'numpy.complex256'>
<class 'numpy.complex128'>
<class 'numpy.complex128'>
<class 'numpy.ndarray'>
Traceback (most recent call last):
  File "/home/cphys/projects/scipy_numpy_bug/scipy_test.py", line 20, in <module>
    print(type(np.linalg.qr(b)[0][0]))
  File "<__array_function__ internals>", line 200, in qr
  File "/home/cphys/.local/lib/python3.10/site-packages/numpy/linalg/linalg.py", line 915, in qr
    t, result_t = _commonType(a)
  File "/home/cphys/.local/lib/python3.10/site-packages/numpy/linalg/linalg.py", line 147, in _commonType
    raise TypeError("array type %s is unsupported in linalg" %
TypeError: array type complex256 is unsupported in linalg

SciPy/NumPy/Python version and system information
1.10.0 1.24.2 sys.version_info(major=3, minor=10, micro=6, releaselevel='final', serial=0)
blas_mkl_info:
  NOT AVAILABLE
blis_info:
  NOT AVAILABLE
openblas_info:
    libraries = ['openblas', 'openblas']
    library_dirs = ['/usr/local/lib']
    language = c
    define_macros = [('HAVE_CBLAS', None)]
    runtime_library_dirs = ['/usr/local/lib']
blas_opt_info:
    libraries = ['openblas', 'openblas']
    library_dirs = ['/usr/local/lib']
    language = c
    define_macros = [('HAVE_CBLAS', None)]
    runtime_library_dirs = ['/usr/local/lib']
lapack_mkl_info:
  NOT AVAILABLE
openblas_lapack_info:
    libraries = ['openblas', 'openblas']
    library_dirs = ['/usr/local/lib']
    language = c
    define_macros = [('HAVE_CBLAS', None)]
    runtime_library_dirs = ['/usr/local/lib']
lapack_opt_info:
    libraries = ['openblas', 'openblas']
    library_dirs = ['/usr/local/lib']
    language = c
    define_macros = [('HAVE_CBLAS', None)]
    runtime_library_dirs = ['/usr/local/lib']
Supported SIMD extensions in this NumPy install:
    baseline = SSE,SSE2,SSE3
    found = SSSE3,SSE41,POPCNT,SSE42,AVX,F16C,FMA3,AVX2
    not found = AVX512F,AVX512CD,AVX512_KNL,AVX512_KNM,AVX512_SKX,AVX512_CLX,AVX512_CNL,AVX1.10.0 1.24.2 sys.version_info(major=3, minor=10, micro=6, releaselevel='final', serial=0)
blas_mkl_info:
  NOT AVAILABLE
blis_info:
  NOT AVAILABLE
openblas_info:
    libraries = ['openblas', 'openblas']
    library_dirs = ['/usr/local/lib']
    language = c
    define_macros = [('HAVE_CBLAS', None)]
    runtime_library_dirs = ['/usr/local/lib']
blas_opt_info:
    libraries = ['openblas', 'openblas']
    library_dirs = ['/usr/local/lib']
    language = c
    define_macros = [('HAVE_CBLAS', None)]
    runtime_library_dirs = ['/usr/local/lib']
lapack_mkl_info:
  NOT AVAILABLE
openblas_lapack_info:
    libraries = ['openblas', 'openblas']
    library_dirs = ['/usr/local/lib']
    language = c
    define_macros = [('HAVE_CBLAS', None)]
    runtime_library_dirs = ['/usr/local/lib']
lapack_opt_info:
    libraries = ['openblas', 'openblas']
    library_dirs = ['/usr/local/lib']
    language = c
    define_macros = [('HAVE_CBLAS', None)]
    runtime_library_dirs = ['/usr/local/lib']
Supported SIMD extensions in this NumPy install:
    baseline = SSE,SSE2,SSE3
    found = SSSE3,SSE41,POPCNT,SSE42,AVX,F16C,FMA3,AVX2
    not found = AVX512F,AVX512CD,AVX512_KNL,AVX512_KNM,AVX512_SKX,AVX512_CLX,AVX512_CNL,AVX512_ICL
512_ICL

That's probably an oversight that came from a contributor with a Windows machine. Because on Win, complex256 is complex 128 anyways. We don't use complex256 anywhere anyways. I'll check.

For reference, this was added intentionally in gh-16294 in response to gh-14853, but as noted in the last comment there, this was probably not the correct thing to do. @AtsushiSakai, can you explain why the precision was doubled in that case? I concur with @WarrenWeckesser's comment. It doesn't seem like the right thing to do to me.

I thought if the input dtype of the sqrtm function is real and the output dtype is complex, the precision had to be doubled to maintain accuracy (e.g., if the input was int32, the output would be complex64).
However, it is true that there may not have been a need to double the precision if both the input and output were complex. In the current implementation, the decision is made based only on the output dtype of the calculation result, so using the input dtype as well might allow for maintaining precision when both input and output are complex.
I will try to implement it that way and see if it works.

Yes, when the input is real floating, the complex output will be twice the itemsize of the input, but that should not be the case when the input is complex. The real part of the input needs to be the same as the real part of the output.
One more thing: in the case of integer input, you need to check the size of the input after it gets converted to a floating point type by _asarray_validated(A, check_finite=True, as_inexact=True). That's the relevant dtype to be looking at, not the integer's itemsize.


Name
Nord
Description
Unofficial nord theme for Zen Browser
Homepage
https://www.nordtheme.com/
Image
https://github.com/nordtheme/visual-studio-code/blob/develop/assets/extension-marketplace-icon-2x.png?raw=true
Type

[X] JSON Color Theme

Theme Styles
{
    "isDarkMode": true,
    "primaryColor": "#5e81ac",
    "secondaryColor": "#4c566a",
    "tertiaryColor": "#3b4252",
    "colorsBorder": "#4c566a",
    "dialogBg": "#2e3440"
}

Readme
Nord theme for Zen Browser

Preferences
No response

It was rejected due to:
https://github.com/zen-browser/theme-store/pull/188#issuecomment-2351527336


Add sanitize_args so credentials are not logged

Sanitizes the value for the --github-token param so it isn't logged.

Codecov Report

Merging #20 (99e1532) into master (c7689e5) will decrease coverage by 3.90%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##           master      #20      +/-   ##
==========================================
- Coverage   88.14%   84.23%   -3.91%
==========================================
  Files           7        7
  Lines         194      203       +9
  Branches       22       24       +2
==========================================
  Hits          171      171
- Misses         21       30       +9
  Partials        2        2

Impacted Files
Coverage Δ

step_exec_lib/utils/processes.py
0.00% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c7689e5...99e1532. Read the comment docs.

Also not needed since github token was removed.


Bad workspace trust layout

Reload the window (seems to only happen the first time I open trust)
Have a split view
Manage Workspace Trust
The folders are truncated with the "Actions" area being giant. I can't resize the "Path" column, nor is there a tooltip for me to see what the paths are, so I'm stuck

I think we should always have a tooltip on the folder, since truncation is possible for any sufficiently-long path.

https://github.com/microsoft/vscode/issues/125406


Fix Ceph Mon

Hi,
This PR fixe issue from ceph module add possibility to set :
"mon.node1" = { "public_addr" = "<IP_ADDRESS>"; };

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Also i propose to try to use something like this for fix generation of config file.
# Function to format values with proper spacing
  formatValue = value:
    if builtins.isString value
    then " = ${value}"
    else " = ${toString value}";

  # Custom INI generator that preserves section order and formatting
  toOrderedINI = sections:
    let
      # Define the desired section order
      sectionOrder = [
        "global"
        "client"
        "client.crash"
      ] ++ (map (id: "mon.${id}") cfg.mon.daemons)
        ++ (map (id: "mds.${id}") cfg.mds.daemons)
        ++ (map (id: "osd.${id}") cfg.osd.daemons)
        ++ (map (id: "mgr.${id}") cfg.mgr.daemons);

      # Format a single key-value pair
      formatKeyValue = name: value: "${name}${formatValue value}";

      # Format a section's content
      formatSection = name: attrs:
        let
          sortedPairs = lib.mapAttrsToList formatKeyValue attrs;
        in
        "[${name}]\n${lib.concatStringsSep "\n" sortedPairs}\n";

      # Get all section names
      allSections = builtins.attrNames sections;

      # Order sections according to sectionOrder, putting unknown sections at the end
      orderedSections =
        let
          # Get sections that match our predefined order
          knownSections = builtins.filter (s: builtins.elem s sectionOrder) allSections;
          # Get remaining sections
          unknownSections = builtins.filter (s: !(builtins.elem s sectionOrder)) allSections;
          # Sort known sections according to sectionOrder
          sortedKnown = builtins.sort (a: b:
            builtins.lessThan
              (builtins.elemIndex a sectionOrder)
              (builtins.elemIndex b sectionOrder)
          ) knownSections;
        in
        sortedKnown ++ unknownSections;

      # Format all sections in order
      formatSections = lib.concatMapStrings
        (name: formatSection name sections.${name})
        orderedSections;
    in
    formatSections;


feat: update missing_trip_edge for flex feed

Summary:
This update modifies the missing_trip_edge notice to ensure it is not triggered if the GtfsStopTime entity has either the startPickupDropOffWindow or endPickupDropOffWindow column defined.
Expected Behavior:
The missing_trip_edge notice will no longer be triggered in cases where one of the above columns is present. For example, using this dataset, the notice is no longer triggered, as shown below:

Please make sure these boxes are checked before submitting your pull request - thanks!

[x] Run the unit tests with gradle test to make sure you didn't break anything
[ ] Add or update any needed documentation to the repo
[x] Format the title like "feat: [new feature short description]". Title must follow the Conventional Commit Specification(https://www.conventionalcommits.org/en/v1.0.0/).
[x] Linked all relevant issues
[x] Include screenshot(s) showing how this pull request works and fixes the issue(s)

@cka-y Cross referencing with the original PR that added these errors  because stop_times.txt was finally parseable for Flex feeds, this acceptance test update looks right! The only one confusing me is missing_required_column for 1136. Any insight into why this PR affects it? Looks like it's already there as a notice on the Mobility Database: https://files.mobilitydatabase.org/mdb-1136/mdb-1136-202406071541/report_5.0.1.html
cc @tzujenchanmbd and @Sergiodero as reference for how we review the acceptance tests :)

@emmambd seems like there was a problem with the acceptance tests. Without updating the code, I ran them again and everything seems to work as expected.


Build statically linked binary

When I run the following command, as suggested in README.md:
cargo build --release

it results in a dynamically linked binary, which runs great locally, but I'm unable to run it on other machines, for example, in a docker image.
The releases page includes statically linked binaries. What is the command or config used to create these? If possible, could these commands be included in the README.md file for future reference?
Thanks for your help!

Static linked instructions is not included in README.md, as this is platform-dependent thing in Rust
For x86 linux - this is just cargo build --release --target=x86_64-unknown-linux-musl, for windows this is RUSTFLAGS=-Ctarget-feature=+crt-static, etc.
There is no universal command, and i think for most users this is enough to use precompiled binaries or build for native platform toolchain (and most platforms don't use static linking by default)

Thanks for getting back to me.
I tried this on Linux, on the v0.4.2 branch:
cargo build --release --target=x86_64-unknown-linux-musl

with this result:
error: cannot produce cdylib for `jsonnet v0.4.2 (/home/.../jrsonnet/bindings/jsonnet)` as the target `x86_64-unknown-linux-musl` does not support these crate types

I have these versions of rust and cargo:
rustc 1.58.1 (db9d1b20b 2022-01-20)
cargo 1.58.0 (f01b232bc 2022-01-19)

Is there something else that I need to have installed for this to work?

Running cargo build inside of workspace will build all projects in workspace, and there is bindings/jsonnet subproject, which is implementation of libjsonnet.so
Dynamic libraries (.so) can't be build statically with musl
You only need to build jrsonnet executable - go to cmds/jrsonnet subdirectory, and execute build here
(Also, i recommend you to use gcmodule branch, as there is several performance regressions in v0.4.2, and there is more supported features on gcmodule branch)


⚠ Bofrost.de has degraded performance

In e1a43b9, Bofrost.de (https://www.bofrost.de) experienced degraded performance:

HTTP code: 200
Response time: 1242 ms

Resolved: Bofrost.de performance has improved in cdfda9e.


Not properly Update Custom User Attributes Upon Page Refresh or Navigating Back from Another Tab/Page

Describe the issue:
Currently, all configured and saved application custom user attributes are disappearing when users refresh the page or navigate back from another tab or page. This
How to reproduce:

Configure and save custom user attributes in the application.
Refresh the page or navigate away to another tab/page.
Return to the original tab/page.
Observe that the previously configured custom user attributes  not updated.

Expected behavior:
Custom user attributes should persist even after page refresh or when users navigate away and then return to the application.
Environment information (Please complete the following information; remove  any unnecessary fields) :

Product Version: Alpha 7.0
OS: Mac, Linux
Database: MySQL, H2, MSSQL, Postgres
Userstore: LDAP, JDBC

updateissue1.webm

Upon further analysis, this seems to be a race condition. When you update the claim configs from the UI, there are 2 update requests are invoked.
curl 'https://localhost:9443/t/carbon.super/api/server/v1/applications/7e254aca-8aae-4d9c-ae4b-5e775c4f5723' \
  -X 'PATCH' \
  -H 'Accept: application/json' \
  -H 'Content-Type: application/json' \
  --data-raw '{"claimConfiguration":{"dialect":"LOCAL","requestedClaims":[],"role":{"claim":{"uri":""},"includeUserDomain":true}}}'

curl 'https://localhost:9443/t/carbon.super/api/server/v1/applications/7e254aca-8aae-4d9c-ae4b-5e775c4f5723/inbound-protocols/oidc' \
  -X 'PUT' \
  -H 'Accept: application/json' \
  -H 'Content-Type: application/json' \
  --data-raw '{"clientId":"JMeiAQWdP8ft5qxd4fKrFQfrF2Qa","clientSecret":"ZoQ9rd2qEmAgKOBsPCfJfx3bH2SpVSRGEy2WWZPfNSIa","state":"ACTIVE","grantTypes":["client_credentials"],"callbackURLs":[],"allowedOrigins":[],"publicClient":false,"pkce":{"mandatory":false,"supportPlainTransformAlgorithm":false},"accessToken":{"type":"Default","userAccessTokenExpiryInSeconds":3600,"applicationAccessTokenExpiryInSeconds":3600,"revokeTokensWhenIDPSessionTerminated":false,"validateTokenBinding":false},"refreshToken":{"expiryInSeconds":86400,"renewRefreshToken":false},"idToken":{"expiryInSeconds":3600,"audience":[],"encryption":{"enabled":false,"algorithm":"","method":""}},"logout":{},"validateRequestObjectSignature":false,"scopeValidators":[],"clientAuthentication":{},"requestObject":{"encryption":{"algorithm":"","method":""}},"pushAuthorizationRequest":{"requirePushAuthorizationRequest":false},"subject":{"subjectType":"public"}}' \

Though these are going as 2 update requests for 2 sections of the application, the whole application is updated with both of the requests.
In the backend first we get the application configuration and then update the application object with the requested changes in the request. Then we delete the application details and recreate it with the updated objects. Now when we take the application for the 2nd request the claims are not modified yet, and when we update the application for the 2nd request, it will override the claims again.


2 Requested Features

Please add these features:

Enable removing & blocking ads from any app then rebuilding it (+ removing ads space).
Enable making any app fully offline (blocking internet access).
Thanks.

Maintenance has been stopped.
获取 Outlook for iOShttps://aka.ms/o0ukef

发件人: USER @.>
发送时间: Monday, September 9, 2024 7:14:25 PM
收件人: PatrickAlex2019/ApkEditor @.>
抄送: Subscribed @.***>
主题: [PatrickAlex2019/ApkEditor] Feature Request (Issue #11)
Please add a tool to disable ads from any app then rebuilding it (+ removing ads space).
Thsnks.
―
Reply to this email directly, view it on GitHubhttps://github.com/PatrickAlex2019/ApkEditor/issues/11, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ANA6W76WOQ674VJ7UJLJGVLZVV7JDAVCNFSM6AAAAABN4JXY2WVHI2DSMVQWIX3LMV43ASLTON2WKOZSGUYTGNRWGMYTQMI.
You are receiving this because you are subscribed to this thread.Message ID: @.***>


Integration with imenu

A simplistic imenu-create-index-function can look like this:
(defun tree-sitter-rust-imenu-index-function ()
  (thread-last (tree-sitter-query [(function_item (identifier) @function)])
    (seq-map (lambda (capture)
               (pcase-let ((`(_ . ,node) capture))
                 (cons (ts-node-text node)
                       (ts-node-start-position node)))))))

A more substantial implementation can use ts-query-matches to capture nested structures.

Hi! I was playing around with this to try and see what kind of imenu integration I can get going, but I am getting a (wrong-type-argument user-ptrp nil), with the following stacktrace:
  ts-make-query(nil [(function_item (identifier) @name)])
  tree-sitter-query([(function_item (identifier) @name)])
  (seq-map #'(lambda (capture) (let* ((x474 (car capture)) (x475 (cdr capture))) (let ((node x475)) (progn (print ts-node-text node) (cons (ts-node-text node) (ts-node-start-position node)))))) (tree-sitter-query [(function_item (identifier) @name)]))
  (save-current-buffer (set-buffer (current-buffer)) (seq-map #'(lambda (capture) (let* ((x474 (car capture)) (x475 (cdr capture))) (let ((node x475)) (progn (print ts-node-text node) (cons ... ...))))) (tree-sitter-query [(function_item (identifier) @name)])))
  ts-identifiers()

It seems that the error is coming from within the Rust module? So I am not sure how I should go about to debug.
For reference, this is what the function I'm testing looks like:
  (defun ts-identifiers()
    (interactive)
    (with-current-buffer (current-buffer)
      (seq-map (lambda (capture)
                 (pcase-let ((`(_ . ,node) capture))
                   (print ts-node-text node)
                   (cons (ts-node-text node) (ts-node-start-position node))))
               (tree-sitter-query [(function_item (identifier) @name)])))
    )

This line  ts-make-query(nil [(function_item (identifier) @name)]) means that tree-sitter-language was nil when tree-sitter-query called ts-make-query .
It probably means tree-sitter-mode was not turned on in that buffer.
Also note that tree-sitter-query is more like a debugging aid currently. It creates a new query object every time. You'd better off create a query object once, and use ts-query-matches or ts-query-captures instead.

I see. Got it working again, thanks! Will start experimenting and move towards using tree-query-matches to get imenu integration working.

tinyts

I'm interested in implementing this. Do you have any tips for how to approach it? I was thinking maybe reusing the queries we use for highlighting (in emacs-tree-sitter/tree-sitter-langs) or adding a new file in the same place for imenu queries.

Imenu support would be really great

@DamienCassou See #199. It's still at the draft stage and I'm not sure what the best approach moving forward should be :/.

I still haven't looked into #199 in details, but like @mohkale noted in that issue, I think arbitrary recursion is the single biggest challenge, specifically capturing the full paths of all nested items, not just the leaf identifiers. Queries alone are probably not expressive enough, so some sort of principled extension would be needed. On that front, I think it's worth exploring the newly introduced stack graphs, and the related graph DSL:
(function_definition
  name: (identifier) @name) @function
{
    node @function.def
    attr (@function.def) kind = "definition"
    attr (@function.def) symbol = @name
    edge @function.containing_scope -> @function.def
}

I went through the video where @dcreager illustrated the concepts very well.

Hi! Is there any update on this feature? Thanks :)


feat-req: Expose superform stores on new method

It would be nice if we could have access to the stores that are created via superforms, so we bind that data or do whatever we wish with it.
I have a global derived svelte store that keeps track of all forms or actions being currently submitted, so I usually bind the $submitting superform store to my own custom store, which then get's picked up by the derived store.
As of right now, we only have access to these values from within the Form.Root, so the only options are either to...
a) Create a new component and insert it as a child
or
b) set the store value within the onSubmit superform event handler
You said...

The big thing is that basically all the options, etc. are all applied beforehand so it can't be reinstantiated

So maybe there could be a method that could be called on the Form.Root so when the Form is initialized, we could have access to any of these stores and bind them if we wanted.
Something like...
<Form.Root onInit={ ({ submitting, delayed, tainted, form }) => {
    // Set what you want or bind to or own stores...
    ...
}}
</Form.Root>

What do you think?

Hey @saturnonearth, if you could expand on your example code block there for how you'd imagine it working, it'll give me a better idea of the ask/specifics behind it!

Hey @saturnonearth, if you could expand on your example code block there for how you'd imagine it working, it'll give me a better idea of the ask/specifics behind it!

<script>
const formSubmitting = writable(false);
const form: formSchema = writable({});

// Compare the data from what the customer entered, so we can keep track of whether data is old or stale
$: $headingFormStale = !compare(itemSaved, $form);
</script>

<Form.Root onInit={ ({ submitting, delayed, tainted, form }) => {
     $formSubmitting = submitting;
     $form = form;
  }

  ...
  </Form.Root>

Upon writing this out, I realize it's a bit weird. I guess my end goal is just to have access to the Superform stores from outside the Form.Root, as the stores give a lot of "super" useful info that is often useful for keeping track of state outside of the form itself.

Okay, I see what you're getting at. I know this is possible and how to do it, I'm just trying to think of the implications of doing so. They wouldn't be stores, but they would update when the stores update, you'd have to do something like:
<script>
	let formSubmitting = false;
</script>

<Form.Root bind:formSubmitting />

which is similar to how things are handled in bits-ui with how we update the prop's value if the store changes.

Going to be revisiting this in relation to #69 as I think the two are more or less related, at least from first glance. I have some ideas that I'll put up in a PR once I have some time for you all to check out!

Closed by #83 :)


Cursor parameter to events request

Is this a parameter that will be added anytime soon?

@zseta thanks for responding!
Basically just keep track of events request/response state using the new cursor pagination "upgrade".
I suppose events_backfill covers that functionality? Btw, excellent job on the development of this library.

If you just want to paginate to download multiple pages of event data you can use the events_backfill() function, example:
api = OpenseaAPI(apikey="")
event_generator = api.events_backfill(start=datetime(2021, 10, 5, 3, 25, tzinfo=timezone.utc),
                                      until=datetime(2021, 10, 5, 3, 20, tzinfo=timezone.utc),
                                      event_type="successful")
for event in event_generator:
    if event is not None:
        print(event) # or do other things with the event data

This will handle the cursor stuff internal.
(Taken from the NFT Starter Kit.)

Ah, nice! Thank you!


Remove the Extension class and associated classes

Remove the Extension class and associated APIs to clear the path for the new Extension API and workflows. Refactored the current Extensions to work as build-time extensions rather than supporting the runtime via the Extension class.
The extension-webpack-tools have also been removed as the new Extension setup will not depend on any specific webpack magic.

@Mergifyio update


merlin-locate fails on me

Not in environment 'x'
How to troubleshoot?
This is mostly the only merlin feature I need to use ...

It only works when the definition is in the same file.
Not very useful since I want to use merlin in big projects (not written by me so I don't know where things are defined).

https://github.com/the-lambda-church/merlin/wiki/vim-from-scratch#source-browsing

OK, the .merlin file was completely outdated, sorry for the noise.


Time machine

Opening a PR to check in the time machine code as it stands now. It's currently hidden behind a flag.
The final UI for this is TBD, but the current UI works. For diffing/patching, I'm using Google's diff-match-patch library which is Apache 2.0 licensed. Currently every single change creates a history entry which is way too much. Once we have the UI finalized I'll have a better idea of what interval we want to use.
Here's an upload target link:
https://arcade.makecode.com/app/49743c2e211ad4a2542a4d91d0a5a3a059db044c-f4a46a5ba2
When using that link, I recommend starting with a fresh project. Also, be sure to only edit said project in the above target link; it will work in other editors but the history will get corrupted. It's on my todo list to figure out a way to detect when this happens

@eanders-ms because we're always just taking a straight diff, whenever the project is saved again that patch will just be included in the user changes. Shouldn't cause any corruption.
I will add the editor version to the history entries, though.


Patch 1

Should fix #83 with respect to #85
(lets see if the checks pass)

@ZerataX is the failed check a result from my code or is this the experimantal docker build test?

nvm of cause it fails, still has the old version of api.py
Marking this as not ready for merge until 85 is resolved

merging this so the check wont fail on 85


What will happening if I use the 5V/16Mhz pro mini

I only have 5v/16mhz pro mini ， how if i can use this board？

You can use it, but make sure the NRF24 radio module is supplied by 3.3V. Also make sure, that your minimum battery voltage is about 5.7V

Thanks，

finally，  I replaced the MIC2505 with the 3.3v version on the board


std contains a duplicate function for getting the gas in the current context

Summary
Two functions exist under different names in two different places. I will argue that the first one should remain (because of the explicit name that differentiates from the global gas) and the one at the bottom should be removed. Although "context_gas" may not be the best name imo.
https://github.com/FuelLabs/sway/blob/a3ffd94c413ca62a8fc130a55b296ee61d6ab1e5/sway-lib-std/src/context/registers.sw#L54-L66
https://github.com/FuelLabs/sway/blob/a3ffd94c413ca62a8fc130a55b296ee61d6ab1e5/sway-lib-std/src/context.sw#L23-L28

The name context_gas is fairly intuitive imo. What about local_gas?

All of those registers.sw functions are named after the opcode that they're wrapping.  They're such small functions I'm not sure it's worth worrying about, we could keep both, though I see everyone up-thumbing it, so I guess gas() can go.

Yeah "context" is a term that is well defined in the specs. It's probably best to keep the name to avoid confusion.


Ad gif between every images Starbound scans

Source name
Starbound scans
Source language
French
Describe your suggested feature
Between every image of a chapter there is an annoying gif added that say « read next chapter only at kewnscans.org », it shouldn’t be there.

Other details
No response
Acknowledgements

[X] I have searched the existing issues and this is a new ticket, NOT a duplicate or related to another open or closed issue.
[X] I have written a short but informative title.
[X] If this is an issue with the app itself, I should be opening an issue in the app repository.
[X] I have updated the app to version 0.15.3.
[X] I will fill out all of the requested information in this form.

closing in favour of #2976


add global

Good morning @louh @migurski! Any idea what this Circle error means?

AssertionError: 4 != 3 : There should be three breadcrumbs at top at /tangram/Scene-file/index.html

It’s a test that I wrote to ensure that the output of the docs “looks right”. It’s complaining that there’s an extra breadcrumb in the navigation up here, which does indeed look really odd to me as well:

Ahh, breadcrumb = navigation level? That does look a bit cramped. Okay, will revise. Thanks!

Wow, that is some funky table of contents formatting!

Replaced by #135.


djl-inference and /ping API endpoint

Hi All, I am deploying a model on SageMaker and getting the following error. Any advice on how to resolve it?
UnexpectedStatusException: Error hosting endpoint gptneox-demo-v1-0-0-endpoint: Failed. Reason: The primary container for production variant AllTraffic did not pass the ping health check. Please check CloudWatch logs for this endpoint..

The cloudwatch says:
java.util.concurrent.CompletionException: java.io.FileNotFoundException: partitioned_model_ file not found in: /opt/ml/model/code
Caused by: java.io.FileNotFoundException: partitioned_model_ file not found in: /opt/ml/model/code

I am using AWS-supplied framework container<PHONE_NUMBER>84.dkr.ecr.us-east-1.amazonaws.com/djl-inference:0.21.0-deepspeed0.8.3-cu117
When I opened a case with support, they said that the container does not respond to requests to /ping endpoint.
How do I supplement, modify the model to respond?

The model has 2 files:
%%writefile {model_base_path}/code/model.py
from djl_python import Input, Output
import os
import deepspeed
import torch
from transformers import pipeline, AutoModelForCausalLM, AutoTokenizer

predictor = None

def get_model():
    model_name = 'EleutherAI/gpt-neox-20b'
    tensor_parallel = int(os.getenv('TENSOR_PARALLEL_DEGREE', '1'))
    local_rank = int(os.getenv('LOCAL_RANK', '0'))
    model = AutoModelForCausalLM.from_pretrained(model_name, revision="float32", torch_dtype=torch.float32)
    tokenizer = AutoTokenizer.from_pretrained(model_name)

    model = deepspeed.init_inference(model,
                                     mp_size=tensor_parallel,
                                     dtype=model.dtype,
                                     replace_method='auto',
                                     replace_with_kernel_inject=True)
    generator = pipeline(task='text-generation', model=model, tokenizer=tokenizer, device=local_rank)
    return generator

def handle(inputs: Input) -> None:
    global predictor
    if not predictor:
        predictor = get_model()

    if inputs.is_empty():
        # Model server makes an empty call to warmup the model on startup
        return None

    data = inputs.get_as_string()
    result = predictor(data, do_sample=True, min_tokens=200, max_new_tokens=256)
    return Output().add(result)

and
%%writefile {model_base_path}/code/serving.properties
engine=DeepSpeed

Thanks for raising this issue @greenpau. We are aware of this bug and it should be resolved in our next container release.
In the meantime, there are a few approaches that can help work past this issue.

In serving.properties file, add another property option.tensor_parallel_degree=<value>. This is the easiest solution

You can leverage the DJLModel from the SageMaker Python SDK to deploy this model with our built in inference handlers. Some sample code for this would be

from sagemaker.djl_inference import DJLModel

role = <your_iam_role_arn>
model = DJLModel(
    "EleutherAI/gpt-neox-20b",
    role,
    number_of_partitions=8,
    dtype="fp32",
)

predictor = model.deploy("ml.g5.48xlarge")

For the gpt-neox-20b model at full precision, the minimum total gpu memory required will be 80GB. Using a g5.12xlarge with tensor parallel degree of 4 will probably not be sufficient since the additional memory required at runtime for inference will likely exceed the gpu memory available. I would recommend at least a g5.48xlarge with tensor parallel degree of 8.

@siddvenk , thank tou for the answer! What if I only have access to 24xlarge? What do I need to change in my code?

@siddvenk , thank you very much! 👍 I went with option 2.


[SUGGESTION] Custom model data for main gui things as arrow

As talked on spigot, it's about the back button, I know some people that needs the same features so I created that to make it easier and less spammy on spigot :)

Oh thx, i was about to make one as well. We rly need this feature

I started uploading my plugins to github and making them open-source so you can add your own specific features ;)

This is such a good idea!!


CrashServer use /rtp BetterRTP 2.11.2 1.16.1

crash.txt

[09:27:00] [Server thread/INFO]: garik4 issued server command: /rtp
[09:27:13] [Paper Watchdog Thread/ERROR]: --- DO NOT REPORT THIS TO PAPER - THIS IS NOT A BUG OR A CRASH  - git-Paper-133 (MC: 1.16.1) ---
[09:27:13] [Paper Watchdog Thread/ERROR]: The server has not responded for 10 seconds! Creating thread dump
[09:27:13] [Paper Watchdog Thread/ERROR]: ------------------------------
[09:27:13] [Paper Watchdog Thread/ERROR]: Server thread dump (Look for plugins here before reporting to Paper!):
[09:27:13] [Paper Watchdog Thread/ERROR]: ------------------------------
[09:27:13] [Paper Watchdog Thread/ERROR]: Current Thread: Server thread
[09:27:13] [Paper Watchdog Thread/ERROR]: 	PID: 22 | Suspended: false | Native: false | State: TIMED_WAITING
[09:27:13] [Paper Watchdog Thread/ERROR]: 	Stack:
[09:27:13] [Paper Watchdog Thread/ERROR]: 		java.base@11.0.7/java.lang.Thread.sleep(Native Method)
[09:27:13] [Paper Watchdog Thread/ERROR]: 		me.SuperRonanCraft.BetterRTP.player.RTPTeleport.loadChunks(RTPTeleport.java:55)
[09:27:13] [Paper Watchdog Thread/ERROR]: 		me.SuperRonanCraft.BetterRTP.player.RTPTeleport.sendPlayer(RTPTeleport.java:21)
[09:27:13] [Paper Watchdog Thread/ERROR]: 		me.SuperRonanCraft.BetterRTP.player.RTP.tp(RTP.java:166)
[09:27:13] [Paper Watchdog Thread/ERROR]: 		me.SuperRonanCraft.BetterRTP.player.Delay.lambda$run$0(Delay.java:93)
[09:27:13] [Paper Watchdog Thread/ERROR]: 		me.SuperRonanCraft.BetterRTP.player.Delay$$Lambda$6385/0x0000000841a1dc40.run(Unknown Source)
[09:27:13] [Paper Watchdog Thread/ERROR]: 		app//org.bukkit.craftbukkit.v1_16_R1.scheduler.CraftTask.run(CraftTask.java:99)
[09:27:13] [Paper Watchdog Thread/ERROR]: 		app//org.bukkit.craftbukkit.v1_16_R1.scheduler.CraftScheduler.mainThreadHeartbeat(CraftScheduler.java:468)
[09:27:13] [Paper Watchdog Thread/ERROR]: 		app//net.minecraft.server.v1_16_R1.MinecraftServer.b(MinecraftServer.java:1292)
[09:27:13] [Paper Watchdog Thread/ERROR]: 		app//net.minecraft.server.v1_16_R1.DedicatedServer.b(DedicatedServer.java:377)
[09:27:13] [Paper Watchdog Thread/ERROR]: 		app//net.minecraft.server.v1_16_R1.MinecraftServer.a(MinecraftServer.java:1213)
[09:27:13] [Paper Watchdog Thread/ERROR]: 		app//net.minecraft.server.v1_16_R1.MinecraftServer.v(MinecraftServer.java:1001)
[09:27:13] [Paper Watchdog Thread/ERROR]: 		app//net.minecraft.server.v1_16_R1.MinecraftServer.lambda$a$0(MinecraftServer.java:177)
[09:27:13] [Paper Watchdog Thread/ERROR]: 		app//net.minecraft.server.v1_16_R1.MinecraftServer$$Lambda$3005/0x000000084082c440.run(Unknown Source)
[09:27:13] [Paper Watchdog Thread/ERROR]: 		java.base@11.0.7/java.lang.Thread.run(Thread.java:834)
[09:27:13] [Paper Watchdog Thread/ERROR]: ------------------------------
[09:27:13] [Paper Watchdog Thread/ERROR]: --- DO NOT REPORT THIS TO PAPER - THIS IS NOT A BUG OR A CRASH ---
[09:27:13] [Paper Watchdog Thread/ERROR]: ------------------------------
[09:27:18] [Paper Watchdog Thread/ERROR]: --- DO NOT REPORT THIS TO PAPER - THIS IS NOT A BUG OR A CRASH  - git-Paper-133 (MC: 1.16.1) ---
[09:27:18] [Paper Watchdog Thread/ERROR]: The server has not responded for 15 seconds! Creating thread dump
[09:27:18] [Paper Watchdog Thread/ERROR]: ------------------------------
[09:27:18] [Paper Watchdog Thread/ERROR]: Server thread dump (Look for plugins here before reporting to Paper!):
[09:27:18] [Paper Watchdog Thread/ERROR]: ------------------------------
[09:27:18] [Paper Watchdog Thread/ERROR]: Current Thread: Server thread
[09:27:18] [Paper Watchdog Thread/ERROR]: 	PID: 22 | Suspended: false | Native: false | State: TIMED_WAITING
[09:27:18] [Paper Watchdog Thread/ERROR]: 	Stack:
[09:27:18] [Paper Watchdog Thread/ERROR]: 		java.base@11.0.7/java.lang.Thread.sleep(Native Method)
[09:27:18] [Paper Watchdog Thread/ERROR]: 		me.SuperRonanCraft.BetterRTP.player.RTPTeleport.loadChunks(RTPTeleport.java:55)
[09:27:18] [Paper Watchdog Thread/ERROR]: 		me.SuperRonanCraft.BetterRTP.player.RTPTeleport.sendPlayer(RTPTeleport.java:21)
[09:27:18] [Paper Watchdog Thread/ERROR]: 		me.SuperRonanCraft.BetterRTP.player.RTP.tp(RTP.java:166)
[09:27:18] [Paper Watchdog Thread/ERROR]: 		me.SuperRonanCraft.BetterRTP.player.Delay.lambda$run$0(Delay.java:93)
[09:27:18] [Paper Watchdog Thread/ERROR]: 		me.SuperRonanCraft.BetterRTP.player.Delay$$Lambda$6385/0x0000000841a1dc40.run(Unknown Source)
[09:27:18] [Paper Watchdog Thread/ERROR]: 		app//org.bukkit.craftbukkit.v1_16_R1.scheduler.CraftTask.run(CraftTask.java:99)
[09:27:18] [Paper Watchdog Thread/ERROR]: 		app//org.bukkit.craftbukkit.v1_16_R1.scheduler.CraftScheduler.mainThreadHeartbeat(CraftScheduler.java:468)
[09:27:18] [Paper Watchdog Thread/ERROR]: 		app//net.minecraft.server.v1_16_R1.MinecraftServer.b(MinecraftServer.java:1292)
[09:27:18] [Paper Watchdog Thread/ERROR]: 		app//net.minecraft.server.v1_16_R1.DedicatedServer.b(DedicatedServer.java:377)
[09:27:18] [Paper Watchdog Thread/ERROR]: 		app//net.minecraft.server.v1_16_R1.MinecraftServer.a(MinecraftServer.java:1213)
[09:27:18] [Paper Watchdog Thread/ERROR]: 		app//net.minecraft.server.v1_16_R1.MinecraftServer.v(MinecraftServer.java:1001)
[09:27:18] [Paper Watchdog Thread/ERROR]: 		app//net.minecraft.server.v1_16_R1.MinecraftServer.lambda$a$0(MinecraftServer.java:177)
[09:27:18] [Paper Watchdog Thread/ERROR]: 		app//net.minecraft.server.v1_16_R1.MinecraftServer$$Lambda$3005/0x000000084082c440.run(Unknown Source)
[09:27:18] [Paper Watchdog Thread/ERROR]: 		java.base@11.0.7/java.lang.Thread.run(Thread.java:834)
[09:27:18] [Paper Watchdog Thread/ERROR]: ------------------------------
[09:27:18] [Paper Watchdog Thread/ERROR]: --- DO NOT REPORT THIS TO PAPER - THIS IS NOT A BUG OR A CRASH ---
[09:27:18] [Paper Watchdog Thread/ERROR]: ------------------------------

I wanted to report this issue, too. I think the cause of this is the Thread.sleep() method being called in the main thread, see the code.

Yeah, I was made aware of this yesterday. It has to do that Delays calling the rtp code on the main thread, unlike when you bypass them it runs async. I will bw getting a hot fix for this today!

Just posted an update, let me know if you have any other issues!