Remove csi-socket

Driver uses var/lib/kubelet/csi-plugins/. This PR has below changes

Move from var/lib/kubelet/csi-plugins/ to var/lib/kubelet/plugins/
Cleanup socket during uninstallation

coverage: 81.999% (-1.1%) from 83.118%
when pulling 36fb9761127a38e836799d8a609c6a12292aa3fc on ambiknai:mastercleanup
into f9246641dbf3d97b618b43968f7390b907c1262a on kubernetes-sigs:master.

upgrade test results
ambikanair@Ambikas-MacBook-Pro plugins % kubectl create -f 5.2.yaml
configmap/ibm-vpc-block-csi-configmap created
configmap/addon-vpc-block-csi-driver-configmap created
serviceaccount/ibm-vpc-block-node-sa created
clusterrole.rbac.authorization.k8s.io/vpc-block-driver-snapshotter-role created
clusterrolebinding.rbac.authorization.k8s.io/vpc-block-driver-snapshotter-binding created
clusterrole.rbac.authorization.k8s.io/vpc-block-driver-registrar-role created
clusterrolebinding.rbac.authorization.k8s.io/vpc-block-driver-registrar-binding created
clusterrole.rbac.authorization.k8s.io/vpc-block-external-resizer-role created
clusterrolebinding.rbac.authorization.k8s.io/vpc-block-external-resizer-binding created
serviceaccount/ibm-vpc-block-controller-sa created
clusterrole.rbac.authorization.k8s.io/vpc-block-provisioner-role created
clusterrolebinding.rbac.authorization.k8s.io/vpc-block-provisioner-binding created
clusterrole.rbac.authorization.k8s.io/vpc-block-external-attacher-role created
clusterrolebinding.rbac.authorization.k8s.io/vpc-block-external-attacher-binding created
csidriver.storage.k8s.io/vpc.block.csi.ibm.io created
daemonset.apps/ibm-vpc-block-csi-node created
deployment.apps/ibm-vpc-block-csi-controller created
storageclass.storage.k8s.io/ibmc-vpc-block-10iops-tier created
storageclass.storage.k8s.io/ibmc-vpc-block-5iops-tier created
storageclass.storage.k8s.io/ibmc-vpc-block-custom created
storageclass.storage.k8s.io/ibmc-vpc-block-general-purpose created
storageclass.storage.k8s.io/ibmc-vpc-block-retain-10iops-tier created
storageclass.storage.k8s.io/ibmc-vpc-block-retain-5iops-tier created
storageclass.storage.k8s.io/ibmc-vpc-block-retain-custom created
storageclass.storage.k8s.io/ibmc-vpc-block-retain-general-purpose created
storageclass.storage.k8s.io/ibmc-vpc-block-metro-10iops-tier created
storageclass.storage.k8s.io/ibmc-vpc-block-metro-5iops-tier created
storageclass.storage.k8s.io/ibmc-vpc-block-metro-custom created
storageclass.storage.k8s.io/ibmc-vpc-block-metro-general-purpose created
storageclass.storage.k8s.io/ibmc-vpc-block-metro-retain-10iops-tier created
storageclass.storage.k8s.io/ibmc-vpc-block-metro-retain-5iops-tier created
storageclass.storage.k8s.io/ibmc-vpc-block-metro-retain-custom created
storageclass.storage.k8s.io/ibmc-vpc-block-metro-retain-general-purpose created
storageclass.storage.k8s.io/ibmc-vpcblock-odf-10iops created
storageclass.storage.k8s.io/ibmc-vpcblock-odf-5iops created
storageclass.storage.k8s.io/ibmc-vpcblock-odf-custom created
storageclass.storage.k8s.io/ibmc-vpcblock-odf-general created
storageclass.storage.k8s.io/ibmc-vpcblock-odf-ret-10iops created
storageclass.storage.k8s.io/ibmc-vpcblock-odf-ret-5iops created
storageclass.storage.k8s.io/ibmc-vpcblock-odf-ret-custom created
storageclass.storage.k8s.io/ibmc-vpcblock-odf-ret-general created
volumesnapshotclass.snapshot.storage.k8s.io/ibmc-vpcblock-snapshot-delete created
volumesnapshotclass.snapshot.storage.k8s.io/ibmc-vpcblock-snapshot-retain created
ambikanair@Ambikas-MacBook-Pro plugins %
ambikanair@Ambikas-MacBook-Pro plugins %
ambikanair@Ambikas-MacBook-Pro plugins % kubectl get pods -n kube-system | grep block
ibm-vpc-block-csi-controller-7d8446bd65-w4h2d         7/7     Running   0             15s
ibm-vpc-block-csi-node-8f2wb                          4/4     Running   0             15s
ibm-vpc-block-csi-node-8mwnv                          4/4     Running   0             15s
ibm-vpc-block-csi-node-rrh29                          4/4     Running   0             15s
ambikanair@Ambikas-MacBook-Pro plugins % kubectl describe pod -n kube-system ibm-vpc-block-csi-node-8mwnv
Name:                 ibm-vpc-block-csi-node-8mwnv
Namespace:            kube-system
Priority:<PHONE_NUMBER>
Priority Class Name:  system-node-critical
Service Account:      ibm-vpc-block-node-sa
Node:                 <IP_ADDRESS>/<IP_ADDRESS>
Start Time:           Tue, 14 Nov 2023 11:22:27 +0530
Labels:               app=ibm-vpc-block-csi-driver
                      controller-revision-hash=6c65546647
                      pod-template-generation=1
Annotations:          cni.projectcalico.org/containerID: f8b0d701e1729dc46c709bc67afb561d34919da9c48662f156470a2bfca84940
                      cni.projectcalico.org/podIP: <IP_ADDRESS>/32
                      cni.projectcalico.org/podIPs: <IP_ADDRESS>/32
Status:               Running
IP:                   <IP_ADDRESS>
IPs:
  IP:           <IP_ADDRESS>
Controlled By:  DaemonSet/ibm-vpc-block-csi-node
Containers:
  csi-driver-registrar:
    Container ID:  containerd://38f684fa53c37c89a3b02c7c0466c6fff15b697f923ca39607300a62944b70b8
    Image:         icr.io/ext/sig-storage/csi-node-driver-registrar:v2.7.0
    Image ID:      icr.io/ext/sig-storage/csi-node-driver-registrar@sha256:5a35d65eea1dc2b8493aacd6276db1cea8b717933b40cfe970b0f2528475871d
    Port:          <none>
    Host Port:     <none>
    Args:
      --v=5
      --csi-address=$(ADDRESS)
      --kubelet-registration-path=$(DRIVER_REGISTRATION_SOCK)
    State:          Running
      Started:      Tue, 14 Nov 2023 11:22:30 +0530
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     40m
      memory:  80Mi
    Requests:
      cpu:     10m
      memory:  20Mi
    Environment:
      ADDRESS:                   /csi/csi.sock
      DRIVER_REGISTRATION_SOCK:  /var/lib/kubelet/csi-plugins/vpc.block.csi.ibm.io/csi.sock
      KUBE_NODE_NAME:             (v1:spec.nodeName)
    Mounts:
      /csi from plugin-dir (rw)
      /registration from registration-dir (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-9jhzr (ro)
  iks-vpc-block-node-driver:
    Container ID:  containerd://7736915c085f9d5f23573f846f3d80dee6344e3bd0b060a92affacbceca3628e
    Image:         us.icr.io/armada-master/ibm-vpc-block-csi-driver:v5.2.10
    Image ID:      us.icr.io/armada-master/ibm-vpc-block-csi-driver@sha256:a1c29fd894cdc065c2693d9619eb6fa33ca3d6ac9c2cbc950a587fa8c316365c
    Port:          9808/TCP
    Host Port:     0/TCP
    Args:
      --v=5
      --endpoint=unix:/csi/csi.sock
      --sidecarEndpoint=$(SIDECAREP)
    State:          Running
      Started:      Tue, 14 Nov 2023 11:22:30 +0530
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     120m
      memory:  300Mi
    Requests:
      cpu:     30m
      memory:  75Mi
    Liveness:  http-get http://:healthz/healthz delay=10s timeout=3s period=10s #success=1 #failure=5
    Environment Variables from:
      ibm-vpc-block-csi-configmap  ConfigMap  Optional: false
    Environment:
      SIDECAREP:                /sidecardir/provider.sock
      KUBE_NODE_NAME:            (v1:spec.nodeName)
      VOLUME_ATTACHMENT_LIMIT:  12
    Mounts:
      /csi from plugin-dir (rw)
      /dev from device-dir (rw)
      /etc/storage_ibmc from customer-auth (ro)
      /etc/storage_ibmc/cluster_info from cluster-info (ro)
      /etc/udev from etcudevpath (rw)
      /lib/udev from libudevpath (rw)
      /run/udev from runudevpath (rw)
      /sidecardir from secret-sidecar-sock-dir (rw)
      /sys from syspath (rw)
      /var/data/kubelet from kubelet-data-dir (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-9jhzr (ro)
  liveness-probe:
    Container ID:  containerd://53a8dfddc72fc2a4b13dae2eaaa301c913d6cd0bc8e2dd811781701b6f320cb8
    Image:         icr.io/ext/sig-storage/livenessprobe:v2.9.0
    Image ID:      icr.io/ext/sig-storage/livenessprobe@sha256:9b351e7c84b9d6f05fbe66b793fce80e464663f9f6a160b5818fa52e32ffbf02
    Port:          <none>
    Host Port:     <none>
    Args:
      --csi-address=/csi/csi.sock
    State:          Running
      Started:      Tue, 14 Nov 2023 11:22:31 +0530
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     20m
      memory:  40Mi
    Requests:
      cpu:        5m
      memory:     10Mi
    Environment:  <none>
    Mounts:
      /csi from plugin-dir (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-9jhzr (ro)
  storage-secret-sidecar:
    Container ID:  containerd://eb02b6780630ae2ec57a84a0fdef0ce0dec40ab16fe1dc8a17c11318911c02a5
    Image:         icr.io/obs/armada-storage-secret:v1.3.2
    Image ID:      icr.io/obs/armada-storage-secret@sha256:f7132d5ec0823aae909563fc61612afc179c74ffe969ee652d9381c6069d39cf
    Port:          <none>
    Host Port:     <none>
    Args:
      --endpoint=$(ENDPOINT)
    State:          Running
      Started:      Tue, 14 Nov 2023 11:22:32 +0530
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     40m
      memory:  80Mi
    Requests:
      cpu:     10m
      memory:  20Mi
    Environment:
      ENDPOINT:           unix:/sidecardir/provider.sock
      TOKEN_EXPIRY_DIFF:  20m
      PROFILE_CAPACITY:   1
    Mounts:
      /sidecardir from secret-sidecar-sock-dir (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-9jhzr (ro)
      /var/run/secrets/tokens from vault-token (rw)
Conditions:
  Type              Status
  Initialized       True
  Ready             True
  ContainersReady   True
  PodScheduled      True
Volumes:
  vault-token:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  600
  secret-sidecar-sock-dir:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:
    SizeLimit:  <unset>
  registration-dir:
    Type:          HostPath (bare host directory volume)
    Path:          /var/lib/kubelet/plugins_registry/
    HostPathType:  Directory
  kubelet-data-dir:
    Type:          HostPath (bare host directory volume)
    Path:          /var/data/kubelet
    HostPathType:  Directory
  plugin-dir:
    Type:          HostPath (bare host directory volume)
    Path:          /var/lib/kubelet/csi-plugins/vpc.block.csi.ibm.io/
    HostPathType:  DirectoryOrCreate
  device-dir:
    Type:          HostPath (bare host directory volume)
    Path:          /dev
    HostPathType:  Directory
  etcudevpath:
    Type:          HostPath (bare host directory volume)
    Path:          /etc/udev
    HostPathType:  Directory
  runudevpath:
    Type:          HostPath (bare host directory volume)
    Path:          /run/udev
    HostPathType:  Directory
  libudevpath:
    Type:          HostPath (bare host directory volume)
    Path:          /lib/udev
    HostPathType:  Directory
  syspath:
    Type:          HostPath (bare host directory volume)
    Path:          /sys
    HostPathType:  Directory
  customer-auth:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  storage-secret-store
    Optional:    false
  cluster-info:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      cluster-info
    Optional:  false
  kube-api-access-9jhzr:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   Burstable
Node-Selectors:              <none>
Tolerations:                 op=Exists
                             node.kubernetes.io/disk-pressure:NoSchedule op=Exists
                             node.kubernetes.io/memory-pressure:NoSchedule op=Exists
                             node.kubernetes.io/not-ready:NoExecute op=Exists
                             node.kubernetes.io/pid-pressure:NoSchedule op=Exists
                             node.kubernetes.io/unreachable:NoExecute op=Exists
                             node.kubernetes.io/unschedulable:NoSchedule op=Exists
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  26s   default-scheduler  Successfully assigned kube-system/ibm-vpc-block-csi-node-8mwnv to <IP_ADDRESS>
  Normal  Pulling    25s   kubelet            Pulling image "icr.io/ext/sig-storage/csi-node-driver-registrar:v2.7.0"
  Normal  Pulled     24s   kubelet            Successfully pulled image "icr.io/ext/sig-storage/csi-node-driver-registrar:v2.7.0" in 1.151652304s (1.151680525s including waiting)
  Normal  Created    24s   kubelet            Created container csi-driver-registrar
  Normal  Started    23s   kubelet            Started container csi-driver-registrar
  Normal  Pulling    23s   kubelet            Pulling image "us.icr.io/armada-master/ibm-vpc-block-csi-driver:v5.2.10"
  Normal  Pulled     23s   kubelet            Successfully pulled image "us.icr.io/armada-master/ibm-vpc-block-csi-driver:v5.2.10" in 651.636435ms (651.659464ms including waiting)
  Normal  Created    23s   kubelet            Created container iks-vpc-block-node-driver
  Normal  Started    23s   kubelet            Started container iks-vpc-block-node-driver
  Normal  Pulled     23s   kubelet            Container image "icr.io/ext/sig-storage/livenessprobe:v2.9.0" already present on machine
  Normal  Created    23s   kubelet            Created container liveness-probe
  Normal  Started    22s   kubelet            Started container liveness-probe
  Normal  Pulling    22s   kubelet            Pulling image "icr.io/obs/armada-storage-secret:v1.3.2"
  Normal  Pulled     21s   kubelet            Successfully pulled image "icr.io/obs/armada-storage-secret:v1.3.2" in 1.193123756s (1.193146654s including waiting)
  Normal  Created    21s   kubelet            Created container storage-secret-sidecar
  Normal  Started    21s   kubelet            Started container storage-secret-sidecar
ambikanair@Ambikas-MacBook-Pro plugins %
ambikanair@Ambikas-MacBook-Pro plugins %
ambikanair@Ambikas-MacBook-Pro plugins % kubectl get pvc
NAME                          STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS                AGE
csi-block-pvc-custom-verify   Bound    pvc-557c814e-ad09-4353-bb9d-d2c262b5132c   20Gi       RWO            ibmc-vpc-block-5iops-tier   20h
ambikanair@Ambikas-MacBook-Pro plugins % kubectl get pods
NAME                                     READY   STATUS    RESTARTS      AGE
cos-s3-csi-controller-7c598bc857-qntnk   3/3     Running   0             4d21h
cos-s3-csi-driver-45d52                  3/3     Running   0             4d21h
cos-s3-csi-driver-94mmp                  3/3     Running   0             4d21h
cos-s3-csi-driver-zdngq                  3/3     Running   3 (12h ago)   4d21h
testcustom-6f55b7688d-z555n              1/1     Running   0             20h
ambikanair@Ambikas-MacBook-Pro plugins % kubectl delete deployment testcustom-
Error from server (NotFound): deployments.apps "testcustom-" not found
ambikanair@Ambikas-MacBook-Pro plugins %
ambikanair@Ambikas-MacBook-Pro plugins % kubectl delete deployment testcustom
deployment.apps "testcustom" deleted
ambikanair@Ambikas-MacBook-Pro plugins % kubectl delete pvc csi-block-pvc-custom-verify
persistentvolumeclaim "csi-block-pvc-custom-verify" deleted
ambikanair@Ambikas-MacBook-Pro plugins %
ambikanair@Ambikas-MacBook-Pro plugins %
ambikanair@Ambikas-MacBook-Pro plugins % kubectl create -f ~/snapshotroks/demo/pvc.yaml
persistentvolumeclaim/csi-block-pvc-custom-verify created
ambikanair@Ambikas-MacBook-Pro plugins % kubectl get pvc
NAME                          STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS                AGE
csi-block-pvc-custom-verify   Bound    pvc-0f1c8334-fc58-47f1-a18c-fd39559ee713   20Gi       RWO            ibmc-vpc-block-5iops-tier   22s
ambikanair@Ambikas-MacBook-Pro plugins % kubectl apply -f 5.2.codecleanup.yaml
Warning: resource configmaps/ibm-vpc-block-csi-configmap is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
configmap/ibm-vpc-block-csi-configmap configured
Warning: resource configmaps/addon-vpc-block-csi-driver-configmap is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
configmap/addon-vpc-block-csi-driver-configmap configured
Warning: resource serviceaccounts/ibm-vpc-block-node-sa is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
serviceaccount/ibm-vpc-block-node-sa configured
Warning: resource clusterroles/vpc-block-driver-snapshotter-role is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
clusterrole.rbac.authorization.k8s.io/vpc-block-driver-snapshotter-role configured
Warning: resource clusterrolebindings/vpc-block-driver-snapshotter-binding is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
clusterrolebinding.rbac.authorization.k8s.io/vpc-block-driver-snapshotter-binding configured
Warning: resource clusterroles/vpc-block-driver-registrar-role is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
clusterrole.rbac.authorization.k8s.io/vpc-block-driver-registrar-role configured
Warning: resource clusterrolebindings/vpc-block-driver-registrar-binding is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
clusterrolebinding.rbac.authorization.k8s.io/vpc-block-driver-registrar-binding configured
Warning: resource clusterroles/vpc-block-external-resizer-role is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
clusterrole.rbac.authorization.k8s.io/vpc-block-external-resizer-role configured
Warning: resource clusterrolebindings/vpc-block-external-resizer-binding is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
clusterrolebinding.rbac.authorization.k8s.io/vpc-block-external-resizer-binding configured
Warning: resource serviceaccounts/ibm-vpc-block-controller-sa is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
serviceaccount/ibm-vpc-block-controller-sa configured
Warning: resource clusterroles/vpc-block-provisioner-role is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
clusterrole.rbac.authorization.k8s.io/vpc-block-provisioner-role configured
Warning: resource clusterrolebindings/vpc-block-provisioner-binding is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
clusterrolebinding.rbac.authorization.k8s.io/vpc-block-provisioner-binding configured
Warning: resource clusterroles/vpc-block-external-attacher-role is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
clusterrole.rbac.authorization.k8s.io/vpc-block-external-attacher-role configured
Warning: resource clusterrolebindings/vpc-block-external-attacher-binding is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
clusterrolebinding.rbac.authorization.k8s.io/vpc-block-external-attacher-binding configured
Warning: resource csidrivers/vpc.block.csi.ibm.io is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
csidriver.storage.k8s.io/vpc.block.csi.ibm.io configured
Warning: resource daemonsets/ibm-vpc-block-csi-node is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
daemonset.apps/ibm-vpc-block-csi-node configured
Warning: resource deployments/ibm-vpc-block-csi-controller is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
deployment.apps/ibm-vpc-block-csi-controller configured
Warning: resource storageclasses/ibmc-vpc-block-10iops-tier is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
storageclass.storage.k8s.io/ibmc-vpc-block-10iops-tier configured
Warning: resource storageclasses/ibmc-vpc-block-5iops-tier is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
storageclass.storage.k8s.io/ibmc-vpc-block-5iops-tier configured
Warning: resource storageclasses/ibmc-vpc-block-custom is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
storageclass.storage.k8s.io/ibmc-vpc-block-custom configured
Warning: resource storageclasses/ibmc-vpc-block-general-purpose is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
storageclass.storage.k8s.io/ibmc-vpc-block-general-purpose configured
Warning: resource storageclasses/ibmc-vpc-block-retain-10iops-tier is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
storageclass.storage.k8s.io/ibmc-vpc-block-retain-10iops-tier configured
Warning: resource storageclasses/ibmc-vpc-block-retain-5iops-tier is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
storageclass.storage.k8s.io/ibmc-vpc-block-retain-5iops-tier configured
Warning: resource storageclasses/ibmc-vpc-block-retain-custom is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
storageclass.storage.k8s.io/ibmc-vpc-block-retain-custom configured
Warning: resource storageclasses/ibmc-vpc-block-retain-general-purpose is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
storageclass.storage.k8s.io/ibmc-vpc-block-retain-general-purpose configured
Warning: resource storageclasses/ibmc-vpc-block-metro-10iops-tier is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
storageclass.storage.k8s.io/ibmc-vpc-block-metro-10iops-tier configured
Warning: resource storageclasses/ibmc-vpc-block-metro-5iops-tier is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
storageclass.storage.k8s.io/ibmc-vpc-block-metro-5iops-tier configured
Warning: resource storageclasses/ibmc-vpc-block-metro-custom is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
storageclass.storage.k8s.io/ibmc-vpc-block-metro-custom configured
Warning: resource storageclasses/ibmc-vpc-block-metro-general-purpose is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
storageclass.storage.k8s.io/ibmc-vpc-block-metro-general-purpose configured
Warning: resource storageclasses/ibmc-vpc-block-metro-retain-10iops-tier is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
storageclass.storage.k8s.io/ibmc-vpc-block-metro-retain-10iops-tier configured
Warning: resource storageclasses/ibmc-vpc-block-metro-retain-5iops-tier is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
storageclass.storage.k8s.io/ibmc-vpc-block-metro-retain-5iops-tier configured
Warning: resource storageclasses/ibmc-vpc-block-metro-retain-custom is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
storageclass.storage.k8s.io/ibmc-vpc-block-metro-retain-custom configured
Warning: resource storageclasses/ibmc-vpc-block-metro-retain-general-purpose is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
storageclass.storage.k8s.io/ibmc-vpc-block-metro-retain-general-purpose configured
Warning: resource storageclasses/ibmc-vpcblock-odf-10iops is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
storageclass.storage.k8s.io/ibmc-vpcblock-odf-10iops configured
Warning: resource storageclasses/ibmc-vpcblock-odf-5iops is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
storageclass.storage.k8s.io/ibmc-vpcblock-odf-5iops configured
Warning: resource storageclasses/ibmc-vpcblock-odf-custom is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
storageclass.storage.k8s.io/ibmc-vpcblock-odf-custom configured
Warning: resource storageclasses/ibmc-vpcblock-odf-general is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
storageclass.storage.k8s.io/ibmc-vpcblock-odf-general configured
Warning: resource storageclasses/ibmc-vpcblock-odf-ret-10iops is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
storageclass.storage.k8s.io/ibmc-vpcblock-odf-ret-10iops configured
Warning: resource storageclasses/ibmc-vpcblock-odf-ret-5iops is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
storageclass.storage.k8s.io/ibmc-vpcblock-odf-ret-5iops configured
Warning: resource storageclasses/ibmc-vpcblock-odf-ret-custom is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
storageclass.storage.k8s.io/ibmc-vpcblock-odf-ret-custom configured
Warning: resource storageclasses/ibmc-vpcblock-odf-ret-general is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
storageclass.storage.k8s.io/ibmc-vpcblock-odf-ret-general configured
Warning: resource volumesnapshotclasses/ibmc-vpcblock-snapshot-delete is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
volumesnapshotclass.snapshot.storage.k8s.io/ibmc-vpcblock-snapshot-delete configured
Warning: resource volumesnapshotclasses/ibmc-vpcblock-snapshot-retain is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
volumesnapshotclass.snapshot.storage.k8s.io/ibmc-vpcblock-snapshot-retain configured
ambikanair@Ambikas-MacBook-Pro plugins % kubectl get pods -n kube-system | grep block
ibm-vpc-block-csi-controller-7d8446bd65-w4h2d         7/7     Running             0             3m18s
ibm-vpc-block-csi-controller-dcbffcc55-2k6k8          0/7     ContainerCreating   0             13s
ibm-vpc-block-csi-node-8f2wb                          0/4     Terminating         0             3m18s
ibm-vpc-block-csi-node-8mwnv                          4/4     Running             0             3m18s
ibm-vpc-block-csi-node-kw8fj                          4/4     Running             0             11s
ambikanair@Ambikas-MacBook-Pro plugins % kubectl describe pod -n kube-system ibm-vpc-block-csi-node-kw8fj
Name:                 ibm-vpc-block-csi-node-kw8fj
Namespace:            kube-system
Priority:<PHONE_NUMBER>
Priority Class Name:  system-node-critical
Service Account:      ibm-vpc-block-node-sa
Node:                 <IP_ADDRESS>/<IP_ADDRESS>
Start Time:           Tue, 14 Nov 2023 11:25:34 +0530
Labels:               app=ibm-vpc-block-csi-driver
                      controller-revision-hash=5dc9d9b655
                      pod-template-generation=2
Annotations:          cni.projectcalico.org/containerID: 1344dfa2870def059019b373f474328e19a1a7daebe73ed469e62dc022b9d0a0
                      cni.projectcalico.org/podIP: <IP_ADDRESS>/32
                      cni.projectcalico.org/podIPs: <IP_ADDRESS>/32
Status:               Running
IP:                   <IP_ADDRESS>
IPs:
  IP:           <IP_ADDRESS>
Controlled By:  DaemonSet/ibm-vpc-block-csi-node
Containers:
  csi-driver-registrar:
    Container ID:  containerd://430e964b48e70bd88985bc58c626f06c87a0633c6cf31402d1199c11b1dbfda5
    Image:         icr.io/ext/sig-storage/csi-node-driver-registrar:v2.7.0
    Image ID:      icr.io/ext/sig-storage/csi-node-driver-registrar@sha256:5a35d65eea1dc2b8493aacd6276db1cea8b717933b40cfe970b0f2528475871d
    Port:          <none>
    Host Port:     <none>
    Args:
      --v=5
      --csi-address=$(ADDRESS)
      --kubelet-registration-path=$(DRIVER_REGISTRATION_SOCK)
    State:          Running
      Started:      Tue, 14 Nov 2023 11:25:36 +0530
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     40m
      memory:  80Mi
    Requests:
      cpu:     10m
      memory:  20Mi
    Environment:
      ADDRESS:                   /csi/csi.sock
      DRIVER_REGISTRATION_SOCK:  /var/lib/kubelet/plugins/vpc.block.csi.ibm.io/csi.sock
      KUBE_NODE_NAME:             (v1:spec.nodeName)
    Mounts:
      /csi from plugin-dir (rw)
      /registration from registration-dir (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-65gjz (ro)
  iks-vpc-block-node-driver:
    Container ID:  containerd://a960bf6733aa665c80eb4608bae381d424935c1f1b2951af19a6b7c80e40a0d3
    Image:         ambikanair/ibm-vpc-block-csi-driver:sockcleanup3
    Image ID:      docker.io/ambikanair/ibm-vpc-block-csi-driver@sha256:2d692eda0cece35b8a71a0521aeb1665dcf365abc1ecb4e6e33f78756f8cbb63
    Port:          9808/TCP
    Host Port:     0/TCP
    Args:
      --v=5
      --endpoint=unix:/csi/csi.sock
      --sidecarEndpoint=$(SIDECAREP)
    State:          Running
      Started:      Tue, 14 Nov 2023 11:25:42 +0530
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     120m
      memory:  300Mi
    Requests:
      cpu:     30m
      memory:  75Mi
    Liveness:  http-get http://:healthz/healthz delay=10s timeout=3s period=10s #success=1 #failure=5
    Environment Variables from:
      ibm-vpc-block-csi-configmap  ConfigMap  Optional: false
    Environment:
      SIDECAREP:                /sidecardir/provider.sock
      KUBE_NODE_NAME:            (v1:spec.nodeName)
      VOLUME_ATTACHMENT_LIMIT:  12
    Mounts:
      /csi from plugin-dir (rw)
      /dev from device-dir (rw)
      /etc/storage_ibmc from customer-auth (ro)
      /etc/storage_ibmc/cluster_info from cluster-info (ro)
      /etc/udev from etcudevpath (rw)
      /lib-kubelet from kubelet-lib-dir (rw)
      /lib/udev from libudevpath (rw)
      /run/udev from runudevpath (rw)
      /sidecardir from secret-sidecar-sock-dir (rw)
      /sys from syspath (rw)
      /var/data/kubelet from kubelet-data-dir (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-65gjz (ro)
  liveness-probe:
    Container ID:  containerd://574cac8897252cc1074824ddda489b9742a606382c2f21408d91fb035310499c
    Image:         icr.io/ext/sig-storage/livenessprobe:v2.9.0
    Image ID:      icr.io/ext/sig-storage/livenessprobe@sha256:9b351e7c84b9d6f05fbe66b793fce80e464663f9f6a160b5818fa52e32ffbf02
    Port:          <none>
    Host Port:     <none>
    Args:
      --csi-address=/csi/csi.sock
    State:          Running
      Started:      Tue, 14 Nov 2023 11:25:42 +0530
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     20m
      memory:  40Mi
    Requests:
      cpu:        5m
      memory:     10Mi
    Environment:  <none>
    Mounts:
      /csi from plugin-dir (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-65gjz (ro)
  storage-secret-sidecar:
    Container ID:  containerd://64c04e80c284383250e0c07a449f3103c14d5c8a14239ff09f88d1bb84f1e9dd
    Image:         icr.io/obs/armada-storage-secret:v1.3.2
    Image ID:      icr.io/obs/armada-storage-secret@sha256:f7132d5ec0823aae909563fc61612afc179c74ffe969ee652d9381c6069d39cf
    Port:          <none>
    Host Port:     <none>
    Args:
      --endpoint=$(ENDPOINT)
    State:          Running
      Started:      Tue, 14 Nov 2023 11:25:44 +0530
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     40m
      memory:  80Mi
    Requests:
      cpu:     10m
      memory:  20Mi
    Environment:
      ENDPOINT:           unix:/sidecardir/provider.sock
      TOKEN_EXPIRY_DIFF:  20m
      PROFILE_CAPACITY:   1
    Mounts:
      /sidecardir from secret-sidecar-sock-dir (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-65gjz (ro)
      /var/run/secrets/tokens from vault-token (rw)
Conditions:
  Type              Status
  Initialized       True
  Ready             True
  ContainersReady   True
  PodScheduled      True
Volumes:
  vault-token:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  600
  secret-sidecar-sock-dir:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:
    SizeLimit:  <unset>
  registration-dir:
    Type:          HostPath (bare host directory volume)
    Path:          /var/lib/kubelet/plugins_registry/
    HostPathType:  Directory
  kubelet-data-dir:
    Type:          HostPath (bare host directory volume)
    Path:          /var/data/kubelet
    HostPathType:  Directory
  plugin-dir:
    Type:          HostPath (bare host directory volume)
    Path:          /var/lib/kubelet/plugins/vpc.block.csi.ibm.io/
    HostPathType:  DirectoryOrCreate
  device-dir:
    Type:          HostPath (bare host directory volume)
    Path:          /dev
    HostPathType:  Directory
  etcudevpath:
    Type:          HostPath (bare host directory volume)
    Path:          /etc/udev
    HostPathType:  Directory
  runudevpath:
    Type:          HostPath (bare host directory volume)
    Path:          /run/udev
    HostPathType:  Directory
  libudevpath:
    Type:          HostPath (bare host directory volume)
    Path:          /lib/udev
    HostPathType:  Directory
  syspath:
    Type:          HostPath (bare host directory volume)
    Path:          /sys
    HostPathType:  Directory
  customer-auth:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  storage-secret-store
    Optional:    false
  cluster-info:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      cluster-info
    Optional:  false
  kubelet-lib-dir:
    Type:          HostPath (bare host directory volume)
    Path:          /var/lib/kubelet/
    HostPathType:  Directory
  kube-api-access-65gjz:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   Burstable
Node-Selectors:              <none>
Tolerations:                 op=Exists
                             node.kubernetes.io/disk-pressure:NoSchedule op=Exists
                             node.kubernetes.io/memory-pressure:NoSchedule op=Exists
                             node.kubernetes.io/not-ready:NoExecute op=Exists
                             node.kubernetes.io/pid-pressure:NoSchedule op=Exists
                             node.kubernetes.io/unreachable:NoExecute op=Exists
                             node.kubernetes.io/unschedulable:NoSchedule op=Exists
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  25s   default-scheduler  Successfully assigned kube-system/ibm-vpc-block-csi-node-kw8fj to <IP_ADDRESS>
  Normal  Pulling    25s   kubelet            Pulling image "icr.io/ext/sig-storage/csi-node-driver-registrar:v2.7.0"
  Normal  Pulled     23s   kubelet            Successfully pulled image "icr.io/ext/sig-storage/csi-node-driver-registrar:v2.7.0" in 1.224286442s (1.224300229s including waiting)
  Normal  Created    23s   kubelet            Created container csi-driver-registrar
  Normal  Started    23s   kubelet            Started container csi-driver-registrar
  Normal  Pulling    23s   kubelet            Pulling image "ambikanair/ibm-vpc-block-csi-driver:sockcleanup3"
  Normal  Pulled     17s   kubelet            Successfully pulled image "ambikanair/ibm-vpc-block-csi-driver:sockcleanup3" in 6.3136206s (6.313637511s including waiting)
  Normal  Created    17s   kubelet            Created container iks-vpc-block-node-driver
  Normal  Started    17s   kubelet            Started container iks-vpc-block-node-driver
  Normal  Pulled     17s   kubelet            Container image "icr.io/ext/sig-storage/livenessprobe:v2.9.0" already present on machine
  Normal  Created    17s   kubelet            Created container liveness-probe
  Normal  Started    17s   kubelet            Started container liveness-probe
  Normal  Pulling    17s   kubelet            Pulling image "icr.io/obs/armada-storage-secret:v1.3.2"
  Normal  Pulled     15s   kubelet            Successfully pulled image "icr.io/obs/armada-storage-secret:v1.3.2" in 1.190740262s (1.190751332s including waiting)
  Normal  Created    15s   kubelet            Created container storage-secret-sidecar
  Normal  Started    15s   kubelet            Started container storage-secret-sidecar
ambikanair@Ambikas-MacBook-Pro plugins % kubectl create -f ~/snapshotroks/demo/pod.yaml
deployment.apps/testcustom created
ambikanair@Ambikas-MacBook-Pro plugins % kubectl get pods
NAME                                     READY   STATUS              RESTARTS      AGE
cos-s3-csi-controller-7c598bc857-qntnk   3/3     Running             0             4d22h
cos-s3-csi-driver-45d52                  3/3     Running             0             4d22h
cos-s3-csi-driver-94mmp                  3/3     Running             0             4d22h
cos-s3-csi-driver-zdngq                  3/3     Running             3 (12h ago)   4d22h
testcustom-6f55b7688d-64qvd              0/1     ContainerCreating   0             9s
ambikanair@Ambikas-MacBook-Pro plugins % kubectl exec -it testcustom-6f55b7688d-64qvd /bin/bash
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
root@testcustom-6f55b7688d-64qvd:/# cd /myvolumepath/
root@testcustom-6f55b7688d-64qvd:/myvolumepath# echo "hi" > new.txt
root@testcustom-6f55b7688d-64qvd:/myvolumepath# exit
exit
ambikanair@Ambikas-MacBook-Pro plugins % kubectl delete pod testcustom-6f55b7688d-64qvd
pod "testcustom-6f55b7688d-64qvd" deleted
ambikanair@Ambikas-MacBook-Pro plugins % kubectl get pods
NAME                                     READY   STATUS    RESTARTS      AGE
cos-s3-csi-controller-7c598bc857-qntnk   3/3     Running   0             4d22h
cos-s3-csi-driver-45d52                  3/3     Running   0             4d22h
cos-s3-csi-driver-94mmp                  3/3     Running   0             4d22h
cos-s3-csi-driver-zdngq                  3/3     Running   3 (12h ago)   4d22h
testcustom-6f55b7688d-hbdfb              1/1     Running   0             50s
ambikanair@Ambikas-MacBook-Pro plugins % kubectl exec -it testcustom-6f55b7688d-hbdfb /bin/bash
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
root@testcustom-6f55b7688d-hbdfb:/# ls /myvolumepath/
lost+found  new.txt
root@testcustom-6f55b7688d-hbdfb:/# exit
exit
ambikanair@Ambikas-MacBook-Pro plugins % kubectl logs ibm-vpc-block-csi-node-kw8fj -n kube-system -c iks-vpc-block-node-driver -f
{"level":"info","timestamp":"2023-11-14T05:55:43.067Z","caller":"cmd/main.go:84","msg":"IBM CSI driver version","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver","DriverVersion":"vpcBlockDriver-"}
{"level":"info","timestamp":"2023-11-14T05:55:43.067Z","caller":"cmd/main.go:85","msg":"Controller Mutex Lock enabled","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver","LockEnabled":true}
{"level":"info","timestamp":"2023-11-14T05:55:43.068Z","caller":"ibmcloudprovider/volume_provider.go:50","msg":"NewIBMCloudStorageProvider-Reading provider configuration...","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver"}
{"level":"warn","timestamp":"2023-11-14T05:55:43.121Z","caller":"ibmcloudprovider/volume_provider.go:63","msg":"Failed to parse VPC_API_VERSION, setting default value","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver"}
{"level":"info","timestamp":"2023-11-14T05:55:43.122Z","caller":"ibmcloudprovider/volume_provider.go:68","msg":"Fetching clusterInfo","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver"}
{"level":"info","timestamp":"2023-11-14T05:55:43.139Z","caller":"ibmcloudprovider/volume_provider.go:75","msg":"Fetched clusterInfo..","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver"}
{"level":"info","timestamp":"2023-11-14T05:55:43.140Z","caller":"utils/init_provider.go:42","msg":"Configuring VPC Block Provider","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver"}
{"level":"info","timestamp":"2023-11-14T05:55:43.140Z","caller":"provider/provider.go:76","msg":"Entering NewProvider","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver"}
{"level":"info","timestamp":"2023-11-14T05:55:43.140Z","caller":"secret_provider/managed_secret_provider.go:54","msg":"Connecting to sidecar","name":"secret-provider","secret-provider-type":"managed-secret-provider"}
{"level":"info","timestamp":"2023-11-14T05:55:46.207Z","caller":"secret_provider/managed_secret_provider.go:92","msg":"Unable to fetch endpoints from cloud-conf","name":"secret-provider","secret-provider-type":"managed-secret-provider","error":"configmaps \"cloud-conf\" not found"}
{"level":"info","timestamp":"2023-11-14T05:55:46.217Z","caller":"secret_provider/managed_secret_provider.go:99","msg":"Initialized managed secret provider","name":"secret-provider","secret-provider-type":"managed-secret-provider"}
{"level":"info","timestamp":"2023-11-14T05:55:46.218Z","caller":"secret_provider/managed_secret_provider.go:54","msg":"Connecting to sidecar","name":"secret-provider","secret-provider-type":"managed-secret-provider"}
{"level":"info","timestamp":"2023-11-14T05:55:46.257Z","caller":"secret_provider/managed_secret_provider.go:92","msg":"Unable to fetch endpoints from cloud-conf","name":"secret-provider","secret-provider-type":"managed-secret-provider","error":"configmaps \"cloud-conf\" not found"}
{"level":"info","timestamp":"2023-11-14T05:55:46.274Z","caller":"secret_provider/managed_secret_provider.go:99","msg":"Initialized managed secret provider","name":"secret-provider","secret-provider-type":"managed-secret-provider"}
{"level":"info","timestamp":"2023-11-14T05:55:46.274Z","caller":"utils/init_provider.go:54","msg":"Configuring IKS-VPC Block Provider","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver"}
{"level":"info","timestamp":"2023-11-14T05:55:46.274Z","caller":"provider/provider.go:76","msg":"Entering NewProvider","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver"}
{"level":"info","timestamp":"2023-11-14T05:55:46.274Z","caller":"secret_provider/managed_secret_provider.go:54","msg":"Connecting to sidecar","name":"secret-provider","secret-provider-type":"managed-secret-provider"}
{"level":"info","timestamp":"2023-11-14T05:55:46.386Z","caller":"secret_provider/managed_secret_provider.go:92","msg":"Unable to fetch endpoints from cloud-conf","name":"secret-provider","secret-provider-type":"managed-secret-provider","error":"configmaps \"cloud-conf\" not found"}
{"level":"info","timestamp":"2023-11-14T05:55:46.397Z","caller":"secret_provider/managed_secret_provider.go:99","msg":"Initialized managed secret provider","name":"secret-provider","secret-provider-type":"managed-secret-provider"}
{"level":"info","timestamp":"2023-11-14T05:55:46.397Z","caller":"secret_provider/managed_secret_provider.go:54","msg":"Connecting to sidecar","name":"secret-provider","secret-provider-type":"managed-secret-provider"}
{"level":"info","timestamp":"2023-11-14T05:55:46.417Z","caller":"secret_provider/managed_secret_provider.go:92","msg":"Unable to fetch endpoints from cloud-conf","name":"secret-provider","secret-provider-type":"managed-secret-provider","error":"configmaps \"cloud-conf\" not found"}
{"level":"info","timestamp":"2023-11-14T05:55:46.429Z","caller":"secret_provider/managed_secret_provider.go:99","msg":"Initialized managed secret provider","name":"secret-provider","secret-provider-type":"managed-secret-provider"}
{"level":"info","timestamp":"2023-11-14T05:55:46.429Z","caller":"provider/provider.go:76","msg":"Entering NewProvider","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver"}
{"level":"info","timestamp":"2023-11-14T05:55:46.429Z","caller":"secret_provider/managed_secret_provider.go:54","msg":"Connecting to sidecar","name":"secret-provider","secret-provider-type":"managed-secret-provider"}
{"level":"info","timestamp":"2023-11-14T05:55:46.457Z","caller":"secret_provider/managed_secret_provider.go:92","msg":"Unable to fetch endpoints from cloud-conf","name":"secret-provider","secret-provider-type":"managed-secret-provider","error":"configmaps \"cloud-conf\" not found"}
{"level":"info","timestamp":"2023-11-14T05:55:46.469Z","caller":"secret_provider/managed_secret_provider.go:99","msg":"Initialized managed secret provider","name":"secret-provider","secret-provider-type":"managed-secret-provider"}
{"level":"info","timestamp":"2023-11-14T05:55:46.469Z","caller":"secret_provider/managed_secret_provider.go:54","msg":"Connecting to sidecar","name":"secret-provider","secret-provider-type":"managed-secret-provider"}
{"level":"info","timestamp":"2023-11-14T05:55:46.486Z","caller":"secret_provider/managed_secret_provider.go:92","msg":"Unable to fetch endpoints from cloud-conf","name":"secret-provider","secret-provider-type":"managed-secret-provider","error":"configmaps \"cloud-conf\" not found"}
{"level":"info","timestamp":"2023-11-14T05:55:46.498Z","caller":"secret_provider/managed_secret_provider.go:99","msg":"Initialized managed secret provider","name":"secret-provider","secret-provider-type":"managed-secret-provider"}
{"level":"info","timestamp":"2023-11-14T05:55:46.498Z","caller":"secret_provider/managed_secret_provider.go:54","msg":"Connecting to sidecar","name":"secret-provider","secret-provider-type":"managed-secret-provider"}
{"level":"info","timestamp":"2023-11-14T05:55:46.588Z","caller":"secret_provider/managed_secret_provider.go:92","msg":"Unable to fetch endpoints from cloud-conf","name":"secret-provider","secret-provider-type":"managed-secret-provider","error":"configmaps \"cloud-conf\" not found"}
{"level":"info","timestamp":"2023-11-14T05:55:46.601Z","caller":"secret_provider/managed_secret_provider.go:99","msg":"Initialized managed secret provider","name":"secret-provider","secret-provider-type":"managed-secret-provider"}
{"level":"info","timestamp":"2023-11-14T05:55:46.601Z","caller":"secret_provider/managed_secret_provider.go:54","msg":"Connecting to sidecar","name":"secret-provider","secret-provider-type":"managed-secret-provider"}
{"level":"info","timestamp":"2023-11-14T05:55:46.616Z","caller":"secret_provider/managed_secret_provider.go:92","msg":"Unable to fetch endpoints from cloud-conf","name":"secret-provider","secret-provider-type":"managed-secret-provider","error":"configmaps \"cloud-conf\" not found"}
{"level":"info","timestamp":"2023-11-14T05:55:46.628Z","caller":"secret_provider/managed_secret_provider.go:99","msg":"Initialized managed secret provider","name":"secret-provider","secret-provider-type":"managed-secret-provider"}
{"level":"info","timestamp":"2023-11-14T05:55:46.628Z","caller":"utils/init_provider.go:65","msg":"Provider registration done!!!","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver"}
{"level":"info","timestamp":"2023-11-14T05:55:46.628Z","caller":"ibmcloudprovider/volume_provider.go:107","msg":"Successfully read provider configuration","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver"}
{"level":"info","timestamp":"2023-11-14T05:55:46.648Z","caller":"ibmcsidriver/ibm_csi_driver.go:55","msg":"IBMCSIDriver-SetupIBMCSIDriver setting up IBM CSI Driver...","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver"}
{"level":"info","timestamp":"2023-11-14T05:55:46.648Z","caller":"ibmcsidriver/ibm_csi_driver.go:120","msg":"IBMCSIDriver-AddVolumeCapabilityAccessModes...","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver","VolumeCapabilityAccessModes":[1]}
{"level":"info","timestamp":"2023-11-14T05:55:46.648Z","caller":"ibmcsidriver/ibm_csi_driver.go:123","msg":"Enabling volume access mode","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver","Mode":"SINGLE_NODE_WRITER"}
{"level":"info","timestamp":"2023-11-14T05:55:46.648Z","caller":"ibmcsidriver/ibm_csi_driver.go:127","msg":"Successfully enabled Volume Capability Access Modes","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver"}
{"level":"info","timestamp":"2023-11-14T05:55:46.648Z","caller":"ibmcsidriver/ibm_csi_driver.go:133","msg":"IBMCSIDriver-AddControllerServiceCapabilities...","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver","ControllerServiceCapabilities":[1,2,3,5,6,9]}
{"level":"info","timestamp":"2023-11-14T05:55:46.648Z","caller":"ibmcsidriver/ibm_csi_driver.go:136","msg":"Adding controller service capability","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver","Capability":"CREATE_DELETE_VOLUME"}
{"level":"info","timestamp":"2023-11-14T05:55:46.648Z","caller":"ibmcsidriver/ibm_csi_driver.go:136","msg":"Adding controller service capability","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver","Capability":"PUBLISH_UNPUBLISH_VOLUME"}
{"level":"info","timestamp":"2023-11-14T05:55:46.648Z","caller":"ibmcsidriver/ibm_csi_driver.go:136","msg":"Adding controller service capability","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver","Capability":"LIST_VOLUMES"}
{"level":"info","timestamp":"2023-11-14T05:55:46.649Z","caller":"ibmcsidriver/ibm_csi_driver.go:136","msg":"Adding controller service capability","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver","Capability":"CREATE_DELETE_SNAPSHOT"}
{"level":"info","timestamp":"2023-11-14T05:55:46.649Z","caller":"ibmcsidriver/ibm_csi_driver.go:136","msg":"Adding controller service capability","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver","Capability":"LIST_SNAPSHOTS"}
{"level":"info","timestamp":"2023-11-14T05:55:46.649Z","caller":"ibmcsidriver/ibm_csi_driver.go:136","msg":"Adding controller service capability","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver","Capability":"EXPAND_VOLUME"}
{"level":"info","timestamp":"2023-11-14T05:55:46.649Z","caller":"ibmcsidriver/ibm_csi_driver.go:140","msg":"Successfully added Controller Service Capabilities","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver"}
{"level":"info","timestamp":"2023-11-14T05:55:46.649Z","caller":"ibmcsidriver/ibm_csi_driver.go:146","msg":"IBMCSIDriver-AddNodeServiceCapabilities...","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver","NodeServiceCapabilities":[1,2,3]}
{"level":"info","timestamp":"2023-11-14T05:55:46.649Z","caller":"ibmcsidriver/ibm_csi_driver.go:149","msg":"Adding node service capability","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver","NodeServiceCapabilities":"STAGE_UNSTAGE_VOLUME"}
{"level":"info","timestamp":"2023-11-14T05:55:46.649Z","caller":"ibmcsidriver/ibm_csi_driver.go:149","msg":"Adding node service capability","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver","NodeServiceCapabilities":"GET_VOLUME_STATS"}
{"level":"info","timestamp":"2023-11-14T05:55:46.649Z","caller":"ibmcsidriver/ibm_csi_driver.go:149","msg":"Adding node service capability","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver","NodeServiceCapabilities":"EXPAND_VOLUME"}
{"level":"info","timestamp":"2023-11-14T05:55:46.649Z","caller":"ibmcsidriver/ibm_csi_driver.go:153","msg":"Successfully added Node Service Capabilities","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver"}
{"level":"info","timestamp":"2023-11-14T05:55:46.649Z","caller":"ibmcsidriver/ibm_csi_driver.go:106","msg":"Successfully setup IBM CSI driver","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver"}
{"level":"info","timestamp":"2023-11-14T05:55:46.663Z","caller":"cmd/main.go:116","msg":"Successfully initialized driver...","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver"}
{"level":"info","timestamp":"2023-11-14T05:55:46.663Z","caller":"cmd/main.go:128","msg":"Starting metrics endpoint","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver"}
{"level":"info","timestamp":"2023-11-14T05:55:46.663Z","caller":"ibmcsidriver/ibm_csi_driver.go:200","msg":"IBMCSIDriver-Run...","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver","Endpoint":"unix:/csi/csi.sock"}
{"level":"info","timestamp":"2023-11-14T05:55:46.663Z","caller":"ibmcsidriver/ibm_csi_driver.go:201","msg":"CSI Driver Name","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver","Name":"vpc.block.csi.ibm.io"}
{"level":"info","timestamp":"2023-11-14T05:55:46.663Z","caller":"ibmcsidriver/server.go:139","msg":"nonBlockingGRPCServer-serve...","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver","Endpoint":"unix:/csi/csi.sock"}
{"level":"info","timestamp":"2023-11-14T05:55:46.663Z","caller":"ibmcsidriver/server.go:83","msg":"nonBlockingGRPCServer-Setup...","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver","Endpoint":"unix:/csi/csi.sock"}
{"level":"info","timestamp":"2023-11-14T05:55:46.663Z","caller":"ibmcsidriver/server.go:112","msg":"Start listening GRPC Server","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver","Scheme":"unix","Addr":"/csi/csi.sock"}
{"level":"info","timestamp":"2023-11-14T05:55:46.663Z","caller":"ibmcsidriver/server.go:145","msg":"Listening GRPC server for connections","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver","Addr":{"Name":"/csi/csi.sock","Net":"unix"}}
I1114 05:55:46.867092       1 server.go:153] GRPC call: /csi.v1.Identity/GetPluginInfo
I1114 05:55:46.867146       1 server.go:154] GRPC request:
{"level":"info","ts":"2023-11-14T05:55:46.867Z","caller":"ibmcsidriver/identity.go:36","msg":"CSIIdentityServer-GetPluginInfo...","RequestID":"9e51de1f-7d39-44f5-af39-e9d78c3060be","Request":{}}
I1114 05:55:46.867275       1 server.go:159] GRPC response: name:"vpc.block.csi.ibm.io" vendor_version:"vpcBlockDriver-"
I1114 05:55:47.092873       1 server.go:153] GRPC call: /csi.v1.Identity/GetPluginInfo
I1114 05:55:47.092939       1 server.go:154] GRPC request:
{"level":"info","ts":"2023-11-14T05:55:47.092Z","caller":"ibmcsidriver/identity.go:36","msg":"CSIIdentityServer-GetPluginInfo...","RequestID":"296815c0-b195-47d1-bd42-84cba30edfb2","Request":{}}
I1114 05:55:47.093055       1 server.go:159] GRPC response: name:"vpc.block.csi.ibm.io" vendor_version:"vpcBlockDriver-"
I1114 05:55:49.887308       1 server.go:153] GRPC call: /csi.v1.Node/NodeGetInfo
I1114 05:55:49.887336       1 server.go:154] GRPC request:
{"level":"info","ts":"2023-11-14T05:55:49.887Z","caller":"ibmcsidriver/node.go:352","msg":"CSINodeServer-NodeGetInfo...
See "man sudo_root" for details.

root@kube-cl5hn4qt0rallj2ub75g-ambikstest-default-00000380:/# ls /var/lib/kubelet/plugins/vpc.block.csi.ibm.io/
root@kube-cl5hn4qt0rallj2ub75g-ambikstest-default-00000380:/#  ls /var/lib/kubelet/csi-plugins/
root@kube-cl5hn4qt0rallj2ub75g-ambikstest-default-00000380:/# exit
exit
job.batch "runon-gmvlxxwz97409xaurybe63q3ro" deleted
ambikanair@Ambikas-MacBook-Pro plugins %

Please Verify changes for following use cases:
1- Deploy current csi driver and Create PVC and mount it to pod  —> then deploy your fix and restart the POD and see every POD is coming in running state, After verifying this stop the CSI driver and verify old and new socket - both should be in deleted.

  2- Check controller pod and provision 10 PVCs and mount to one node during that process delete one CSI controller POD and see how this is impacting socket cleanup.


Sponsorship Roadmap

Outline of work to be done on Kobalte, sponsored by @davedbase.
Will keep this issue updated with linked PRs rather than a single huge PR.
Roadmap
Components that are planned for the sponsorship as well as implementation requirements and resources.
Pagination ✅+⏳
https://github.com/kobaltedev/kobalte/pull/239

Feature request https://discord.com/channels/722131463138705510/1063803756388548709/1140407086052823080 (always show same number of items to avoid content shift)

Work already done ~10h + estimate 1h for feature request.
MenuBar
Will be based on https://www.radix-ui.com/primitives/docs/components/menubar and follow https://www.w3.org/WAI/ARIA/apg/patterns/menubar/.

Extra option boolean than radix for focus on Alt.

PR will also include documentation and tests.

While it is much bigger than Pagination, most of the work has already been done in the ContextMenu & DropdownMenu component. Only the MenuBar itself and its interaction need to be added.
Estimate 10h of work.
RangeSlider
Based on https://www.radix-ui.com/primitives/docs/components/slider
Follow https://www.w3.org/WAI/ARIA/apg/patterns/slider-multithumb/

PR will also include documentation and tests.
Finish Slider implementation if not done.

Work estimate TBD.
Additional work
DatePicker, Calendar, ComboBox and bug fixes / other chore will be added in the future.
Anything else mentioned by @fabien-ml or @davedbase and approved can be added to this list.

For Calendar: tests & doc are missing.
For DatePicker (my current WIP): multiple selection, range selection, tests & doc are missing.
I'm ok with the rest mentioned above.

Closing this part as done, new issues to be created for continuation.


ci: bump NodeJS v14 and v16 minor versions to latest

Ensure that all the NodeJS versions that we

Use for the CI
Mention in the docs
Use for the VSCode dev container

are all upgraded to the latest minor versions (so don't upgrade out of the LTS major versions)
Why? Because the Angular CLI started requiring v16.10.x or later and right now
we are running the CI with something below that which makes this PR fail the CI
https://github.com/hyperledger/cactus/pull/1971
Logs: upgrade_angular_node_16_10_minimum_version_build-containers.log

Hi @petermetz  Can you assign me on this one. Thank you

Hi @petermetz Can you assign me on this one. Thank you

@aldousalvarez Yup, done. Thank you!


Getting Error: "Couldn't find that dyno."

After updating to the latest version, whenever I try to run:
heroku run bash -a my-private-shield-app

I get Error: Couldn't find that dyno..
However I've been running this in previous versions without problems.
Here's the output with HEROKU_DEBUG= flag:
Error: Couldn't find that dyno.

Error ID: not_found
    at Function.request (/usr/local/Cellar/heroku/7.33.0/libexec/node_modules/@heroku-cli/command/lib/api-client.js:93:27)
    at processTicksAndRejections (internal/process/task_queues.js:86:5)

The latest version homebrew installed is 7.33. Latest working version is 7.31.2

Upgrade to v7.33.1, thanks for reporting!


Switch to using the "system font stack"

This is a first attempt to close #8634 by switching to a system font stack.
Should I put a line break for the list of fonts? I ran make reformat hoping it would sort out my formatting issues but no luck :D

I copied меншого to before the <em> tag to make it easier to compare (that is why it appears twice). I think it now uses the same font.
What should I do next? Are there things that can be cleaned up now that Source Sans Pro isn't in use any more?
Are there any conventions around adding [WIP] to the PR title or marking it as draft for stuff which needs a bit of hand-holding from an experienced person?

I think it now uses the same font.

Have you checked this with DevTools? The italics version seems visually smaller which suggests that the font may still be different.

Should I put a line break for the list of fonts? I ran make reformat hoping it would sort out my formatting issues but no luck :D

I don't think it knows how to format CSS. I guess you can change it any way you want as long as it remains readable.

Are there any conventions around adding [WIP] to the PR title or marking it as draft for stuff which needs a bit of hand-holding from an experienced person?

I think you can mark PRs as Draft when you yourself think that it's not ready for review.
P.S. I don't think I currently have the warehouse repo set up on my laptop but I've made this change via DevTools on the production instance. And in my env, it now selects Arial for both normal paragraph text and the emphasized one.

I think it now uses the same font.

Have you checked this with DevTools? The italics version seems visually smaller which suggests that the font may still be different.

Should I put a line break for the list of fonts? I ran make reformat hoping it would sort out my formatting issues but no luck :D

I don't think it knows how to format CSS. I guess you can change it any way you want as long as it remains readable.

Are there any conventions around adding [WIP] to the PR title or marking it as draft for stuff which needs a bit of hand-holding from an experienced person?

I think you can mark PRs as Draft when you yourself think that it's not ready for review.
P.S. I don't think I currently have the warehouse repo set up on my laptop but I've made this change via DevTools on the production instance. And in my env, it now selects Arial for both normal paragraph text and the emphasized one.

Devtools for the two bits of text below. I think this means they are using the same font (not quite sure why FA shows up for the normal text)

This is with firefox on OSX

Devtools for the two bits of text below. I think this means they are using the same font (not quite sure why FA shows up for the normal text)

This is with firefox on OSX

As far as I can tell this is ready for reviewing. System font stack is used for all normal text (normal == not monospaced).

As far as I can tell this is ready for reviewing. System font stack is used for all normal text (normal == not monospaced).

Thanks @betatim for taking a look at this!

Are there things that can be cleaned up now that Source Sans Pro isn't in use any more?

Looks like it's still in use by the admin interface CSS as well as some SVG logos and other templates.
I'd be fine with dropping it in all those places and removing our dependency on fonts.google.com for all Warehouse pages (except our 404 page, this should remain).
However I'll leave this for @nlhkabu to approve since ultimately it has the effect of changing the font all of PyPI will render with.

Thanks @betatim for taking a look at this!

Are there things that can be cleaned up now that Source Sans Pro isn't in use any more?

Looks like it's still in use by the admin interface CSS as well as some SVG logos and other templates.
I'd be fine with dropping it in all those places and removing our dependency on fonts.google.com for all Warehouse pages (except our 404 page, this should remain).
However I'll leave this for @nlhkabu to approve since ultimately it has the effect of changing the font all of PyPI will render with.

On my side, I'd prefer to maintain Source Sans Pro where we can.
Are we able to target this per language (at least for now), as per https://www.w3.org/International/questions/qa-css-lang ?
FYI, support in this font is coming (slowly....) - see:
https://github.com/adobe-fonts/source-sans/issues/87 and
https://github.com/google/fonts/issues/1744

oh, and I should have said - thanks for your work on this @betatim! :tada:

Are we able to target this per language (at least for now), as per w3.org/International/questions/qa-css-lang ?

Oh wow! I didn't know that this existed. I'll do some reading to find out what the answer is. If someone knows the answer to this question please post it (don't wait for me to find out first :D )

We could close this pull request and instead rely on the new version of Source Code Pro as @nlhkabu described on 23 Apr 2021.
The two third-party changes that @nlhkabu are now resolved:

adobe-fonts/source-sans#87 was resolved in 2019
google/fonts#1744 was merged on 15 Sep 2021

I confirmed today that https://pypi.org/help/#file-size-limit now displays italic ”меншого” using:

Chrome 100.0 on macOS 12.3
Firefox 98.0 on Ubuntu 20.04 LTS
Safari 15.4 on macOS 12.3

We could close this pull request and instead rely on the new version of Source Code Pro

That would work for me. I'll close this PR. Thanks for following up!


shiftfs EOVERFLOW

Required information

Distribution: Alpine
Distribution version: edge
The output of "lxc info":

config: {}
api_extensions:
- storage_zfs_remove_snapshots
- container_host_shutdown_timeout
- container_stop_priority
- container_syscall_filtering
- auth_pki
- container_last_used_at
- etag
- patch
- usb_devices
- https_allowed_credentials
- image_compression_algorithm
- directory_manipulation
- container_cpu_time
- storage_zfs_use_refquota
- storage_lvm_mount_options
- network
- profile_usedby
- container_push
- container_exec_recording
- certificate_update
- container_exec_signal_handling
- gpu_devices
- container_image_properties
- migration_progress
- id_map
- network_firewall_filtering
- network_routes
- storage
- file_delete
- file_append
- network_dhcp_expiry
- storage_lvm_vg_rename
- storage_lvm_thinpool_rename
- network_vlan
- image_create_aliases
- container_stateless_copy
- container_only_migration
- storage_zfs_clone_copy
- unix_device_rename
- storage_lvm_use_thinpool
- storage_rsync_bwlimit
- network_vxlan_interface
- storage_btrfs_mount_options
- entity_description
- image_force_refresh
- storage_lvm_lv_resizing
- id_map_base
- file_symlinks
- container_push_target
- network_vlan_physical
- storage_images_delete
- container_edit_metadata
- container_snapshot_stateful_migration
- storage_driver_ceph
- storage_ceph_user_name
- resource_limits
- storage_volatile_initial_source
- storage_ceph_force_osd_reuse
- storage_block_filesystem_btrfs
- resources
- kernel_limits
- storage_api_volume_rename
- macaroon_authentication
- network_sriov
- console
- restrict_devlxd
- migration_pre_copy
- infiniband
- maas_network
- devlxd_events
- proxy
- network_dhcp_gateway
- file_get_symlink
- network_leases
- unix_device_hotplug
- storage_api_local_volume_handling
- operation_description
- clustering
- event_lifecycle
- storage_api_remote_volume_handling
- nvidia_runtime
- container_mount_propagation
- container_backup
- devlxd_images
- container_local_cross_pool_handling
- proxy_unix
- proxy_udp
- clustering_join
- proxy_tcp_udp_multi_port_handling
- network_state
- proxy_unix_dac_properties
- container_protection_delete
- unix_priv_drop
- pprof_http
- proxy_haproxy_protocol
- network_hwaddr
- proxy_nat
- network_nat_order
- container_full
- candid_authentication
- backup_compression
- candid_config
- nvidia_runtime_config
- storage_api_volume_snapshots
- storage_unmapped
- projects
- candid_config_key
- network_vxlan_ttl
- container_incremental_copy
- usb_optional_vendorid
- snapshot_scheduling
- snapshot_schedule_aliases
- container_copy_project
- clustering_server_address
- clustering_image_replication
- container_protection_shift
- snapshot_expiry
- container_backup_override_pool
- snapshot_expiry_creation
- network_leases_location
- resources_cpu_socket
- resources_gpu
- resources_numa
- kernel_features
- id_map_current
- event_location
- storage_api_remote_volume_snapshots
- network_nat_address
- container_nic_routes
- rbac
- cluster_internal_copy
- seccomp_notify
- lxc_features
- container_nic_ipvlan
- network_vlan_sriov
- storage_cephfs
- container_nic_ipfilter
- resources_v2
- container_exec_user_group_cwd
- container_syscall_intercept
- container_disk_shift
- storage_shifted
- resources_infiniband
- daemon_storage
- instances
- image_types
- resources_disk_sata
- clustering_roles
- images_expiry
- resources_network_firmware
- backup_compression_algorithm
- ceph_data_pool_name
- container_syscall_intercept_mount
- compression_squashfs
- container_raw_mount
- container_nic_routed
- container_syscall_intercept_mount_fuse
- container_disk_ceph
- virtual-machines
- image_profiles
- clustering_architecture
- resources_disk_id
- storage_lvm_stripes
- vm_boot_priority
- unix_hotplug_devices
- api_filtering
- instance_nic_network
- clustering_sizing
- firewall_driver
- projects_limits
- container_syscall_intercept_hugetlbfs
- limits_hugepages
- container_nic_routed_gateway
- projects_restrictions
- custom_volume_snapshot_expiry
- volume_snapshot_scheduling
- trust_ca_certificates
- snapshot_disk_usage
- clustering_edit_roles
- container_nic_routed_host_address
- container_nic_ipvlan_gateway
- resources_usb_pci
- resources_cpu_threads_numa
- resources_cpu_core_die
- api_os
- container_nic_routed_host_table
- container_nic_ipvlan_host_table
- container_nic_ipvlan_mode
- resources_system
- images_push_relay
- network_dns_search
- container_nic_routed_limits
- instance_nic_bridged_vlan
- network_state_bond_bridge
- usedby_consistency
- custom_block_volumes
- clustering_failure_domains
- resources_gpu_mdev
- console_vga_type
- projects_limits_disk
- network_type_macvlan
- network_type_sriov
- container_syscall_intercept_bpf_devices
- network_type_ovn
- projects_networks
- projects_networks_restricted_uplinks
- custom_volume_backup
- backup_override_name
- storage_rsync_compression
- network_type_physical
- network_ovn_external_subnets
- network_ovn_nat
- network_ovn_external_routes_remove
- tpm_device_type
- storage_zfs_clone_copy_rebase
- gpu_mdev
- resources_pci_iommu
- resources_network_usb
- resources_disk_address
- network_physical_ovn_ingress_mode
- network_ovn_dhcp
- network_physical_routes_anycast
- projects_limits_instances
- network_state_vlan
- instance_nic_bridged_port_isolation
- instance_bulk_state_change
- network_gvrp
- instance_pool_move
- gpu_sriov
- pci_device_type
- storage_volume_state
- network_acl
- migration_stateful
- disk_state_quota
- storage_ceph_features
- projects_compression
- projects_images_remote_cache_expiry
- certificate_project
- network_ovn_acl
- projects_images_auto_update
- projects_restricted_cluster_target
- images_default_architecture
- network_ovn_acl_defaults
- gpu_mig
- project_usage
- network_bridge_acl
- warnings
- projects_restricted_backups_and_snapshots
- clustering_join_token
- clustering_description
- server_trusted_proxy
- clustering_update_cert
- storage_api_project
- server_instance_driver_operational
- server_supported_storage_drivers
- event_lifecycle_requestor_address
- resources_gpu_usb
- clustering_evacuation
- network_ovn_nat_address
- network_bgp
- network_forward
- custom_volume_refresh
- network_counters_errors_dropped
- metrics
- image_source_project
- clustering_config
- network_peer
- linux_sysctl
- network_dns
- ovn_nic_acceleration
- certificate_self_renewal
- instance_project_move
- storage_volume_project_move
- cloud_init
- network_dns_nat
- database_leader
- instance_all_projects
- clustering_groups
- ceph_rbd_du
- instance_get_full
- qemu_metrics
- gpu_mig_uuid
- event_project
- clustering_evacuation_live
- instance_allow_inconsistent_copy
- network_state_ovn
- storage_volume_api_filtering
- image_restrictions
- storage_zfs_export
- network_dns_records
- storage_zfs_reserve_space
- network_acl_log
- storage_zfs_blocksize
- metrics_cpu_seconds
- instance_snapshot_never
- certificate_token
- instance_nic_routed_neighbor_probe
- event_hub
- agent_nic_config
- projects_restricted_intercept
- metrics_authentication
- images_target_project
- cluster_migration_inconsistent_copy
- cluster_ovn_chassis
- container_syscall_intercept_sched_setscheduler
- storage_lvm_thinpool_metadata_size
- storage_volume_state_total
- instance_file_head
- instances_nic_host_name
- image_copy_profile
- container_syscall_intercept_sysinfo
- clustering_evacuation_mode
- resources_pci_vpd
api_status: stable
api_version: "1.0"
auth: trusted
public: false
auth_methods:
- tls
environment:
  addresses: []
  architectures:
  - aarch64
  - armv7l
  certificate: REDACTED
  certificate_fingerprint: 77144852aa5a4705ad12c28520e193e826bf6323654d0c726cee763a9cd95813
  driver: lxc | qemu
  driver_version: 4.0.12 | 7.0.0
  firewall: nftables
  kernel: Linux
  kernel_architecture: aarch64
  kernel_features:
    idmapped_mounts: "true"
    netnsid_getifaddrs: "true"
    seccomp_listener: "true"
    seccomp_listener_continue: "true"
    shiftfs: "true"
    uevent_injection: "true"
    unpriv_fscaps: "true"
  kernel_version: 5.15.59-0-lts
  lxc_features:
    cgroup2: "true"
    core_scheduling: "true"
    devpts_fd: "true"
    idmapped_mounts_v2: "true"
    mount_injection_file: "true"
    network_gateway_device_route: "true"
    network_ipvlan: "true"
    network_l2proxy: "true"
    network_phys_macvlan_mtu: "true"
    network_veth_router: "true"
    pidfd: "true"
    seccomp_allow_deny_syntax: "true"
    seccomp_notify: "true"
    seccomp_proxy_send_notify_fd: "true"
  os_name: Alpine Linux
  os_version: 3.17_alpha20220715
  project: default
  server: lxd
  server_clustered: false
  server_event_mode: full-mesh
  server_name: lxd
  server_pid: 3207
  server_version: "5.2"
  storage: btrfs | zfs
  storage_version: 5.18.1 | 2.1.5-1
  storage_supported_drivers:
  - name: btrfs
    version: 5.18.1
    remote: false
  - name: dir
    version: "1"
    remote: false
  - name: zfs
    version: 2.1.5-1
    remote: false

Issue description
All containers using shiftfs are unable to create any files in the rootfs because that always fails with EOVERFLOW:
# mkdir /tmp/test
mkdir: can't create directory '/tmp/test': Value too large for data type

If I nsenter -t INITPID -m sh I can see that at least the UIDs are actually correct:
# ls -lah
total 72K
drwxr-xr-x   19 655360   655360        19 Aug  1 13:04 .
drwxr-xr-x   19 655360   655360        19 Aug  1 13:04 ..
drwxr-xr-x    2 655360   655360        88 Aug  1 13:01 bin
drwxr-xr-x    7 655360   655360       440 Aug 10 06:31 dev
drwxr-xr-x   21 655360   655360        48 Aug  1 13:04 etc
drwxr-xr-x    2 655360   655360         2 May 23 16:53 home
drwxr-xr-x    8 655360   655360        19 Aug  1 13:01 lib
drwxr-xr-x    5 655360   655360         5 May 23 16:53 media
drwxr-xr-x    2 655360   655360         2 May 23 16:53 mnt
drwxr-xr-x    2 655360   655360         2 May 23 16:53 opt
dr-xr-xr-x  330 root     root           0 Aug 10 06:31 proc
drwx------    2 655360   655360         2 May 23 16:53 root
drwxr-xr-x    4 655360   655360       240 Aug 10 06:31 run
drwxr-xr-x    2 655360   655360       106 Aug  1 13:01 sbin
drwxr-xr-x    2 655360   655360         2 May 23 16:53 srv
dr-xr-xr-x   13 root     root           0 Aug 10 06:31 sys
drwxrwxrwt    2 655360   655360         2 May 23 16:53 tmp
drwxr-xr-x    8 655360   655360         8 Aug  1 13:01 usr
drwxr-xr-x   11 655360   655360        12 Aug 10 06:31 var

Steps to reproduce
I'm not sure why alpine has that issue and a x86 ubuntu-VM using snap doesn't. Hints welcome.
I'll try to reproduce the issue in a x86 alpine VM if nobody has any immediate ideas.

Can you show the output of findmnt from inside the container?

 That is very odd because I would think that the 5.15 kernel on Alpine does support them.

That's because idmapped mounts are not supported by ZFS yet.
ubuntu seems to be able to use shiftfs on top of ZFS though.

So you're using an Alpine vm and the Alpine vm uses zfs as the root filesystem?

no I'm using a physical aarch64 device with an ext4 rootfs and a zfs storage pool for LXD.

no I'm using a physical aarch64 device with a squashfs(+overlaytmpfs) rootfs and a zfs storage pool for LXD.

So you're running an Alpine container on top of zfs, right?

Oh you meant the guest, yes that's an images:alpine/3.16 container(not a VM)

Can you show me findmnt on the physical aarch64 if that's not something you'd rather not share/

here you go: (I removed unrelated containers)
# findmnt
TARGET                                                     SOURCE                       FSTYPE  OPTIONS
/                                                          overlayfs                    overlay rw,rela
├─/sys                                                     sysfs                        sysfs   rw,nosu
│ ├─/sys/kernel/security                                   securityfs                   securit rw,nosu
│ ├─/sys/kernel/debug                                      debugfs                      debugfs rw,nosu
│ │ └─/sys/kernel/debug/tracing                            tracefs                      tracefs rw,nosu
│ ├─/sys/fs/pstore                                         pstore                       pstore  rw,nosu
│ └─/sys/fs/cgroup                                         none                         cgroup2 rw,nosu
├─/dev                                                     devtmpfs                     devtmpf rw,nosu
│ ├─/dev/pts                                               devpts                       devpts  rw,nosu
│ ├─/dev/shm                                               shm                          tmpfs   rw,nosu
│ └─/dev/mqueue                                            mqueue                       mqueue  rw,nosu
├─/proc                                                    proc                         proc    rw,nosu
├─/media/root-ro                                           /dev/mmcblk1p7               squashf ro,rela
├─/media/root-rw                                           root-tmpfs                   tmpfs   rw,rela
├─/run                                                     tmpfs                        tmpfs   rw,nosu
├─/var                                                     /dev/sda1                    ext4    rw,rela
│ ├─/var/lib/lxcfs                                         lxcfs                        fuse.lx rw,nosu
│ ├─/var/lib/lxd/shmounts                                  tmpfs                        tmpfs   rw,rela
│ ├─/var/lib/lxd/devlxd                                    tmpfs                        tmpfs   rw,rela
│ ├─/var/lib/lxd/storage-pools/btrfs                       /dev/sda3                    btrfs   rw,rela
│ └─/var/lib/lxd/storage-pools/default/containers/nice-chamois
│                                                          default/containers/nice-chamois
│                                                                                zfs     rw,rela
├─/boot                                                    /dev/mmcblk1p1               vfat    rw,rela
└─/media/config                                            /dev/mmcblk1p5               ext4    rw,rela

I managed to create a cloud-init based LXD VM where the issue can be reproduced. To be clear: The issue happens inside the LXD container inside the LXD VM.
config:
  cloud-init.user-data: |
    #cloud-config
    write_files:
      - path: /etc/lxc/default.conf
        permissions: '0644'
        content: |
          lxc.net.0.type = empty
          lxc.idmap = u 0<PHONE_NUMBER>000000
          lxc.idmap = g 0<PHONE_NUMBER>000000

      - path: /etc/subuid
        permissions: '0644'
        content: |
          root:100000:1000000000

      - path: /etc/subgid
        permissions: '0644'
        content: |
          root:100000:1000000000

      - path: /etc/local.d/cgroup-initscope.start
        permissions: '0755'
        content: |
          #!/bin/sh
          mkdir -m 0755 -p /sys/fs/cgroup/init.scope

      - path: /etc/modules-load.d/shiftfs.conf
        permissions: '0644'
        content: |
          shiftfs

    runcmd:
      - echo "https://dl-cdn.alpinelinux.org/alpine/edge/testing" >> /etc/apk/repositories
      - apk update
      - apk upgrade
      - apk add
        apparmor
        apparmor-profiles
        apparmor-utils
        chrony
        cloud-utils-growpart
        e2fsprogs
        e2fsprogs-extra
        eudev
        eudev-netifnames
        git
        linux-virt-dev
        lxcfs
        lxd-feature
        make
        nftables
        zfs
        zfs-udev

      - |
        cat >> /etc/rc.conf <<EOF
        rc_cgroup_mode="unified"
        rc_logger="YES"
        rc_parallel="YES"
        EOF

      - growpart /dev/sda 2
      - resize2fs /dev/sda2

      - rc-update del mdev sysinit
      - rc-update add localmount sysinit
      - rc-update add zfs-import sysinit
      - rc-update add zfs-mount sysinit

      - rc-update add cgroups boot
      - rc-update add local boot

      - rc-update add chronyd default
      - rc-update add lxcfs default
      - rc-update add lxd default

      - git clone https://github.com/toby63/shiftfs-dkms.git -b k5.16
      - make -C shiftfs-dkms
      - ln -s /shiftfs-dkms/shiftfs.ko /lib/modules/$(uname -r)/
      - depmod -a

to reproduce:
lxc launch --vm images:alpine/edge/cloud -c security.secureboot=false alpine < alpine-shiftfs-bug.yaml
lxc exec alpine -- sh
lxd init --auto --storage-backend zfs
lxc launch images:alpine/3.16 a1
lxc exec a1 -- sh
touch /testfile

I'm closing this issue, not because we don't care about but because it's not a LXD bug.
I'm sure @brauner will still keep the chat going on here. Given we're not seeing this on Ubuntu, I wonder if it may be an incorrect port to 5.16 (Ubuntu is on 5.15)?
I don't know how easy it would be for you to transplant an Ubuntu 5.15 kernel onto your Alpine VM, but if doable, that'd be an easy way to see if it's a kernel problem or something odd with the mount layout in userspace.

FTR Alpine is on Kernel 3.15 as well and shiftfs.c has the same checksum as in Ubuntu. The shifts repo just uses the same branch for 5.15 and 5.16

I just booted the alpine VM with ubuntus 5.15.0-43-generic and it works :thinking:
Alpine uses an (almost) unpatched 5.15.59 kernel. Do you already know which of Ubuntus patches might be necessary for this to work? If not I guess I'm gonna read the whole git history to try and find something relevant.

ok alpine kernel 5.15.39 works, and 5.15.59 doesn't. That means that Ubuntu will probably have the same issue as soon as the latest patch version gets merged.

it starts to break with 5.15.52. The commit that causes that is 38753e9173a5903e902c856b41fb325762bf5945.
I'm not yet sure why exactly that causes it or where exactly the EOVERFLOW comes from. It's entirely possible that the problem is in zfs and not in shiftfs. They have plenty of EOVERFLOWs in their code that make way more sense than shift_acl_ids inside shiftfs.c. I'm currently testing with ZFS 2.1.5.
Is there any way to force using shiftfs even when idmapped mounts are available so I can test this with a filesystem like ext4?
@stgraber IMO you should reopen this issue since at this point I'm pretty sure that ubuntu will have this issue after the next kernel update.

the EOVERFLOW comes from here.
That totally makes sense since the breaking commit changed the implementation of the function fsuidgid_has_mapping to check against fs_userns instead of init_user_ns.
I don't yet know why that's an issue since I don't yet understand that code but this sounds like shiftfs and idmapping have some sort of conflict here and that shiftfs basically assumes that idmappings don't exist.

okay I got it working with the latest shiftfs.c from the ubuntu kinetic kernel.
e1b92741ef11bccde558ac7b16d72981a1e020b7 fixed it and the commit description matches everything I've seen so far.
To me that's good news since it means I don't have to maintain my own alpine kernel fork and can just update the shiftfs module instead.

@stgraber renames are still broken on the ubuntu kernel and this change is required:
diff --git a/shiftfs.c b/shiftfs.c
index a5338dc..46a7d05 100644
--- a/shiftfs.c
+++ b/shiftfs.c
@@ -632,10 +632,10 @@ static int shiftfs_rename(struct user_namespace *ns,
        struct inode *loweri_dir_old = lowerd_dir_old->d_inode,
                     *loweri_dir_new = lowerd_dir_new->d_inode;
        struct renamedata rd = {
-               .old_mnt_userns = ns,
+               .old_mnt_userns = &init_user_ns,
                .old_dir        = loweri_dir_old,
                .old_dentry     = lowerd_old,
-               .new_mnt_userns = ns,
+               .new_mnt_userns = &init_user_ns,
                .new_dir        = loweri_dir_new,

I'm not going to submit that to ubuntu since I their contribution barrier is way too high due to their complicated processes, documentation and software.

This fix referenced above in e1b92741ef11bccde558ac7b16d72981a1e020b7 leaves me slightly concerned. The analysis isn't correct. A stacking filesystem like shifts or overlayfs calls vfs_* helpers for the lower filesystem. And when it does so it needs to account for the properties of the lower filesystem not of shiftfs. IOW, passing down information from the shiftfs layer is almost always a bug. They frankly wouldn't have noticed this but on newer kernels an idmapped mount is either identified by having the init_user_ns or fs_userns != mnt_userns. So if shiftfs is mounted in a userns then fs_userns == mnt_userns (!= init_user_ns) meaning that they passed down shifts specific inofrmation to the lower filesystem.
The fix that they are outlined means that you're still allowing shiftfs to be mounted on top of idmapepd mounts which means things are broken there as well as the mount's idmapping isn't taken into account. So you either want to do what I did for overlayfs upstream to allow idmapped lower layers or for now at least you want sm like (untested):
diff --git a/shiftfs.c b/fixes.next
index a5338dc..71355d3 100644
--- a/shiftfs.c
+++ b/fixes.next
@@ -2083,6 +2083,17 @@ static int shiftfs_fill_super(struct super_block *sb, void *raw_data,
                        cap_lower(cred_tmp->cap_effective, CAP_SYS_RESOURCE);
                        sbinfo->creator_cred = cred_tmp;
                }
+
+               /*
+                * Supporting idmapped lower layers requires a decent amount of
+                * rework that involves passing down the mnt_userns from the
+                * lower layer into vfs_*() helpers.
+                */
+               if (is_idmapped_mnt(sbinfo->mnt)) {
+                       err = -EINVAL;
+                       printk(KERN_ERR "shiftfs: idmapped lower layers not supported\n");
+                       goto out_put_path;
+               }however
shiftfs logic is still relying on the fact that these functions need to
use the main filesystem namespace.

        } else {
                /*
                 * This leg executes if we're admin capable in the namespace,

Thanks for the suggested check. I think it is sufficient because time should be invested into making every Filesystem idmapping compatible instead of doing the same for an obsolete Filesystem(=shiftfs). ZFS has a PR pending already though the latest comment brings up valid concerns which could delay things further.

For those coming here from Google after having their containers blown up by Ubuntu 5.15.0-52 update: workaround is to stop lxd, rmmod shiftfs, add shiftfs to modules blacklist, restart lxd.
(For reasons I can't really understand, two containers were working fine, but one with seemingly identical config was getting EOVERFLOW.)

https://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/jammy/commit/fs/shiftfs.c?h=master-next&id=6d1703cb41f2a7720ce227c5f86b9d2af989c7c4
https://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/jammy/commit/fs/shiftfs.c?h=master-next&id=4246c2678b2271f769b124e5a0c410e8a0fd5c3c


Change High Ground

High Ground should provide half cover against enemies attacking from below (+2 AC) and the usual range extension (which is already given by default).
The Advantage on attack rolls should be removed, together with the Disadvantage from Low Ground

This seems a good early solution, but I think we can find a better way to implement cover. High-low ground leaves so many important factors out?

I highly doubt we can implement better cover solutions without Larian wanting to do it themselves. Cover checks require a processing on the positions of each character and LoS checks, which is very hard to do outside the low level layer of the game


Feature: Log size

Allow to specify number/size of logged requests with rotating.

Thanks for the feedback. This is nothing we plan to have for mitmproxy's core, but surely can be implemented in an independent addon! :smiley:


Position

Is this working with Tailwindcss 2+?  The position is fixed at top right.  When trying to show more that one notification, only one appears.  Using Vue 2.x and Tailwindcss 2.x

Resolve issue.

@bobking256 Did you get this working? I'm attempting to change position to bottom but it doesn't appear to have any impact:
<notificationGroup position="bottom">...</notificationGroup>


feat(meetings): add 'No assignments yet' in the assignment history

Description
Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.
Fixes # (issue)
Type of change
Please delete options that are not relevant.

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

Checklist:

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[x] Any dependent changes have been merged and published in downstream modules

Task linked: CU-86c16na6b Add 'No assignments yet' in the assignment history – Person page and more

:tada: This PR is included in version 3.3.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


NR-10893 fix social icons alignment as home page

JIRA ticket: https://issues.newrelic.com/browse/NR-10893
Description: Social media icons should be in 2 rows like in HP. Needs to be in sync with NR Homepage.
Previous:

Now:

A full fledge footer can be seen on merging the below two PRs

https://github.com/newrelic/instant-observability-website/pull/200
https://github.com/newrelic/instant-observability-website/pull/201

ll fledge footer can be seen on merging the below two PRs

NR-10888: TOS, DMCA, Privacy should be in one column. Cookie, UK Slavery Act in 2nd column. #200
NR-10893 fix social icons alignment as home page #201

ll fledge footer can be seen on merging the below two PRs
https://github.com/newrelic/instant-observability-website/pull/200
https://github.com/newrelic/instant-observability-website/pull/201


update children props when experiment rerender + update babel cli

Issue https://github.com/pushtell/react-ab-test/issues/10

Published version 1.7.0. Thanks for the PR!


browserify with jquery.headroom.js?

I build lasted headroom, it work with browserify smooth.
but browserify how to work with jquery.headroom.js? if I require this file,
error: object is not a function.

Good point! Forgot about the plugins :)

Any updates here?

This would be great.