Build arm64 binaries to run on Raspbian OS

Usecase: Highly available ownCloud running on Kubernetes cluster
Raspberry Pi 3 model b is cost & power efficient to run ownCloud, but the issue is neither storage nor the nodes can be configured as HA.
I'm running ownCloud on 6 nodes Kubernetes cluster on Raspbian Lite, application availability is achieved, but i can't protect my storage since there is only single SSD attached to each node.
I want to try OpenEBS to see if I can achieve storage availability by replicating with OpenEBS.
Use case is niche and unproven since the resources on Raspberry Pi 3 is limited. OpenFaaS is another common usecase popular on Raspberry.
My cluster specs:
6 nodes K8s cluster running on Raspbian Lite
Each node:
Quad Core 1.2GHz Broadcom BCM2837 64bit CPU
1GB RAM
32 GB SD for OS
512 GB SSD for data

Quick update for arm64 enthusiasts:
Made some progress here fixing dependencies and building deps for arm64 architecture. Instead of RPi 3 I've used a little bit more powerful SoC called Le Potato from Libre Computer.
Need more testing, and integration with OpenEBS build setup before I can send a PR, but in the meantime if you want to play with it, my images are here

My request to get a CI setup hw from WorksOnArm Project is approved.
https://github.com/WorksOnArm/cluster/issues/54
Next week I am hoping to spend some time with OpenEBS build team to unify amd64 and arm64 builds and get official builds started.

Any update on this?  I'm working on an ARM cluster for my home lab and would love to get a storage solution like OpenEBS working.

Hi @muratkars @jmreicha I'm hopeful that there's an update here - let me know if there's anything I can do to help from the arm64 cluster side at Packet.

Hi @vielmetti @jmreicha - Team is working on this. Currently, build and instructions provided here seems to be functional.
Once the 0.6 release is out, a new arm64 build will be released and we will focus back on getting it into CI run on packet and adding few more validation steps for arm64.

Thanks @muratkars ! Remind me what your CI infrastructure looks like? I want to make sure we have the right sort of resources ready.

@muratkars Can you point me in the right direction for how/where builds happen?  I'd like to take see if I can take a stab at this as part of Octoberfest.

@muratkars Hello!
I see this is still open and the issue also interests me a lot as I'm working on a k8s cluster on Rock64 boards.
I've tried the instructions at https://blog.openebs.io/arming-kubernetes-with-openebs-1-b450f41e0c1f but it looks like the yml provided aren't available anymore.
Is there something I can do to help with this?
Cheers!

Thanks @andrei-dascalu . Team is working to get automation of multi-architecture images up soon. In the meantime, I'll try to get the v0.8.1 arm64 built this week and update the issue here. cc @harshvkarn

@muratkars Anything new for us waiting for arm64? I'm curious to give OpenEBS a try on my cluster!

Thanks @bwolf for checking back on this. Added this for 0.9.

@muratkars worth asking about another update? This seems to be slipping without any resolution in sight. With the new RPi4 and other more powerful SBCs using arm64 being available now, this is quite a limitation to wider adoption.

@dewet22 This wasn't on the highly active issue until now :) Builds will be coming soon. I'll keep this issue posted. Thanks.

yaml files https://raw.githubusercontent.com/muratkars/openebs/lepotato-arm64/k8s/openebs-operator-arm64.yaml & https://raw.githubusercontent.com/muratkars/openebs/lepotato-arm64/k8s/openebs-storageclasses.yaml no longer appear to be present. Is there any update on the arm64 builds?

@harshvkarn is there an update on this? Thanks!

@vielmetti
We were able to build the bare minimum OpenEBS w/ Jiva. I will commit for the official build process very soon.
These are the images for now if you want to test it.
harshvkarn/jiva:arm
harshvkarn/provisioner-localpv:alpine-arm64
harshvkarn/admission-server:arm
harshvkarn/m-apiserver:arm
harshvkarn/node-disk-manager:arm
harshvkarn/node-disk-operator:arm
harshvkarn/openebs-k8s-provisioner:arm

Attempting a deploy in k3s on RPI3B running Ubuntu 18.04.3/aarch64, maya-apiserver dumps out in preflight:
$ kubectl logs maya-apiserver-5d9b995c9-r6nvz
+ MAYA_API_SERVER_NETWORK=eth0
+ grep inet
+ awk {print $2}
+ ip -4 addr show scope global dev eth0
+ cut -d / -f 1
+ CONTAINER_IP_ADDR=<IP_ADDRESS>
+ exec /usr/local/bin/maya-apiserver start --bind=<IP_ADDRESS>
I0911 01:22:01.742542       6 start.go:153] Initializing maya-apiserver...
preflight checks failed:   --  precheck for bdc failed: the server could not find the requested resource (get blockdeviceclaims.openebs.io)

I could be missing something obvious, so feel free to point out what I could be doing wrong.  :)
-- T

@harshvkarn Redeployed k8s 1.15.3 w/ kubeadm on ubuntu 18.04/aarch64, .  maya-apiserver errors our the same as k3s above (same platform).
ndm-operator errors out:
{"level":"info","ts":1568435605.2400267,"logger":"ndm-operator","msg":"Go Version: go1.12.5"}
{"level":"info","ts":1568435605.240167,"logger":"ndm-operator","msg":"Go OS/Arch: linux/arm64"}
{"level":"info","ts":1568435605.2402081,"logger":"ndm-operator","msg":"operator-sdk Version: v0.5.0"}
{"level":"info","ts":1568435605.2474062,"logger":"leader","msg":"Trying to become the leader."}
{"level":"info","ts":1568435605.8234258,"logger":"leader","msg":"Found existing lock with my name. I was likely restarted."}
{"level":"info","ts":1568435605.8236134,"logger":"leader","msg":"Continuing as the leader."}
{"level":"info","ts":1568435606.252126,"logger":"ndm-operator","msg":"Registering Components"}
{"level":"info","ts":1568435606.2535322,"logger":"kubebuilder.controller","msg":"Starting EventSource","controller":"blockdevice-controller","source":"kind source: /, Kind="}
{"level":"error","ts":1568435606.2538614,"logger":"kubebuilder.source","msg":"if kind is a CRD, it should be installed before calling Start","kind":"BlockDevice.openebs.io","error":"no matches for kind \"BlockDevice\" in version \"openebs.io/v1alpha1\"","stacktrace":"github.com/openebs/node-disk-manager/vendor/github.com/go-logr/zapr.(*zapLogger).Error\n\t/root/go/src/github.com/openebs/node-disk-manager/vendor/github.com/go-logr/zapr/zapr.go:128\ngithub.com/openebs/node-disk-manager/vendor/sigs.k8s.io/controller-runtime/pkg/source.(*Kind).Start\n\t/root/go/src/github.com/openebs/node-disk-manager/vendor/sigs.k8s.io/controller-runtime/pkg/source/source.go:89\ngithub.com/openebs/node-disk-manager/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Watch\n\t/root/go/src/github.com/openebs/node-disk-manager/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:122\ngithub.com/openebs/node-disk-manager/pkg/controller/blockdevice.add\n\t/root/go/src/github.com/openebs/node-disk-manager/pkg/controller/blockdevice/blockdevice_controller.go:53\ngithub.com/openebs/node-disk-manager/pkg/controller/blockdevice.Add\n\t/root/go/src/github.com/openebs/node-disk-manager/pkg/controller/blockdevice/blockdevice_controller.go:36\ngithub.com/openebs/node-disk-manager/pkg/controller.AddToManager\n\t/root/go/src/github.com/openebs/node-disk-manager/pkg/controller/controller.go:13\nmain.main\n\t/root/go/src/github.com/openebs/node-disk-manager/cmd/manager/main.go:88\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:200"}
{"level":"error","ts":1568435606.2542424,"logger":"ndm-operator","msg":"","error":"no matches for kind \"BlockDevice\" in version \"openebs.io/v1alpha1\"","stacktrace":"github.com/openebs/node-disk-manager/vendor/github.com/go-logr/zapr.(*zapLogger).Error\n\t/root/go/src/github.com/openebs/node-disk-manager/vendor/github.com/go-logr/zapr/zapr.go:128\nmain.main\n\t/root/go/src/github.com/openebs/node-disk-manager/cmd/manager/main.go:89\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:200"}

FWIW, ndm-operator gives the same error on k3s, so damnfino what the problem is.  Any help?
-- T

Seems to be working with these images:
harshvkarn/m-apiserver:arm
harshvkarn/openebs-k8s-provisioner:arm
harshvkarn/node-disk-manager-arm64:v0.4.1
harshvkarn/node-disk-operator-arm64:v0.4.1
harshvkarn/admission-server:arm
harshvkarn/provisioner-localpv:alpine-arm64

By "working" I mean all the pods come up, and all the expected CRDs are registered.  I'll burn it all down again and repeat on k3s.  (Bob bless Ansible developers. :)
-- T

@Cerebus @DeanHao thanks for using and verifying with the above images. Taking this up for 1.3 release to make setup travis builds and generate images.

Working on k3s as well, to the same level as before.  I haven't yet tried deploying a storage backed workload.  :)

Hi @muratkars - what is the prognosis on getting support for arm64 released? Arm Techcon is coming up, would be lovely to have some announcement synced by then. (I don't see a due date for the 1.3 milestone, so I don't know what your expected time frame is at all.)

@vielmetti - tentatively planned for 2nd week of october, we are on monthly cadence.  Are you referring to oct 8-10 techcon?

Just a quick update here an many thanks to @DeanHao for contributions, @vielmetti and community being patient with us.
Although manual builds were possible and done by the community members and me and in the past (quick Docker Hub search shows that) automating multi-architectural builds for key OpenEBS container images took a little bit longer than we anticipated.
Target release is still 1.3 and we are very close to deliver it this time.

Are there updated images available for use now, even if just in testing at this stage? I'm looking forward to arm64 support for my home lab and would be interested in helping sanity test.

@radicand thanks for your interest. Do look out relevant Dockerfiles in this location
https://github.com/openebs/maya/tree/master/buildscripts

Thanks @AmitKumarDas , I was hoping there would be a published image to use for testing vs. building one; but I've looked at the repo without much luck on determining how to build it myself. It appears to need some binaries built before the docker image can be built, and I have not been successful with determining how to do this yet. Any guidance (or links to where images may already exist) would be appreciated.

@radicand Let me get back with proper links/contributors. Do you mind connecting us on openebs slack for quicker response/discussions?

Sorry for the offtopic, but is there any chance to get openebs running on armhf(or armv7l ) architecture?
Thanks in advance for any information!

I am working on a document that explains how to build the binaries and will get it out for the 1.14 release. A short summary is that local pv and jiva volumes related containers can now be built on arm machines.

Sample YAML for installing OpenEBS using the images built for arm64 is here:
https://github.com/openebs/charts/blob/master/docs/openebs-operator-arm-dev.yaml
Looking for help from community to test these images and provide us a list of issues/enhancements. Please comment her e or join our Slack arm channel.

@kmova Thank you a lot ! Additionally for jiva volume, we need two more containers:

The pvc controller require maya-exporter arm64 container, which is currently possible by building the source code openbs/slp openebs/libcstor openebs/cstor to have the arm64 base image and finally build via openebs/maya the m-exporter.
The ScrubJob needs also openebs-tools arm64  to propagate the deletion.

@Guigouu -- thanks for reporting the issue with the jiva scrub image. Fixed maya to override the scrub image using the IO_HELPER_IMAGE env.
The exporter image will be fixed in 1.6, along with the support for cstor arm builds.

is it live? how do I install?  https://raw.githubusercontent.com/muratkars/openebs/lepotato-arm64/k8s/openebs-operator-arm64.yaml is not working anymore :(

@carnei-ro - can you try this? https://github.com/openebs/charts/blob/master/docs/openebs-operator-arm-dev.yaml.

@kmova it kinda works, when I create the StoragePoolClaim, it launches cstor-pool, cstor-pool-mgmt and m-exporter, but as x86_64 I think. Should I try to use Jiva? Are there any docs?

I've edited the yaml and now it seems to work. Thanks
        - name: OPENEBS_IO_CSTOR_TARGET_IMAGE
          value: "mayankrpatel/cstor-istgt:arm-ci"
        - name: OPENEBS_IO_CSTOR_POOL_IMAGE
          value: "mayankrpatel/cstor-pool:arm-ci"
        - name: OPENEBS_IO_CSTOR_POOL_MGMT_IMAGE
          value: "mayankrpatel/cstor-pool-mgmt:arm-ci"
        - name: OPENEBS_IO_CSTOR_VOLUME_MGMT_IMAGE
          value: "quay.io/openebs/cstor-volume-mgmt:1.5.0"
        - name: OPENEBS_IO_VOLUME_MONITOR_IMAGE
          value: "goaoza/m-exporter-arm64"
        - name: OPENEBS_IO_CSTOR_POOL_EXPORTER_IMAGE
          value: "goaoza/m-exporter-arm64"

I haven't yet tested the cstor! thanks for trying it out. Do let us know here or in the openebs slack #arm channel about any question you have. There are a few more folks in the arm channel that can help.

With release 1.8, the arm builds for NDM and Jiva have been automated.
Pending repos are:

cstor
istgt
maya
external-storage

Awesome news @kmova - I'm a bit confused on where to find the builds though, the tags on dockerhub appear to be CI and not version pinned (yet?). Is this just technically done, but not published out, or am I looking in the wrong place for the images?

Since with 1.9 there are automated builds for cstor, is there somewhere an updated yaml to test it ?

@moritz31 -- The updated YAML is here: https://openebs.github.io/charts/openebs-operator-arm-dev.yaml

Could anyone share their experience running the images on Rpi4? What does the cpu/memory consumption look like? Is it feasible to run OpenEBS pods alongside other workloads on the same node?

@jjbubudi , I run 1.5 currently, with plans to update to 1.9 this weekend, on a 3 now cluster consisting of a RPi4(4gb), and two RPi3(1gb). With external storage for each node, I have not found openebs to place any significant overhead; I run several deployments in this setup using a mix of Jiva (3 replications), and local storage depending on the workload. OpenEBS replication pod CPU is negligible, and memory reports range from 15mb to 25mb. The management pods are similar, 15mb to 35mb memory, CPU slightly higher (1-8m vs 1m for replication) but still negligible. Hope that helps.

Hello, when taking  The updated YAML is here: https://openebs.github.io/charts/openebs-operator-arm-dev.yaml, on Raspberry Pi 4  k3s Cluster in aarch64 OS architecture, i am getting all good from the  openebs, except for the maya-apiserver
pi@pi1:~ $ kubectl get all -A
NAMESPACE     NAME                                               READY   STATUS             RESTARTS   AGE
openebs       pod/openebs-snapshot-operator-546b77dc67-g54t7     2/2     Running            1          14m
openebs       pod/openebs-ndm-kckk4                              1/1     Running            0          14m
kube-system   pod/coredns-8655855d6-5qdbk                        1/1     Running            1          13h
openebs       pod/openebs-admission-server-7b897bfcdc-l5pzn      1/1     Running            0          14m
openebs       pod/openebs-ndm-nzg44                              1/1     Running            2          14m
openebs       pod/openebs-ndm-operator-76d8945d4-h4ktq           1/1     Running            4          14m
openebs       pod/openebs-provisioner-64cd65d58f-v6nwn           1/1     Running            5          14m
openebs       pod/openebs-ndm-559ww                              1/1     Running            3          14m
openebs       pod/openebs-localpv-provisioner-6f7cd99684-hq79b   1/1     Running            3          14m
openebs       pod/maya-apiserver-5d9f6f96db-l7bqb                0/1     CrashLoopBackOff   9          14m

NAMESPACE     NAME                             TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)                  AGE
default       service/kubernetes               ClusterIP   <IP_ADDRESS>      <none>        443/TCP                  13h
kube-system   service/kube-dns                 ClusterIP   <IP_ADDRESS>     <none>        53/UDP,53/TCP,9153/TCP   13h
openebs       service/maya-apiserver-service   ClusterIP   <IP_ADDRESS>    <none>        5656/TCP                 14m
openebs       service/admission-server-svc     ClusterIP   <IP_ADDRESS>   <none>        443/TCP                  9m55s

NAMESPACE   NAME                         DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
openebs     daemonset.apps/openebs-ndm   3         3         3       3            3           <none>          14m

NAMESPACE     NAME                                          READY   UP-TO-DATE   AVAILABLE   AGE
kube-system   deployment.apps/coredns                       1/1     1            1           13h
openebs       deployment.apps/openebs-admission-server      1/1     1            1           14m
openebs       deployment.apps/openebs-snapshot-operator     1/1     1            1           14m
openebs       deployment.apps/openebs-ndm-operator          1/1     1            1           14m
openebs       deployment.apps/openebs-provisioner           1/1     1            1           14m
openebs       deployment.apps/openebs-localpv-provisioner   1/1     1            1           14m
openebs       deployment.apps/maya-apiserver                0/1     1            0           14m

NAMESPACE     NAME                                                     DESIRED   CURRENT   READY   AGE
kube-system   replicaset.apps/coredns-8655855d6                        1         1         1       13h
openebs       replicaset.apps/maya-apiserver-5d9f6f96db                1         1         0       14m
openebs       replicaset.apps/openebs-admission-server-7b897bfcdc      1         1         1       14m
openebs       replicaset.apps/openebs-snapshot-operator-546b77dc67     1         1         1       14m
openebs       replicaset.apps/openebs-ndm-operator-76d8945d4           1         1         1       14m
openebs       replicaset.apps/openebs-provisioner-64cd65d58f           1         1         1       14m
openebs       replicaset.apps/openebs-localpv-provisioner-6f7cd99684   1         1         1       14m

I get  a :
pi@pi1:~ $ kubectl logs pod/maya-apiserver-5d9f6f96db-l7bqb -n openebs
standard_init_linux.go:211: exec user process caused "exec format error"

And description of the pod gives :
pi@pi1:~ $ kubectl describe pod/maya-apiserver-5d9f6f96db-l7bqb -n openebs
Name:         maya-apiserver-5d9f6f96db-l7bqb
Namespace:    openebs
Priority:     0
Node:         pi2/<IP_ADDRESS>
Start Time:   Tue, 26 May 2020 11:41:25 +0200
Labels:       name=maya-apiserver
              openebs.io/component-name=maya-apiserver
              openebs.io/version=1.10.0
              pod-template-hash=5d9f6f96db
Annotations:  <none>
Status:       Running
IP:           <IP_ADDRESS>
IPs:
  IP:           <IP_ADDRESS>
Controlled By:  ReplicaSet/maya-apiserver-5d9f6f96db
Containers:
  maya-apiserver:
    Container ID:   containerd://c6397012b008f3439dc341091a3fa04dc3019d192bcbb6b6d333831e7518a43f
    Image:          quay.io/openebs/m-apiserver:1.10.0
    Image ID:       quay.io/openebs/m-apiserver@sha256:332bb8af16797ee017ef707b007e094dc799c02d69c1363194ccc9d7d3b8d667
    Port:           5656/TCP
    Host Port:      0/TCP
    State:          Waiting
      Reason:       CrashLoopBackOff
    Last State:     Terminated
      Reason:       Error
      Exit Code:    1
      Started:      Tue, 26 May 2020 11:56:51 +0200
      Finished:     Tue, 26 May 2020 11:56:51 +0200
    Ready:          False
    Restart Count:  10
    Liveness:       exec [/usr/local/bin/mayactl version] delay=30s timeout=1s period=60s #success=1 #failure=3
    Readiness:      exec [/usr/local/bin/mayactl version] delay=30s timeout=1s period=60s #success=1 #failure=3
    Environment:
      OPENEBS_NAMESPACE:                             openebs (v1:metadata.namespace)
      OPENEBS_SERVICE_ACCOUNT:                        (v1:spec.serviceAccountName)
      OPENEBS_MAYA_POD_NAME:                         maya-apiserver-5d9f6f96db-l7bqb (v1:metadata.name)
      OPENEBS_IO_CREATE_DEFAULT_STORAGE_CONFIG:      true
      OPENEBS_IO_INSTALL_DEFAULT_CSTOR_SPARSE_POOL:  false
      OPENEBS_IO_JIVA_CONTROLLER_IMAGE:              quay.io/openebs/jiva-arm64:1.10.0
      OPENEBS_IO_JIVA_REPLICA_IMAGE:                 quay.io/openebs/jiva-arm64:1.10.0
      OPENEBS_IO_JIVA_REPLICA_COUNT:                 3
      OPENEBS_IO_CSTOR_TARGET_IMAGE:                 quay.io/openebs/cstor-istgt-arm64:1.10.0
      OPENEBS_IO_CSTOR_POOL_IMAGE:                   quay.io/openebs/cstor-pool-arm64:1.10.0
      OPENEBS_IO_CSTOR_POOL_MGMT_IMAGE:              quay.io/openebs/cstor-pool-mgmt-arm64:1.10.0
      OPENEBS_IO_CSTOR_VOLUME_MGMT_IMAGE:            quay.io/openebs/cstor-volume-mgmt-arm64:1.10.0
      OPENEBS_IO_VOLUME_MONITOR_IMAGE:               quay.io/openebs/m-exporter-arm64:1.10.0
      OPENEBS_IO_CSTOR_POOL_EXPORTER_IMAGE:          quay.io/openebs/m-exporter-arm64:1.10.0
      OPENEBS_IO_HELPER_IMAGE:                       quay.io/openebs/linux-utils-arm64:1.10.0
      OPENEBS_IO_ENABLE_ANALYTICS:                   true
      OPENEBS_IO_INSTALLER_TYPE:                     openebs-operator-arm
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from openebs-maya-operator-token-x5z9k (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  openebs-maya-operator-token-x5z9k:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  openebs-maya-operator-token-x5z9k
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason                  Age                  From               Message
  ----     ------                  ----                 ----               -------
  Normal   Scheduled               <unknown>            default-scheduler  Successfully assigned openebs/maya-apiserver-5d9f6f96db-l7bqb to pi2
  Normal   Pulling                 15m                  kubelet, pi2       Pulling image "quay.io/openebs/m-apiserver:1.10.0"
  Normal   Pulled                  15m                  kubelet, pi2       Container image "quay.io/openebs/m-apiserver:1.10.0" already present on machine
  Normal   Created                 15m (x2 over 15m)    kubelet, pi2       Created container maya-apiserver
  Normal   Started                 15m (x2 over 15m)    kubelet, pi2       Started container maya-apiserver
  Warning  BackOff                 15m (x2 over 15m)    kubelet, pi2       Back-off restarting failed container
  Warning  FailedCreatePodSandBox  13m                  kubelet, pi2       Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "a4ba7f5c20f14aa6c18df53bf43fa5e47fcd0604125a48f31a9bff70392264dc": open /run/flannel/subnet.env: no such file or directory
  Normal   SandboxChanged          13m (x2 over 13m)    kubelet, pi2       Pod sandbox changed, it will be killed and re-created.
  Normal   Pulled                  9m56s (x4 over 13m)  kubelet, pi2       Container image "quay.io/openebs/m-apiserver:1.10.0" already present on machine
  Normal   Created                 9m56s (x4 over 13m)  kubelet, pi2       Created container maya-apiserver
  Normal   Started                 9m56s (x4 over 13m)  kubelet, pi2       Started container maya-apiserver
  Warning  BackOff                 80s (x48 over 11m)   kubelet, pi2       Back-off restarting failed container
pi@pi1:~ $

Can you help me ?
It seems that a sandbox pod is not able to run correctly, don't know why

"exec format error" is usually a signal that the binary in a container is for the wrong architecture, which in turn is usually a signal that the container in the registry does not have multiarch support.
quay.io/openebs/m-apiserver:1.10.0
Looking at
https://github.com/openebs/maya/blob/master/buildscripts/apiserver/Makefile.mk
it appears that the arm64 binaries are in a separate container in the registry. Quay for the longest time did not have arm64 support, but now it does.

the latest touch in that repo was from @kmova - are there any technical blockers to making a single multiarch image for m-apiserver  that would include both arm64 and amd64 files?

Thanks  it works !
Editing :
        image: quay.io/openebs/m-apiserver:1.10.0

To
        image: quay.io/openebs/m-apiserver-arm64:1.10.0

@vielmetti @armandleopold - the multi-arch support is under development. One challenge we have currently is with managing the dependencies on the CGO / gcc for some components like NDM. The development is being tracked under: #3023
@xUnholy is currently helping to sort this out. https://github.com/openebs/node-disk-manager/pull/428

@armandleopold The fix for maya apiserver arm image is taken in this PR

I have multi-arch k8s cluster.
2 nodes amd64 and 4 nodes arm64.
In deployment I can choose only one architecture, therefore, have issue with openebs on other arch.
For example, when I use "local pv hostpath" and "openebs-localpv-provisioner" deployed on amd64, it's not working on arm64 device as "linux-utils" defined for amd64:
Normal  Pulled     25s   kubelet, worker02  Container image "quay.io/openebs/linux-utils@sha256:c43eda3de479248100c85b37b123bb405226185003fe7f57c1011ccb46def110" already present on machine
...
init-pvc-42ade16b-92e3-4048-9fae-cfad0ca7e317                     0/1     Error               0          3s      <IP_ADDRESS>   worker02   <none>           <none>
...
➜  ~ kubectl logs init-pvc-42ade16b-92e3-4048-9fae-cfad0ca7e317 -n openebs
standard_init_linux.go:211: exec user process caused "exec format error"

Any way to deploy OpenEBS on multi-arch cluster now or need wait for multi-arch containers?
Any ETA? Thanks.

Bumping this issue which has gone stale, but @kmova has some earlier work on e2e pipelines for testing at https://github.com/openebs/e2e-tests/issues/405 which should be of interest.

@kmova @nsathyaseelan Are there updates on the e2e pipelines for ARM.?

Will this work also cover multi-arch for https://github.com/openebs/cstor-operators as well?  Currently seeing the exec format error messages for openebs-cstor-csi-controller, and openebs-csi-node after installing with kubectl apply -f https://openebs.github.io/charts/cstor-operator.yaml on an ARM64 cluster.


Ad Exchange Buyer APIs

Prepare for authorization
Complete the following steps to prepare for authentication using OAuth 2.0. The API supports many types of credentials; for this example we'll use a service account.

Go to the Google API Console Enabled APIs page.
From the project drop-down, select a project or create a new one.
In the list of Enabled APIs, make sure Ad Exchange Buyer API is listed. If it's not listed, click the Google APIs tab, search for and select the Ad Exchange Buyer API, and click Enable API.
Next, in the sidebar on the left select Credentials.
Select the Create credentials drop-down, then choose Service account key.
In the Service account drop-down, choose New service account.
Enter a Name for the service account. The service account ID is automatically generated from the name and project name.
Note the service account ID: you'll need it to grant access to the new service account in the Authorized Buyers UI in step 11.
Choose a Key type, either the recommended JSON file, or P12 if backward compatibility with code using P12 format is needed.
Click Create. The JSON or P12 file with the account's public/private key pair is saved to your Downloads folder. Keep the generated JSON or P12 file in a safe place.
You must grant the service account access in the Authorized Buyers UI for it to work. Select Settings > Account Settings, then choose User Management > Account Users, and click +Service Account. Enter the service account ID you noted above in step 8. This creates a new user with the service account role.

Source ;
https://developers.google.com/authorized-buyers/apis/guides/start#prepare-for-authorization

Thanks. What exactly are you trying to accomplish with this PR? It looks like it includes changes from an old development branch that are already implemented.


Uniform mesh deepcopy fix

Description

Addressing "deepcopy is extremely wasteful" statement in uniformMesh.py:::UniformMeshGeometryConverter::initNewReactor
Now creates a new, empty, reactor with same settings as sourceReactor.
A timing test (via timeit with 10 executions) shows that this new approach is about 10x faster than the deepcopy approach.
also updating docstrings for methods changed to be more verbose.

Checklist

[x] The code is understandable and maintainable to people beyond the author.
[x] Tests have been added/updated to verify that the new or changed code works.
[x] There is no commented out code in this PR.
[x] The commit message follows good practices.
[x] All docstrings are still up-to-date with these changes.

If user exposed functionality was added/changed:

[ ] Documentation added/updated in the doc folder.
[ ] New or updated dependencies have been added to setup.py.

@john-science @jakehader @ntouran

Just a small comment - Why do we need to copy k-effective? I can understand the library probably. Can we just remove that?

@jakehader , commit https://github.com/terrapower/armi/pull/645/commits/0087b552320ab97e28cb34551122c4d4e99e8a43 should cover everything in your initial review.

Just a note, after this PR our auto-docs failed to build. And our code coverage is no longer reporting correctly.
So, I am not sure what happened. I will try another build and see if it was a fluke.


Support writes to Azure Storage

Description
The delta-rs Azure Storage backend currently doesn't support write operations:
https://github.com/delta-io/delta-rs/blob/50a53cad084f2aa65ed0d6c0f1f5ad620a785530/rust/src/storage/azure.rs#L1
As a result, Delta Tables can not be written to Azure Storage.
Several methods need to be implemented:
https://github.com/delta-io/delta-rs/blob/50a53cad084f2aa65ed0d6c0f1f5ad620a785530/rust/src/storage/azure.rs#L192
https://github.com/delta-io/delta-rs/blob/50a53cad084f2aa65ed0d6c0f1f5ad620a785530/rust/src/storage/azure.rs#L196
https://github.com/delta-io/delta-rs/blob/50a53cad084f2aa65ed0d6c0f1f5ad620a785530/rust/src/storage/azure.rs#L200
The delta-rs Azure Storage backend currently uses the Blob API to access the storage account (ContainerClient), which does not support atomic rename (rename_obj_noreplace). We need to use the ADLS Gen2 API, which does support atomic rename.
Both APIs can be used in parallel with ADLS Gen2 storage accounts, but we may want to simply switch all operations over to the ADLS Gen2 API unless there is an advantage to using the Blob API for some operations.
The other operations required by the delta-rs Azure Storage backend are:
https://github.com/delta-io/delta-rs/blob/50a53cad084f2aa65ed0d6c0f1f5ad620a785530/rust/src/storage/azure.rs#L122
https://github.com/delta-io/delta-rs/blob/50a53cad084f2aa65ed0d6c0f1f5ad620a785530/rust/src/storage/azure.rs#L141
https://github.com/delta-io/delta-rs/blob/50a53cad084f2aa65ed0d6c0f1f5ad620a785530/rust/src/storage/azure.rs#L157
For atomic rename to work, the actual Azure Storage Account resource being used must have hierarchical namespaces enabled, which is what makes it an "ADLS Gen2" storage account.
Apologies for any remaining confusion, this is a bit tricky to explain.
Also, the Azure Rust SDK doesn't yet support write operations via the ADLS Gen API, but work is underway.
Use Case
Write Delta Tables to Azure Storage
Related Issue(s)
None

I am considering looking into this once (if :)) #486 gets merged. MY assumption would be that delete_obj and put_obj could be implemented without the need for hierarchical namespaces enabled. However I am still a bit confused what constitutes gen2... As far as I understand, the *.dfs.core.windows.net endpoint is available on all recent (v2) storage accounts, if HNS is enabled, it becomes ADLS Gen 2, and the file_system APIs / directories become available? If HNS is a requirement, likely #60 should then be addressed as well somewhere.

Thanks @roeap, that would be awesome. Your assumptions above are all correct. Only HNS supports atomic renames, which will obviate the need for the lock service that we have to use for S3.

PR for #60 is #492

@thovoll, thanks for the pointers, and excited to see that the azure-sdk-for-rust repo seems to see some increased activity recently - I'm guessing pushing for a release of the crates... hopefully we can bring the azure support throughout the arrow-rust-data-ecosystem en par with S3 :).

Keep track of ADLS Gen2 support in the Azure Rust SDK here: https://github.com/Azure/azure-sdk-for-rust/issues/496

do we know if the atomic rename in gen2 api is a cheap pointer change or will actually require data copy behind the scene? If it's the latter, we might be better off using simple put if absent semantic instead. @blogle started a good discussion around this redesign in slack for the GCS backend.

Yes, file rename is a cheap and atomic operation in ADLS Gen2.

FYI, we now have the required operation in the Azure Rust SDK that enables us to implement rename_obj_noreplace in the delta-rs Azure Storage backend:
https://github.com/Azure/azure-sdk-for-rust/pull/506
I know #486 is ready to merge now, so we will be able to start working on this soon.

I just want to let you know there is another 3rd party Azure SDK implementation here - https://crates.io/crates/azure_sdk_for_rust - and it has crates released.

I think these crates are actually the foundation for the development going on under the Azure github org right now. If you go to the repo, you'll see a notice that they migrated. Looking at the efforts in the new repo, they are working hard on stabilizing APIs and hopefully soon cutting a 0.1 release.

@roeap @houqp what do you think about the auth methods we should support?
After #486 we have:

Static auth: shared key, shared access signature (SAS), and connection string (shared key)
Auto-refreshing auth based on DefaultAzureCredential

None of these should be used in production applications in my opinion, for various reasons. The static methods present key rotation and authz challenges. DefaultAzureCredential is just a convenience abstraction with non-deterministic behavior.
I propose we use managed identity credentials (AMI). The downside is that they are hard to use during development, but we can support az cli credentials for this and activate that path using a suitable switch in delta-rs, maybe an environment variable.
Making these work at runtime requires an identity in Azure (which the key based methods don't), but this is the right direction since it better supports key rotation and authz.
I realize we should probably align this with how we do things for AWS and GCP, but I wanted to present this unadulterated argument first.
Any thoughts?

Thanks for starting this discussion @thovoll . I think we should definitely have first class support for AMI due to production requirements. @thovoll does Azure sdk support the concept of credential chain? Basically the way it works for AWS is the client will automatically identify the best auth method based on provided environment variable and credential config file across different languages and platforms: see https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/credentials.html and https://github.com/rusoto/rusoto/blob/master/AWS-CREDENTIALS.md. If we have something similar provided by the azure sdk out of the box, that would be ideal. If not, we can implement a temp workaround within delta-rs to simulate something similar. For example, go for AMI if it's available, if not try other routes that are easier to prepare for development environments.

The DefaultCredential we use is an credential chain. Only authorization via KEY or or SAS is not part of this chain. I am not sure about priorities in the chained credential, but assume this can be either configured or achieved with a simple custom credential.

Correct, default credential chain from the Azure Rust SDK can be configured but it's still rudimentary, also in terms of the methods it supports. There's no custom chain in the Azure Rust SDK yet but the Azure .Net SDK for example has more complete default and custom credential chains which will be implemented in the Azure Rust SDK at some point.
Regardless, my argument is that credential chains are convenient but shouldn't be used for production code. I'd rather explicitly state what credentials I want to use than falling through a hidden switch statement based on environmental conditions. The potential 'decoy' log entries alone are distracting.
If we only support a couple of types of credentials this should be pretty easy but if we want to support all the flavors it would be a little tedious to implement a deterministic approach - but I'd probably still prefer it. Since delta-rs is a library we should probably support all the credential types (but without using a chain).
The AWS docs that @houqp linked recommend using the default chain but also mention the other approaches:

Use the default credential provider chain (recommended).
Use a specific credential provider or provider chain (or create your own).
Supply the credentials yourself. These can be root account credentials, IAM credentials, or temporary credentials retrieved from AWS STS.

The Azure docs at least hint at the tradeoff:

Note: DefaultAzureCredential is intended to simplify getting started with the SDK by handling common scenarios with reasonable default behaviors. Developers who want more control or whose scenario isn't served by the default settings should use other credential types.

As for a concrete long-term proposal (long-term because the Azure Rust SDK doesn't fully support this yet):

Take TokenCredential as a parameter in delta-rs and pass it to the DataLakeClient (not possible yet but here is what that looks like in the Azure .Net SDK).
This still allows delta-rs users to specify a chain since DefaultAzureCredential is a TokenCredential. But it doesn't allow delta-rs users to use other methods like shared keys or connection strings (which the Azure SDK supports but we would be hiding from delta-rs users, indicating our industrial strength intent).
Maybe we should support SAS but this method is intended for "limited delegated access" (potentially a scenario that delta-rs would be used in).
Also need to revisit how we can auto-refresh the credential, which will likely involve constructing a new DataLakeClient at the right time.
In kafka-delta-ingest, don't pass DefaultAzureCredential to delta-rs but implement a more opinionated and deterministic approach. Only support AMI (for prod) and CLI (for dev), possibly supported by a command line argument or environment variable.

Again, this is a long-term proposal but we can still discuss this in principle. If we agree, we can plot a path towards this future.

As for a concrete short-term proposal:

Obtain a token (via DefaultAzureCredential) in delta-rs
Pass the token to DataLakeClient (this is the only method supported by the Azure Rust SDK at the moment)
This is the quickest way to enable writes to Azure Storage.
Improve from there, i.e. re-implement auto-refreshing credentials and everything else mentioned above over time.

It's definitely doable and I'm sure many people do the same in Azure. Irrespective of how common it is, what do you think about the argument otherwise?
For S3, I think you meant this one?
https://github.com/delta-io/delta-rs/blob/cba4e3d316c56c65dc1cb660dc7e6d380ddd3a6b/rust/src/storage/s3/mod.rs#L425-L429
I'll take a closer look and try to keep it close.
Do we support auto-refreshing credentials in S3 and GCP?

Yeah, sorry @thovoll , new_with is the method that I was trying to use as an example. Auto refresh of credential should be supported by cloud SDK if possible.

The way we implemented for the Azure credentials is actually heavily inspired by the internals of the rusto s3 crate 🙂.

Does the AWS and GCP SDK support auto-refresh?

Yes, I remember the AWS SDK has an auto refreshing credential provider builtin.

I'm looking into how to best implement auto-refresh.

Ok, so auto-refresh in the Azure Rust SDK will take some time, filed an issue here: https://github.com/Azure/azure-sdk-for-rust/issues/543
In the meantime, we'll have to implement auto-refresh in delta-rs.

@thovoll - this is done by now, right?


Unable to connect to mysql server.

getting same error on Server version: 8.0.18 MySQL Community Server - GPL
with caching_sha2_password


Support for Python 3.10+

Is support for newer python versions planned?

Thanks

@generall This project is very interesting and I was wondering if you are planning to continue development at some point. Or if there are any active forks?

Hello @simonhessner,
Unfortunately, we currently don't have such plans

Dnag, sorry to hear that, this looks perfect for me.  For starting a new project, does it makes sense to try and get python10 working and still use this, or would anyone suggest an alternative?  Google doesn't work well with "quaterion alternatives' since it picks up the dataframe idea of the word.


Create threshold module

Description

Create separate module for logic of checking threshold of scores

Closes #231
Changes

Create files for native and halo2 version of threshold module
Implement the native version

Can we add 1 or two tests?

I think converge and converge_rational are redundant. We just need to test the Threshold in isolation

@duguorong009 I added new test cases to help you understand how threshold checks work.

@duguorong009 I will merge once the print statements are removed

@duguorong009 I will merge once the print statements are removed

Ouch.
Yes, I will remove the print statements.


"ask-sdk-local-debug package not found" when running debugger. Packages are installed.

Describe the bug
When running local debugger, ASK Toolkit output warns
ask-sdk-local-debug package not found. Please install the ask-sdk-local-debug package from npm.
To Reproduce
cd <skill lambda>
npm install --save<EMAIL_ADDRESS>npm install --save-dev ask-sdk-local-debug

Follow this to install launcher:
https://developer.amazon.com/en-US/docs/alexa/ask-toolkit/vs-code-testing-simulator.html#prepare
Open skill in its own workspace. All folders appear present and correct
.ask
.git
.gitignore
.vscode
ask-resources.json
lambda
node_modules
package-lock.json
package.json
skill-package

I have tried being in the lambda folder and the top level folder. I have also tried creating a package.json at the package root (above lambda) and npm install-ing model and debug at that level too. It still thinks ask-sdk-local-debug package is not installed. Debug won't run, so it'll only ever debug using the remote lambda, which defeats the point.
Extension Logs
[info - 2021-02-07 18:12:34.960Z] Activating extension
[debug - 2021-02-07 18:12:34.962Z] Registering commands in the extension
[debug - 2021-02-07 18:12:34.966Z] Registering views in the extension
[debug - 2021-02-07 18:12:34.997Z] Calling method: ask.welcome
[debug - 2021-02-07 18:12:34.998Z] Calling method: welcomeScreen.getHtmlForView
[verbose - 2021-02-07 18:12:34.998Z] Calling method: checkGitInstallation
[verbose - 2021-02-07 18:12:34.998Z] Calling method: isGitInstalled
[verbose - 2021-02-07 18:12:34.998Z] Executing git command : git --version
[debug - 2021-02-07 18:12:35.007Z] Calling method: renderView, args:  [{"name":"welcomeScreen","js":false,"args":{"enabled":"checked"}}]
[debug - 2021-02-07 18:12:35.009Z] Calling method: interpolate, args:  [{"enabled":"checked"}]
[debug - 2021-02-07 18:12:35.011Z] Registering event handlers in the extension
[debug - 2021-02-07 18:12:35.012Z] Registering statusbar in the extension
[verbose - 2021-02-07 18:12:35.013Z] Updating profile icon in the extension
[debug - 2021-02-07 18:12:35.013Z] Registering treeviews in the extension
[verbose - 2021-02-07 18:12:35.019Z] Calling method: findSkillFolders
[verbose - 2021-02-07 18:12:35.418Z] Calling method: getOrInstantiateGitApi
[verbose - 2021-02-07 18:12:35.420Z] Calling method: findSkillFolders
[debug - 2021-02-07 18:12:35.421Z] Checking if extension version is latest
[debug - 2021-02-07 18:12:39.025Z] Calling method: ask.debugAdapterPath, args: [{"name":"Debug Alexa Skill (Node.js)","type":"node","request":"launch","program":"${command:ask.debugAdapterPath}","args":["--accessToken","${command:ask.accessToken}","--skillId","${command:ask.skillIdFromWorkspace}","--handlerName","handler","--skillEntryFile","/home/ubuntu/scripts/alexa/MyTalkingNewspaperV7/lambda/index.js","--region","EU"],"cwd":"/home/ubuntu/scripts/alexa/MyTalkingNewspaperV7/lambda","__configurationTarget":5,"__debuggablePatterns":["*.js","*.es6","*.jsx","*.mjs"],"__workspaceFolder":"/home/ubuntu/scripts/alexa/MyTalkingNewspaperV7"}]
[error - 2021-02-07 18:12:39.330Z] ask-sdk-local-debug package not found. Please install the ask-sdk-local-debug package from npm.

[info - 2021-02-07 18:12:39.856Z] skill package schema has been updated.
[verbose - 2021-02-07 18:12:39.858Z] Calling method: getSkillMetadataSrc
[verbose - 2021-02-07 18:12:39.858Z] Calling method: getAskResourceConfig
[info - 2021-02-07 18:12:39.860Z] SkillPackageWatcher is started
[debug - 2021-02-07 18:12:44.202Z] Failed to upload telemetry data.
[debug - 2021-02-07 18:12:48.473Z] Failed to upload telemetry data.

Expected behavior

Your Environment

ASK SDK for Node.js used: 2.22.4
Operating System and version: Ubuntu 20.04

Node.js and NPM Info

Node.js version used for development: 14.15.4
NPM version used for development:  7.5.2

launch.json
{
    // Use IntelliSense to learn about possible attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Debug Alexa Skill (Node.js)",
            "type": "node",
            "request": "launch",
            "program": "${command:ask.debugAdapterPath}",
            "args": [
                "--accessToken",
                "${command:ask.accessToken}",
                "--skillId",
                "${command:ask.skillIdFromWorkspace}",
                "--handlerName",
                "handler",
                "--skillEntryFile",
                "${workspaceFolder}/lambda/index.js",
                "--region",
                "EU"
            ],
            "cwd": "${workspaceFolder}/lambda"
        },
        {
            "type": "pwa-node",
            "request": "launch",
            "name": "Launch Program",
            "skipFiles": [
                "<node_internals>/**"
            ],
            "program": "${file}"
        }
    ]
}

@talkingnews , my hypothesis is that the node_modules folder in the project root is resulting in the ask.debugAdapterPath command to look for ask-sdk-local-debug package in the wrong folder. So lets start out with just deleting the node_modules, package.json and package-lock.json files at the root folder.

Hi @sattpat . Thank you for your good suggestion.
However, at first I did not have that extra package.json and node modules folder in the root. I only added that after it wasn't working. I can also confirm that the the node packages ar4 installed correctly in the modules folder

This is very interesting. Would it be possible for you to share the entire skill package as a zip?

Sure. In fact, I used ASK Toolkit for VSCode to create a brand new template skill and the result is the same.
I've zipped it complete with the node_modules so you can see the debugger installed correctly.
https://www.dropbox.com/s/c1859de4ka6vkab/VisualStudioTest.zip?dl=1
Many thanks.

Hi @sattpat - I have found the solution. In launch.json, change
"program": "${command:ask.debugAdapterPath}",
to
"program": "node_modules/ask-sdk-local-debug/dist/LocalDebuggerInvoker.js",
Now it works.
I am using remote development on VSCode - is it possible that the Toolkit has problems reading the debugAdapterPath variable?

So the command ask.debugAdapterPath is supposed to resolve the path to the local debug packages. Its interesting that the other vscode commands are working. Are you running this in a windows linux subsystem?

Hi @sattpat . Yes, it's very strange but I checked and double checked.
No wsl, this is just a straight Ubuntu 20.04 dev box.

Gotcha and just so I understand this correct, you are running the extension on a remote machine?

Correct. So, I am running vscode locally on windows, but the toolkit extension is running on the remote Linux host via the vscode remote extension. please let me know if I can send you any logs or screenshots that might help. I can even do a short video showing me toggling between the two bits of code and the log window output. In all other respects the toolkit works ok. It is just this one environment variable

Thanks @talkingnews . I will try to reproduce this issue and get back to you some time next week.

Please help!

Please help!

Hi @mengtongun, thanks for the sharing your screenshot. But I feel like this is a separate issue and not related to the current one.
Do you mind creating a new issue along with detailed information provided in the template and also steps to reproduce ?

Actually I changed to Ubuntu and everything is working fine for me now.


Bioinformatics QC page & simplfy analysis finalise

Minor Software Update
Description

Change the drop down where sample analyses were closed to have a series of drop downs so that less mistakes are made

Added send back button with breakglass
Close sample button now pops up a modal
Modal will do checks that used to happen on submit as a ajax calls, will only carry on to next step if the checks pass/ will limit options if they fail

Added a bioinformatics QC step before results are released

idea is that they can be auto uploaded here but only visible to scientists once QC is done
added permission so only bioinf team can do the QC
Added form to duplicate info from AutoQC, in future would like to automate this
Added bioinf QC form to simplify the bioinf QC step
This hasnt been fully tested so might be a bit clunky, might try to drop the patient name at the same time as deploying this to simplify further

Simplified worksheet status so it'll hopefully be quicker to display

Outstanding:

tests have been broken - fixtures changes and more model functions mean logic has moved, will need overhaul
Hasnt had really detailed testing, particularly bioinf QC bits

Justification
Not changing the logic, just how the database functions

Testing
Description :
Ongoing, see above, just an interim code review please
Data Location:
N/A

Checklist
< Place an X in the appropriate boxes, if not applicable state N/A >

[ ] Is the test data easily accesible?
[ ] Has the pipeline/software been run end to end by the tester?
[ ] Has the pipeline/software been code reviewed on github?
[ ] Has the pipeline/software version number been incremented?
[ ] Are all relevant doccuments updated e.g. SOP, ReadME, Pull request

Additional checklist for pipeline updates (if applicable)

[ ] Is the sensitivity or other key performance metrics unaffected by the changes?
[ ] If the environment changes has this been deployed and tested in place?
[ ] Has the pipeline version in the variables file been updated e.g. development > master

Doing this review as a comment per chunk of testing.
User testing - bioinformatics QC checks (note have not checked underlying code yet)
User Groups / Display

Users not in the QC group can see what needs bioinformatics QC but cannot click on to the runs
Users in the QC group can click on the runs and perform QC
COMMENTS/ISSUES
Add instructions to the docs on how to set up the QC user group locally

Performing QC Checks

I really like this section I think it's going to make failing more straightforward and querying better in the future
Have tried combinations of run pass all samples pass, run pass some samples fail, whole run fail
COMMENTS/ISSUES
Can't edit the patient name within the bioinformatics QC page (but losing it altogether might be the easiest way to solve this)
You can use the same AutoQC primary key for different runs  - do we want this or should we force it to be unique?
AutoQC URL needs changing: should be <IP_ADDRESS>:5000/run_analysis/
Samples aren't finalising properly from the bioinformatics QC checks. The status in the table changes from bioinformatics check 2 to bioinformatics QC fail, but when you go in to the sample you've still got the check interface open and can't download the report/XML

User testing - updated pass/fail interface (note have not checked any of the underlying code yet)
General Interface

Can't finalise at all without checking patient demographics and updating name
Can't pass RNA without checking all the fusions
Can't pass DNA without doing all the DNA checks
Send back check button works as expected

PASSING SAMPLES

If everything agrees after 2 checks it closes nicely
If last 2 checks agree after 3 checks it closes nicely
COMMENTS/ISSUES
Will finalise if user 1 and user 3 are the same person, or if user 2 and user 3 are the same person which means it can finalise with some variants only having been checked by 1 person

FAILING

The status of a sample awaiting a fail is now 'IGV check 2' and the button's less obvious - this might be the intended behaviour though?
2 fails works correctly
pass then fail then fail works correctly (yay!)
fail then pass then pass works correctly
fail then pass then fail then fail works BUT checker 3 and 4 can be the same person (see above - same thing with 2 checks rather than 2 checks by independent users)
scientists can't reopen a completed fail sample - i can't remember if we only added that functionality in on complete samples for a reason

COMMENTS/ISSUES

Overall very clean but would be good if it was 2 independent matching checks not just 2 matching checks
This is more of a GUI one - but would it be possible to make some of the tickboxes a little bit bigger? e.g. on the demographics page tickbox. Lots of dexterity required on a big screen!

My unit tests do work - don't know if that's concerning if they're meant to have broken


Question: override variables in plan

Can we override variables in terraform plan from jenkinsfile?
For example: Canary deployment with weighted targets.

Yes.  The way that we've done it on our teams is to store the weights externally, then pass them in as environment variables.  You can pass in any variable to terraform, by prefixing the name of the variable with TF_VAR_ - See: https://www.terraform.io/docs/commands/environment-variables.html
terraform-pipeline offers a few different plugins, to retrieve values from an external store, and then convert them to environment variables.  See: https://github.com/manheim/terraform-pipeline#credentials-and-configuration-management.  Eg: We've stored record weights in ParameterStore, and adjusted the weights and redeployed environments to shift traffic in a controlled fashion.
To put everything together more concretely:

Your terraform code could define a variable with a default weight.  That variable would be used to set the weight value for a Route53 record.  variable ROUTE_53_WEIGHT { type = String }
Your terraform-pipeline could use any plugin that will inject environment variables.  In this example, I'll use ParameterStoreBuildWrapperPlugin.  In ParameterStore, I'd create a key/value pair TF_VAR_ROUTE_53_WEIGHT, and set it to my desired weight for the environment that I want to deploy.
I'd run terraform.  ParameterStoreBuildWrapper will set an environment variable TF_VAR_ROUTE_53_WEIGHT=<weight>, and terraform-plan/terraform-apply will automatically set that value for the terraform variable ROUTE_53_WEIGHT in my code.

Thank you @kmanning


Don't compress the storybook Prismic images

We wondered if the compress attribute is non-deterministic generating slightly different images each time which in turn causes Chromatic to flag a diff.
Removing the compress attribute from the images we're using as mocks in Storybook to see if this helps.

We use a few more more images with that param in (just did a =compress search within Cardigan folder), would it be good to move them to use the mocks?

good point. I've just removed the attribute everywhere inline for now (can do a proper tidy in a separate piece of work)

Are we worried about format doing the same sort of thing?

Are we worried about format doing the same sort of thing?

I wasn't because I thought it would be jpeg all the way. But maybe that's not always true. We could try this and if it doesn't work try removing the format param as well?

We could try this and if it doesn't work try removing the format param as well?

Absolutely, go for it!


Test from Arti

Test from Arti

Can one of the admins verify this patch?

Can one of the admins verify this patch?

Can one of the admins verify this patch?


Changed displayed time format

Changes format to yyyy-mm-dd hh:mm:ss
See https://mushroomobserver.slack.com/archives/C040TH9FV/p1678670009303949 et seq.

I will merge and deploy this as soon as CI checks are complete.

Coverage: 93.578%. Remained the same when pulling 1497b27b79747511c133f54c549b5a5d48796675 on jdc-display_time-format into 8cd2cebe00c4e3db8120ea596cd6ffa494ae4789 on main.


Gradients in Grad Rollout

Hi,
For the gradients list in this fuction: https://github.com/jacobgil/vit-explain/blob/main/vit_grad_rollout.py#L9
do we need to reverse the gradients? Since the attention is accumulated in the forward path, while gradient is accumulated in the backward path. To multiply attention with gradient, we need to reverse the gradient: gradients = gradients[::-1].
What do you think? Thanks

I agree with @ilovecv
I check the address of the module using the following code and found that the order is reversed.


"Go To Symbol in Workspace" command provides inconvenient UX in some cases.

VSCode Version: Code 1.7.2 (7ba55c5860b152d999dda59393ca3ebeb1b5c85f, 2016-11-21T22:06:37.428Z)
OS Version: Windows_NT ia32 6.3.9600
Extensions:

Extension
Author
Version

vscode-file-peek
abierbaum
1.0.1

html-snippets
abusaidm
0.0.18

project-manager
alefragnani
0.12.2

path-intellisense
christian-kohler
1.1.0

githistory
donjayamanne
0.1.2

vscode-generate-getter-setter
DSKWRK
0.4.1

EditorConfig
EditorConfig
0.3.4

tslint
eg2
0.7.1

RelativePath
jakob101
0.11.0

docthis
joelday
0.3.5

Angular2
johnpapa
1.0.2

refactorix
krizzdewizz
0.3.0

debugger-for-chrome
msjsdiag
2.3.2

typescript-hero
rbbit
0.11.0

vscode-icons
robertohuertasm
4.0.5

vscode-terminal-here
Tyriar
0.0.5

gitblame
waderyan
1.3.0

"Go To Symbol in Workspace" command (CTRL+T) is one of the most useful commands used for navigation in workspace, but in some cases it can be not so useful as it could be. Here is a couple of examples (usually I use VS code for typescript projects):

When I run vs code and there are no opened files, CTRL+T doesn't provide any result. For new users of VS code it's a bit unobvious fact, that you should open some file before perform some navigation.
If I have several tsconfig.json files in my workspace (atleast two), it performs search only within nearest one relative to an editing file. So, I can't see any classes and interfaces from the first project if I'm editing a file from the second project.

I suppose that this a bit strange behaviour origins from Typescript Language Service (tsserver), but can these issues be resolved on VS Code side? Most of users expect that "Go To Symbol in Workspace" performs search through whole workspace folder and through all projects in this workspace.
Thanks for awesome editor, it's really great!

That a problem specific to TypeScripts WorkspaceSymbolProvider and how tsserver works

When I run vs code and there are no opened files, CTRL+T doesn't provide any result. For new users of VS code it's a bit unobvious fact, that you should open some file before perform some navigation.

This also confused me. I have a habit of closing all open files every now and then, but then CTRL+T wont work, so now I mostly use Go to file... (CTRL+P) to avoid this confusion all together.
I don't know how tsserver works, but wouldn't it be possible to use the project root folder as the tsserver root? Especially considering I have a tsconfig.json there.

When I run vs code and there are no opened files, CTRL+T doesn't provide any result. For new users of VS code it's a bit unobvious fact, that you should open some file before perform some navigation.

Indeed. All my dev team just tried the new VSCode version to check out if this broken feature has been repaired; but we was quite disappointed seeing that "Goto symbol in workspace" still doesn't return any result.
If I understand well, "Goto symbol in workspace" is only looking in opened files ? Maybe should it be renamed to "Goto symbol in opened files" instead ?

I would love to be able to search from any symbol from the project and have them appear first, before other NPM modules or external imports.

Years since first reported?
Still no commits?
How come, VScode Team?

I run into the same problem. cmd+t can only search in opened files and maybe some related files.

The same confusing... I expect that Go To Symbol in Workspace is not depend on opened files

@jrieken this is incredibly annoying but easy enough to work around. How do we fix this? What specifically is it that the tsserver does that is preventing this feature from working as if I had all my TS file opened in the first place? For the most time it does find stuff but whenever you switch workspace or restart VS code it's back to nothing.

Anyone looking into fixing this?

I would be happy to help if someone can point to the cause in the source.

has this been looked into, ever since?

This has been open for a while! Why can't I search for symbol when no file is open in workspace?

Not searching all projects is a duplicate of #11026 (fixed with TS 3.9)
Search requiring a file is a duplicate of #11173

@mjbvz does it work only with TS?
What's about python/es6 classes for example?

This issue (and the linked ones) are about JS/TS
If you see this same issue but for Python, please file an issue against the python extension


Ambiguity checker is unaware of run criteria (including States)

Bevy version
348e2a3d404bc61afe534371b100dc173f6d076f
Operating system & version
Irrelevant.
What you did
use bevy::ecs::schedule::ReportExecutionOrderAmbiguities;
use bevy::prelude::*;

#[derive(Clone, Eq, PartialEq)]
enum AppState {
    Menu,
    InGame,
}

#[derive(Default)]
struct Data;

fn main() {
    App::build()
        .add_plugins(DefaultPlugins)
        .add_state(AppState::Menu)
        .init_resource::<Data>()
        .insert_resource(ReportExecutionOrderAmbiguities)
        .add_system_set(SystemSet::on_update(AppState::Menu).with_system(menu.system()))
        .add_system_set(SystemSet::on_update(AppState::InGame).with_system(game.system()))
        .run();
}

fn menu(mut _data: ResMut<Data>) {}
fn game(mut _data: ResMut<Data>) {}

What you expected to happen
No ambiguities should be found.
What actually happened
A system-order ambiguity between menu_system and gameplay_system was found.
 * Parallel systems:
 -- "&bevy_scratchpad::game" and "&bevy_scratchpad::menu"
    conflicts: ["bevy_scratchpad::Data"]

Additional information
First reported here.

Obviously, the state case is the most important example of this. But it also occurs with other run criteria:
use bevy::ecs::schedule::{ReportExecutionOrderAmbiguities, ShouldRun};
use bevy::prelude::*;
#[derive(Default)]
struct Data;

fn main() {
    App::build()
        .add_plugins(DefaultPlugins)
        .init_resource::<Data>()
        .insert_resource(ReportExecutionOrderAmbiguities)
        .add_system_set(
            SystemSet::new()
                .with_system(never.system())
                .with_run_criteria(never_run.system()),
        )
        .add_system_set(SystemSet::new().with_system(always.system()))
        .run();
}

fn never_run() -> ShouldRun {
    ShouldRun::No
}

fn never(mut _data: ResMut<Data>) {}
fn always(mut _data: ResMut<Data>) {}

 * Parallel systems:
 -- "&bevy_scratchpad::always" and "&bevy_scratchpad::never"
    conflicts: ["bevy_scratchpad::Data"]

As @cart said, I think the correct answer for the short and probably medium term is to just hard-code the systems associated with each enum variant of a state / each "step" (e.g. on_enter, on_update) as being disjoint.
That said, this might automatically be fixed by a stageless approach (#1375), where we determine ambiguities via hard sync points instead (of which you should get between each state / step within a state).

That said, this might automatically be fixed by a stageless approach (#1375), where we determine ambiguities via hard sync points instead (of which you should get between each state / step within a state).

No, sync points have nothing to do with states. Sync points (dependencies, ambiguities...) are about execution order, states are about execution fact.
We could slightly-less-hard-code a fix for states: automatically define explicit execution order between systems with the appropriate run criteria. This is not that different, in the end, but it makes sure these relationships are reflected in the chached data instead of being simply hard-coded in the ambiguity checker. This won't reduce parallelization potential: systems in question are by definition not supposed to be executable at the same time.
This is solvable for states because we actually know the relationships systems with those run criteria will have. A general solution is impossible if we don't won't to overcomplicate the API by requiring users to provide relationship tables of some kind.
A form of "define stuff by doing things to a label" API (lets call that "label properties"?) could make this more elegant, but we shouldn't be considering that as a thing-that-could-happen yet. (Stageless and label properties are orthogonal.)

We could slightly-less-hard-code a fix for states: automatically define explicit execution order between systems with the appropriate run criteria. This is not that different, in the end, but it makes sure these relationships are reflected in the chached data instead of being simply hard-coded in the ambiguity checker. This won't reduce parallelization potential: systems in question are by definition not supposed to be executable at the same time.

I like this: it solves the problem and provides more useful information in other places we may want to display / work off of execution order.

What's the status of this now that we have Schedule v3? OnEnter/OnExit systems now live in separate schedules so there should be no ambiguities. And of course, OnUpdate no longer exists.

This is still technically true, but no longer a practical or fixable concern.

True. If we had some way of adding metadata to run conditions, we could hard-code in_state (and state_exists_and_equals) to mark systems as mutually exclusive with one another. We could add metadata by changing fn in_state() -> impl FnMut() to return an impl Condition wrapper type that includes the extra metadata.

We could maybe add a implicit set somehow for each in_state value and add ambiguous with for all those sets with each other. Might be tricky now that we removed the iterator over all the states.


Documentation for services and battery controls

These are surprisingly confusing (thanks GE).
In particular, "Eco" means two things:

The low-level toggle which controls a single register value (battery_power_mode)
The high-level toggle which controls multiple register values (battery_power_mode and enable_discharge)

Completed in 54fd99f697861abfda0d4e52185dc1d717f3da02


Changing image cycle button behavior

Changing behavior of the image cycle buttons so that they no longer roll over
from last to first or vice versa. Instead, there is no wrapping, but there
are extra buttons to jump to the first or last picture.

Coverage increased (+0.8%) to 56.14% when pulling e64d1b6ea8bbb9204bf811aba616bd286a0fe1cb on mtlynch:more-buttons into 7e9142b5fcfcc06389fc193e005cdaf49f205ab7 on JeetShetty:master.


Update banner to mention MIT license

Update built engine banner to explicitly call out that the build is under an MIT license.
I confirm I have read the contributing guidelines and signed the Contributor License Agreement.

I would prefer to actually insert the full license, because if the engine is used in the context of a website. There is no source tree.
I know this will add a few bytes to the file size, but is the proper way if I understand correctly.

@lucaheft If it's good enough for React, isn't it sufficient here?
<EMAIL_ADDRESS>

Welp i guess so. I'm no legal expert, but if they are doing it like that it's propably alright. Thanks for adding!


[Build] Use central package management  - missed a couple of projects

PR Details
Continuation of #2045, where a few projects were missed.

Thanks for the aftercare!


Allow more direct access to ObjectMapper or allow controlling its creation

We use Logstash Logback Encoder to log all requests to our API into Elasticsearch.
Our service (Spring-based) has an object mapper with several customizations: lots of feature toggles, custom modules, custom naming strategy, etc. and we would like to re-use that object mapper with all its configurations in our Logstash Logback Encoder. Although we can access Logstash Logback Encoder's object mapper via
... implements JsonFactoryDecorator {
    @Override
    public JsonFactory decorate(JsonFactory factory) {
        ObjectMapper objectMapper = (ObjectMapper) factory.getCodec();
        ...
   }
}

we would like to be able to replace it entirely or have full control over its creation. For example, re-use the object mapper we use for serializing and deserializing all requests and responses in Logstash Logback Encoder. Right now we have to keep two and make sure we configure them the same way.
Another use case is when we don't fully own the object mapper and get it via dependency injection from an interface if we're offering our service as a library, for example. In that case it's more difficult to make sure the object mapper used by Logstash Logback Encoder matches our global one. For example, in Spring we can just customize the object mapper as here or we can totally control its creation.
We're also doing it programatically (since we configure both Jackson and Logback programatically instead of via configuration files):
LoggingEventCompositeJsonEncoder encoder = new LoggingEventCompositeJsonEncoder();
encoder.setJsonFactoryDecorator(...);

Hi @joca-bt ,
You can return a different JsonFactory from a JsonFactoryDecorator with a different ObjectMapper configured.  As an example, you can take a look at the decorator that switches from JSON to Smile format.

We ended up replacing the object mapper used by the default json factory, this way we keep the customizations made by logstash (JsonGenerator.Feature.FLUSH_PASSED_TO_STREAM), and just swap the object mapper.


Unable to Download Retriever-Pretrained-Checkpoint or Training Data

Hi, I am attempting to replicate the remarkable achievement, but it appears that everything on Tsinghua Cloud has expired. I am unable to download the pre-trained checkpoint by simply clicking the URL in README.md or using the download.py script, nor can I access the training data. Can anyone help me? I would greatly appreciate it.

@alvis233 Hi,
Thanks for your interest in WebGLM. We are sorry that the cloud storage is experiencing a migration this morning, and it will soon be available. We will inform you once it is ready.

@alvis233
The download is up now.


syscfg to configure NFC pins to be GPIO

Break out a syscfg value to configure the NFC pins as GPIOs

@wes3 spoke to Sam, it does make sense. I do agree as well. Let's merge it ?

Yep; looks good!


improve doc for DragListener interrupt

DragListener and PressListener (the superclass) currently have identical documentation for interrupt:
* Interrupts the listener, releasing it (canceling behavior).

This is not helpful for DragListener.  Please include information specific to the drag behavior in  -- anything that we need to know that we can rely on happening when interrupt is called.  For example, I was interested in whether end is called.  It took me 45 minutes to determine "yes", but I feel like I'm relying on undocumented behavior.
Assigning to @ariel-phet to prioritize and assign.

Slack discussion

Chris Malley 11:12 AM
Scenery question for @jonathanolson or anyone else.  If interruptSubtreeInput is called while a DragListener drag cycle is in progress, is end callled? (I don’t have a multitouch device with me to test.)
I see that Node interruptSubtreeInput calls interruptInput for all Nodes in the subtree.  And interruptInput calls listener.interrupt for each listener.   DragListener interrupt chains to superclass PressListener interrupt.  And then I sort of lose it from there.
Chris Malley 11:23 AM
More generally, how can I guarantee that something is done when a drag cycle ends, regardless of how it ended?  I have a least 1 DragListener that will leave the sim in a bad state if interruptSubtreeInput doesn’t result in end being called.
Chris Malley 11:44 AM
The answer is apparently, yes, end is called when a DragListener is interrupted.  I verified by calling interruptSubtreeInput in the Nth call to drag, and observed that end is called.
Chris Malley 11:50 AM
And looking at DragListener code… Superclass PressListener interrupt calls release, which DragListener overrides to called options.end.  Hopefully I’m relying on intentional behavior here, since it’s not documented anywhere.  interrupt doc is identical in PressListener and DragListener:

Interrupts the listener, releasing it (canceling behavior).

It would be nice if the doc in DragListener said something about how this relates to the drag cycle (the “behavior”), and end specifically.

Is the above documentation change satisfactory, or is there more you would recommend?

Looks great, thank you.  Closing.


Cant set multidate

Is there a way i can select more than one date ?

No there's no way to select multiple dates with one picker


Removes https get accept header when using Ruby version < 3.0

This takes @lauratpa's original fix and adds tests — in this case easy to do since the calls that differ by Ruby version are all mocked.

Hello @swinner2 /  @fphilipe
I saw this PR is approved since Augus 22; any chance to get it merged and having a new release of premailer-rails before new year ? :pray:
Thanks for your help !


Backend - Event Creation - Store start and end times

allow start and end times for the event to including dates.

<add-to-calendar-button
name="Title"
options="'Apple','Google'"
location="World Wide Web"
startDate="2024-04-19"
endDate="2024-04-19"
startTime="10:15"
endTime="23:30"
timeZone="America/Los_Angeles"

https://add-to-calendar-button.com/use-with-react

********TESTING

these are being stored & returned back to FE appropriately :)


copy over events

Whenever we morph elements we should probs be copying over the events. From yo-yo:
// update-events.js
module.exports = [
  // attribute events (can be set with attributes)
  'onclick',
  'ondblclick',
  'onmousedown',
  'onmouseup',
  'onmouseover',
  'onmousemove',
  'onmouseout',
  'ondragstart',
  'ondrag',
  'ondragenter',
  'ondragleave',
  'ondragover',
  'ondrop',
  'ondragend',
  'onkeydown',
  'onkeypress',
  'onkeyup',
  'onunload',
  'onabort',
  'onerror',
  'onresize',
  'onscroll',
  'onselect',
  'onchange',
  'onsubmit',
  'onreset',
  'onfocus',
  'onblur',
  'oninput',
  // other common events
  'oncontextmenu',
  'onfocusin',
  'onfocusout'
]

// copy.js
  function copier (f, t) {
    // copy events:
    var events = opts.events || defaultEvents
    for (var i = 0; i < events.length; i++) {
      var ev = events[i]
      if (t[ev]) { // if new element has a whitelisted attribute
        f[ev] = t[ev] // update existing element
      } else if (f[ev]) { // if existing element has it and new one doesnt
        f[ev] = undefined // remove it from existing element
      }
    }
    // copy values for form elements
    if ((f.nodeName === 'INPUT' && f.type !== 'file') || f.nodeName === 'TEXTAREA' || f.nodeName === 'SELECT') {
      if (t.getAttribute('value') === null) t.value = f.value
    }
  }

How would you plan to make this customizable? Assembling a list of custom events to pass to the differ is a pain and it's pretty much impossible to maintain an authoritative list when you consider browser differences.
What do you think about initially iterating over an element once (even document.body) and getting a list of keys that match /^on/?
On Chrome that gets me:
["onblur", "onerror", "onfocus", "onload", "onresize", "onscroll", "onbeforeunload", "onhashchange", "onlanguagechange", "onmessage", "onoffline", "ononline", "onpagehide", "onpageshow", "onpopstate", "onstorage", "onunload", "onrejectionhandled", "onunhandledrejection", "onabort", "oncancel", "oncanplay", "oncanplaythrough", "onchange", "onclick", "onclose", "oncontextmenu", "oncuechange", "ondblclick", "ondrag", "ondragend", "ondragenter", "ondragleave", "ondragover", "ondragstart", "ondrop", "ondurationchange", "onemptied", "onended", "oninput", "oninvalid", "onkeydown", "onkeypress", "onkeyup", "onloadeddata", "onloadedmetadata", "onloadstart", "onmousedown", "onmouseenter", "onmouseleave", "onmousemove", "onmouseout", "onmouseover", "onmouseup", "onmousewheel", "onpause", "onplay", "onplaying", "onprogress", "onratechange", "onreset", "onseeked", "onseeking", "onselect", "onshow", "onstalled", "onsubmit", "onsuspend", "ontimeupdate", "ontoggle", "onvolumechange", "onwaiting", "onbeforecopy", "onbeforecut", "onbeforepaste", "oncopy", "oncut", "onpaste", "onsearch", "onselectstart", "onwheel", "onwebkitfullscreenchange", "onwebkitfullscreenerror"]

The only existing events not caught by that method:
["onfocusin", "onfocusout"]

Looks like you can copy over only the events being used by an element, but it requires looping over the elements keys. That may be faster than iterating over a laonger list of event types, but I am not sure. https://github.com/kristoferjoseph/nanomorph/blob/master/morph.js#L16

We could take a page out of the backbone playbook and allow the user to pass in an events array as an option.

I kinda like the initial approach you took for that; looping over ^on should be fast-ish I think hmmm

If it's only done on a single element and then cached there shouldn't be any meaningful perf difference. That assumes all elements are identical and caching document.body would get the same results as iterating through any other element. I'm not actually sure that's true, haven't tested it.

Hmm interesting thought. Not sure how we would go about doing that without holding onto a reference internally since update is handed two previously created nodes it seems like we would have to run this loop on every update otherwise.

@kristoferjoseph looping should be rare tho; if we implement morphdom's .isSameNode() trick then only the hot path will ever be looped for equal nodes only which even pessimistically isn't that many nodes.
Optimistically it should be fine I think; doing the existing approach of yo-yo we're already miles ahead of every rendering engine in terms of perf so I'm not too worried for real world perf, even though from a purist standpoint I wholly agree with what you're saying - maybe mark it as something to revisit later?

This should work now! :sparkles:


At OTAA, LoRa Server returns Unconffrimed data down to Confirmed data.

#Bug
When LoRa node sends unconfirmed data no reply is sent from server immediately, but after sending confirmed data from LoRa node LoRa server sends unconfirmed data.

Which looks valid to me as LoRa Server needs to acknowledge the confirmed uplink.

Please refer below attached LoRa_sequence.png ..

The sequence diagram is given, For ConfirmedUp we are not receiving ConfirmedDown Data rather we are seeing UnconfirmedDown.
Any reason for the attached Is there some delay
Attached the images that was taken on LoRaWAN packet decoder for each sequence.


chore(main): release 0.1.1

:robot: I have created a release beep boop
0.1.1 (2024-01-03)
Features

establishing initial version (f0e5522)
establishing initial version (#13) (701c02f)
project created (5bf4964)

Bug Fixes

fixes types; fixes reactivity (c9b438f)

Code Refactoring

adds comments to utils (#24) (59f18ce)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/oqx/qwik-table/releases/tag/v0.1.1 :sunflower:


connecting to HikVision cameras

Hi,
I've install Tensorflow lite using you very well detailed guide, thank you for that.
it is possible to connect to them hikvision my link i use within home assistant is
"http://admin:password@IP:PORT/ISAPI/Streaming/channels/301/picture"
I’ve tried using your steam.py but I keep getting an error.
I would really appreciate  your help

I managed to connect to my cameras using this: https://stackoverflow.com/a/52428164

Hi,
thank you for the reply.
can I please confirm.
you used the python-vlc to capture an image second and used the TFLite_detection_image.py?

I used OpenCV to access my IP cameras via their IP addresses, and I passed the image to the TF inferencing as per Edje's tutorial.


POST Activation changes

Change:

Created a new API endpoint /api/activation/ to create new activations

Test:

Send a POST request to /api/activation/ with the following request body:

{
   "id": 1,
   "name": "demo",
   "rule_set_file_id": 1,
   "inventory_id": 1,
   "extra_var_id": 1,
   "description": "demo activation",
   "execution_env_id": 1,
   "restart_policy_id": 1,
   "playbook_id": 1,
   "activation_status": "success",
   "activation_enabled": true,
   "created": "2022-08-22 23:15:53",
   "last_modified": "2022-08-22 23:15:53"
}

NOTE: The code assumes that you already have inventory, extra vars, restart policy, playbook and execution environment created.

@Dostonbek1 Since this is your first time contribution, an approval from @benthomasson is needed to run workflows, but please don't hesitate to run linters and autoformatters locally on your own. You can do it by calling task lint and task.format if you have the taskfile CLI tool configured.
Refer to the README for more details:
https://github.com/benthomasson/ansible-events-ui/blob/main/README.md#1-install-taskfile


Cannot restart rsyslog as per the article. Needs more clarity.

[Enter feedback here]
After issuing this command in the page: sudo systemctl restart rsyslog
getting an error called: Failed to restart rsyslog.service: Unit rsyslog.service is masked.
Troubleshooting information in this article for the same would be helpful
Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 99dac403-887a-c5e3-9130-3d395184c51d
Version Independent ID: 72d81e24-e046-d243-190e-fa8507e384f4
Content: Syslog troubleshooting on Azure Monitor Agent for Linux - Azure Monitor
Content Source: articles/azure-monitor/agents/azure-monitor-agent-troubleshoot-linux-vm-rsyslog.md
Service: azure-monitor
Sub-service: agents
GitHub Login: @guywi-ms
Microsoft Alias: guywild

@mimequake
Thanks for your feedback! We will investigate and update as appropriate.

@mimequake, Thanks for opening an issue in the Azure technical documentation repository. We use GitHub issues as the primary channel for customer and community feedback about the Azure documentation and as the current feedback is not related to Azure Doc feedback but more of OS related tech issue, I will close this issue for now. The following are some information that should help you in this regard though:
It seems that the OS masked this service to prevent it from starting when editing the conf file. I am not sure of the exact reason for it, but here is an overview of masking in Linux and how to unmask a service - https://en.linuxportal.info/tutorials/troubleshooting/how-can-we-manually-unmask-services-that-are-masked-service-is-masked-that-we-cannot-unmask-with-the-unmask-option-of-the-systemctl-command
I would suggest posting this query on the respective OS community (depending on the distro being used), so that the community experts there can help you, in case you are unable to unmask the service using the "unmask" option with systemctl.


more AS improvements

I added some improvements which potentially can do by LLVM/emscripten. Hope this lead to more closer result for AS also on Chrome

It looks like the Emscripten and AssemblyScript versions are running at the same speed on Chrome. I should sync up the other versions with the modified logic though.

Sure

btw try add -O3 for emcc


Add moment object to dateTime PropTypes

This eliminates the React warning when passing a moment object as the value of the dateTime prop.

Lost interest in this.


cargo-sweep: fix cargoSha256

Motivation for this change
Update cargoSha256.
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[x] Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Assured whether relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

Thanks! I get the same hash, but just in case @GrahamcOfBorg build cargo-sweep

@marsam Thank you!!


Fix : cine notif order

Description
Please explain the changes you made here.
Checklist

[ ] Created tests which fail without the change (if possible)
[ ] All tests passing
[ ] Extended the documentation, if necessary

There is a formatting issue


Can't find generators under Win32 (reopened)

https://github.com/yeoman/yo/issues/243

Please give details that are unique to your case here. The other issue was 1 year ago so probably unrelated.
Also FWIW, it's an issue with your own setup. We have a lot of other windows user who're not having issue and we haven't made a release in the past two weeks.

I'm sure it's probably my own set-up. I've discovered that grunt also can't find any of it's modules either. I have a feeling it has something to do with the path separator, but I can't be completely sure.
I installed on windows with choclatey perhaps that has something to do with it. Still trying to figure it out.

uninstalled node and all modules from my system and reinstalled the official MSI from the node home page, still getting the same issues.
not doing anything too strange:
yo doctor still says everything is good.
all modules installed locally to project
generator wont find any of the generators.
If it's my personal setup then I can't for the life of me figure out what's wrong.

Check your NODE_PATH value, and check if modules are installed in that folder.
If they're not, then you know what's wrong.

I know the node pat is set correctly, and yo isn't able to find then in either the local modules or the global
I've tried installing them in both places, and just one or the other
nothing

@elparkino, yes, it is. I can't reproduce your problem on Windows.


Improve loading of 3P scripts

🚀 Feature request
Command (mark with an x)

[ ] new
[x] build
[x] serve
[ ] test
[ ] e2e
[ ] generate
[ ] add
[ ] update
[ ] lint
[ ] extract-i18n
[ ] run
[ ] config
[ ] help
[ ] version
[ ] doc

Description
Loading third-party scripts in a suboptimal way can cause performance regressions. A typical example is adding an SDK as script tag directly to the page head, which will block the loading of first-party scripts and delay hydration/CSR if done incorrectly.
Describe the solution you'd like
Expand scripts the functionality already available in angular.json, allowing developers to configure different script loading strategies. Project Aurora identified three script loading strategies that we can reuse:

afterInteractive - this strategy would prioritize first-party scripts by executing third-party scripts after the page has been hydrated or client-side rendered.
beforeInteractive - execute third-party scripts before first-party scripts. This strategy is particularly important for loading polyfills. Since we already have polyfill.ts, I'm a little hesitant to include it in the Angular CLI as part of the first feature iteration. I'd suggest keeping it out of scope and collecting feedback in the meantime.
lazyOnload - lowest priority. We load these scripts and execute them in requestIdleCallback. Examples for libraries using this strategy for prefetching are quicklink, ngx-quicklink, and guess-js.

We'd sometimes need a hook after the script has been loaded and executed if we load it asynchronously (afterInteractive or lazyOnload). For example, rendering a payment dialog after we've downloaded a third-party SDK.
Using angular.json does not provide an evident approach that would let us accomplish this. A lower-level API that would provide the necessary functionality is to allows developers to specify id's in the script declaration. Using the id, developers will discover the script element in their code and hook to its onload event.
This way, the scripts property in angular.json would change from an array of strings or objects to an array of strings or objects with the following properties:

input - the source URL or path of the script. The CLI will perform different actions depending on whether input is a local path or a remote URL
strategy - afterInteractive, beforeInteractive, or lazyOnload
inject - specifies if the script should be injected or not. We preserve the current semantics of the inject property in the script object
bundleName - applicable for local scripts only. For external scripts, we can throw an error
id - an optional identifier that the developer can use to get a hold of the script and hook to its onload event

Suppose we find a string rather than an object for a given script. In that case, we can treat it as a script object with an input the specified string, strategy equal to beforeInteractive for the current major, or afterInteractive for the next major. Ideally, we'd want the default strategy to be afterInteractive to prevent folks from causing performance regressions, but since this would be a breaking change, we don't want to do this until v13.
In v13, we can migrate all the scripts from objects to strings using the beforeInteractive strategy.
Describe alternatives you've considered
Project Aurora considered using a script component for alternative frameworks. Their approach easier accommodates the onLoad callback use case but seems to fit less naturally in the Angular CLI model.

Based on our discussion today, I want to add a few more details.
Motivation
Currently, folks can accomplish the same by updating index.html manually and adding script references. This technique, however, doesn't guarantee that people will be following best practices. Many application developers add blocking scripts to the page head, which causes regression of CWV.
Additionally, best practices for loading scripts evolve, which means that to get the best from CWV, developers will have to be constantly following the progress of these low-level browser APIs.
Conformance
As part of the solution, we can introduce a conformance rule that will fail the build if folks are loading blocking scripts in the application head. Even though this would be helpful, we still don't provide a solution that aligns with the best set of practices for script loading.
Ideally, we'd want to have both:

Functionality for properly loading scripts with proper defaults
Conformance to ensure that index.html is free of blocking scripts

After some discussion with the Web SDK team, there are some concerns about not optimizing scripts which are directly authored in the index.html file. We'll need conformance tooling to enforce this anyways, so the main question is around how we want to evolve the angular.json script loading.
After some evaluation of the API, we're not entirely sure how this should continue to fit in to Angular tooling and how we should evolve the API over time. So we're planning to write an RFC to gather additional information about how devs currently use angular.json script loading to better understand how we can optimize those scripts and how script loading should fit into Angular tooling. @clydin offered to write an initial RFC proposal for this.

@mgechev, is this still something we want to do?


Error-highlighting not working

I am serving GraphiQL with express-graphql (version 0.6.11). I noticed that GraphiQL, in this setup, does not highlight errors as it does for example at http://graphql.org/swapi-graphql/. Specifically, even if I construct a query with wrong fields or pass arguments of the wrong type, GraphiQL does neither underline the wrong parts of the query in red, nor display error-messages when I hover over the erroneous parts. Here is an example, where I would expect wrongField to be highlighted:

This behavior is reproducible across different GraphQL schemas being served and across browsers (Chrome 61.0.3163.100 (Official Build) (64-bit) and Safari 11.0 (126<IP_ADDRESS>.7)). express-graphql loads the following versions of GraphiQL and support libraries:
<link<EMAIL_ADDRESS>rel="stylesheet">
<script src="//cdn.jsdelivr.net/fetch/0.9.0/fetch.min.js"></script>
<script src="//cdn.jsdelivr.net/react/15.4.2/react.min.js"></script>
<script src="//cdn.jsdelivr.net/react/15.4.2/react-dom.min.js"></script>
<script<EMAIL_ADDRESS>
I cannot find options to configure error-highlighting in express-graphql or in GraphiQL.
Any idea what might be going wrong?

I assume this could be related to #534...?!

The problem seems to be caused by express-graphql relying on version 0.11.2 of GraphiQL. Updating the version should fix the problem. I have filed a corresponding issue here: https://github.com/graphql/express-graphql/issues/396


Remove dead links

Maybe move these to another section to see if they have been moved or see if in archive.org?

I couldn't find any archives hosted on other sites, but I reached out to the authors individually. I'll update this as soon as I hear back.


split and customize bench rules for vtcomposite vs node-mapnik

Addresses #43
This PR attempts to create a more accurate comparison when benching locally between node-mapnik and vtcomposite, specifically with options params such as buffer_size and compression.
cc @springmeyer

@artemp These changes enable a more accurate comparison between node-mapnik and vtcomposite...! vtcomposite is significantly faster than previous benching because we were not calling getData on the resulting node-mapnik tile buffer, which is necessary to access the data and should be called with each "rule".
@springmeyer suggested the following architecture changes below to reduce the chance that bench differences are due to differences in the rules or script logic, which I will work on!

collapse to just one “rules.js” and one “bench.js” script
integrate node-mapnik and vtcomposite into the same script and have one or the other enabled via a command line switch

@millzpaugh this branch is looking great. Did a quick further review however and have a few extra thoughts. Things that would bring the two comparisons even more in-line:

The node-mapnik.bench.js has memory tracking, the vtcomposite.bench.js does not. This would only have an impact if we passed the --mem arg however. Nevertheless this should be sync'd. Or if you merge the two scripts into one (recommended), that would deal with the discrepancy as well.
We don't currently test passing in compressed tiles. It looks like only uncompressed tiles are passed. Accepting compressed tiles directly will more correctly mirror production use (and will likely give another performance edge to vtcomposite). Note: if we do add support for passing in compressed tiles then to be fair (fair=correctly mirror current production) we'd want to move the addDataSync calls with node-mapnik to addData async calls, so that the decompression is done in the threadpool (like the current implementation in tilelive-vector (https://github.com/mapbox/tilelive-vector/blob/1e6a7487b5f4938d7e307b70f1b8d1935aefda0a/backend.js#L113)

Just added this ticket to capture the decompression feature mentioned above.  https://github.com/mapbox/vtcomposite/issues/62 I'm hoping to add the decompression option in a separate PR and then merge this in.
@springmeyer if you could review, that would be great!


IOBWS 3.0 > Claims

The use of "Recreation" would at the very least raise questions by native English speakers as the correct term is "ReCreate". The endpoints, if they survive, should be renamed.

Has been fixed.