Libyang version to build latest netopeer2 and sysrepo

I have a question about what version of libyang to use to build the latest netopeer2 and sysrepo.
With version 112 I have errors building sysrepo
-fdebug-prefix-map=/workdir/build/tmp/sysroots/qemux86=   -m32 -march=i586  --sysroot=/workdir/build/tmp/sysroots/qemux86 -Wall -Wpedantic -std=gnu11 -Wno-strict-aliasing   -fPIC -o CMakeFiles/COMMON.dir/utils/xpath.c.o   -c /workdir/uap-sysrepo/src/utils/xpath.c
| /workdir/uap-sysrepo/src/common/sr_mem_mgmt.c: In function 'sr_msg_free':
| /workdir/uap-sysrepo/src/common/sr_mem_mgmt.c:508:28: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
|      sr_mem_ctx_t *sr_mem = (sr_mem_ctx_t *)msg->_sysrepo_mem_ctx;
|                             ^
| /workdir/uap-sysrepo/src/common/sr_protobuf.c: In function 'sr_gpb_req_alloc':
| /workdir/uap-sysrepo/src/common/sr_protobuf.c:372:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
|          msg->_sysrepo_mem_ctx = (uint64_t)sr_mem;
|                                  ^
| /workdir/uap-sysrepo/src/common/sr_protobuf.c: In function 'sr_gpb_resp_alloc':
| /workdir/uap-sysrepo/src/common/sr_protobuf.c:640:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
|          msg->_sysrepo_mem_ctx = (uint64_t)sr_mem;
|                                  ^
| ERROR: oe_runmake failed
| /workdir/uap-sysrepo/src/common/sr_protobuf.c: In function 'sr_gpb_notif_alloc':
| /workdir/uap-sysrepo/src/common/sr_protobuf.c:729:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
|          msg->_sysrepo_mem_ctx = (uint64_t)sr_mem;
|                                  ^
| /workdir/uap-sysrepo/src/common/sr_protobuf.c: In function 'sr_gpb_notif_ack_alloc':
| /workdir/uap-sysrepo/src/common/sr_protobuf.c:778:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
|          msg->_sysrepo_mem_ctx = (uint64_t)sr_mem;
|                                  ^
| /workdir/uap-sysrepo/src/common/sr_utils.c: In function 'sr_dup_datatree_to_ctx':
| /workdir/uap-sysrepo/src/common/sr_utils.c:762:19: warning: implicit declaration of function 'lyd_dup_to_ctx' [-Wimplicit-function-declaration]
|              dup = lyd_dup_to_ctx(root, 1, ctx);
|                    ^~~~~~~~~~~~~~
| /workdir/uap-sysrepo/src/common/sr_utils.c:762:17: warning: assignment makes pointer from integer without a cast [-Wint-conversion]
|              dup = lyd_dup_to_ctx(root, 1, ctx);
|                  ^
| /workdir/uap-sysrepo/src/common/sr_utils.c:764:13: warning: implicit declaration of function 'lyd_merge_to_ctx' [-Wimplicit-function-declaration]
|              lyd_merge_to_ctx(&dup, root, LYD_OPT_EXPLICIT, ctx);
|              ^~~~~~~~~~~~~~~~
| /workdir/uap-sysrepo/src/common/sr_protobuf.c: In function 'sr_gpb_internal_req_alloc':
| /workdir/uap-sysrepo/src/common/sr_protobuf.c:877:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
|          msg->_sysrepo_mem_ctx = (uint64_t)sr_mem;
|                                  ^
| /workdir/uap-sysrepo/src/common/sr_utils.c: In function 'sr_find_schema_node':
| /workdir/uap-sysrepo/src/common/sr_utils.c:2450:47: warning: passing argument 2 of 'lys_find_xpath' from incompatible pointer type [-Wincompatible-pointer-types]
|      struct ly_set *set = lys_find_xpath(NULL, node, expr, options);
|                                                ^~~~
| In file included from /workdir/build/tmp/sysroots/qemux86/usr/include/libyang/libyang.h:20:0,
|                  from /workdir/uap-sysrepo/src/common/sr_utils.c:36:
| /workdir/build/tmp/sysroots/qemux86/usr/include/libyang/tree_schema.h:1683:16: note: expected 'const char *' but argument is of type 'const struct lys_node *'
|  struct ly_set *lys_find_xpath(const struct lys_node *node, const char *expr, int options);
|                 ^~~~~~~~~~~~~~
| /workdir/uap-sysrepo/src/common/sr_utils.c:2450:53: warning: passing argument 3 of 'lys_find_xpath' makes integer from pointer without a cast [-Wint-conversion]
|      struct ly_set *set = lys_find_xpath(NULL, node, expr, options);
|                                                      ^~~~
| In file included from /workdir/build/tmp/sysroots/qemux86/usr/include/libyang/libyang.h:20:0,
|                  from /workdir/uap-sysrepo/src/common/sr_utils.c:36:
| /workdir/build/tmp/sysroots/qemux86/usr/include/libyang/tree_schema.h:1683:16: note: expected 'int' but argument is of type 'const char *'
|  struct ly_set *lys_find_xpath(const struct lys_node *node, const char *expr, int options);
|                 ^~~~~~~~~~~~~~
| /workdir/uap-sysrepo/src/common/sr_utils.c:2450:26: error: too many arguments to function 'lys_find_xpath'
|      struct ly_set *set = lys_find_xpath(NULL, node, expr, options);
|                           ^~~~~~~~~~~~~~
| In file included from /workdir/build/tmp/sysroots/qemux86/usr/include/libyang/libyang.h:20:0,
|                  from /workdir/uap-sysrepo/src/common/sr_utils.c:36:
| /workdir/build/tmp/sysroots/qemux86/usr/include/libyang/tree_schema.h:1683:16: note: declared here
|  struct ly_set *lys_find_xpath(const struct lys_node *node, const char *expr, int options);
|                 ^~~~~~~~~~~~~~
With version 174 I have errors building netopeer2:
/workdir/build/tmp/work/i586-poky-linux-musl/libnetconf2/1.0+gitAUTOINC+64bad2c9df-r0/git/src/session_client.c:293:5: error: unknown type name 'ly_module_clb'
ly_module_clb old_clb = NULL;
 ^~~~~~~~~~~~~

/workdir/build/tmp/work/i586-poky-linux-musl/libnetconf2/1.0+gitAUTOINC+64bad2c9df-r0/git/src/messages_server.c: In function 'nc_err_libyang':
/workdir/build/tmp/work/i586-poky-linux-musl/libnetconf2/1.0+gitAUTOINC+64bad2c9df-r0/git/src/messages_server.c:471:14: error: 'LYVE_INVALATTR' undeclared (first use in this function)
case LYVE_INVALATTR:
^~~~~~~~~~~~~~
Is there a single version of libyang that can build last week's releases of netopeer2 and sysrepo?

Hello,
towards the end, I think you are talking about the compilation of libnetconf2, not Netopeer2, right?
Anyway, I believe your actual problem is that your version of libnetconf2 is too old. I think so because of the  last error message:

/workdir/build/tmp/work/i586-poky-linux-musl/libnetconf2/1.0+gitAUTOINC+64bad2c9df-r0/git/src/messages_server.c: In function 'nc_err_libyang':
/workdir/build/tmp/work/i586-poky-linux-musl/libnetconf2/1.0+gitAUTOINC+64bad2c9df-r0/git/src/messages_server.c:471:14: error: 'LYVE_INVALATTR' undeclared (first use in this function)
case LYVE_INVALATTR:

I took a look at the commit history of libnetconf2 and found this commit from 2nd of March, This commit renamed LYVE_INVALATTR to LYVE_INMETA.
Your sources of libnetconf2 are old (they do not include this commit), but your version of libyang is current, therefore it contains the LYVE_INMETA instead of the old LYVE_INVALATTR identifier.
The solution is straightforward - update your libnetconf2 sources to current version, then recompile and install it and then you should be able to compile both Sysrepo and Netopeer2 without any problem.

Hi,
if you have the latest revisions of all the libraries (from either devel or master branches), and something fails to compile, please create an issue. If not, then update it and try again. Also, you can fairly easily check whether some symbols are defined in a library, so if it fails to compile, but the symbols are there, you have something wrong on your system.
Regards,
Michal

I was planning to create libyang release when other projects were released to have compatible "packages", but I forget about it. I'm going to create the release.

libyang now has release 0.12-r2


Canary deployments with Gateway API between namespaces

The Gateway API spec seems to expect each team to work on a single namespace, which feels a bit too rigid for certain use cases.
In my case, I would like to be able to do canary deployments that involve services that are not directly getting external traffic. This is a lot easier to do if you can do a different deployment in a second namespace, something like this:

kind: HTTPRoute
apiVersion: networking.x-k8s.io/v1alpha1
metadata:
  name: httproute1
  namespace: ns1
  labels:
    app: ecommerce
spec:
  gateways:
    allow: "All"
  hostnames:
    - "www.example.org"
  rules:
    - matches:
        - path:
            type: Prefix
            value: /
      forwardTo:
        - serviceName: service1
          port: 80
          weight: 50

kind: HTTPRoute
apiVersion: networking.x-k8s.io/v1alpha1
metadata:
  name: httproute2
  namespace: ns2
  labels:
    app: ecommerce
spec:
  gateways:
    allow: "All"
  hostnames:
    - "www.example.org"
  rules:
    - matches:
        - path:
            type: Prefix
            value: /
      forwardTo:
        - serviceName: service1
          port: 80
          weight: 50

This is not currently possible with the current spec and something that will be useful to have (to put an example, the NGINX Ingress controller allows to do this).
Similar, I am wondering if this other way to implement the same thing is allowed by the spec:

Basically, having a single HTTPRoute object, but that points to an ExternalService, pointing to the service in the second namespace
kind: HTTPRoute
apiVersion: networking.x-k8s.io/v1alpha1
metadata:
  name: httproute-external
  namespace: ns1
  labels:
    app: ecommerce
spec:
  gateways:
    allow: "All"
  hostnames:
    - "www.example.org"
  rules:
    - matches:
        - path:
            type: Prefix
            value: /
      forwardTo:
        - serviceName: service1
          port: 80
          weight: 50
        - serviceName: service1-external
          port: 80
          weight: 50

With service1-external looking like this:
kind: Service
apiVersion: v1
metadata:
  name: service1-external
spec:
  type: ExternalName
  externalName: service1.ns2.svc.cluster.local

We are currently discussion the addition of "inclusion" or "delegation" of HTTPRoute which may meet your criteria.
In your example, your HTTPRoute objects are the same -- how is this not the same as a simple traffic split?

    +-----------------+           +----------------+
    |                 |           |                |
    |  HTTPRoute   50%-------------> Service 1     |
    |              50%---------+  |                |
    +-----------------+        |  +----------------+
                               |  +----------------+
                               |  |                |
                               +-->> Service 2     |
                                  |                |
                                  +----------------+

Sure, the difference would be that service 2 is on a different namespace, to make it easier to do traffic splits on whole applications, when the service that changes is not the first one.
These would be the nginx configs:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: minimal-ingress
  namespace: ns1
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
  - host: www.example.org
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: service1
            port:
              number: 80

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: minimal-ingress
  namespace: ns2
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-weight: "50"
spec:
  rules:
  - host: www.example.org
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: service1
            port:
              number: 80

There are actually two active discussions about making this possible as a 1st class capability of the API. This is the proposal for cross-Namespace Route->Service (by @robscott) and this  is the proposal for cross-Namespace Route->Route (by @youngnick). Both are a bit early proposals but making steady progress.
Today there can be a cross-Namespace relationship between Gateways and Routes but not between any other resources. There are several other cross-Namespace relationships that are very useful though.

Gateway-Route is useful when merging different Routes to the same Gateway, different teams that want full control of their routing config but want to share the same load balancer
Route-Route is useful for Route delegation, or when a portion of the Route should be defined in one Namespace and sub-portion of it defined in another Namespace
Route-Service is useful for having routing configured in different Namespace where the app is deployed (your use-case)
Gateway-Secret is useful for when secrets shouldn't be accessed directly by the Gateway-owner. They can be referenced but not directly accessed.

The syntax could likely change and this is not in the API today, but your use-case could look something like the following. Here I've put the HTTPRoute in ns1 while the Services are in ns2 and ns3.
kind: HTTPRoute
apiVersion: networking.x-k8s.io/v1alpha1
metadata:
  name: cross-ns-route
  namespace: ns1
  labels:
    gateway: external-http
spec:
  hostnames:
    - "www.example.org"
  rules:
  - forwardTo:
    - serviceName: service1
      namespace: ns2
      port: 8080
      weight: 50
    - serviceName: service1
      namespace: ns3
      port: 8080
      weight: 90

@arapulido can I ask why you would like to do canaries between different Namespaces? Is this because it's simple to canary an entire stack because every service is using the same name within a Namespace?

Another good use cases is having my mail.ns Service and mail.google.com external service (ExternalName or similar), and all external services are defined in some external-services namespace or similar.

@arapulido can I ask why you would like to do canaries between different Namespaces? Is this because it's simple to canary an entire stack because every service is using the same name within a Namespace?

Yes, it makes things clearer and if you happen to have some service hostnames hardcoded in the code, it would still work


Move dUSD modules to Ardana dApps

We want the price-feeder and any other relevant modules copied to Ardana Apps such that each one gets a dev shell, tests, and a build.

@Geometer1729 Good point. Removed.


2.2 still requires .NET

I may be missreading the release notes here, but the tool is still prompting me to install .NET on win 10

Encountered the same issue


olm install - operatorhubio-catalog  Readiness probe failed

Bug Report

What did you do?

Install the  olm using the  "operator-sdk olm install"
What did you expect to see?
The pods in olm namespace in "ready" state.

What did you see instead? Under which circumstances?
The operatorhubio-catalog is not ready:
k get pods -n olm

->   describe pod operatorhubio-catalog-sxpwz  -n olm

Environment
Minikube  1.25.2
Operator type:

/language go

Kubernetes cluster type:

$ operator-sdk version

operator-sdk version: "v1.22.1", commit: "46ab175459a775d2fb9f0454d0b4a8850dd745ed", kubernetes version: "1.24.1", go version: "go1.18.3", GOOS: "linux", GOARCH: "amd64"
$ go version (if language is Go)

go version go1.18 linux/amd64
$ kubectl version

Client Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.4", GitCommit:"e87da0bd6e03ec3fea7933c4b5263d151aafd07c", GitTreeState:"clean", BuildDate:"2021-02-18T16:12:00Z", GoVersion:"go1.15.8", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.3", GitCommit:"816c97ab8cff8a1c72eccca1026f7820e93e0d25", GitTreeState:"clean", BuildDate:"2022-01-25T21:19:12Z", GoVersion:"go1.17.6", Compiler:"gc", Platform:"linux/amd64"}
Possible Solution

Additional context

@fredcsfaria we would need more context on the cluster which is being used. Usually this error occurs when there are not enough hosts available to schedule the pods on, or maybe when there is a networking issue with the node where the particular pod is being scheduled. Are there any logs from the pods which is being started but not healthy? Or any errors from the API server which can be used to dig into this?

I'm also seeing this behaviour:
$ kubectl get po

NAME                                READY   STATUS             RESTARTS   AGE
catalog-operator-7f6fdbc478-6b8j2   1/1     Running            0          110m
olm-operator-7f5fbcf9f9-bhljf       1/1     Running            0          110m
operatorhubio-catalog-v5w7b         0/1     CrashLoopBackOff   7          14m
operatorhubio-catalog-x2hbh         0/1     CrashLoopBackOff   19         49m
packageserver-7cf6d4dc5d-4w7xc      1/1     Running            0          109m
packageserver-7cf6d4dc5d-lr8qv      1/1     Running            0          109m

$ kubectl describe po operatorhubio-catalog-v5w7b
Name:         operatorhubio-catalog-v5w7b
Namespace:    olm
Priority:     0
Node:         node1/<IP_ADDRESS>
Start Time:   Tue, 09 Aug 2022 00:12:37 +0000
Labels:       olm.catalogSource=operatorhubio-catalog
              olm.pod-spec-hash=6bfddc89d8
Annotations:  cluster-autoscaler.kubernetes.io/safe-to-evict: true
              cni.projectcalico.org/podIP: <IP_ADDRESS>/32
              cni.projectcalico.org/podIPs: <IP_ADDRESS>/32
Status:       Running
IP:           <IP_ADDRESS>
IPs:
  IP:  <IP_ADDRESS>
Containers:
  registry-server:
    Container ID:   docker://abc02bb38b14eb1fd7b93f14a8209223eeb587d057d75ccefe8c919740309600
    Image:          quay.io/operatorhubio/catalog:latest
    Image ID:       docker-pullable://quay.io/operatorhubio/catalog@sha256:b6492a9fd6fc07ac0975939cd5828151e47b730eda130c75a109bcee3b93a969
    Port:           50051/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Tue, 09 Aug 2022 00:27:19 +0000
    Last State:     Terminated
      Reason:       Error
      Exit Code:    2
      Started:      Tue, 09 Aug 2022 00:21:31 +0000
      Finished:     Tue, 09 Aug 2022 00:22:07 +0000
    Ready:          False
    Restart Count:  8
    Requests:
      cpu:        10m
      memory:     50Mi
    Liveness:     exec [grpc_health_probe -addr=:50051] delay=10s timeout=5s period=10s #success=1 #failure=3
    Readiness:    exec [grpc_health_probe -addr=:50051] delay=5s timeout=5s period=10s #success=1 #failure=3
    Environment:  <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from operatorhubio-catalog-token-xrjtx (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  operatorhubio-catalog-token-xrjtx:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  operatorhubio-catalog-token-xrjtx
    Optional:    false
QoS Class:       Burstable
Node-Selectors:  kubernetes.io/os=linux
Tolerations:     node-role.kubernetes.io/master:NoSchedule op=Exists
                 node.kubernetes.io/not-ready:NoExecute op=Exists for 120s
                 node.kubernetes.io/unreachable:NoExecute op=Exists for 120s
Events:
  Type     Reason     Age                   From               Message
  ----     ------     ----                  ----               -------
  Normal   Scheduled  15m                   default-scheduler  Successfully assigned olm/operatorhubio-catalog-v5w7b to node1
  Normal   Pulled     15m                   kubelet            Successfully pulled image "quay.io/operatorhubio/catalog:latest" in 405.142183ms
  Normal   Created    14m (x2 over 15m)     kubelet            Created container registry-server
  Normal   Pulled     14m                   kubelet            Successfully pulled image "quay.io/operatorhubio/catalog:latest" in 361.608421ms
  Normal   Started    14m (x2 over 15m)     kubelet            Started container registry-server
  Warning  Unhealthy  13m (x7 over 14m)     kubelet            Readiness probe failed: timeout: failed to connect service ":50051" within 1s
  Normal   Pulling    13m (x3 over 15m)     kubelet            Pulling image "quay.io/operatorhubio/catalog:latest"
  Warning  Unhealthy  13m (x6 over 14m)     kubelet            Liveness probe failed: timeout: failed to connect service ":50051" within 1s
  Normal   Killing    13m (x2 over 14m)     kubelet            Container registry-server failed liveness probe, will be restarted
  Normal   Pulled     13m                   kubelet            Successfully pulled image "quay.io/operatorhubio/catalog:latest" in 514.682595ms
  Warning  BackOff    4m56s (x28 over 11m)  kubelet            Back-off restarting failed container

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.7", GitCommit:"132a687512d7fb058d0f5890f07d4121b3f0a2e2", GitTreeState:"clean", BuildDate:"2021-05-12T12:40:09Z", GoVersion:"go1.15.12", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.7", GitCommit:"132a687512d7fb058d0f5890f07d4121b3f0a2e2", GitTreeState:"clean", BuildDate:"2021-05-12T12:32:49Z", GoVersion:"go1.15.12", Compiler:"gc", Platform:"linux/amd64"}

I ran the following experiment, manually running the entrypoint of the container
$ docker run --rm -it --entrypoint=sh quay.io/operatorhubio/catalog:latest
sh-4.4$ /bin/opm serve /configs/
WARN[0000] unable to set termination log path            error="open /dev/termination-log: permission denied"

INFO[0048] serving registry                              configs=/configs/ port=50051

It took around 50 seconds for the serving registry log message to appear. I believe that is the source of the issue I am experiencing - the liveness/readiness probes are failing because it takes the pod too long to initialize and the pod gets stuck in a restart loop because the initialDelaySeconds for the probes is too short.

Looks like my issue will be fixed in the next release: This PR tackles the issue 👍 Thanks

Fixed by https://github.com/operator-framework/operator-lifecycle-manager/pull/2791

I'm still facing this issue
kubectl create -f \ https://raw.githubusercontent.com/operator-framework/operator-lifecycle-manager/master/deploy/upstream/quickstart/crds.yaml kubectl create -f \ https://raw.githubusercontent.com/operator-framework/operator-lifecycle-manager/master/deploy/upstream/quickstart/olm.yaml
`Events:
Type     Reason     Age                From               Message

Normal   Scheduled  75s                default-scheduler  Successfully assigned olm/operatorhubio-catalog-n2cp2 to aks-systempool-27289948-vmss00000c
Normal   Pulled     73s                kubelet            Successfully pulled image "quay.io/operatorhubio/catalog:latest" in 716.734417ms (716.844818ms including waiting)
Normal   Pulling    34s (x2 over 74s)  kubelet            Pulling image "quay.io/operatorhubio/catalog:latest"
Normal   Killing    34s                kubelet            Container registry-server failed liveness probe, will be restarted
Warning  Unhealthy  34s                kubelet            Readiness probe errored: rpc error: code = NotFound desc = failed to exec in container: failed to load task: no running task found: task d796b71bb52dc8516ad4f1b347934703c842b488d855fde20918bd1021c7172a not found: not found
Normal   Created    33s (x2 over 73s)  kubelet            Created container registry-server
Normal   Started    33s (x2 over 73s)  kubelet            Started container registry-server
Normal   Pulled     33s                kubelet            Successfully pulled image "quay.io/operatorhubio/catalog:latest" in 605.963458ms (605.976558ms including waiting)
Warning  Unhealthy  4s (x7 over 64s)   kubelet            Readiness probe failed: timeout: failed to connect service ":50051" within 1s
Warning  Unhealthy  4s (x5 over 54s)   kubelet            Liveness probe failed: timeout: failed to connect service ":50051" within 1s
`

I have the same issue.  I'm just trying to do a basic setup to get through a tutorial.  Microk8s in case it matters.
operator-sdk version: "v1.35.0", commit: "e95abdbd5ccb7ca0fd586e0c6f578e491b0a025b", kubernetes version: "v1.28.0", go version: "go1.21.11", GOOS: "linux", GOARCH: "amd64"
Normal   Started    3m55s (x2 over 5m36s)  kubelet  Started container registry-server
Warning  Unhealthy  37s (x30 over 5m26s)   kubelet  Startup probe failed: timeout: failed to connect service ":50051" within 1s


Include subject DID on follow and block deletion records

If possible, please include the subject DID in the records of follow and block deletions.
There are use cases where one wants to know who was unfollowed and who was unblocked.
At the time of the deletion record appearing in the firehose, the actual follow / block record has been deleted, and one cannot use the rkey to retrieve the subject DID any longer.

See https://github.com/bluesky-social/jetstream/issues/29


Support sub-package managers (e.g. pip)

Tl;dr : support pip specifications of packages as well

I received the following email:
I was wondering though if it's possible to include pip packages in the environment specification? For example, my environment.yml file looks like:
dependencies:
- python=3
- requests
- xlsxwriter
- pip
- pip:
    - dateparser==0.4.0
    - tweepy
    - ujson

Would be especially nice to be able to do fancy pip URIs such as git+https://github.com/...


Fix: Load Tutorial URLs with https

I don't think this is sufficient as it assumes that existing URLs already use https in projects, which they don't. What I think we need to do instead is compare the current page scheme to the scheme of the tutorial URL and then make the URL HTTPS if the current page is HTTPS (or if the original tutorial url was, but this is a no-op). This way if we roll out legacy.appinventor over http tutorials will continue to work without having to provide two different AIAs.
Sent from my iPhone

On Dec 19, 2020, at 08:56, Pavitra Golchha<EMAIL_ADDRESS>wrote:

You can view, comment on, or merge this pull request online at:
https://github.com/mit-cml/appinventor-sources/pull/2353
Commit Summary
Fix: Load Tutorial URLs with https
File Changes
M appinventor/appengine/src/com/google/appinventor/client/Ode.java (2)
Patch Links:
https://github.com/mit-cml/appinventor-sources/pull/2353.patch
https://github.com/mit-cml/appinventor-sources/pull/2353.diff
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

Would protocol-independent URLs work? //:appinventor.mit.edu

I think what we want is to use HTTPS if either the current page is HTTPS or the TutorialURL is HTTPS. If App Inventor is served over HTTPS, the tutorials will also have to be served over HTTPS else the browser won't load them. Like wise, if the tutorial URL is HTTPS then we can't necessarily count (and shouldn't) downgrade to HTTP just because App Inventor is served over HTTP.
I think your logic in the proposed change to the if statement is good, but we will want to apply the above logic to adjust the URL to compensate for the presence of HTTPS on the host page.

If I am not mistaken, then this should be the truth table for the behaviour you asked

Current page
Tutorial URL
Effective URL

HTTPS
HTTPS
HTTPS

HTTPS
HTTP
HTTPS

HTTP
HTTPS
HTTPS

HTTP
HTTP
HTTP

Right. It's effectively an inclusive or with HTTPS==true and HTTP==false


about nacos's namespace settings

Please answer these questions before submitting your issue.

Why do you submit this issue?
[ √ ] Question or discussion
[ ] Bug
[ ] Requirement
[ ] Feature or performance improvement

Question

about nacos config in version6.3, is it support namespace settings？current is 'public' as default.

Nacos for what? Cluster management or Dyanmic configuration?

I think both of them are not supporting namespace. If you want, the contribution is welcome.

ok，i know，thx.


bug: ion-router does not work with query params

Bug Report
Ionic version:
4.9.1
Current behavior:
I use ion-router component in stencilJs.
Here is ion-router config:
<ion-router useHash={false}>
    <ion-route url="/home" component="app-home-page" />
</ion-router>

I need to open /home page with query params (/home?p=1)
Navigation to /home?p=1 does not work. I tried to navigate using ion-router-link component:
<ion-router-link href={"/home?p=1"}>Home page</a>
and using ion-router component
const nav = document.querySelector('ion-router');
nav.push("/home?p=1")

In both cases there is an error console: route not found. But if I reload the page in browser then page is successfully opened.
Expected behavior:

Navigation to /home?p=1 must work.
Steps to reproduce:

In stencilJs add ion-router:

<ion-router useHash={false}>
    <ion-route url="/home" component="app-home-page" />
</ion-router>

try to navigate to "/home?p=1"

Same here, I have defined like this:
    render() {
        return (
            <ion-app>
                <ion-router useHash={true}>
                    <ion-route url='/home' component='app-home' />
                    <ion-route url='/home/profile' component='app-profile' />
                    <ion-route-redirect from='/' to='/home'></ion-route-redirect>
                </ion-router>
                <ion-nav />
            </ion-app>
        );
    }

When loading web page with /#/home/profile?age=21 it simply stays in /#/home.

Any updates on this bug? It's pretty weird that something straightforward and commonly used as a querystring is not supported by the ion-router.

I just tested this in<EMAIL_ADDRESS>and it navigates correctly and sets the query string.
This might have been fixed with eab3373 which, according to the changelog was released with 5.1.0,
Also see #21062
@mslourens Can you test it in 5.1+?

@tricki the issue has been solved in 5.1.1. I just tested it, the querystring gets passed along with the route.

@bubbleupdotnet Can you confirm this works and close the issue? Thanks.

Thanks for the issue. This was resolved in Ionic Framework v5.1.0.


Replace 'xvid' with 'H264' in filename

I am kinda picky about what my file names look like, so I made a feeble attempt to replace any instance of 'xvid' with 'H264', since the files will no longer use the xvid codec.  It will also remove the old extension coming before the new .mp4 extension.
I am not an experienced programmer. I just did some googling to find a few lines of code that could work.  So, there may be a more professional way of doing this.  Also, I don't use redis and queue my files, so I didn't test tasks.py, but I think it should work.

Thanks for putting this together. Now that most of ffproc has been rewritten, this isn't really applicable anymore.
I'll add the features you've implemented here to the to-do list in the readme, because there are a lot of good suggestions (renaming, log file, space difference especially)


Missing Domain layer entities for new deployment

Describe the bug
A clear and concise description of what the bug is.
To Reproduce
Steps to reproduce the behavior:

Stop devlake
Delete lake database from mysql
Start devlake
Missing domain layer entities

Expected behavior
Domain layer entities should be created automatically for new users
Screenshots

fixed


Site Infrastructure

Select, implement, and maintain the backing systems which accomplish Phase 2.

Frameworks
Our Code
CI/CD
Testing
Monitoring

This umbrella issue establishes DRI for this work and may be broken up into more tangible issues.

Done: #345 #267


Add additional output to stdout

I'm finding this output quite useful in working out at-a-glance what has
been deployed to where.

Thank you for your submission.


Fix up pubsub API docs to show returned 'next_page_token'.

Explicit paging is required for scalability.

Changes Unknown when pulling 0d5b1ce32ad26a4acc141f8054a526c9f370c51f on tseaver:pubsub-api_usage_fixup into ** on GoogleCloudPlatform:master**.

LGTM


Status?

Hi,
Is there any future updates to this application?

Yes, I intend to keep this project alive. I've just been very busy lately but will work on this very soon. I have actually started implementing the decryption functionality in the dev branch. Check #1 out.
Once that has been implemented, Crypter version 2.0 will be released (https://github.com/HR/Crypter/milestones/v2.0).
You are welcome to contribute and suggest any ideas for version 2.0

@HR I will do so

Hi @di48l069 , just implemented the decryption functionality. Should ship more updates and finalise everything in coming days for 2.0


question on zig-fn.zig stack usage

https://github.com/natecraddock/ziglua/blob/a3e518f950a89c24ca1b827d7513411acfb230f8/examples/zig-fn.zig#L46
Shouldn't the stack index be -1 instead of 1 here to get the return value of the previously called function?  I think if you had something else in the stack before the pcall, lua.toInteger(1) would return something else than the result of the adder function?
It now looks like it works only by accident, and might be a misleading example.  Unless I'm completely misunderstanding how return values get passed.

Oh good catch, thank you. In this case index 1 does work because there is nothing else on the stack, but it would be best to give an example of code that is more reliable and generally useful
I also see a few other misleading comments in this file (note about stable pointers from when it required an allocator pointer), so I'll clean it up soon.


Fixing ifNotNilDo: when loading Server

This requires fixing the version of MongoTalk to be able to run in Pharo7


Fail within if-block

Sadly master actually seems broken, but we missed that because of Travis's exit code wasn't correct.

It's a quick fix, I'll make a PR.


Update upload.py

s/must be have/must have the/

Merged, thanks @vaibhavsagar!

Always a pleasure to contribute to Zulip!


knative HelloWorld Serving Code Example

Deploy a clean microk8s snap deployment:
snap install microk8s --current
Enable DNS, Istio + kNative
sudo microk8s.enable dns istio knative
Deploy HelloWorld go service example from:
apiVersion: serving.knative.dev/v1alpha1 # Current version of Knative
kind: Service
metadata:
  name: helloworld-go # The name of the app
  namespace: default # The namespace the app will use
spec:
  template:
    spec:
      containers:
        - image: gcr.io/knative-samples/helloworld-go # The URL to the image of the app
          env:
            - name: TARGET # The environment variable printed out by the sample app
              value: "Go Sample v1"

The pods will create and fail due to below error:
Readiness probe errored: rpc error: code = Unknown desc = failed to exec in container: failed to start exec "fc02dba843b84f907e3054501f078791474de71dce1d68e37734af3ef30fcf22": OCI runtime exec failed: exec failed: container_linux.go:345: starting container process caused "apparmor failed to apply profile: write /proc/self/attr/exec: operation not permitted": unknown
Result from kubectl get ksvc is "RevisionMissing".
Digging further but has anyone else experienced this?

Ok I have done some further testing, this works with clean 18.04.3 LTS (bionic) however issue exists in 19.10 (eoan).

I'm getting a similar error attempting to exec -ti into a pod, something I did regularly during development while under minikube (attempting to switch). The pod has the status of "Running" and has a single container.
/snap/bin/microk8s.kubectl exec -t -i web-6d64dd86f-dzb6q  -- bash
error: Internal error occurred: error executing command in container: failed to exec in container: failed to start exec "78186bf219f66adc86f6255eb98bbbe4683f6875c58022feb6cdced38b30f594": OCI runtime exec failed: exec failed: container_linux.go:345: starting container process caused "apparmor failed to apply profile: write /proc/self/attr/exec: operation not permitted": unknown

@rbt can you please attach the tarball from microk8s.inspect. I would like to see the error in dmesg you are probably seeing.

1.16/stable: inspection-report-20191111_173728.tar.gz
After fiddling with the above, I realized some pods worked. I've narrowed it down to the below setting in the failing deployments. It's possible minikube was incomplete/buggy by allowing access and microk8s is working as expected; albeit with a terrible error message.

       securityContext:
         allowPrivilegeEscalation: false

Same issue on a fresh microk8s installation with any exec probe.
$ snap list microk8s
Name      Version  Rev   Tracking  Publisher   Notes
microk8s  v1.16.3  1079  stable    canonical✓  classic

$ cat /etc/os-release
NAME="Ubuntu"
VERSION="19.10 (Eoan Ermine)"

# ------------------------ >8 ------------------------
          readinessProbe:
            exec:
              command:
              - /ko-app/queue
              - -probe-period
              - "0"
# ------------------------ >8 ------------------------

$ kubectl describe pod
Events:
  Type     Reason     Age                 From               Message
  ----     ------     ----                ----               -------
  Normal   Started    70s                 kubelet, manny     Started container queue-proxy
  Warning  Unhealthy  69s                 kubelet, manny     Readiness probe errored: rpc error: code = Unknown desc = failed to exec in container: failed to start exec "66a30c11deb99fd1a8e6929c83e7d9e59512a506a8693766c7e3e873c6808c08": OCI runtime exec failed: exec failed: container_linux.go:345: starting container
 process caused "apparmor failed to apply profile: write /proc/self/attr/exec: operation not permitted": unknown
  Warning  Unhealthy  68s                 kubelet, manny     Readiness probe errored: rpc error: code = Unknown desc = failed to exec in container: failed to start exec "95a2a1c687a12900fa1f9bd0725971ff51418b99d9638ea38673f36240a46b86": OCI runtime exec failed: exec failed: container_linux.go:345: starting container
 process caused "apparmor failed to apply profile: write /proc/self/attr/exec: operation not permitted": unknown
  Warning  Unhealthy  67s                 kubelet, manny     Readiness probe errored: rpc error: code = Unknown desc = failed to exec in container: failed to start exec "8d7e2c2ae8a0e9161b70061a1c7bc2231c47dca2fe3e88a148a481e6379c6a20": OCI runtime exec failed: exec failed: container_linux.go:345: starting container
 process caused "apparmor failed to apply profile: write /proc/self/attr/exec: operation not permitted": unknown
  Warning  Unhealthy  66s                 kubelet, manny     Readiness probe errored: rpc error: code = Unknown desc = failed to exec in container: failed to start exec "9c4658f5c2352f41d335bddadb84341acc328ddfde5adc107160bb497f3fdc7e": OCI runtime exec failed: exec failed: container_linux.go:345: starting container
 process caused "apparmor failed to apply profile: write /proc/self/attr/exec: operation not permitted": unknown
  Warning  Unhealthy  65s                 kubelet, manny     Readiness probe errored: rpc error: code = Unknown desc = failed to exec in container: failed to start exec "a62f586b56f11a04828bb9b97652ce4da36a750cdb07a02d5b1d720288212018": OCI runtime exec failed: exec failed: container_linux.go:345: starting container
 process caused "apparmor failed to apply profile: write /proc/self/attr/exec: operation not permitted": unknown
  Warning  Unhealthy  64s                 kubelet, manny     Readiness probe errored: rpc error: code = Unknown desc = failed to exec in container: failed to start exec "de440bd965a761b7faf1dc17d668be728015cc885ef63c92f4e6bc9b4db6f925": OCI runtime exec failed: exec failed: container_linux.go:345: starting container
 process caused "apparmor failed to apply profile: write /proc/self/attr/exec: operation not permitted": unknown
  Warning  Unhealthy  63s                 kubelet, manny     Readiness probe errored: rpc error: code = Unknown desc = failed to exec in container: failed to start exec "ef93e3317c6cc9c28a133f9d6e3c89709314e740ecdff62bde8acf903f765bde": OCI runtime exec failed: exec failed: container_linux.go:345: starting container
 process caused "apparmor failed to apply profile: write /proc/self/attr/exec: operation not permitted": unknown
  Warning  Unhealthy  62s                 kubelet, manny     Readiness probe errored: rpc error: code = Unknown desc = failed to exec in container: failed to start exec "ed74b405332d2909a9acd5d6f6bd6e292959d23dd6bf3cc3be398e74775e416b": OCI runtime exec failed: exec failed: container_linux.go:345: starting container
 process caused "apparmor failed to apply profile: write /proc/self/attr/exec: operation not permitted": unknown
  Warning  Unhealthy  61s                 kubelet, manny     Readiness probe errored: rpc error: code = Unknown desc = failed to exec in container: failed to start exec "7f86e575600d371b92b14696c0e38b27164ef2730ae54d848e1ef9acd95b0f42": OCI runtime exec failed: exec failed: container_linux.go:345: starting container
 process caused "apparmor failed to apply profile: write /proc/self/attr/exec: operation not permitted": unknown

🗄 inspection-report-20191202_151718.tar.gz

I am also having this issue on fresh Microk8s

Im facing exactly the same issue regarding apparmor when starting the hello demo from knative (https://knative.dev/docs/serving/getting-started-knative-app/)
Readiness probe errored: rpc error: code = Unknown desc = failed to exec in container: failed to start exec "7612ded2f1b669ec2610e2f1fddd53fe39cf26c8f05c35c7626676fea1daf021": OCI runtime exec failed: exec failed: container_linux.go:345: starting container process caused "apparmor failed to apply profile: write /proc/self/attr/exec: operation not permitted": unknown
Are there already solutions for this issue or is a downgrade to 18.04.3 LTS (bionic) the only solution at the moment?

I am seeing the following in the dmesg logs (inspection-report attached above in the apparmor folder):
[ 2640.194427] audit: type=1400 audit(1575281926.659:229): apparmor="ALLOWED" operation="exec" info="no new privs" error=-1 profile="snap.microk8s.daemon-containerd" name="/ko-app/activator" pid=11686 comm="runc:[2:INIT]" requested_mask="x" denied_mask="x" fsuid=65532 ouid=0 target="cri-containerd.apparmor.d"

@joedborg, @jdstrand have you seen this before?

CC @anonymouse64

@ktsakalozos both the cri-containerd.apparmor.d and snap.microk8s.daemon-kubelet (which I couldn't find in the snap) AppArmor profiles are in complain mode, yet nothing gets raised in the audit logs.
aa-logprof doesn't suggest any addition either.

A new finding: if I disable all profiles with aa-teardown (or /lib/apparmor/apparmor.systemd stop), and let containerd reload the cri-containerd.apparmor.d profile (happens eventually), the container starts without any issue.
$ sudo aa-status
apparmor module is loaded.
1 profiles are loaded.
1 profiles are in enforce mode.
   cri-containerd.apparmor.d
0 profiles are in complain mode.
2 processes have profiles defined.
2 processes are in enforce mode.
   /go/bin/helloworld (14862) cri-containerd.apparmor.d
   /ko-app/queue (14917) cri-containerd.apparmor.d
0 processes are in complain mode.
0 processes are unconfined but have a profile defined.

Now comes the interesting part: if I reload all profiles with /lib/apparmor/apparmor.systemd reload and re-create the Deployment (Knative Service) from scratch, the container also starts.
$ sudo aa-status
apparmor module is loaded.
51 profiles are loaded.
13 profiles are in enforce mode.
   /sbin/dhclient
   /usr/bin/man
   /usr/lib/NetworkManager/nm-dhcp-client.action
   /usr/lib/NetworkManager/nm-dhcp-helper
   /usr/lib/connman/scripts/dhclient-script
   /usr/sbin/tcpdump
   cri-containerd.apparmor.d
   lsb_release
   man_filter
   man_groff
   nvidia_modprobe
   nvidia_modprobe//kmod
   snap.core.hook.configure
38 profiles are in complain mode.
   /snap/core/8268/usr/lib/snapd/snap-confine
   /snap/core/8268/usr/lib/snapd/snap-confine//mount-namespace-capture-helper
   /usr/lib/snapd/snap-confine
   /usr/lib/snapd/snap-confine//mount-namespace-capture-helper
   snap-update-ns.core
   snap-update-ns.microk8s
   snap.microk8s.add-node
   snap.microk8s.cilium
   snap.microk8s.config
   snap.microk8s.ctr
   snap.microk8s.daemon-apiserver
   snap.microk8s.daemon-apiserver-kicker
   snap.microk8s.daemon-cluster-agent
   snap.microk8s.daemon-containerd
   snap.microk8s.daemon-controller-manager
   snap.microk8s.daemon-etcd
   snap.microk8s.daemon-flanneld
   snap.microk8s.daemon-kubelet
   snap.microk8s.daemon-proxy
   snap.microk8s.daemon-scheduler
   snap.microk8s.disable
   snap.microk8s.enable
   snap.microk8s.helm
   snap.microk8s.hook.configure
   snap.microk8s.hook.install
   snap.microk8s.hook.remove
   snap.microk8s.inspect
   snap.microk8s.istioctl
   snap.microk8s.join
   snap.microk8s.juju
   snap.microk8s.kubectl
   snap.microk8s.leave
   snap.microk8s.linkerd
   snap.microk8s.remove-node
   snap.microk8s.reset
   snap.microk8s.start
   snap.microk8s.status
   snap.microk8s.stop
2 processes have profiles defined.
2 processes are in enforce mode.
   /go/bin/helloworld (2654) cri-containerd.apparmor.d
   /ko-app/queue (2704) cri-containerd.apparmor.d
0 processes are in complain mode.
0 processes are unconfined but have a profile defined.

If I do the same procedure after executing snap restart microk8s.daemon-containerd, the container fails to start again.
Somehow I believe the order in which AppArmor profiles are loaded matters here.

@ktsakalozos @joedborg, @jdstrand The culprit is the snap.microk8s.daemon-containerd profile. Containers start shortly after unloading it with sudo apparmor_parser -R /var/lib/snapd/apparmor/profiles/snap.microk8s.daemon-containerd.
The same profile can then be loaded (same command with -a flag) without any issue. A long as cri-containerd.apparmor.d is loaded first containers will start.

The comment about tearing down the apparmor policy from underneath microk8s could explain things. Put another way, when microk8s is started, it is supposed to be started under an apparmor profile (lenient with classic snap). microk8s starts up and eventually loads cri-containerd.apparmor.d and containerd is allowed to start containers (via runc) under this profile. If some of these are not in place (eg, something isn't being setup right on start), then weird things can happen. Eg, if containerd isn't setting up runc right to use it, then might see the 'no new privs' issue. The 'failed to apply profile' issue might be because cri-containerd.apparmor.d isn't loaded yet.
That said, this comment from @rbt is important:

After fiddling with the above, I realized some pods worked. I've narrowed it down to the below setting in the failing deployments. It's possible minikube was incomplete/buggy by allowing access and microk8s is working as expected; albeit with a terrible error message.

       securityContext:
         allowPrivilegeEscalation: false

https://kubernetes.io/docs/concepts/policy/pod-security-policy/#privilege-escalation tells us: "These options control the allowPrivilegeEscalation container option. This bool directly controls whether the no_new_privs flag gets set on the container process."
When nnp is invoked, it can't be revoked and it can complicate profile transitions that can block. OTOH, I don't know the order of operations, but it sounds like nnp is being applied before the profile transition to cri-containerd.apparmor.d, and the kernel is applying its heuristics to determine if that is ok and deciding it is not.

Sorry for not seeing this earlier but I think @jdstrand 's analysis is spot-on, nnp is indeed applied by runC before the apparmor profile transition to cri-containerd. I would recommended to not set that pod security setting

@jdstrand @anonymouse64 you both explained the case for the "no new privs" error, which is great and I thank you for the lengthy explanations, but it's not what this issue is about. This issue is about "exec" probes not working on microk8s. Please see my findings above.

Is it possible there are multiple cri-containerd.apparmor.d profiles? The microk8s snap seems to copy over the cri-containerd.apparmor.d profile and load that when containerd starts:
https://github.com/ubuntu/microk8s/blob/fd296d64b37c72621a2a2de6c0865b4d84cbdb94/microk8s-resources/wrappers/run-containerd-with-args#L14-L22
When you say it works above, perhaps that is because when containerd is already running and it is asked to start a new container with an apparmor profile, it will generate a default one which is different from the one that the snap is copying in.
All that being said, it would be great to see a diff between the profile that is in the snap and the one that containerd generates for itself.
@antoineco also note that the exec probe issue you are having could be due to the no new privs issue, because you are seeing a failure to transition to the profile, which can happen if containerd drops privileges because of that config, and then the profile it tries to transition to has more privileges. Are you certain that your containers don't have that allowPrivilegeEscalation: false setting set for your containers?

@anonymouse64 please ignore the first case where I unload everything, this was only the first step of my troubleshooting. The profile that I unload (an reload) is the one matching the systemd service, not the CRI profile loaded by containerd (the one that gets copied, as that you mentioned). At that point containerd is using the profile included with the snap, not its own defaults.
Yes, I'm certain the containers created by Knative Services do not contain any specific security context options, unless set explicitly by the user.

@anonymouse64 the diff between the cri-containerd.apparmor.d from the snap and the one included with containerd doesn't contain anything relevant:
33c33
<   deny /sys/firmware/efi/efivars/** rwklx,
---
>   deny /sys/firmware/** rwklx,
36,37d35
<
<   # suppress ptrace denials when using 'docker ps' or using 'ps' inside a container
39,41d36
<
<   signal (receive) peer=snap.microk8s.daemon-kubelet,
<   signal (receive) peer=snap.microk8s.daemon-containerd,

My bad, the queue-proxy sidecar has "allowPrivilegeEscalation": false defined, but this is controlled by Knative and can not be changed. Knative is a microk8s addon, so I would expect it to work out-of-the-box.

As an aside, the way I helped @joedborg handle this in the kubernetes-worker snap temporarily is to patch runC to never drop privileges. Perhaps microk8s needs to do that as well in order to work with knative.

@anonymouse64 where can I find the runC patches you suggested? By not allowing runC to drop privileges are there any restrictions/limitations to the workloads we can serve?

Besides, I'm still puzzled about the fact that simply reloading the snap.microk8s.daemon-containerd profile resolves the issue durably:
$ sudo apparmor_parser -R /var/lib/snapd/apparmor/profiles/snap.microk8s.daemon-containerd

Indeed, this immediately solved the problem for me as well.

$ sudo apparmor_parser -R /var/lib/snapd/apparmor/profiles/snap.microk8s.daemon-containerd

Indeed, this immediately solved the problem for me as well.

Please note that apparmor_parser -R removes the profile. To reload it, use -r.

Please note that apparmor_parser -R removes the profile. To reload it, use -r.

I was terribly sloppy in my quoting of Antoine above (which I've fixed).  I did execute -a after -R to re-add it.  Although perhaps -r would work too, and more conveniently.

Im facing the same problem

@ktsakalozos sorry I forgot to respond to your ping, were you able to sync with @joedborg about the runC patches ?

@ktsakalozos sorry I forgot to respond to your ping, were you able to sync with @joedborg about the runC patches ?

Yes, thank you.

The same issue applies to Ubuntu 20.04 (focal) and prevents us from upgrading to the fresh LTS version of Ubuntu

Apparently something changed between Linux kernel 4.15.0 (works) and 4.18.0 (doesn't work)

I am experiencing same issue after upgrading to 20.04.
Luckily, fix mentioned earlier with removing/adding apparmor seems to work for me.

Besides, I'm still puzzled about the fact that simply reloading the snap.microk8s.daemon-containerd profile resolves the issue durably:

This 'works' for you because when you remove the profile (-R), the containerd process is put under the 'unconfined' profile and adding (-a) the profile back does not reattach the existing process to the added profile (ie, it is still unconfined). Whenever you stop/start containerd, I would expect the issue to come back up.

When people see this issue, are there any security violations in the logs? Eg: 'journalctl --since yesterday |grep audit'.

@jdstrand unfortunately not: https://github.com/ubuntu/microk8s/issues/784#issuecomment-571704446

Ok, I tried to reproduce this with the instructions in the initial report and I see many denials related to no new privs when enabling the various services. While I didn't see the same errors others did when creating the helloworld-go pod (perhaps I didn't create it properly?), I can say for sure that unless containerd is handling nnp properly, the kernel nnp feature will sometimes prevent containerd from transitioning a container into the cri-containerd.apparmor.d profile (regardless of if apparmor claims it is ALLOWed). The fact that it works when removing the snap.microk8s.daemon-containerd profile very strongly supports this.
I believe this issue will be resolved when microk8s (classic) is updated to include https://github.com/ubuntu/microk8s/blob/feature/jdb/strict/patches/runc-patches/snap-runc-no-prctl.patch. @ktsakalozos and I discussed this last week actually and I know that it is planned to incorporate this patch, but I don't know about the timelines.

Hi all, thank you for your time, effort and patience. Could anyone provide feedback on the fix available through:
sudo snap install microk8s --classic --channel=latest/edge/runc-nnp

Many thanks.

@ktsakalozos, latest/edge/runc-nnp works for me

Will try to dig in over the weekend.

@giner any chance you gave it a run over the weekend? Thank you.

@ktsakalozos, I've done some tests. The original issue has gone however some of my workloads are failing when running on the mentioned version of microk8s and I haven't been able to find relevant logs or other helpful details so far.

@giner can you give me an example workload I could look at?

I'm trying to run this https://github.com/cloudfoundry-community/eirini-on-microk8s
vagrant up works well and all components are eventually up however cf push is failing as eirini is not able to create containers (don't remember the exact message)

The app specific error:
   2020-09-07T14:54:19.75+0000 [API/0] ERR Failed to stage build: Container 'opi-task-downloader' in Pod 'hello-myspace-fxm7f' failed: CreateContainerError

k8s events (errors and warnings only):
42s         Warning   Failed             pod/hello-myspace-fxm7f   Error: failed to get sandbox container task: no running task found: task 4e9c20a086b140b86d2836d0d43ac6ddf4fb53f983063b34d4b24ec339e20ecf not found: not found
35s         Warning   Failed             pod/hello-myspace-fxm7f   Error: cannot find volume "certs-volume" to mount into container "opi-task-downloader"

Thank you @giner . Any chance you could attach the microk8s.inspect tarball? I wonder if there are apparmor denials.

inspection-report-20200907_151848.tar.gz

@giner it is not so easy for me to verify the issue you report. A search on the error pointed to a fix on containerd https://github.com/containerd/containerd/issues/2392 . Is it possible to narrow down the steps needed to reproduce what you see?

Install latest virtualbox and vagrant, then:
git clone https://github.com/cloudfoundry-community/eirini-on-microk8s
cd eirini-on-microk8s
git checkout nnp
vagrant up
... wait for some time ...
There will be a few comands printed, just run them one after another
(wait for all containers to be up before running cf commands)

Note that the above requires at least 16G RAM and good internet connection

Hi @giner , everyone, just pushed a new snap under latest/edge/runc-nnp. It seems to work over here. Please, let me know if it works for you too:
sudo snap install microk8s --classic --channel=latest/edge/runc-nnp

Thank you

It seems that revision 1710 doesn't have the fix yet and 1722 has it. Is there a generic way to know which revision has a fix without installing it?
  1.19/stable:      v1.19.2         2020-09-26 (1710) 214MB classic
  1.19/candidate:   v1.19.2         2020-09-22 (1710) 214MB classic
  1.19/beta:        v1.19.2         2020-09-22 (1710) 214MB classic
  1.19/edge:        v1.19.2         2020-09-23 (1722) 214MB classic

Is there a generic way to know which revision has a fix without installing it?

Not without downloading and extracting the snap. You should expect the fix to land on the stable channel with v1.19.3

using the comme sudo snap install microk8s --classic --channel=latest/edge/runc-nnp fixes the "RevisionMissing" issue that I had, but I am now with an error when running microk8s kubectl get ksvc
NAME                URL                                            LATESTCREATED             LATESTREADY               READY     REASON
helloworld-python   http://helloworld-python.default.example.com   helloworld-python-8sqdh   helloworld-python-8sqdh   Unknown   IngressNotConfigured

Should I open new issue to track this specific error?


Request to re-adopt the right terms for master-slave relation between nodes

The lobby of americans biggots that are asking everywhere to change nomenclature from master-slave to something else wrongly assume to have the right to appropriate the terms and linking it only with the background they choose, in this case the slavery in the american culture.
But in other context those names aren't offensive at all, and they have no linking to that history. In the BDSM community the master-slave terms are used to indicate a consensual sexual behaviour in relationships between adult people, and we, masters and slaves of the BDSM community, find highly offensive to be associated with the american history of slavery.
In the same manner, in the software history master-slave has ever been used in a different meaning, a non-offensive one, and censuring that term only to follow a biggot american sub-culture is highly offensive for everyone that isn't in that culture and feel the appropriation of the terms like an abuse by them.

No!


test: Update the cloud-provider-oci image for e2e tests

What this PR does / why we need it:
Version 1.23.0 of CCM was just released. We need to stay up-to-date
https://github.com/oracle/oci-cloud-controller-manager/releases/tag/v1.23.0
Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #106

unit tests

e2e test

CCM specific test
• [SLOW TEST:994.952 seconds]
Workload cluster creation
/home/ubuntu/projects/cluster-api-provider-oci/test/e2e/cluster_test.go:50
  Cloud Provider OCI testing [PRBlocking]
  /home/ubuntu/projects/cluster-api-provider-oci/test/e2e/cluster_test.go:202
------------------------------
STEP: Tearing down the management cluster

Ran 1 of 22 Specs in 1089.704 seconds
SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 21 Skipped


using the BIDS-schema to simplify creating the dataset layout

I want the data structures to be formally described (in json/yaml) and not in the MATLAB code, that representation should be shared with pybids (MATLAB has support for JSON - see here

This would rely on using the BIDS schema that exists as a set of yml files for now and should therefore be converted to something that matlab can handle like JSON.
Questions

where to do the conversion and hosting of the JSON schema?

See other comments in the "wish-list" issue #60
https://github.com/bids-standard/bids-matlab/issues/60#issuecomment-713249223
https://github.com/bids-standard/bids-matlab/issues/60#issuecomment-715282521
https://github.com/bids-standard/bids-matlab/issues/60#issuecomment-717327122
https://github.com/bids-standard/bids-matlab/issues/60#issuecomment-717348527
https://github.com/bids-standard/bids-matlab/issues/60#issuecomment-717390428

matbids already has the BIDS scheme apart from the implementation.
It is a one-on-one port from the (older) pybids implementation and uses json files.
A suggestion is to maybe use the bids-validator json files.

For the record, here is list of BIDS schemas or 'pseudo-schemas':

official YML based schema in the specification repo with some doc HERE

regex based rules of the BIDS-validator are in several JSON files here

pybids has some rules encoded in several json files HERE to help with folder and filename parsing.

matbids has equivalent ones HERE

From my understanding and seeing that other tools will progressively try to rely only on the BIDS official schema, I think it makes sense to alreay start moving in that direction.
Question:

who (which repo) is in charge of converting and hosting a json version of the schema?
do we want to have an "interface" layer that is just in charge of reading the schema and passes its output to bids.layout or other functions? (Just thinking that this could be a way to keep bids.layout insulated from any changes happening at the schema level (and could also help with testing).

On converting the schema.
This can be done fairly simply with the python package ruamel
Virtual env (because everything is better with a virtual env)
virtualenv -p python3 convert_schema
source  convert_schema/bin/activate
pip install ruamel.yaml

Example to convert the anat.yml from the specs repo.
from ruamel.yaml import YAML
import json

in_file = 'src/schema/datatypes/anat.yaml'
out_file = 'anat.json'

yaml = YAML(typ='safe')
with open(in_file) as fpi:
    data = yaml.load(fpi)
with open(out_file, 'w') as fpo:
    json.dump(data, fpo, indent=2)

Gives:
[
  {
    "suffixes": [
      "T1w",
      "T2w",
      "T1rho",
      "T1map",
      "T2map",
      "T2star",
      "FLAIR",
      "FLASH",
      "PD",
      "PDmap",
      "PDT2",
      "inplaneT1",
      "inplaneT2",
      "angio"
    ],
    "extensions": [
      ".nii.gz",
      ".nii",
      ".json"
    ],
    "entities": {
      "sub": "required",
      "ses": "optional",
      "run": "optional",
      "acq": "optional",
      "ce": "optional",
      "rec": "optional"
    }
  },
  {
    "suffixes": [
      "defacemask"
    ],
    "extensions": [
      ".nii.gz",
      ".nii",
      ".json"
    ],
    "entities": {
      "sub": "required",
      "ses": "optional",
      "run": "optional",
      "acq": "optional",
      "ce": "optional",
      "rec": "optional",
      "mod": "optional"
    }
  }
]

FYI: there is a kanban on the specs repo about anything scheme related.
https://github.com/bids-standard/bids-specification/projects/4
This includes a PR to cover derivatives in the schema.
https://github.com/bids-standard/bids-specification/pull/645

One issue that is not addressed by this though is how do we deal with things that are in the BIDS-BEP pipeline but not yet in the schema.
For example I have to deal with MP2RAGE images that are in BEP001 but not yet in the schema.
Do we want to a sort of "fall back" solution when we are dealing with a dataset that is not "vanilla BIDS".
Should we then rely on a JSON file to define "patterns" to deal with those cases a bit like pybids does at the moment for the regular stuff.

fixed by #124


Add prefilled id column when creating a new table

Since most tables will have an id column, now when creating a new table, it will have one prefilled to avoid adding it manually.

that's a good one :D


Weekly Report Milestone - 2024-11-07

What did you do this week?

This week I created the merge request for the API implementation and had my code reviewed and merged into main. Next week I will be working on the frontend UI a bit more.

This week I implemented feathers into my JSON handling file and started thinking about the deployment process of our application. This week, I plan on testing our feathers API to extract data from the database and display it on the front end.

This week I started finding the best way to make new script files to replace the Powershell scripts. This is so we can run them on Linux.


Fix typo in line 43, "se" to "she"

Hi @dtriana, I'm your friendly neighborhood Azure Pull Request Bot (You can call me AZPRBOT). Thanks for your contribution!

This seems like a small (but important) contribution, so no contribution license agreement is required at this point. Real humans will now evaluate your PR.

TTYL, AZPRBOT;

@dtriana : Thanks for your contribution to the Azure documentation! The author, @viv-liu, has been notified to review your proposed change.

#sign-off


[Improvement] Rename some package named with ‘logs’

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Search before asking

[X] I have searched in the issues and found no similar issues.

What would you like to be improved?
Usually ，the directory name ‘logs’  is included in the file ‘.gitignore’ .  Kyuubi contians some package names with logs , for example , ’org.apache.kyuubi.jdbc.hive.logs‘ and ‘org.apache.hive.beeline.logs’ . Some git operations will lose processing of files in this package.
How should we improve?
Rename package name with 'logs'  will avoid some misoperations and offer some convenience. For example , maybe we can rename 'logs' to 'logutils'.
Are you willing to submit PR?

[X] Yes I am willing to submit a PR!

@yulongz, IMO, you could update the .gitnore for above, not rename the package. cc @yaooqinn

@SteNicholas  Whether the file ‘https://github.com/apache/incubator-kyuubi/blob/master/.gitignore’ should contain the line 'logs/'?

@yulongz, you could add the path to the .gitignore.

@yaooqinn, this issue has no response. IMO. this could be closed.

+1 to close this，free free to open it if youthink this is still an issue that needs to be fixed


Update release notes for 1.2

Create/update release notes for 1.2

Switch the release notes around to make 1.2 current


Premint - if no mint recipient, default to msg.sender

Current dependencies on/for this PR:

main

PR #310

PR #305

PR #307
PR #262

PR #299

PR #312   👈

PR #289

PR #303

PR #309

PR #294

PR #295

PR #273

PR #288

PR #284

PR #280

This stack of pull requests is managed by Graphite.

Just trying to understand the motivation for this pr.

In case it's not set by accident

hmm i wonder if its better to leave this explicit and let it revert if a recipient isn't passed

Merge activity

Nov 15, 2:48 PM: @oveddan started a stack merge that includes this pull request via Graphite.


Move/Remove the Great Prophet panel in GP view to the last slot

 Issue by chaorace
Friday Nov 04, 2016 at 23:53 GMT
Originally opened as https://github.com/CQUI-Org/cqui/issues/40

Barring the very early game, great prophets don't really bear any relevance to someone visiting the GP view. Because of this, it makes sense to move the great prophet panel to the last slot on the GP view... and to possibly even hide it entirely once all available great prophets have been created

 Comment by jacks0nX
Saturday Nov 05, 2016 at 16:55 GMT

Moved:

Removed when religion already founded:

 Comment by chaorace
Saturday Nov 05, 2016 at 17:15 GMT

Yep, that looks pretty good!

 Comment by zyxpsilon
Monday Nov 07, 2016 at 17:09 GMT

I'll be adding a new mockup structure (with precise calibration details) for each Panels.. soon. But we'll still have to wait for the SDK assets to adapt their modal principles to our needs, i gather. ;)

 Comment by zyxpsilon
Monday Nov 07, 2016 at 20:34 GMT

Soooo.. here's that new structure -- although i had explained it all in another "post" that was deleted or that i simply couldn't find from lack of GitHub experience, it should be obvious how this could spare plenty of pixels on the whole framework.

 Comment by jacks0nX
Monday Nov 07, 2016 at 21:14 GMT

Looks great, this definitely is more pleasing to see than the current solution, in my opinion. Would maybe still include a "Progress" above the progress bars though, looks kind of empty down there.

 Comment by zyxpsilon
Tuesday Nov 08, 2016 at 14:06 GMT

Yep.. maintaining the terms "PROGRESS" and "With Patronage" above the costs duo would be preferable too i guess.
The trick i gather will be to calibrate the Modal dispatch and possibly create a new DDS image (BLP packaged) to fit the narrower framing -- and for that, we'd need the SDK for sure. Unless you'd have some other magic indirect solution.
Don't forget the big round GP faces can also be discarded or re-sized if need be. IIRC, the glower effects (top banners & GP circles) are just thrown individually as well.
Good luck! :)

 Comment by jacks0nX
Tuesday Nov 08, 2016 at 19:11 GMT

Opened a new issue for the redesign.

 Comment by vttale
Thursday Nov 17, 2016 at 01:00 GMT

Do the purple dots above the recruit button signify anything?  If not, should they just go away?  If so, should they be more clear about what they represent?


Segregated multi-site support

The current multi-site support in Wagtail assumes that all sites are for the same organization.  If I wanted to create sites for multiple organizations of the same type (for instance, Chambers of Commerce) on a single Wagtail installation, then the images and documents uploaded by one organization are visible to all.  Also, the pages of all organizations are visible to everyone else in the Wagtail Explorer.  Creating multiple sites for organizations of the same type on a single Wagtail installation would provide the capability to view statistics across multiple organizations and to promote cross-organization communication among members (and probably other interesting things as well).  Please consider implementing segregated multi-site support in which users who are assigned editing permissions per site can only view the pages and assets of that site.

Looks like Django sites framework's CurrentSiteManager might be useful.  Perhaps the requested functionality could simply be implemented as an option in Settings.

Hi @DanAtShenTech - thanks for the suggestion.
This is indeed something we're keen to implement, and there are various strands of work going towards this (#2869, #4206, #4207) - although it looks like we don't currently have an open issue for the feature as a whole since #2463 was closed, so I'll leave this one open.
It's possible to achieve most of this through user groups and permissions - in particular, a user who's been assigned permission over a single sub-site won't see other sites in the explorer. The main thing that's missing at the moment is introducing a 'choose' permission level so that we can restrict the pages shown in the 'choose a page' popup. Note that we want to avoid tying our model of limited-access users too closely to the concept of 'sites', since we want to keep things flexible enough to allow for other arrangements - for example, an organisation might want to enforce this kind of compartmentalisation between sections of a site, without having a distinct domain name for each one.

One other thought to consider with this request:  It would be useful to be able to have themes placed outside of app-specific files.  This would allow multiple sites to share the same theme, which is a very real possibility in my use case.  I'm thinking of a general themes folder at the same level as the general templates folder.  I've accomplished this in my current Wagtail app, but because of how Django handles templates versus what I'll call other static assets (aren't templates really static assets too?) meant that I had to prefix all css and js references with the folder name of the theme and then do an alias in my Apache VirtualHost setup from the prefix to the location of the folder (along with a Require all granted directive following the alias).  I would really like to get away from having to create the Apache alias and directive in this type of structure, but this might be a limitation of Django because of how it differentiates between templates and other static assets and doesn't treat the combination of the three (html, js, and css) as a theme.

SNIPPETS:  In order for the requested feature to be complete, it seems that snippets would also have to be made specific to a site in  some way.

in particular, a user who's been assigned permission over a single sub-site won't see other sites in the explorer.

Could you elaborate on this? I'm having trouble finding any clear avenues for assigning permissions to a site. Thank you! :)
( +1 to fleshing this feature out further, also 👍 )

My friend and I hacked together something that sort of worked:
https://github.com/mjlabe/bwagtail-cms
We ended up filtering via SQL:
https://github.com/mjlabe/bwagtail-cms/blob/master/bwagtail/bwagtail/get_user_sites.py
And overriding the views for switching the site:
https://github.com/mjlabe/bwagtail-cms/blob/master/bwagtail/wag_custom/views.py
https://github.com/mjlabe/bwagtail-cms/blob/master/bwagtail/wag_custom/forms.py

This video might be useful in implementing this feature.

I ran a bit farther with @mjlabe's implementation, and made a few small updates:

updated edit and edit_current_site to match updated arguments ( circa 2.6.2 )
monkeypatches / re-sets the view function instead of requiring re-registering all the Wagtail URLs ( though this approach is downright disgusting lol and I would love to hear a cleaner solution here )
removes the references to django-currentuser

Not production ready, but if this is a helpful stepping stone in getting someone else toward a workable prototype, here you go:
https://gist.github.com/bahoo/5f073c15d0e3ff742ab14ecc5a671de9

While the tree-based permissions of the page tree and collections allow a great deal of flexibility in defining how multi-tenancy should work for a project - I can't help bit think we're missing something to pull it all together.
I would like to propose a new CMSInstance model to fulfil this role - which would represent an 'instance of the CMS' accessed by editors (as opposed to a Site which is accessed by front-end users). And like a Site,  a CMSInstance would be identifiable by the domain of the current request, but it would only apply to the CMS (which might be at an entirely different domain to any of the Site objects).
Like we did for images and documents when Collections were introduced, we could create a single CMSInstance by default and associate all Site objects with it (via a new ForeignKey field on the Site model), and things like the site switcher in wagtail.contrib.settings (and perhaps the site list itself) would be limited to sites associated with the current CMSInstance.
All views and choosers would use the current CMSInstance to filter out anything from other instances, and automatically associate content with the correct instance. And, like we do with wagtail.search.models.Indexed, we could include a model mixin (e.g. CMSInstanceSpecific) that developers could use to create models with content unique to each instance.
We'd probably also need to provide a version of CMSInstanceSpecific to help create tenant-specific user models, (username would have to be unique per CMSInstance, rather than completely unique), which would be important for managing users / configuring admin permissions on a per-instance basis (admin authentication would need a rejig too).
A few other things that would need thinking about are:

What the relationship between CMSInstance / Collection / Image / Document should be
What we'd do about tags (each CMSInstance should probably have it's own tag set/cloud)\
How would non-CMS users be associated with a CMSInstance - maybe indirectly via Site?
Whether there would be a UI to manage this, or whether (like Collections), developers would manage CMSInstance objects outside of Wagtail.

Interested to hear what folks think about this general approach? @mjlabe @bahoo @DanielSwain Does this sound like something that would fit your use-cases? If feedback is good, I'll work up these ramblings into a proper RFC.

It sounds great to me. I think it would be best to have dropdowns to assign
a site to a CMS instance. Even if Django admin were left as is, I still
think it would work for me. The Django admin site is for true admins with
access to everything and the wagtail admin would work on a per CMSInstance
basis.
On Sun, Nov 24, 2019, 6:57 PM Andy Babic<EMAIL_ADDRESS>wrote:

While the tree-based permissions of the page tree and collections allow a
great deal of flexibility in defining how multi-tenancy should work for a
project - I can't help bit think we're missing something to pull it all
together.
I would like to propose a new CMSInstance model to fulfil this role -
which would represent an 'instance of the CMS' accessed by editors (as
opposed to a Site which is accessed by front-end users). And like a Site,
a CMSInstance would be identifiable by the domain of the current request,
but it would only apply to the CMS (which might be at an entirely different
domain to any of the Site objects).
Like we did for images and documents when Collections were introduced, we
could create a single CMSInstance by default and associate all Site
objects with it (via a new ForeignKey field on the Site model), and
things like the site switcher in wagtail.contrib.settings (and perhaps
the site list itself) would be limited to sites associated with the current
CMSInstance.
All views and choosers would use the current CMSInstance to filter out
anything from other instances, and automatically associate content with the
correct instance. And, like we do with wagtail.search.models.Indexed, we
could include a model mixin (e.g. CMSInstanceSpecific) that developers
could use to create models with content unique to each instance.
We'd probably also need to provide a version of CMSInstanceSpecific to
help create tenant-specific user models, (username would have to be unique
per CMSInstance, rather than completely unique), which would be important
for managing users / configuring admin permissions on a per-instance basis
(admin authentication would need a rejig too).
A few other things that would need thinking about are:

What the relationship between CMSInstance / Collection / Image /
Document should be
What we'd do about tags (each CMSInstance should probably have it's
own tag set/cloud)\
How would non-CMS users be associated with a CMSInstance - maybe
indirectly via Site?
Whether there would be a UI to manage this, or whether (like
Collections), developers would manage CMSInstance objects outside of
Wagtail.

Interested to hear what folks think about this general approach? @mjlabe
https://github.com/mjlabe @bahoo https://github.com/bahoo @DanielSwain
https://github.com/DanielSwain Does this sound like something that
would fit your use-cases? If feedback is good, I'll work up these ramblings
into a proper RFC.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/wagtail/wagtail/issues/4288?email_source=notifications&email_token=AG3DIIIDTQYYBPQND7WRW6LQVMIE3A5CNFSM4EQXZWH2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEFAYQ5I#issuecomment-557942901,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AG3DIIPPKXJ4AX24447IBS3QVMIE3ANCNFSM4EQXZWHQ
.

To add to this discussion, I am including here references to my two comments from July 9, 2019.  I also plan to add some other thoughts here as soon as I have time to write them up.

@ababic did your proposal ever make it into an RFC (I looked [here](https://github.com/wagtail/rfcs/pulls and couldn't find it)? My group (in a clinical informatics lab in Boston) are just (i.e. in the past 2 weeks) getting started with Wagtail to deal with a sudden surge of content related to the pandemic across our network of hospitals. Loving it so far, but the enhancement discussed here sounds like it might be ideal for some of our uses case both now and longer term. In our case it's not different Chambers of Commerce (the example used by @DanielSwain above), but different hospitals, academic centers, and other organizational units that we support that may have similar content, needs, but need to be segregated in the admin UI.
Would love to contribute to the discussion around this and pick a direction for our urgent need that is compatible with community plans to address this.

Hi @carlb. Thanks for posting! It just so happens I implemented something like the above for a client project recently. It made a few things clearer, although many elements were probably a bit too bespoke to apply to many projects.
I haven't put together an RFC yet, as the issue is so multi-facetted that it probably demands 3 or 4 RFCs to tackle the various different aspects, and I simply haven't had the time to put them together (although I'd certainly still like to).
The CMSInstance idea (I went with CMSSite in the end) definitely has legs, but I only managed to make it work with each site being administered at a separate CMS domain, e.g.

cms.foo.com manages content for foo.com
cms.bar.org manages content for bar.org

Making things completely segregated is really quite a challenge, I've found. If you want true segregation between each site, I would highly recommend investing the time in your infrastructure instead, and simply redeploy the same codebase multiple times (especially if free hosting plans will do).

I am still very hopeful to see this feature come to pass.  In preparation for this eventuality, I want to mention my comment here about use of a themes folder.

@DanielSwain Broadly speaking, I would say the remaining pieces of the puzzle are:

Support for a can_choose Page permission for pages (allowing true hiding of pages between sites)
Support for "Swappable site identification policies" - Allowing developers to customise how sites are matched to requests (e.g. by the CMS domain being used)
Something like the CMSInstance model discussed above, allowing you to recognise the site by the domain it is administered on, rather than the domain it is viewed on.
Support for a can_choose_from permission for collections (you now have site-specific media that can be made private for each site)
Support for nested collection in the Wagtail UI (RFC)
Support for site-specific collections (each site can specify it's own root collection, and a site's collections will be selected by default in various places)
Support for site-specific tag clouds
Adding some mixin models / utilities to help implement site-specific models and manage them via Wagtail
Perhaps a native (but optional) version of https://github.com/moorinl/wagtail-themes
A way to manage permissions for multiple sites that doesn't require duplication/management of lots of site-specific groups

Even with all these things in place, I think Wagtail might still have to recommend separate hosting as the favoured way to segregate sites, not least because all it takes is for someone to make the Django admin area available (or some other Django app that doesn't care about Wagtail's site-specific view logic), and you immediately have important security and privacy implications.

To handle some restrictions I created the folowing models.
Rootpage is the toplevel – independent of any site – to add home pages to.
class ProxySite(Site):

    objects = ProxySiteManager() < see below

    class Meta:
        proxy = True

    @property
    def home_page(self):
        return self.root_page

class RootPage(Page): < toplevel
    parent_page_types = []
    subpage_types = ['cms.home_page')]
    is_movable = False
    is_copyable = False
    is_deletable = False
    is_unpublishable = False

    @property
    def can_add_homepage(self):
        return self.get_children().count() < MAX_SITES

class HomePage(Page):

    max_count = MAX_SITES
    is_movable = False
    is_unpublishable = False
    @property
    def is_copyable(self):
        return self.get_parent().specific.can_add_homepage

    @property
    def is_deletable(self):
        return self.get_parent().get_children().count() > 1

    subpage_types = [
            'cms.search_page',
            'cms.default_page',
            'cms.album_page',
            'cms.news_page'
        ]

class DefaultPage(Page):

        parent_page_types = [
            'cms.home_page',
            'cms.default_page'
        ]
        subpage_types = [
            'cms.default_page',
        ]

class ProxySiteQuerySet(models.query.QuerySet):

    def default(self):
        try:
            return self.get(is_default_site=True)
        except:
            return

    def not_default(self):
        return self.filter(is_default_site=False)

    def default_or_first(self):
        return self.default() or self.first()

    def get_user_sites(self, user):
        user_perms = UserPagePermissionsProxy(user)
        if user_perms.can_edit_pages():
            site_pks = list(page.get_site().pk for page in user_perms.editable_pages().all())
            site_pks = list(sorted(set(site_pks)))
            return self.filter(pk__in=site_pks).distinct().all()

        return self.none()

    def get_current_user_site(self, user, site):
        user_site = self.none()
        user_sites = self.get_user_sites(user)
        try:
            user_site = user_sites.get(hostname=site.hostname)
        except ObjectDoesNotExist:
            user_site = user_sites.first()
        return user_site

    def get_sites_for_hostname(self, hostname, port):
        return self.annotate(
            match = Case(
                When(hostname = hostname, port = port, then = MATCH_HOSTNAME_PORT),
                When(hostname = hostname, is_default_site = True, then = MATCH_HOSTNAME_DEFAULT),
                When(is_default_site = True, then = MATCH_DEFAULT),
                default=MATCH_HOSTNAME,
                output_field=IntegerField()
            )
        ).filter(
            Q(hostname=hostname) | Q(is_default_site=True)
        ).order_by(
            'match'
        ).select_related(
            'root_page'
        )

class ProxySiteManager(models.Manager):

    """ Is in wagtail’s SiteManager """
    def get_by_natural_key(self, hostname, port):
        return self.get(hostname=hostname, port=port)

    def get_queryset(self):
        return ProxySiteQuerySet(self.model, using=self._db)

    def default(self):
        return self.get_queryset().default()

    def not_default(self):
        return self.get_queryset().not_default()

    def default_or_first(self):
        return self.get_queryset().default_or_first()

    def get_user_sites(self, user):
        return self.get_queryset().get_user_sites(user)

    def get_current_user_site(self, user, site):
        return self.get_queryset().get_current_user_site(user, site)

    def get_sites_for_hostname(self, hostname, port):
        return self.get_queryset().get_sites_for_hostname(hostname, port)

Editors are linked to one or more home page using group and permissions.
And some hooks prevent unwanted actions.

@hooks.register('before_delete_page')
def before_delete_home_page(request, page):
    if request.method == 'POST':
        return

    home_page = request.proxy_site.home_page.specific
    specific_page = page.specific
    if specific_page.__class__ == home_page.__class__:
        if request.user.is_superuser:
            messages.warning(
                request,
                _('This is the {home_page} of {site_name}. When deleted the site will be removed to.').format(
                    home_page  = home_page._meta.verbose_name.lower(),
                    site_name = request.proxy_site.site_name
                )
            )
        else:
            messages.error(
                request,
                _('You are not allowed to delete the {home_page}.').format(
                    home_page = home_page._meta.verbose_name.lower()
                )
            )
            return redirect('wagtailadmin_explore', page.id)

@hooks.register('before_copy_page')
def before_copy_home_page(request, page):

    home_page = request.proxy_site.home_page.specific
    specific_page = page.specific
    if specific_page.__class__ == home_page.__class__:
        if request.user.is_superuser:
            root_page = home_page.get_parent().specific
            if root_page.can_add_homepage:
                messages.warning(
                    request,
                    _('Please note that you are about to copy the {home_page}.').format(
                        home_page = home_page._meta.verbose_name.lower()
                    )
                )
            else:
                messages.error(
                    request,
                    _('The root page doesn’t allow you to copy the {home_page}.').format(
                        home_page = home_page._meta.verbose_name.lower()
                    )
                )
                return redirect('wagtailadmin_explore', page.id)

        else:
            messages.error(
                request,
                _('You are not allowed to copy the {home_page}.').format(
                    home_page = home_page._meta.verbose_name.lower()
                )
            )
            return redirect('wagtailadmin_explore', page.id)

Unfortunately the above code shows an errormessage in the navigator due to 404 for the rootpage raised by the api:
class PagesAdminAPIViewSet(PagesAPIViewSet):

    def get_queryset(self):
        queryset = super().get_queryset()
        # Hide root page
        # TODO: Add "include_root" flag
        queryset = queryset.exclude(depth=1).specific()
        return queryset


Fixing and reviewing PR #260

This PR fixes PR #260.

This PR is the result of reviewing PR#260. Therefore I'm merging without additional reviewer.


Issues with keyboard on SM-G950F

Hello guys,
i have an issue with my Samsung SM-G950F.
When i want to write message through scrcpy, i can't write uppercase with shift. I can use capslock, but it's a big problem when i'm writing messages.
When i'll use shift, i have littlecase.
Also i have an issue with latin characters like ą, ę, ć, ó, ł etc.
When i'll write ć and ó, i have something like this:

In consola i find these errors:
[server] WARN: Could not inject char u+0119```

There are many disscusion in issue #431 .

For any text problem, first try with scrcpy --prefer-text.
Only few non-ASCII characters can be emulated: https://blog.rom1v.com/2018/03/introducing-scrcpy/#handle-accented-characters
See related issues: https://github.com/Genymobile/scrcpy/issues?q=is%3Aopen+is%3Aissue+label%3Aunicode
(I'm closing this one as a duplicate)


Substack Import Fails with "Invalid file type"

Issue Summary
I want to import users and posts from my old substack account, but I get an "Invalid File" error despite it being literally the zipfile I got from substack and it's not invalid as I can decompress and look at the content inside.
Steps to Reproduce

Open the Substack Importer in the Labs section of Ghost
Enter a valid URL for a substack
Try to upload Zip File
Have it fail and look like this

Ghost Version
5.59.1
Node.js Version
18.16.1
How did you install Ghost?
Ubuntu v20.04.4 LTS, manually installed
Database type
MySQL 5.7
Browser & OS version
No response
Relevant log / error output
No response
Code of Conduct

[X] I agree to be friendly and polite to people in this repository

Hi @mgattozzi - thanks for the report. We're looking at a fix for this and should get back to you soon 🙂

Hi @mgattozzi, I have just pushed a fix for this.

@mgattozzi We pushed the fix to the separate project, so it should already be live. (It's not a feature within Ghost)

Oh! Excellent thanks so much


Deduplicate shellscript code in Mycroft module

Also make the script POSIX sh compatible
Lastly, only set Mycroft permissions if Mycroft actually exists in
/opt/mycroft. On some systems, like postmarketOS, Mycroft is installed
differently and doesn't actually require fixing of permissions like that

@AIIX Can you please review this PR?

@AIIX Can you please review this PR?

Looks good to me.

Great. Thanks 👍


Fixing Ali Cloud download blob method

Pull Request Test Coverage Report for Build 2883

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 90.066%

Totals

Change from base Build 2881:
0.0%

Covered Lines:
9103

Relevant Lines:
9772

💛  - Coveralls


fix: rspack ecosystem-ci failed

Summary
https://github.com/web-infra-dev/rspack-ecosystem-ci/actions/runs/7014628924/job/19082632444
Related Links

Checklist

[ ] I have added changeset via pnpm run change.
[ ] I have updated the documentation.
[ ] I have added tests to cover my changes.

!bench