TaskRun fails during initialization when disable-home-env-overwrite=true

This is closely related to the on-going Tekton $HOME issue (https://github.com/tektoncd/pipeline/issues/2013#issuecomment-585908031). I am testing disable-home-env-overwrite before it gets flipped.
This comment says

With this new flag Tekton will no longer interfere with HOME - it will be whatever you expect it to be when the container runs in a Pod.
Previously $HOME would have been set to /tekton/home but now it won't be. So I would expect $HOME/.docker/config.json to be written to /root/.docker/config.json if the user is root and the image doesn't specify its own HOME.

I don't think this is the case. I am testing gcr.io/cloud-builders/gradle, but Tekton fails as it tries to create a directory /.docker.
"level":"fatal",
"ts":1583431818.4164164,
"caller":"creds-init/main.go:41",
"msg":"Error initializing credentials: mkdir /.docker: permission denied",
"stacktrace":
main.main
    github.com/tektoncd/pipeline/cmd/creds-init/main.go:41
runtime.main
    runtime/proc.go:203

Note the "permission denied" error is not the issue here. The issue is that it is /.docker instead of /root/.docker.

I tested this after defining HOME=/root on top of gcr.io/cloud-builders/gradle, but it doesn't seem to make any difference. It's still /.docker.

Note that this error is from the credential-initializer init container. So credential-initializer needs to get the user home information from my main step.
But then, we will have to answer this question: what if I define multiple steps whose images have different user home directories (say, the home of image A in the first step is /root, while image B in the next step is /home/foo)? Where should credential-initializer put .docker/config.json? Perhaps both /root/.docker/config.json and /home/foo/.docker/config.json.

I think relying on creds-init might actually be the wrong approach here. You're right that multiple home directories in multiple steps make things complicated.  You might consider passing a workspace into your pipelines instead.  This workspace could be from a Secret or could be from a persistent volume.  You could mount it at a known location in your Task (such as /home/foo) and then symlink or copy it explicitly as needed in each Step.  Users of the Pipeline could then provide the credentials through this workspace.
To me, it seems like creds-init is simply not capable of injecting credentials in a reliable way for this use case and I'm not sure that copying these creds everywhere that they might be used is the best way to go.  I'm still thinking about possible solutions but something about the way it works currently doesn't feel right to me.

Thanks. I'll look into mounting a Secret, but I'm not sure if this will work nicely as a tekton catalog writer (jib-gradle and jib-maven). This sounds like my task will add a a special contract only applicable for my catalog for providing Docker credentials. I'd like a general solution in a document way at the Tekton level, but I'll look into mounting a Secret anyways.
In any case, this error is blocking me from testing disable-home-env-overwrite at all. My cluster has some Docker Secrets by default (which I don't want to remove because I don't know what they are), and I think creds-init just crashes because of their presence.

I'm not sure that copying these creds everywhere that they might be used is the best way to go.

BTW, isn't this the current behavior? The Docker credentials are exposed to every steps at /home/tekton. Doesn't seem like it will make things worse.

I'm not sure that copying these creds everywhere that they might be used is the best way to go.

BTW, isn't this the current behavior? The Docker credentials are exposed to every step at /home/tekton. Doesn't seem like it will make things worse.

I meant more the copying of creds to every possible HOME directory that steps of a task might use. Having a single root-owned location seems different to me than having 10 copies with random owners.  Maybe it's a distinction that doesn't matter after all, I dunno.
To me this feels like there should be an explicit "opt-in" - Step A declares that it will use creds X so we do copy X into that step's ~/.X directory.  But Step B does not explicitly request any creds so we do not copy any into that container user's home.

This sounds like my task will add a special contract only applicable for my catalog for providing Docker credentials. I'd like a general solution in a document way at the Tekton level, but I'll look into mounting a Secret anyways.

Yeah that's a fair point and I understand not wanting to take this path if this isn't an approach that everyone uses.  I'm wondering whether this should become a recommendation for catalog authors though - to expose (optional) workspaces for credentials to be mounted into.  If everyone was doing it then it might not be bad?

I'm wondering whether this should become a recommendation for catalog authors though - to expose (optional) workspaces for credentials

I can do this (as I commented in https://github.com/tektoncd/pipeline/issues/2119#issuecomment-595819831), and I will do this if this is the only option. However, I'd like to throw a couple points for the sake of discussions. In my case, the Docker config file is just one of many ways to get credentials for remote Docker registries. Taking the optional workspaces approach means having to declare 5+ optional workspaces. The task needs to clone a git repo, and pull from/push to remote Docker registry over HTTPS, and docker credential (of type kubernets.io/basic-auth, kubernetes.io/dockerconfigjson, `kubernetes.io/dockerconfigjson, etc) for base image and target image repositories at least. That's a lot of clutters.
Now, when you flip disable-home-env-overwrite, you certainly have to decide how the current Tekton auth support will work and do something about it. If you decide to get rid of the entire auth support, that's fair. The point is, the you will make the auth support work in one way or another (as long as you don't get rid of it), and I wonder what it would look like and what your plan is. If it doesn't go away, I think most likely I will be able to make some use of it. Implementing my own optional workspaces might become a wast of time.
How about this: the current behavior is to create these credential files under /tekton/home no matter what. When flipping disable-home-env-overwrite, you basically do the same thing: create these files at a fixed location, be it /tekton/home, /workspace/credentials, /mnt/secrets, etc. The location doesn't matter much to me, as long as it is a pre-determined documented location. Maybe optionally a Task could configure this base directory. And whether you copy such files into the container or mount the secrets doesn't matter much to me either. Then, my task could just reference or copy such files.

To me this feels like there should be an explicit "opt-in" - Step A declares that it will use creds X so we do copy X into that step's ~/.X directory. But Step B does not explicitly request any creds so we do not copy any into that container user's home.

Also note, I think copying a file into a "home" directory needs more thought as I explained in https://github.com/tektoncd/pipeline/issues/2013#issuecomment-595472162. For example, OpenShift runs containers as a random UID like<PHONE_NUMBER>.

Taking the optional workspaces approach means having to declare 5+ optional workspaces in my Task. The Task needs to clone a git repo, pull from/push to remote Docker registries over HTTPS, and use docker credential (of type kubernetes.io/basic-auth, kubernetes.io/dockerconfigjson, kubernetes.io/dockercfg, etc) for base image and target image repositories at least. That's a lot of clutters.

I think "clutter" is a mis-characterization.  I view this as a Task Author explicitly declaring support for, and dependence on, credentials.  It becomes obvious what the Task accepts and expects.  With creds-init  the end-user has to learn this odd system of annotations on Secrets with names like "tekton.dev/git-0" that are utterly removed from the Task they're trying to run.  They have to digest auth.md in its entirety in order to make this work.
So in other words I feel like the burden should be on catalog Task authors to explicitly declare what creds they support in the Tasks they write.  In my view this shouldn't be something that happens quietly behind the scenes.  There may be much better ways for Tekton to support authors doing this but I definitely think explicit is better than implicit here.
So having written all this, creds-init as it stands today should 100% work as it's documented to.  I'm still debugging the issue with the HOME var mentioned at the top of this post and will update as I figure out what's up or if I have more questions.

So in other words I feel like the burden should be on catalog Task authors to explicitly declare what creds they support in the Tasks they write.

Thanks. If Tekton recommends this approach, I am all for it. I hope the optional workspace feature is implemented soon. For now, I think I'm blocked on the issue to implement this approach.

So having written all this, creds-init as it stands today should 100% work as it's documented to.

Anyways, I'm still super curious how it will work when disable-home-env-overwrite is flipped. I appreciate your update once you make a decision.

I've gone with the approach of placing the creds in a fixed location (/tekton/home) and made a PR here: https://github.com/tektoncd/pipeline/pull/2180

Design doc for this problem to be discussed in WG on wednesday: https://docs.google.com/document/d/1SVuDt-SXPHymz41dveSXFSPrx5Z-Wzb9eHliJAyYg4o

Just to reiterate from the Pull Request that closed this Issue:

Credentials are now written to /tekton/creds when the disable-home-env-overwrite flag is "true".
A new variable has been exposed, $(credentials.path) which points to the place where creds-init wrote the credentials.
Our entrypoint binary will automatically copy credentials from $(credentials.path) to the Step's HOME.  We find the HOME directory using go-homedir rather than relying on just the $HOME env var.

@chanseokoh once v0.11.0-rc3 is released this fix will be available to try out.  Very keen to hear your feedback / experience with the changes!

Pipelines Beta RC3 has been released and includes this fix https://github.com/tektoncd/pipeline/releases/tag/v0.11.0-rc3

I'm testing this now and also the case where I run a pipeline as a non-root while forcing $HOME to /workspace (which I think is a reasonable choice). One problem is that the volumes I mount for caching have ownership and permissions of drwxr-xr-x root root and my process can't write things into it. For example, in my task spec,
    env:
    - name: HOME
      value: /workspace
    volumeMounts:
    - name: $(inputs.params.CACHE)
      mountPath: /workspace/.gradle/caches
      subPath: gradle-caches
    - name: $(inputs.params.CACHE)
      mountPath: /workspace/.gradle/wrapper
      subPath: gradle-wrapper

And /workspace/.gradle is not writable by a non-root user.
[build-and-push] drwxr-xr-x 3 root root 4096 Apr 29 16:53 .gradle

What's the right solution to this?

FYI, I was setting securityContext: runAsUser: 1234 in a Task spec.
Now, if I set the following in a TaskRun spec,
    securityContext:
      runAsUser: 1111
      runAsGroup: 2222
      fsGroup: 3333

I get the following errors when disable-home-env-overwrite and disable-working-directory-overwrite are set to true.
[create-dir-image-ppl7m] 2020/04/29 17:17:03 unsuccessful cred copy: ".docker" from "/tekton/creds" to "/": unable to create destination directory: mkdir /.docker: permission denied
[create-dir-image-ppl7m] 2020/04/29 17:17:03 unsuccessful cred copy: ".gitconfig" from "/tekton/creds" to "/": unable to open destination: open /.gitconfig: permission denied
[create-dir-image-ppl7m] 2020/04/29 17:17:03 unsuccessful cred copy: ".git-credentials" from "/tekton/creds" to "/": unable to open destination: open /.git-credentials: permission denied
[create-dir-image-ppl7m] 2020/04/29 17:17:03 unsuccessful cred copy: ".ssh" from "/tekton/creds" to "/": unable to create destination directory: mkdir /.ssh: permission denied

[git-source-source-v75bx] 2020/04/29 17:17:03 unsuccessful cred copy: ".docker" from "/tekton/creds" to "/": unable to create destination directory: mkdir /.docker: permission denied
[git-source-source-v75bx] 2020/04/29 17:17:03 unsuccessful cred copy: ".gitconfig" from "/tekton/creds" to "/": unable to open destination: open /.gitconfig: permission denied
[git-source-source-v75bx] 2020/04/29 17:17:03 unsuccessful cred copy: ".git-credentials" from "/tekton/creds" to "/": unable to open destination: open /.git-credentials: permission denied
[git-source-source-v75bx] 2020/04/29 17:17:03 unsuccessful cred copy: ".ssh" from "/tekton/creds" to "/": unable to create destination directory: mkdir /.ssh: permission denied
[git-source-source-v75bx] {"level":"error","ts":1588180626.7862072,"caller":"git/git.go:41","msg":"Error running git [config --global http.sslVerify true]: exit status 255\nerror: could not lock config file //.gitconfig: Permission denied\n","stacktrace":"github.com/tektoncd/pipeline/pkg/git.run\n\tgithub.com/tektoncd/pipeline/pkg/git/git.go:41\ngithub.com/tektoncd/pipeline/pkg/git.Fetch\n\tgithub.com/tektoncd/pipeline/pkg/git/git.go:82\nmain.main\n\tgithub.com/tektoncd/pipeline/cmd/git-init/main.go:53\nruntime.main\n\truntime/proc.go:203"}
[git-source-source-v75bx] {"level":"warn","ts":1588180626.7865357,"caller":"git/git.go:83","msg":"Failed to set http.sslVerify in git config: exit status 255"}
[git-source-source-v75bx] {"level":"fatal","ts":1588180626.7866344,"caller":"git-init/main.go:54","msg":"Error fetching git repository: exit status 255","stacktrace":"main.main\n\tgithub.com/tektoncd/pipeline/cmd/git-init/main.go:54\nruntime.main\n\truntime/proc.go:203"}

container step-git-source-source-v75bx has failed  : [{"key":"StartedAt","value":"2020-04-29T17:17:06Z","resourceRef":{}}]

One problem is that the volumes I mount for caching have ownership and permissions of drwxr-xr-x root root and my process can't write things into it.

Sorry, I need to correct this. It is the "non-existing parent directories of the volumes". For example, I am mounting /workspace/.gradle/cache where /wokspace/.gradle doesn't exist. I see that /workspace/.gradle/cache is globally writable, but the parent directory /wokspace/.gradle that did not exist before is not writable (having drwxr-xr-x root root). I wonder if Tekton should also make /wokspace/.gradle writable? Otherwise, what would be the solution or a temporary workaround?

Volume mounts can be nested, so /workspace/.gradle can be a volumeMount even while /workspace/.gradle/cache and /workspace/.gradle/wrapper are as well.  This results in all three directories being world-writable.
For the "unsuccessful cred copy" messages, can you confirm whether $HOME was set in your Step's env?  If so I'm baffled how the cred copy didn't figure out what HOME was and why it decided to write to / instead.  If HOME wasn't set though, what was the user's home directory expected to be resolved to?
The $(credentials.path) variable is available for cases where Tekton just straight up does the wrong thing. In this case your Task can take charge of copying the credentials out of that directory into the correct HOME location.

Volume mounts can be nested, so /workspace/.gradle can be a volumeMount even while /workspace/.gradle/cache and /workspace/.gradle/wrapper are as well. This results in all three directories being world-writable.

The problem is that I don't want to cache /workspace/.gradle. But now I realized that probably I can just always mount an emptyDir.

For the "unsuccessful cred copy" messages, can you confirm whether $HOME was set in your Step's env? If so I'm baffled how the cred copy didn't figure out what HOME was and why it decided to write to / instead. If HOME wasn't set though, what was the user's home directory expected to be resolved to?

$HOME is set to /workspace in a Task spec as you can see in my previous example, and everything seems to work OK when I don't set securityContext in a TaskRun spec. That is, I can see /workspace/.docker. So, what is weird is that, without touching anything (i.e., $HOME still set to /workspace), setting securityContext causes the "unsuccessful cred copy" into /.
This error just prevents my task from running at all, so I think there's even no opportunity of using $(credentials.path). In any case, I think this looks like a bug that prevents you from using securityContext.

And sorry for asking a bit unrelated question: my task gets a git project source at /workspace/source (git resource), but /workspace/source is not writable by a non-root. I can work around this by mounting an emptyDir to /workspace/source only for the sake of making it globally writable, but I think there should be a better way?

Let's take this one complete example at a time.  It's difficult to know where to start otherwise.  I've set both disable-home-env-overwrite and disable-working-directory-overwrite to "true" in my cluster.  I do not see the same thing you do where it tries to write to /.ssh though.
Here's the complete TaskRun and TaskSpec which I'm running. It uses two securityContexts, one in the TaskRun's podTemplate and one in the Step:
apiVersion: tekton.dev/v1beta1
kind: TaskRun
metadata:
  generateName: test-creds-
spec:
  podTemplate:
    securityContext:
      runAsUser: 1111
      runAsGroup: 2222
      fsGroup: 3333
  taskSpec:
    steps:
    - name: check-dirs
      image: ubuntu
      env:
      - name: HOME
        value: /workspace
      script: |
        #!/usr/bin/env bash
        set -xe
        id
        ls -lahR $(credentials.path)
        echo ~
        echo $HOME
        ls -lahR $HOME
      securityContext:
        runAsUser: 1234

Here're the relevant log lines I see in the when I run this TaskRun:
[check-dirs] 2020/04/29 19:41:55 unsuccessful cred copy: ".docker" from "/tekton/creds" to "/workspace": unable to open source: open /tekton/creds/.docker/config.json: permission denied
[check-dirs] 2020/04/29 19:41:55 unsuccessful cred copy: ".gitconfig" from "/tekton/creds" to "/workspace": unable to open source: open /tekton/creds/.gitconfig: permission denied
[check-dirs] 2020/04/29 19:41:55 unsuccessful cred copy: ".git-credentials" from "/tekton/creds" to "/workspace": unable to open source: open /tekton/creds/.git-credentials: permission denied
[check-dirs] 2020/04/29 19:41:55 unsuccessful cred copy: ".ssh" from "/tekton/creds" to "/workspace": unable to open source: open /tekton/creds/.ssh/known_hosts: permission denied

The only one of these that I'm actually interested in (because my ServiceAccount / Secret specify it) is this line:
[check-dirs] 2020/04/29 19:41:55 unsuccessful cred copy: ".ssh" from "/tekton/creds" to "/workspace": unable to open source: open /tekton/creds/.ssh/known_hosts: permission denied

So what's happening here?
Looking at the directory structure of /tekton/creds I see the following:
[check-dirs] /tekton/creds:
[check-dirs] total 8.0K
[check-dirs] drwxrwsrwt 4 root 3333  120 Apr 29 19:41 .
[check-dirs] drwxr-xr-x 8 root root 4.0K Apr 29 19:41 ..
[check-dirs] drwxr-sr-x 2 1111 3333   60 Apr 29 19:41 .docker
[check-dirs] -rw------- 1 1111 3333    0 Apr 29 19:41 .git-credentials
[check-dirs] -rw------- 1 1111 3333   29 Apr 29 19:41 .gitconfig
[check-dirs] drwxr-sr-x 2 1111 3333  100 Apr 29 19:41 .ssh

I'm only interested in the .ssh directory.  It's interesting that the owner of the directory is 1111. My Step has user set to 1234. But I see that the read and exec bits are set for both group and global so I'd assume we're ok here.
Looking at the directory structure of /tekton/creds/.ssh I see this:
[check-dirs] /tekton/creds/.ssh:
[check-dirs] total 12K
[check-dirs] drwxr-sr-x 2 1111 3333 100 Apr 29 19:41 .
[check-dirs] drwxrwsrwt 4 root 3333 120 Apr 29 19:41 ..
[check-dirs] -rw------- 1 1111 3333 110 Apr 29 19:41 config
[check-dirs] -rw------- 1 1111 3333  23 Apr 29 19:41 id_fake-ssh-directory
[check-dirs] -rw------- 1 1111 3333  28 Apr 29 19:41 known_hosts

Hm, OK so this explains why the error message open /tekton/creds/.ssh/known_hosts: permission denied showed up.  My Step runs as user 1234 but the files in /tekton/creds/.ssh are owned by user 1111 and there are no group or global bits set.
So I go check the docs. Reading there it sounds like written files will be owned by the user specified in the securityContext's runAsUser field. Now, the way that /tekton/creds/.ssh ended up being there is that an init container, credentials-initializer copied the directory out of a Secret volume mount.  That init container won't have had the runAsUser of 1234, it would've been the Pod securityContext's runAsUser value, 1111.  So that probably explains what's happening here: The creds-init helper runs as the PodTemplate's user, 1111, and writes files with that UID to /tekton/creds. Then our Step container comes along and specifies its own runAsUser, 1234, which isn't allowed to read files created by 1111.  Our entrypoint tries to copy the file /tekton/creds/.ssh/known_hosts but isn't allowed.
Right, we've got a pretty good idea of what's happening now. What's a good solution?  At the moment our creds-init helper writes files with permission 0600 and clearly that isn't going to work if the UID can be different for every single Step. We could change these permissions to be world-readable but that seems insecure.  It does work, I've just tested it, but it seems like a garbage solution.
Right now I'm thinking maybe we should drop creds-init completely.  Every Step could get the Secrets as volume mounts and do the same work that creds-init does but in the context of their own isolated securityContext, writing to their own $HOME directory, as their respective runAsUser user.  Is this better than giving the files 0666 permissions in creds-init though?  I dunno.  I'm going to sleep on it.  I really wish creds-init could just go away because it's really not making the transition away from /tekton/home any easier.
Also, if possible, could you post a complete Task + TaskRun with as few Steps / other stuff in it as possible, which reproduces the issue you're seeing.  It'll be much easier to provide help if we have a single complete example to work from.

Thanks a lot for looking into this! I couldn't agree more on what you said. And I am also not sure what the best solution should be.
I get the exact same result with your Task + TaskRun. So I compared the difference with my Task and was able to come up with a sample that reproduces my error. Hope this helps.
 apiVersion: tekton.dev/v1beta1
 kind: TaskRun
 metadata:
-  generateName: test-creds-
+  name: test-creds
 spec:
+  serviceAccountName: registry-admin
   podTemplate:
     securityContext:
       runAsUser: 1111
       runAsGroup: 2222
       fsGroup: 3333
+  resources:
+    inputs:
+    - name: source
+      resourceSpec:
+        type: git
+        params:
+        - name: url
+          value: https://github.com/che-samples/console-java-simple
   taskSpec:
+    resources:
+      inputs:
+      - name: source
+        type: git
     steps:
     - name: check-dirs
       image: ubuntu

The pipeline halts with this error:
All TaskRuns deleted in namespace "default"
taskrun.tekton.dev/test-creds created
[git-source-source-rlv2l] 2020/04/29 21:58:10 unsuccessful cred copy: ".docker" from "/tekton/creds" to "/": unable to create destination directory: mkdir /.docker: permission denied
[git-source-source-rlv2l] 2020/04/29 21:58:10 unsuccessful cred copy: ".gitconfig" from "/tekton/creds" to "/": unable to open destination: open /.gitconfig: permission denied
[git-source-source-rlv2l] 2020/04/29 21:58:10 unsuccessful cred copy: ".git-credentials" from "/tekton/creds" to "/": unable to open destination: open /.git-credentials: permission denied
[git-source-source-rlv2l] 2020/04/29 21:58:10 unsuccessful cred copy: ".ssh" from "/tekton/creds" to "/": unable to create destination directory: mkdir /.ssh: permission denied
[git-source-source-rlv2l] {"level":"error","ts":1588197492.548074,"caller":"git/git.go:41","msg":"Error running git [config --global http.sslVerify true]: exit status 255\nerror: could not lock config file //.gitconfig: Permission denied\n","stacktrace":"github.com/tektoncd/pipeline/pkg/git.run\n\tgithub.com/tektoncd/pipeline/pkg/git/git.go:41\ngithub.com/tektoncd/pipeline/pkg/git.Fetch\n\tgithub.com/tektoncd/pipeline/pkg/git/git.go:82\nmain.main\n\tgithub.com/tektoncd/pipeline/cmd/git-init/main.go:53\nruntime.main\n\truntime/proc.go:203"}
[git-source-source-rlv2l] {"level":"warn","ts":1588197492.5484045,"caller":"git/git.go:83","msg":"Failed to set http.sslVerify in git config: exit status 255"}
[git-source-source-rlv2l] {"level":"fatal","ts":1588197492.5505333,"caller":"git-init/main.go:54","msg":"Error fetching git repository: exit status 255","stacktrace":"main.main\n\tgithub.com/tektoncd/pipeline/cmd/git-init/main.go:54\nruntime.main\n\truntime/proc.go:203"}

container step-git-source-source-rlv2l has failed  : [{"key":"StartedAt","value":"2020-04-29T21:58:12Z","resourceRef":{}}]

What's interesting is that, if I comment out securityContext in TaskRun, it doesn't crash as I said.
#   podTemplate:
#     securityContext:
#       runAsUser: 1111
#       runAsGroup: 2222
#       fsGroup: 3333

Excellent, I've been able to reproduce the problem exactly.
create-dir-image-XXXXX is injected into a Task when either the GCS PipelineResource is used or Tekton decides it needs to create an extra directory during PipelineResource linking.  It doesn't have a HOME when the home override flag is "true" and it doesn't run as root when securityContext sets non-zero user ID. So it reports errors when the entrypoint tries to copy credentials out of /tekton/creds into /.
git-source-XXXXX is placed into a Task when the Git PipelineResource is used. It shares the same problems as above and also adds another wrinkle: it can't lock the $HOME/.gitconfig file for setting configuration options. This is again because $HOME isn't set, it defaults to /, and it's running as a non-zero user ID. Unlike create-dir-image- this is a fatal error for the Git PipelineResource and the Task dies here.
Ultimately the errors with these two Steps are happening because PipelineResources don't have a HOME set and they're trying to write to / as a non-root user due to the securityContext.
So summarizing the various problems that have been discovered here:

The Git PipelineResource needs to be able to lock and write files in $HOME.  Specifically $HOME/.gitconfig.

Credentials need to be written to /tekton/creds using the UID of the currently running Step.  creds-init can't do this because UID can differ from container to container.

And the likely solutions seem to me:

PipelineResources need to have their HOME set somewhere they can always write regardless of UID. I'm thinking /tekton/home since it's always mounted (even when the override flag is true) and it's always world-writeable since it's an emptyDir.

creds-init probably needs to go away completely and have its logic moved into the entrypointer. This is the only solution I can think of that will allow UID to be random, creds copied out of secret volumes with the correct file permissions, and HOME to be discovered at runtime.

Ideally the entrypointer could copy credentials straight out of secret volumes and into wherever they think $HOME is.  Unfortunately an annoying extra problem that I've brought upon myself is that I've introduced $(credentials.path), which I've documented as pointing to a single location.  So the entrypointer is going to need to copy the creds to /tekton/creds as well as copying them to wherever $HOME is.
I'll create issues for each of these problems and then start working on fixes for both.

Awesome! I can see what's going on.

Ultimately the errors with these two Steps are happening because PipelineResources don't have a HOME set and they're trying to write to / as a non-root user due to the securityContext.

One thing that still seems strange to me: as I said, when not setting securityContext in a TaskRun (not Task), I see that it correctly copied .docker, .gitconfig, etc into /workspace instead of /. This is reproducible with my sample when

using serviceAccountName
removing securityContext from TaskRun

[check-dirs] 2020/05/01 14:53:01 unsuccessful cred copy: ".docker" from "/tekton/creds" to "/workspace": unable to open source: open /tekton/creds/.docker/config.json: permission denied
[check-dirs] 2020/05/01 14:53:01 unsuccessful cred copy: ".gitconfig" from "/tekton/creds" to "/workspace": unable to open source: open /tekton/creds/.gitconfig: permission denied
[check-dirs] 2020/05/01 14:53:01 unsuccessful cred copy: ".git-credentials" from "/tekton/creds" to "/workspace": unable to open source: open /tekton/creds/.git-credentials: permission denied
[check-dirs] 2020/05/01 14:53:01 unsuccessful cred copy: ".ssh" from "/tekton/creds" to "/workspace": unable to open source: open /tekton/creds/.ssh/known_hosts: permission denied

(If you also remove securityContext from Task, they will be copied successfully into /workspace).
So, without securityContext in TaskRun, it can actually see the right $HOME value. I think something else is also involved to make this difference.

So, without securityContext in TaskRun, it can actually see the right $HOME value. I think something else is also involved to make this difference.

check-dirs Step always sees the correct $HOME value, /workspace.  We set this explicitly on the Step.  In some of our runs above, though, the Task dies before it gets to check-dirs.  The log output we see in these cases is only for git-source-xxxx and create-dir-image-xxxx.  They fail writing to /.

(If you also remove securityContext from Task, they will be copied successfully into /workspace).

One small nit here: we don't set the securityContext on the Task but on the Step.  So check-dirs, the step container, receives the securityContext. But creds-init (an injected initContainer) and pipeline resource injected containers do not receive that securityContext.  If you remove the securityContext from the check-dirs Step then the TaskRun's securityContext is applied to all containers equally.
This is all immeasurably confusing.  I'm going to try to illustrate the different scenarios here:

First scenario:
disable-home-override: "true"
No TaskRun securityContext: UID=root
No check-dirs securityContext: UID=root
Order of operations:

creds-init initContainer writes credentials to /tekton/creds.  Those files all get root ownership.
PipelineResource containers, run as root, copy creds from /tekton/creds to /.
Git PipelineResource, run as root, writes to /.gitconfig successfully.
check-dirs container, runs as root, read creds from /tekton/creds, write creds to /workspace.

Second scenario:
disable-home-override: "true"
No TaskRun securityContext: UID=root
check-dirs has securityContext: UID=1234
Order of operations:

creds-init initContainer writes credentials to /tekton/creds.  Those files all get root ownership.
PipelineResource containers, run as root, copy creds from /tekton/creds to /.
Git PipelineResource, run as root, writes to /.gitconfig successfully.
check-dirs container, runs as 1234, dies copying creds from /tekton/creds to /workspace because they're owned by root.
Error: [check-dirs] 2020/05/01 14:53:01 unsuccessful cred copy: ".ssh" from "/tekton/creds" to "/workspace": unable to open source: open /tekton/creds/.ssh/known_hosts: permission denied

Third scenario:
disable-home-override: "true"
TaskRun has securityContext: UID=1111
check-dirs has securityContext: UID=1234
Order of operations:

creds-init initContainer writes credentials to /tekton/creds.  Those files all get 1111 ownership.
PipelineResource containers, run as 1111, fail to copy creds from /tekton/creds to /.
Messages: unsuccessful cred copy: ".ssh" from "/tekton/creds" to "/": unable to create destination directory: mkdir /.ssh: permission denied
Git PipelineResource, run as 1111, fatal error: dies writing to /.gitconfig.
Error: {"level":"error","ts":1588180626.7862072,"caller":"git/git.go:41","msg":"Error running git [config --global http.sslVerify true]: exit status 255\nerror: could not lock config file //.gitconfig: Permission denied\n","stacktrace":"github.com/tektoncd/pipeline/pkg/git.run\n\tgithub.com/tektoncd/pipeline/pkg/git/git.go:41\ngithub.com/tektoncd/pipeline/pkg/git.Fetch\n\tgithub.com/tektoncd/pipeline/pkg/git/git.go:82\nmain.main\n\tgithub.com/tektoncd/pipeline/cmd/git-init/main.go:53\nruntime.main\n\truntime/proc.go:203"}

Here check-dirs never runs.  Therefore no mention of /workspace.

This is a really confusing dance and there's quite a bit of work to do to get all of Tekton's movements in lock-step.

@chanseokoh I think this would be worth checking in on again now that 0.14 has been released. Some of the warnings may remain in the logs (e.g. unsuccessful cred copy may continue to show up) but Tasks should hopefully 🤞 no longer fail at one of these initialization stages. And credentials should be initialized in each Step with the UID of that Step's first process.
There's still some trickiness with the meaning of "$HOME" though. If a container is run with a randomized UID then that user isn't going to have a $HOME directory, since they won't have an entry in /etc/passwd. From what I understand reading around a little bit it sounds like this is fixed in OpenShift 4+ but I'm not sure how much Tekton can otherwise realistically do to handle scenarios like this.  If there's no user home dir and therefore no valid $HOME directory then I'm left unsure what the correct fallback behavior is.

@sbwsg thanks for letting me know. I've tested 0.14 against GKE with Task and TaskRun, and it seems mostly working.
However, I've noticed a recent change in the jib-gradle catalog task that replaced git input resource with a "workspace" source, which forced me to try Pipeline and PipelineRun. I've never used Pipeline or PipelineRun before, so it took a lot of time to make it running. I generally followed the test run. However, after I changed the PipelineRun to run as non-root (the usual uid 1111 / gid 2222 / fsgroup 3333 stuff), I hit the following error, which seems a bit familiar:
error: could not lock config file //.gitconfig: Permission denied

full log:
[clone] 2020/07/07 20:10:10 unsuccessful cred copy: ".docker" from "/tekton/creds" to "/": unable to create destination directory: mkdir /.docker: permission denied
[clone] 2020/07/07 20:10:10 unsuccessful cred copy: ".gitconfig" from "/tekton/creds" to "/": unable to open destination: open /.gitconfig: permission denied
[clone] 2020/07/07 20:10:10 unsuccessful cred copy: ".git-credentials" from "/tekton/creds" to "/": unable to open destination: open /.git-credentials: permission denied
[clone] 2020/07/07 20:10:10 unsuccessful cred copy: ".ssh" from "/tekton/creds" to "/": unable to create destination directory: mkdir /.ssh: permission denied
[clone] + CHECKOUT_DIR=/workspace/output/
[clone] + '[[' true '==' true ]]
[clone] + cleandir
[clone] + '[[' -d /workspace/output/ ]]
[clone] + rm -rf /workspace/output//build /workspace/output//build.gradle /workspace/output//pom.xml /workspace/output//src
[clone] + rm -rf /workspace/output//.git /workspace/output//.gradle
[clone] + rm -rf '/workspace/output//..?*'
[clone] + test -z
[clone] + test -z
[clone] + test -z
[clone] + /ko-app/git-init -url https://github.com/chanseokoh/appengine-diagnosis -revision master -refspec  -path /workspace/output/ '-sslVerify=true' '-submodules=true' -depth 1
[clone] {"level":"error","ts":1594152611.4998615,"caller":"git/git.go:41","msg":"Error running git [config --global http.sslVerify true]: exit status 255\nerror: could not lock config file //.gitconfig: Permission denied\n","stacktrace":"github.com/tektoncd/pipeline/pkg/git.run\n\tgithub.com/tektoncd/pipeline/pkg/git/git.go:41\ngithub.com/tektoncd/pipeline/pkg/git.Fetch\n\tgithub.com/tektoncd/pipeline/pkg/git/git.go:84\nmain.main\n\tgithub.com/tektoncd/pipeline/cmd/git-init/main.go:52\nruntime.main\n\truntime/proc.go:203"}
[clone] {"level":"warn","ts":1594152611.4999757,"caller":"git/git.go:85","msg":"Failed to set http.sslVerify in git config: exit status 255"}
[clone] {"level":"fatal","ts":1594152611.4999988,"caller":"git-init/main.go:53","msg":"Error fetching git repository: exit status 255","stacktrace":"main.main\n\tgithub.com/tektoncd/pipeline/cmd/git-init/main.go:53\nruntime.main\n\truntime/proc.go:203"}

container step-clone has failed  : [{"key":"StartedAt","value":"2020-07-07T20:10:11.453Z","resourceRef":{}}]

I also wonder if I have to use Pipeline and PipelineRun if I want to fetch workspace source for the jib-gradle task from GitHub. Previously, I was able to define a git resource in TaskRun, but now I'm not sure how I can easily achieve getting source from GitHub in TaskRun if the task is using a workspace.

However, after I changed the PipelineRun to run as non-root (the usual uid 1111 / gid 2222 / fsgroup 3333 stuff), I hit the following error, which seems a bit familiar:

This is an example of where "$HOME" doesn't exist.  I'm guessing user 1111 doesn't have an entry in /etc/passwd.  And therefore doesn't have a home directory.  What's happened is that the git-init binary is delegating to git, running the git config --global http.sslVerify true command. --global tells git to try and access config in ~. ~ doesn't exist for UID 1111, and so just resolves to /. So git in turn isn't able to get a lock on the global git config file since UID 1111 doesn't have write access to /. Hm.  Without ~ it's tricky to know precisely what to tell git to do here.

I also wonder if I have to use Pipeline and PipelineRun if I want to fetch workspace source for the jib-gradle task from GitHub. Previously, I was able to define a git resource in TaskRun, but now I'm not sure how I can easily achieve getting source from GitHub in TaskRun if the task is using a workspace.

Yeah, right now PipelineResources are getting a heavy amount of attention / redesign so pipelines + workspaces are preferred for this kind of "fetch-before-use" behaviour. There's nothing stopping you using a jib-gradle Task with PipelineResources but I guess @vdemeester thought it better to put the beta version of the catalog more in line with the beta APIs that Tekton exposes (which doesn't include PipelineResources - they're still alpha).

Thanks for the explanation. The git behavior seems fair. I guess the UID 1111 example isn't something we will see in practice and probably the image should have a home for it. I think then this is basically working in my case.
One last question: Gradle auto-creates ~/.gradle, and jib-gradle is designed to cache two subdirectories (~/.gradle/caches and ~/.gradle/wrapper) through volume mounts. When running the task as non-root, I noticed that ~/.gradle becomes owned by root:root (while ~/.gradle/caches and ~/.gradle/wrapper are world-writable) and Gradle can't access it. The workaround I use for testing is to mount an emptyDir volume at ~/.gradle only to make it globally writable, as you mentioned in https://github.com/tektoncd/pipeline/issues/2165#issuecomment-621397910. Is this an acceptable and reasonable approach?

Interesting that ~/.gradle ends up owned by root:root, I'm not sure where that's happening. I've had a quick dig through the kubernetes source code but couldn't find anything specifically that sets a parent directory of a mount to be root:root owned.
Regardless, I think mounting the writable emptyDir to ~/.gradle is the right way to go here.  Acceptable and reasonable!


Update HTTP.jl dep to v1.0+

Hello, I'm using Makie.jl and Pluto.jl together. Pluto has update their dependent of HTTP.jl to v1.0.2 or later, but JSServe and WGLMakie still depends on v0.9. This prevented me to install latest Pluto. Could this get an update so we can work with HTTP v1.0+?
As a remainder, HTTP 1.0+ has integrated WebSockets functionalities as part of it, replacing old WebSockets.jl. So the version update may not be trivial.
Thank you.

Seems WebSockets.jl is still maintained, in next version it will bump dep of HTTP to v1, so maybe we only need to wait that to be released. :)

Yeah, I switched a couple of times to HTTP only without relying on Websockets.jl... Gotta figure out in which branches that code has ended up, to not start from scratch when upgrading to HTTP 1.0

Close since v2.0.0 works fine.


GL_Texture cut's off edges of texture

As I have posted on Stack Overflow the GL_Texture cuts off the edges of the textures.
http://stackoverflow.com/questions/36283627/opengl-es-2-0-cuts-off-edges-of-texture
To prevent this issue just generate the textures with a buffer of 1 pixel on each side.
Any help would be appreciated.

probably resolved with the newest version


Legend in arma plots

Most of the figures in Covariance Stationary Process lecture (arma) have odd legends, for example,

They used to be correct
https://github.com/QuantEcon/lecture-source-jl/commit/b53a336270436ce5c507b45178ce657aa18ab6bc#diff-5deb0d38adabea182b6a883a673e1926R248

Hey @Shunsuke-Hori, thanks for the issue.
It looks like the only difference in the legends is that the old ones used LaTeXStrings and these don't. That was actually a conscious decision on our part. See #327.
Will close this issue, but please re-open if I've missed something.

Actually, now that I think about it, we could probably do better by interpolating the actual value into the legend (which is likely what you had in mind?). Will make that change.

@Shunsuke-Hori Yes, that's what I meant, like python version:

Perfect. Sorry, saw the latexstring call and thought that's what you meant. But in any case, took care of it in the commit above.

@arnavs thank you for the commit. I think next figure should be fixed as well

Yup, that was also included :-)
https://github.com/QuantEcon/lecture-source-jl/commit/d7808395153ac5d80f805803434d93580e19daec#diff-5deb0d38adabea182b6a883a673e1926R488

@arnavs Oh, sorry. I didn't see that. Thank you for all the work!!


HTTP APIs for avalon/pyblish (aport)

Aport is abbreviation for Avalon-port
based on Pico server pico on github
it is run as server on http://localhost:4242 and ip and port could be changed in environment variables
it is using simple HTTP APIs. Example:  http://localhost:4242/<python_module>/<module_function>?<argument1>=<value1>?<argument2>=<value2>
It could be better connected to javascript, and Extended Scrpt used in Adobe CC so any of their software could be integrated.
a prototype we have and testing at the moment is able to:

set context - set avalon session into particular context and host application
deregister pyblish plugin directory paths - remove all paths from PUBLISH_PATH
register pyblish plugin directory paths - adding path for host application plugins into PUBLISH_PATH
publish from input json file - it will open pyblish qml and load instances from json file

Aport could be run as part of the Avalon Tray menu and it could be waiting for any host to submit new publishing subset
this way it could be used for by: NukeStudio, Adobe Premiere, Davinci Resolve, rest of Adobe products and other software which is not supporting python

Example of use from adobe Premiere Pro:

Example of function from api.py (using some functions from pype's avalon config):

@pico.expose()
def context(project, asset, task, app):
    # url: `http://localhost:4242/api/context?project=jakub_projectx&asset=shot02&task=rotopaint&app=premiere`

    os.environ["AVALON_PROJECT"] = project

    avalon.update_current_task(task, asset, app)

    project_code = pype.get_project_code()
    pype.set_project_code(project_code)
    hierarchy = pype.get_hierarchy()
    pype.set_hierarchy(hierarchy)
    fix_paths = {k: v.replace("\\", "/") for k, v in SESSION.items()
                 if isinstance(v, str)}
    SESSION.update(fix_paths)
    SESSION.update({"AVALON_HIERARCHY": hierarchy,
                    "AVALON_PROJECTCODE": project_code,
                    "current_dir": os.getcwd().replace("\\", "/")
                    })

    return SESSION

It's very unclear what this is doing.

http://localhost:4242/<python_module>/<module_function>?<argument1>=<value1>?<argument2>=<value2>

Is the idea to facilitate calls to arbitrary Python from JavaScript? It sounds like we need to narrow the scope of this somewhat, has it's scope expanded beyond Premiere and Pyblish? Is it still being used? What is the current state of it?

@jezscha @mkolar are you using this in production? Is it working well?

we are using it at one site. However we didn't have capacity to spend some good time on it, so it's a bit temperamental and not very stable.

Thanks for the info. At least for the moment I won't spend time investigating this. :)


how to get stuffthingmaps/train2017/ dataset?

data = dict(
imgs_per_gpu=1,
workers_per_gpu=1,
train=dict(
type=dataset_type,
ann_file=data_root + 'annotations/instances_train2017.json',
img_prefix=data_root + 'train2017/',
seg_prefix=data_root + 'stuffthingmaps/train2017/',
pipeline=train_pipeline),
how to get stuffthingmaps/train2017/ dataset?

Please take a look at https://github.com/joe-siyuan-qiao/DetectoRS/tree/master/configs/htc to prepare the dataset.

Is it the same as the dataset in this link:
http://images.cocodataset.org/annotations/stuff_annotations_trainval2017.zip
?

Is it the same as the dataset in this link:
http://images.cocodataset.org/annotations/stuff_annotations_trainval2017.zip
?


vite cli doesn't work with --host

vite --host <IP_ADDRESS>

will lead to
  > Local:    http://localhost:3001/

in addition the open in the browser thing will crash the container in docker, I have to add --open 0 to make it work, which is a bit weird as I thought --open false would work (and also I wouldn't even need that in docker)

For the --host issue, I guess it's actually working as expected, <IP_ADDRESS> being the same than localhost.
Vite is replacing <IP_ADDRESS> by localhost here: https://github.com/vitejs/vite/blob/main/packages/vite/src/node/server/index.ts#L526 (and I'm not an expert but I guess this SO thread provides a good explanation of why).

indeed, it works but I thought it didn't due to another issue. Replacing localhost by <IP_ADDRESS> in the terminal would be a good idea though ^^

Replacing localhost by <IP_ADDRESS> in the terminal would be a good idea though ^^

From my understanding, "<IP_ADDRESS>" is not exactly the same thing than "localhost" since it potentially represents multiple IP addresses (even if I guess most of the time it only represents localhost).
In the context of a web server, you probably don't (never?) want to listen to <IP_ADDRESS> but only localhost or any specific IP address, that's why they are replacing "<IP_ADDRESS>" by "localhost".
Again, this is just my understanding of a subject I don't master at all so if you want to know more please do your research or wait for someone else's response.

<IP_ADDRESS> means your listening to all IP addresses. This is necessary if you're running vitejs in a docker container (which I'm doing right now).

@mimoo, for reference, there is an open PR for supporting --open false, that fixes #2552
Would you confirm that <IP_ADDRESS> is acting as expected and we can close this issue?


Revoke select on cdb_tablemetadata from PUBLIC_DB_USER

Frontend tests were OK :+1: (details)

@Kartones please review this one :-)
Although I removed the grant from the extension (https://github.com/CartoDB/cartodb-postgresql/commit/3cf62ecd2e9e23d4d3292c1f21e093c698080f33#diff-d2b9f52e335786f15f702ec492b45311R5) we were still giving select on all tables on cartodb schema.

+1


ECMAScript and UMD versions for library

Thanks for amazing library.
Right now testing it for pet project and find out, that package contain only commonjs version. Problem with this, that such version cannot be used with ES6 projects - system just fail to import it. Will be good, if package will provide also ES6 and UMD versions.
I did such approach for my last library re2js - https://github.com/le0pard/re2js/blob/main/package.json and https://github.com/le0pard/re2js/blob/main/rollup.config.js#L27-L52
This allow to anyone use it with commonjs, es6 or udm from single package.

Right, yeah. I'm exceptionally poor with these kind of things, but I'll figure it out!

Could you check if what I did now works for you? I've published multiple formats with rollup:
IIFE (for browser<EMAIL_ADDRESS>UMD<EMAIL_ADDRESS>CommonJS<EMAIL_ADDRESS>ESM<EMAIL_ADDRESS>AMD<EMAIL_ADDRESS>

Thanks, @ilkkapeltola
I will check in several hours, when will reach computer.
About formats - you just need esm, cjs and umd.

umd – Universal Module Definition, works as amd, cjs and iife all in one

So umd cover amd and iife.
Also will be good to add exports section in package.json :  https://github.com/le0pard/re2js/blob/6fe44bd68f101d6993de9a60eb220ae04f334fbd/package.json#L26
BTW, unpkg key in package.json can be used for umd package : https://github.com/le0pard/re2js/blob/6fe44bd68f101d6993de9a60eb220ae04f334fbd/package.json#L25

Thanks for the tips. Yeah, I was wondering if it's better to just provide umd and not the iife and amd separately at all. It doesn't seem to add too many bytes. Maybe for simplicity I'll do it that way. I'll look into the exports and unpkg too in a moment.

Checked. ESM and CJS version works without issues:
const { getFingerprint } = require('@thumbmarkjs/thumbmarkjs');

(async () => {
  const data = await getFingerprint()
  console.log(data)
})()

and
import { getFingerprint } from '@thumbmarkjs/thumbmarkjs'

(async () => {
  const data = await getFingerprint()
  console.log(data)
})()

both output 12arglk, so it works


Cannot start Container which depends on Network - netavark: invalid version number

Issue Description
I tried to install either homeassistant or nginx using podman-compose up.
In both cases (they both require network access to be accessible obviously), the podman-compose (and podman) command gets interrupted by stating "netavark: invalid version number"
If I run the podman-compose command (I already created the container previously)
podman-compose up

I get
['podman', '--version', '']
using podman version: 4.3.1
** excluding:  set()
['podman', 'network', 'exists', 'nginx_podman']
podman create --name=nginx --label io.podman.compose.config-hash=123 --label io.podman.compose.project=nginx --label io.podman.compose.version=0.0.1 --label com.docker.compose.project=nginx --label com.docker.compose.project.working_dir=/home/podman/compose/nginx --label com.docker.compose.project.config_files=compose.yml --label com.docker.compose.container-number=1 --label com.docker.compose.service=server --net nginx_podman --network-alias server -p 8080:80 nginx:latest
Error: creating container storage: the container name "nginx" is already in use by c112347ac237594a5ad74d907a5b3657053cc393b09cc09ddecf9852d42f4feb. You have to remove that container to be able to reuse that name: that name is already in use
exit code: 125
podman start -a nginx
Error: unable to start container c112347ac237594a5ad74d907a5b3657053cc393b09cc09ddecf9852d42f4feb: netavark: invalid version number
exit code: 125

podman start -a nginx --log-level=debug

Yields
INFO[0000] podman filtering at log level debug
DEBU[0000] Called start.PersistentPreRunE(podman start -a nginx --log-level=debug)
DEBU[0000] Merged system config "/usr/share/containers/containers.conf"
DEBU[0000] Merged system config "/etc/containers/containers.conf"
DEBU[0000] Using conmon: "/usr/bin/conmon"
DEBU[0000] Initializing boltdb state at /home/podman/storage/libpod/bolt_state.db
DEBU[0000] systemd-logind: Unknown object '/'.
DEBU[0000] Using graph driver overlay
DEBU[0000] Using graph root /home/podman/storage
DEBU[0000] Using run root /run/user/1002/containers
DEBU[0000] Using static dir /home/podman/storage/libpod
DEBU[0000] Using tmp dir /run/user/1002/libpod/tmp
DEBU[0000] Using volume path /home/podman/storage/volumes
DEBU[0000] Set libpod namespace to ""
DEBU[0000] [graphdriver] trying provided driver "overlay"
DEBU[0000] overlay: mount_program=/usr/bin/fuse-overlayfs
DEBU[0000] backingFs=extfs, projectQuotaSupported=false, useNativeDiff=false, usingMetacopy=false
DEBU[0000] Initializing event backend journald
DEBU[0000] Configured OCI runtime runj initialization failed: no valid executable found for OCI runtime runj: invalid argument
DEBU[0000] Configured OCI runtime kata initialization failed: no valid executable found for OCI runtime kata: invalid argument
DEBU[0000] Configured OCI runtime runsc initialization failed: no valid executable found for OCI runtime runsc: invalid argument
DEBU[0000] Configured OCI runtime krun initialization failed: no valid executable found for OCI runtime krun: invalid argument
DEBU[0000] Using OCI runtime "/usr/bin/crun"
INFO[0000] Setting parallel job count to 25
DEBU[0000] Made network namespace at /run/user/1002/netns/netns-4fdb5096-29f8-d735-8e39-acd0063dfda3 for container c112347ac237594a5ad74d907a5b3657053cc393b09cc09ddecf9852d42f4feb
DEBU[0000] [graphdriver] trying provided driver "overlay"
DEBU[0000] overlay: mount_program=/usr/bin/fuse-overlayfs
DEBU[0000] backingFs=extfs, projectQuotaSupported=false, useNativeDiff=false, usingMetacopy=false
DEBU[0000] Ignoring global metacopy option, the mount program doesn't support it
DEBU[0000] Cached value indicated that idmapped mounts for overlay are not supported
DEBU[0000] Check for idmapped mounts support
DEBU[0000] overlay: mount_data=lowerdir=/home/podman/storage/overlay/l/ZGGUQHP7NZTNGQEGXEJVQN5MKT:/home/podman/storage/overlay/l/UFXPLA6A72EP54GHONOF54QZYV:/home/podman/storage/overlay/l/E6R3CSIVFUM3ZAY6PJT222I57R:/home/podman/storage/overlay/l/LSFY23TJPH2ILHNNLETVUMFMIC:/home/podman/storage/overlay/l/YJGO6ZGWCR5235NYUQS7ONIWCU:/home/podman/storage/overlay/l/ASO75UGOYELOLL5RGZAQGRQN77:/home/podman/storage/overlay/l/DEMKWP5JLMUGXIRJKJDRKZDIDD,upperdir=/home/podman/storage/overlay/0dd7c67cbdfd8d13d3a4b4cf0275c7a4a93308883d73a6d6511027d1d64f5665/diff,workdir=/home/podman/storage/overlay/0dd7c67cbdfd8d13d3a4b4cf0275c7a4a93308883d73a6d6511027d1d64f5665/work,nodev
DEBU[0000] The path of /etc/resolv.conf in the mount ns is "/etc/resolv.conf"
DEBU[0000] Successfully loaded network homeassistant_podman: &{homeassistant_podman 74eec6bde38539e6d1d68600ad571b619567d19e1115cd0e08a4e7fb4b96c100 bridge podman1 2023-10-29 17:27:18.676551877 +0000 UTC [{{{<IP_ADDRESS> ffffff00}} <IP_ADDRESS> <nil>}] false false true map[com.docker.compose.project:homeassistant io.podman.compose.project:homeassistant] map[] map[driver:host-local]}
DEBU[0000] Successfully loaded network nginx_podman: &{nginx_podman c1794cfa9fb4e4ab0cfdf56e605f38cb697f61ce27537818dad1dfe2f1f98562 bridge podman2 2023-10-29 17:38:42.187250173 +0000 UTC [{{{<IP_ADDRESS> ffffff00}} <IP_ADDRESS> <nil>}] false false true map[com.docker.compose.project:nginx io.podman.compose.project:nginx] map[] map[driver:host-local]}
DEBU[0000] Successfully loaded 3 networks
DEBU[0000] Mounted container "c112347ac237594a5ad74d907a5b3657053cc393b09cc09ddecf9852d42f4feb" at "/home/podman/storage/overlay/0dd7c67cbdfd8d13d3a4b4cf0275c7a4a93308883d73a6d6511027d1d64f5665/merged"
[DEBUG netavark::network::validation] "Validating network namespace..."
[DEBUG netavark::commands::setup] "Setting up..."
[INFO  netavark::firewall] Using iptables firewall driver
DEBU[0000] Created root filesystem for container c112347ac237594a5ad74d907a5b3657053cc393b09cc09ddecf9852d42f4feb at /home/podman/storage/overlay/0dd7c67cbdfd8d13d3a4b4cf0275c7a4a93308883d73a6d6511027d1d64f5665/merged
DEBU[0000] Unmounted container "c112347ac237594a5ad74d907a5b3657053cc393b09cc09ddecf9852d42f4feb"
DEBU[0000] Network is already cleaned up, skipping...
DEBU[0000] Cleaning up container c112347ac237594a5ad74d907a5b3657053cc393b09cc09ddecf9852d42f4feb
DEBU[0000] Network is already cleaned up, skipping...
DEBU[0000] Container c112347ac237594a5ad74d907a5b3657053cc393b09cc09ddecf9852d42f4feb storage is already unmounted, skipping...
Error: unable to start container c112347ac237594a5ad74d907a5b3657053cc393b09cc09ddecf9852d42f4feb: netavark (exit code 1): invalid version number

Steps to reproduce the issue
Steps to reproduce the issue

Install Debian Bookworm (preferably on aarch64 / ARM64 device)
Install podman & podman-compose
Run podman-compose up with one of the provided compose YML files
podman-compose-nginx-homeassistant.zip

Describe the results you received
Received an error netavark (exit code 1): invalid version number
Depending if podman-compose or podman start is used to try to bring up the container, the exit code number can vary.
It seems to be exit code 1 or exit code 125.
Describe the results you expected
I expected podman-compose up to correctly and successfully bring up the container & its network.
podman info output
yaml
host:
  arch: arm64
  buildahVersion: 1.28.2
  cgroupControllers:
  - cpu
  - memory
  - pids
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon_2.1.6+ds1-1_arm64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.6, commit: unknown'
  cpuUtilization:
    idlePercent: 99.57
    systemPercent: 0.2
    userPercent: 0.23
  cpus: 8
  distribution:
    codename: bookworm
    distribution: debian
    version: "12"
  eventLogger: journald
  hostname: Rock5B-01
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1002
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1002
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
  kernel: 6.6.0-rc4-1-arm64
  linkmode: dynamic
  logDriver: journald
  memFree:<PHONE_NUMBER>0
  memTotal:<PHONE_NUMBER>2
  networkBackend: netavark
  ociRuntime:
    name: crun
    package: crun_1.8.1-1+b1_arm64
    path: /usr/bin/crun
    version: |-
      crun version 1.8.1
      commit: f8a096be060b22ccd3d5f3ebe44108517fbf6c30
      rundir: /run/user/1002/crun
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL
  os: linux
  remoteSocket:
    path: /run/user/1002/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: false
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns_1.2.0-1_arm64
    version: |-
      slirp4netns version 1.2.0
      commit: 656041d45cfca7a4176f6b7eed9e4fe6c11e8383
      libslirp: 4.7.0
      SLIRP_CONFIG_VERSION_MAX: 4
      libseccomp: 2.5.4
  swapFree: 0
  swapTotal: 0
  uptime: 0h 12m 4.00s
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  volume:
  - local
registries:
  search:
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - docker.io
  - quay.io
store:
  configFile: /home/podman/.config/containers/storage.conf
  containerStore:
    number: 4
    paused: 0
    running: 0
    stopped: 4
  graphDriverName: overlay
  graphOptions:
    overlay.mount_program:
      Executable: /usr/bin/fuse-overlayfs
      Package: fuse-overlayfs_1.10-1_arm64
      Version: |-
        fusermount3 version: 3.14.0
        fuse-overlayfs: version 1.10
        FUSE library version 3.14.0
        using FUSE kernel interface version 7.31
    overlay.mountopt: nodev,metacopy=on
  graphRoot: /home/podman/storage
  graphRootAllocated:<PHONE_NUMBER>360
  graphRootUsed:<PHONE_NUMBER>
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 3
  runRoot: /run/user/1002/containers
  volumePath: /home/podman/storage/volumes
version:
  APIVersion: 4.3.1
  Built: 0
  BuiltTime: Thu Jan  1 00:00:00 1970
  GitCommit: ""
  GoVersion: go1.19.8
  Os: linux
  OsArch: linux/arm64
  Version: 4.3.1

Podman in a container
No
Privileged Or Rootless
Rootless
Upstream Latest Release
No
Additional environment details
Hardware: Rock 5B aarch64 / 16GB RAM / SD Card (128GB) with 2TB NVMe for Podman Storage (EXT4)
uname -a
Linux Rock5B-01 6.6.0-rc4-1-arm64 containers/podman#1 SMP PREEMPT Debian 6.6.0-rc4-1 (2023-10-01) aarch64 GNU/Linux

podman-compose --version
['podman', '--version', '']
using podman version: 4.3.1
podman-composer version  1.0.3
podman --version
podman version 4.3.1
exit code: 0

aptitude show netavark
Package: netavark
Version: 1.4.0-3
State: installed
Automatically installed: yes
Priority: optional
Section: net
Maintainer: Reinhard Tartler<EMAIL_ADDRESS>Architecture: arm64
Uncompressed Size: 3552 k
Depends: libc6 (>= 2.34), libgcc-s1 (>= 4.2)
Recommends: aardvark-dns
Breaks: aardvark-dns (< 1.4)
Enhances: podman
Provides: container-network-stack (= 2)
Description: Rust based network stack for containers
 Netavark is designed to work with but is also applicable for other OCI container management applications.
Homepage: https://github.com/containers/netavark

Additional information
Additional information like issue happens only occasionally or issue happens with a particular architecture or on a particular setting

Just found in my BASH history.
This is the command I used to fix the problem, when nothing would work:
update-alternatives --set iptables /usr/sbin/iptables-legacy

Netavark should work with both the legacy and the nftables compatibility iptables command
This seems to be the version output on debian right now so I cannot see why this would ever fail:
root@0cb9cc26c560:/# iptables-legacy --version
iptables v1.8.9 (legacy)
root@0cb9cc26c560:/# iptables-nft --version
iptables v1.8.9 (nf_tables)

I have the same version on my fedora install and that works flawless.

I only did it for IPv4 so I also ran now for IPv6
root@Rock5B-01:~# update-alternatives --set ip6tables /usr/sbin/ip6tables-legacy
update-alternatives: using /usr/sbin/ip6tables-legacy to provide /usr/sbin/ip6tables (ip6tables) in manual mode

Then the error changed with podman-compose up
['podman', '--version', '']
using podman version: 4.3.1
** excluding:  set()
['podman', 'network', 'exists', 'nginx_podman']
podman create --name=nginx --label io.podman.compose.config-hash=123 --label io.podman.compose.project=nginx --label io.podman.compose.version=0.0.1 --label com.docker.compose.project=nginx --label com.docker.compose.project.working_dir=/home/podman/compose/nginx --label com.docker.compose.project.config_files=compose.yml --label com.docker.compose.container-number=1 --label com.docker.compose.service=server --net nginx_podman --network-alias server -p 8080:80 nginx:latest
Error: creating container storage: the container name "nginx" is already in use by c112347ac237594a5ad74d907a5b3657053cc393b09cc09ddecf9852d42f4feb. You have to remove that container to be able to reuse that name: that name is already in use
exit code: 125
podman start -a nginx
Error: unable to start container c112347ac237594a5ad74d907a5b3657053cc393b09cc09ddecf9852d42f4feb: netavark: code: 1, msg: Warning: Extension comment revision 0 not supported, missing kernel module?
iptables: No chain/target/match by that name.

exit code: 125

A bit more details with podman start -a nginx --debug
Error: unknown flag: --debug
See 'podman start --help'
podman@Rock5B-01:~/compose/nginx$ podman start -a nginx --log-level=debug
INFO[0000] podman filtering at log level debug
DEBU[0000] Called start.PersistentPreRunE(podman start -a nginx --log-level=debug)
DEBU[0000] Merged system config "/usr/share/containers/containers.conf"
DEBU[0000] Merged system config "/etc/containers/containers.conf"
DEBU[0000] Using conmon: "/usr/bin/conmon"
DEBU[0000] Initializing boltdb state at /home/podman/storage/libpod/bolt_state.db
DEBU[0000] systemd-logind: Unknown object '/'.
DEBU[0000] Using graph driver overlay
DEBU[0000] Using graph root /home/podman/storage
DEBU[0000] Using run root /run/user/1002/containers
DEBU[0000] Using static dir /home/podman/storage/libpod
DEBU[0000] Using tmp dir /run/user/1002/libpod/tmp
DEBU[0000] Using volume path /home/podman/storage/volumes
DEBU[0000] Set libpod namespace to ""
DEBU[0000] [graphdriver] trying provided driver "overlay"
DEBU[0000] overlay: mount_program=/usr/bin/fuse-overlayfs
DEBU[0000] backingFs=extfs, projectQuotaSupported=false, useNativeDiff=false, usingMetacopy=false
DEBU[0000] Initializing event backend journald
DEBU[0000] Configured OCI runtime runj initialization failed: no valid executable found for OCI runtime runj: invalid argument
DEBU[0000] Configured OCI runtime kata initialization failed: no valid executable found for OCI runtime kata: invalid argument
DEBU[0000] Configured OCI runtime runsc initialization failed: no valid executable found for OCI runtime runsc: invalid argument
DEBU[0000] Configured OCI runtime krun initialization failed: no valid executable found for OCI runtime krun: invalid argument
DEBU[0000] Using OCI runtime "/usr/bin/crun"
INFO[0000] Setting parallel job count to 25
DEBU[0000] Made network namespace at /run/user/1002/netns/netns-677353d2-f825-c9c9-5014-aaa3a4ddb886 for container c112347ac237594a5ad74d907a5b3657053cc393b09cc09ddecf9852d42f4feb
DEBU[0000] The path of /etc/resolv.conf in the mount ns is "/etc/resolv.conf"
DEBU[0000] Successfully loaded network homeassistant_podman: &{homeassistant_podman 74eec6bde38539e6d1d68600ad571b619567d19e1115cd0e08a4e7fb4b96c100 bridge podman1 2023-10-29 17:27:18.676551877 +0000 UTC [{{{<IP_ADDRESS> ffffff00}} <IP_ADDRESS> <nil>}] false false true map[com.docker.compose.project:homeassistant io.podman.compose.project:homeassistant] map[] map[driver:host-local]}
DEBU[0000] Successfully loaded network nginx_podman: &{nginx_podman c1794cfa9fb4e4ab0cfdf56e605f38cb697f61ce27537818dad1dfe2f1f98562 bridge podman2 2023-10-29 17:38:42.187250173 +0000 UTC [{{{<IP_ADDRESS> ffffff00}} <IP_ADDRESS> <nil>}] false false true map[com.docker.compose.project:nginx io.podman.compose.project:nginx] map[] map[driver:host-local]}
DEBU[0000] Successfully loaded 3 networks
DEBU[0000] [graphdriver] trying provided driver "overlay"
DEBU[0000] overlay: mount_program=/usr/bin/fuse-overlayfs
[DEBUG netavark::network::validation] "Validating network namespace..."
[DEBUG netavark::commands::setup] "Setting up..."
DEBU[0000] backingFs=extfs, projectQuotaSupported=false, useNativeDiff=false, usingMetacopy=false
DEBU[0000] Ignoring global metacopy option, the mount program doesn't support it
[INFO  netavark::firewall] Using iptables firewall driver
DEBU[0000] Cached value indicated that idmapped mounts for overlay are not supported
DEBU[0000] Check for idmapped mounts support
DEBU[0000] overlay: mount_data=lowerdir=/home/podman/storage/overlay/l/ZGGUQHP7NZTNGQEGXEJVQN5MKT:/home/podman/storage/overlay/l/UFXPLA6A72EP54GHONOF54QZYV:/home/podman/storage/overlay/l/E6R3CSIVFUM3ZAY6PJT222I57R:/home/podman/storage/overlay/l/LSFY23TJPH2ILHNNLETVUMFMIC:/home/podman/storage/overlay/l/YJGO6ZGWCR5235NYUQS7ONIWCU:/home/podman/storage/overlay/l/ASO75UGOYELOLL5RGZAQGRQN77:/home/podman/storage/overlay/l/DEMKWP5JLMUGXIRJKJDRKZDIDD,upperdir=/home/podman/storage/overlay/0dd7c67cbdfd8d13d3a4b4cf0275c7a4a93308883d73a6d6511027d1d64f5665/diff,workdir=/home/podman/storage/overlay/0dd7c67cbdfd8d13d3a4b4cf0275c7a4a93308883d73a6d6511027d1d64f5665/work,nodev
[DEBUG netavark::network::bridge] Setup network nginx_podman
[DEBUG netavark::network::bridge] Container interface name: eth0 with IP addresses [<IP_ADDRESS>/24]
[DEBUG netavark::network::bridge] Bridge name: podman2 with IP addresses [<IP_ADDRESS>/24]
[DEBUG netavark::network::core_utils] Setting sysctl value for net.ipv4.ip_forward to 1
DEBU[0000] Mounted container "c112347ac237594a5ad74d907a5b3657053cc393b09cc09ddecf9852d42f4feb" at "/home/podman/storage/overlay/0dd7c67cbdfd8d13d3a4b4cf0275c7a4a93308883d73a6d6511027d1d64f5665/merged"
DEBU[0000] Created root filesystem for container c112347ac237594a5ad74d907a5b3657053cc393b09cc09ddecf9852d42f4feb at /home/podman/storage/overlay/0dd7c67cbdfd8d13d3a4b4cf0275c7a4a93308883d73a6d6511027d1d64f5665/merged
[DEBUG netavark::network::core_utils] Setting sysctl value for /proc/sys/net/ipv6/conf/eth0/autoconf to 0
[INFO  netavark::network::netlink] Adding route (dest: <IP_ADDRESS>/0 ,gw: <IP_ADDRESS>, metric 100)
[DEBUG netavark::firewall::varktables::helpers] chain NETAVARK-9D9EED5AEF6BF exists on table nat
[DEBUG netavark::firewall::varktables::helpers] chain NETAVARK-9D9EED5AEF6BF exists on table nat
[DEBUG netavark::firewall::varktables::helpers] chain NETAVARK_FORWARD exists on table filter
[DEBUG netavark::firewall::varktables::helpers] chain NETAVARK_FORWARD exists on table filter
DEBU[0000] Unmounted container "c112347ac237594a5ad74d907a5b3657053cc393b09cc09ddecf9852d42f4feb"
DEBU[0000] Network is already cleaned up, skipping...
DEBU[0000] Cleaning up container c112347ac237594a5ad74d907a5b3657053cc393b09cc09ddecf9852d42f4feb
DEBU[0000] Network is already cleaned up, skipping...
DEBU[0000] Container c112347ac237594a5ad74d907a5b3657053cc393b09cc09ddecf9852d42f4feb storage is already unmounted, skipping...
Error: unable to start container c112347ac237594a5ad74d907a5b3657053cc393b09cc09ddecf9852d42f4feb: netavark (exit code 1): code: 1, msg: Warning: Extension comment revision 0 not supported, missing kernel module?
iptables: No chain/target/match by that name.

Output of zgrep NETFILTER /proc/config.gz
CONFIG_NETFILTER=y
CONFIG_NETFILTER_ADVANCED=y
CONFIG_BRIDGE_NETFILTER=m
CONFIG_NETFILTER_INGRESS=y
CONFIG_NETFILTER_EGRESS=y
CONFIG_NETFILTER_SKIP_EGRESS=y
CONFIG_NETFILTER_FAMILY_BRIDGE=y
CONFIG_NETFILTER_BPF_LINK=y
# CONFIG_NETFILTER_NETLINK_ACCT is not set
# CONFIG_NETFILTER_NETLINK_QUEUE is not set
# CONFIG_NETFILTER_NETLINK_LOG is not set
# CONFIG_NETFILTER_NETLINK_OSF is not set
CONFIG_NETFILTER_XTABLES=m
# CONFIG_NETFILTER_XTABLES_COMPAT is not set
CONFIG_NETFILTER_XT_MARK=m
# CONFIG_NETFILTER_XT_CONNMARK is not set
# CONFIG_NETFILTER_XT_TARGET_AUDIT is not set
CONFIG_NETFILTER_XT_TARGET_CHECKSUM=m
# CONFIG_NETFILTER_XT_TARGET_CLASSIFY is not set
# CONFIG_NETFILTER_XT_TARGET_CONNMARK is not set
# CONFIG_NETFILTER_XT_TARGET_DSCP is not set
# CONFIG_NETFILTER_XT_TARGET_HL is not set
# CONFIG_NETFILTER_XT_TARGET_HMARK is not set
# CONFIG_NETFILTER_XT_TARGET_IDLETIMER is not set
# CONFIG_NETFILTER_XT_TARGET_LED is not set
CONFIG_NETFILTER_XT_TARGET_LOG=m
# CONFIG_NETFILTER_XT_TARGET_MARK is not set
CONFIG_NETFILTER_XT_NAT=m
# CONFIG_NETFILTER_XT_TARGET_NETMAP is not set
# CONFIG_NETFILTER_XT_TARGET_NFLOG is not set
# CONFIG_NETFILTER_XT_TARGET_NFQUEUE is not set
# CONFIG_NETFILTER_XT_TARGET_RATEEST is not set
# CONFIG_NETFILTER_XT_TARGET_REDIRECT is not set
CONFIG_NETFILTER_XT_TARGET_MASQUERADE=m
# CONFIG_NETFILTER_XT_TARGET_TEE is not set
# CONFIG_NETFILTER_XT_TARGET_TPROXY is not set
# CONFIG_NETFILTER_XT_TARGET_TCPMSS is not set
# CONFIG_NETFILTER_XT_TARGET_TCPOPTSTRIP is not set
CONFIG_NETFILTER_XT_MATCH_ADDRTYPE=m
# CONFIG_NETFILTER_XT_MATCH_BPF is not set
# CONFIG_NETFILTER_XT_MATCH_CGROUP is not set
# CONFIG_NETFILTER_XT_MATCH_CLUSTER is not set
# CONFIG_NETFILTER_XT_MATCH_COMMENT is not set
# CONFIG_NETFILTER_XT_MATCH_CONNBYTES is not set
# CONFIG_NETFILTER_XT_MATCH_CONNLABEL is not set
# CONFIG_NETFILTER_XT_MATCH_CONNLIMIT is not set
# CONFIG_NETFILTER_XT_MATCH_CONNMARK is not set
CONFIG_NETFILTER_XT_MATCH_CONNTRACK=m
# CONFIG_NETFILTER_XT_MATCH_CPU is not set
# CONFIG_NETFILTER_XT_MATCH_DCCP is not set
# CONFIG_NETFILTER_XT_MATCH_DEVGROUP is not set
# CONFIG_NETFILTER_XT_MATCH_DSCP is not set
# CONFIG_NETFILTER_XT_MATCH_ECN is not set
# CONFIG_NETFILTER_XT_MATCH_ESP is not set
# CONFIG_NETFILTER_XT_MATCH_HASHLIMIT is not set
# CONFIG_NETFILTER_XT_MATCH_HELPER is not set
# CONFIG_NETFILTER_XT_MATCH_HL is not set
# CONFIG_NETFILTER_XT_MATCH_IPCOMP is not set
# CONFIG_NETFILTER_XT_MATCH_IPRANGE is not set
CONFIG_NETFILTER_XT_MATCH_IPVS=m
# CONFIG_NETFILTER_XT_MATCH_L2TP is not set
# CONFIG_NETFILTER_XT_MATCH_LENGTH is not set
# CONFIG_NETFILTER_XT_MATCH_LIMIT is not set
# CONFIG_NETFILTER_XT_MATCH_MAC is not set
# CONFIG_NETFILTER_XT_MATCH_MARK is not set
# CONFIG_NETFILTER_XT_MATCH_MULTIPORT is not set
# CONFIG_NETFILTER_XT_MATCH_NFACCT is not set
# CONFIG_NETFILTER_XT_MATCH_OSF is not set
# CONFIG_NETFILTER_XT_MATCH_OWNER is not set
# CONFIG_NETFILTER_XT_MATCH_PHYSDEV is not set
# CONFIG_NETFILTER_XT_MATCH_PKTTYPE is not set
# CONFIG_NETFILTER_XT_MATCH_QUOTA is not set
# CONFIG_NETFILTER_XT_MATCH_RATEEST is not set
# CONFIG_NETFILTER_XT_MATCH_REALM is not set
# CONFIG_NETFILTER_XT_MATCH_RECENT is not set
# CONFIG_NETFILTER_XT_MATCH_SCTP is not set
# CONFIG_NETFILTER_XT_MATCH_SOCKET is not set
# CONFIG_NETFILTER_XT_MATCH_STATE is not set
# CONFIG_NETFILTER_XT_MATCH_STATISTIC is not set
# CONFIG_NETFILTER_XT_MATCH_STRING is not set
# CONFIG_NETFILTER_XT_MATCH_TCPMSS is not set
# CONFIG_NETFILTER_XT_MATCH_TIME is not set
# CONFIG_NETFILTER_XT_MATCH_U32 is not set

Is it complaining about # CONFIG_NETFILTER_XT_MATCH_COMMENT is not set specifically ? Or it's a generic error that doesn't provide much information ? It just says msg: Warning: Extension comment revision 0 not supported ...

I just found the scripts:

https://raw.githubusercontent.com/opencontainers/runc/main/script/check-config.sh
https://raw.githubusercontent.com/moby/moby/master/contrib/check-config.sh

These throw some missing (optional) modules. None under the "generally necessary" section though.
I will likely need to recompile the kernel for this to work though, which most likely brings its own set of problem on a SBC (Rock 5B). Last time I tried I couldn't even get a custom kernel to boot, because the default would always be selected, no matter what I did to the bootloader (extlinux).

Note if you run rootless iptables is not allowed to load the modules for you, so in this case they must already be loaded before that by root. So first I would make sure if this runs as root, if it does you just need to make sure the right module is loaded don boot, i.e. https://github.com/containers/podman/blob/main/contrib/modules-load.d/podman-iptables.conf
Anyway this is clearly not an upstream issue so I close it.


TopNavLink should include to support

TopNavLink and SideNavLink should support to parameter for dynamic routing.

Hey @ajturner! I think this may just be a hole in our docs.
While we don't think Calcite React should be in the router game, it is very easy to integrate with router libraries by using the as prop. It looks like we haven't done a great job documenting as... but it allows you to change the rendered component - and since we pass all your props to the rendered component, you can do things like this:
<TopNavLink as={NavLink} to="/plans">Plans</TopNavLink>

Here's a simple example:
https://codesandbox.io/s/wwx035kyx8

Closing this one for now (will work on #236). @ajturner please let us know if this doesn't meet your need.


[Bug] Duplicates in issue report

Description
There appear to be some duplicates in the new issue reporting page.
Reproduction

Attempt to create a new issue
See duplicates of "Bug report" and "Feature request"

Expectation
We should probably have just one per type.
Screenshots

Additional context

Resolution

[ ] 🙋 I would be interested in resolving this bug.

The list of issue templates are here: https://github.com/WordPress/openverse-catalog/tree/main/.github/ISSUE_TEMPLATE


Update Spanish README.md

Fixes #3314
All Submissions:

[x] Have you followed the guidelines stated in CONTRIBUTING.md file?
[ x Have you checked to ensure there aren't any other open Pull Requests for the desired changed?

Type of change
Please delete options that are not relevant.

[x] This change requires a documentation update

Context
Update the docs/es/README.md to match the latest version of the main README.md

Thanks for contribution.
We've just merged #3313 which slightly change the demo code, if you can adjust your PR to match we'll be able to merged it.

Thanks for contribution.
We've just merged #3313 which slightly change the demo code, if you can adjust your PR to match we'll be able to merged it.

Done 🤘🏼


11474 Nature Of Business component

Issue #: /bcgov/entity#11474
Description of changes:

implemented Nature Of Business component
updated some interfaces
added some theme variables
some lint fixes

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of the business-filings-ui license (Apache 2.0).

Storybook:


Stable versions 0.x and roadmap to 1.0

Is there a plan to release stable versions 0.x? Currently Flutter is downloaded from master branch which is actively developed. After each flutter upgrade I am not sure if some critical bugs won't be introduced and block the development. Would be nice to start releasing Flutter with some QA involved.

Is there any roadmap to 1.0 with features that are "must haves" to release final version?

What exactly do you mean by "stable" and "final"? (I'm not being facetious, these things mean different things to different people so it's hard to answer your question without specific definitions.)

By 0.x stable i mean: it has no critical bugs
By 1.0 final i mean: it is ready for production apps

People have shipped production apps with Flutter already.
If there are any critical bugs that you are running into, please let us know, and we will prioritise them.

Is there a plan to release stable versions 0.x?

We're just starting to spin up a release process for an alpha branch, which has some automated vetting:
https://github.com/flutter/flutter/wiki/Release-process
Once that is in place, it should be more stable than the master branch, which receives changes as they are made.

Is there any roadmap to 1.0 with features that are "must haves" to release final version?

You can see how we've broken down the outstanding issues into numbered milestones:
https://github.com/flutter/flutter/milestones
I'm sure how those milestones related to numbered releases, but they should give you a rough sense for which issue we perceive are more critical.  (If you think we've misprioritized, please let us know.)

@abarth Thank you!

My client wants to do hybrid apps and he is looking at ionic or react native, I am looking at flutter and tried to do bring him into it. He trusts my word but he is feeling uncomfortable because flutter is still in beta. for developers and programmers flutter will be very interesting to try their pet projects or some organizations may courageous enough to try something in beta for production apps. But for most people, it is one major factor to decide whether o use it or not.
The fact that someone doing production app on our beta couldn't be an excuse for not having a stable release plan. I think flutter is stable enough so why don't we just release version 1.0 and work on the features incrementally? (Remember what happened to angular 2, its year-long development is what gave its place to react(IMHO).)

@agiratech-vigneshm What's the point in making a Flutter final release, when the underlying language (Dart 2) is not even beta? I don't see how rushing would provide better stability guarantees.

@zoechi So does it mean that if Dart 2 is not beta or released Flutter wont be released or should not be released? I think @agiratech-vigneshm is looking for some form guarantees for a client which could equally be provided with Flutter in beta...since someone already did it with alpha

I think Dart 2 is quite close. I guess they're working hard to be able to make an announcement at I/O 2018, but such discussions are pointless anyway. It's ready when it's ready (https://en.wikipedia.org/wiki/Forward-looking_statement)

and I think you can use it as is right now as long there's nothing breaking or blocking you at the as is now

@agiratech-vigneshm React native is not in 1.0 neither. And look at Facebook Parse, shut down quite abruptly, so it does not mean much anyway.
Flutter should not reach 1.0 yet as it is not feature complete and it would probably slow down the current development pace (major release version usually gets support commitment without any major or breaking change).
We are a development studio and we are starting to onboard some of our clients on Flutter. Best way is to be plain honest and transparent on pros and cons regarding 2 natives apps written in Swift/Kotlin VS 1 native app written in Flutter. Depending on the project requirements, budget and time to market, Flutter can make lot of sense.
There is always a risk when choosing an alternative technology, and it applies the same to Ionic or React native. The big difference with Flutter right now is the missing features: map support, solid IAP support, native push notifications support (not firebase), proper database support, etc.

Two questions for those of you interested in this:

What are the key characteristics of a release that you would want to see in a "Flutter 1.0"?
What questions do you want to see answered in a roadmap?

Parity on development possibilities with android/ios SDK:
Every feature that can developed in an app written with the platform SDK should be possible to be developed in a Flutter app, even it requires heavy custom plugin development.
One thing customers hate more than the risk of Flutter being abandoned is the risk of not being able to develop a feature because it's technically not possible in Flutter (without being a flutter core developer). An obvious example is the native map support.
Personally I would also like to have a crash reporting tool, which I think is a must.

Short/mid/long term visibility on development priorities, high level goals, what's the direction of the project. Having everything open and accessible on Github is great, but because of the project's activity following the github issues and discussions is a daily job 😝

Thanks for the response guys, I really love to work with flutter and I am doing my side projects in it moving from ionic and I stopped learning Native script as flutter seems the better choice. But I just can't convince my client to flutter his excuse is (it's in alpha/beta) and he is not heard about it. I just expected a roadmap or a vision that when a stable release will be. I understand plans can always change but we must have a plan and try to stick with it. I know people working on flutter core knows better than me, I just wanted to convey what I felt. I meant no offense. salute to all your hard work and I am looking forward to working more with flutter.

@hixie : For me, I would expect good documentation for every widgets and features. In my opinion Flutter is already ready for production use..
Good documentation will help everyone to quickly understand and implement. It's easy to convince your clients too.

@lukaspili https://github.com/flutter/sentry works great for me.

Totally missed out the sentry plugin, thanks @zoechi!

I have tested the beta version and it does seem production ready. However, after having experienced moving from AngularJS to Angular in the past (I confess I didn't really like AngularJS but love Angular so totally worth it), my main concern is whether Flutter may undergo the same major change between beta and 1.0.
Considering how time consuming it was to move a project from AngularJS to Angular, I am afraid creating a project with Flutter today may lead to a total rewrite when 1.0 is released.

@zoechi I like that approach. I am aware that there is no 100% safe code or framework, even with native (remember Objectiv C? ;-)) and my interrogation is probably more related to Angular=Google, Flutter=Google so I wondered if Flutter may undergo the same rebuild for v.1.0. Probably an irrational fear but nevertheless an interrogation.
This being said, it will not stop me from exploring Flutter even more as I am becoming more and more a fan of the framework.
Also, I DO keep in mind that there is no "one solution fits all" framework or language but rather a thought through choice to be made per project.

@benjaminhorner so was what I mentioned about Angular what caused your concerns? Just curious.
If they have good reasons, like with Angular (actually wasn't that bad with Angular for me), I prefer breaking changes over being stuck with something known to not be as good as it could be and hurting maintainability and progress in the future.

@zoechi yes you were spot on.


Vulnerability roundup 98: mutt-1.14.7: 1 advisory [5.3]

search, files

[ ] CVE-2020-28896 CVSSv3=5.3 (nixos-20.09)

Scanned versions: nixos-20.09: aa5b9cd16b9.
Cc @rnhmjoj

This should have been fixed by: https://github.com/NixOS/nixpkgs/pull/104584.

This should have been fixed by: https://github.com/NixOS/nixpkgs/pull/104584.


iPad Screenshots?

Great work Brenden :+1:
My only question is, why no iPad support?

@madclouds @bartl-git @nunofgs @aichholzer I've added iPad screenshots to the template in this pull request: https://github.com/clusterinc/SketchToAppStore/pull/8


Improve console and migration logging

This pull request introduces a new plainFormatter to make logs more human-readable. Compare how logs look like when you debug the system (without CloudWatch/Kibana)
Gist Commit 1 = before
Gist Commit 2 = after
https://gist.github.com/lzap/5d13da5e4ccc3184580cfd3f7c1e510e
Edit: Please merge https://github.com/RedHatInsights/edge-api/pull/2497 first there will be conflicts due to those mocks, I would rather rebase this one than the other.

LGTM, but I'm afraid to ack so the bot doesn't merge it. In hindsight, I'm going to turn that off xD

Please merge https://github.com/RedHatInsights/edge-api/pull/2497 first there will be conflicts due to those mocks, I would rather rebase this one than the other.


"Save as type" is very very wide

Only because of all the extensions in XML, do we really need to show all of them?

#12248

Limiting to at most 10 extensions now.


Consider to support http delete method with body by using OkHttp 2.3.0

Can you consider this feature?

This is added to wasp. in the next version all http methods are supported.

added in release v1.11


[BUG] : cant handle multi return values

so basically code is
const factory = new LuaFactory()
const engine = await factory.createEngine();

const fn= await engine.doString(`
      return function ()
        return '22', '33', {};
      end;
      `);

const result = fn(); // result is '22' while expected ['22','33', {}]
// same there
const result2 = await engine.doString(`
      return '11','22',{};
      `);
// result  is '11' while expected ['11','22',{}]

after some time i figure out that it always return ONLY one result for functions (which is wrong. bc .call() returns multiresult)
this is fix for functions
engine.global.typeExtensions[4].extension.__proto__.getValue = (thread: Thread, index: number) => {
        thread.lua.lua_pushvalue(thread.address, index)
        const func = thread.lua.luaL_ref(thread.address, LUA_REGISTRYINDEX)

        const jsFunc = (...args: any[]): any => {
          if (thread.isClosed()) {
            console.warn('Tried to call a function after closing lua state')
            return
          }

          const internalType = thread.lua.lua_rawgeti(thread.address, LUA_REGISTRYINDEX, func)
          if (internalType !== LuaType.Function) {
            const callMetafieldType = thread.lua.luaL_getmetafield(thread.address, -1, '__call')
            thread.pop()
            if (callMetafieldType !== LuaType.Function) {
              throw new Error(`A value of type '${internalType}' was pushed but it is not callable`)
            }
          }

          for (const arg of args) {
            thread.pushValue(arg)
          }

          const base = thread.getTop();
          const status = thread.lua.lua_pcallk(thread.address, args.length, LUA_MULTRET, 0, 0, null)
          if (status === LuaReturn.Yield) {
            throw new Error('cannot yield in callbacks from javascript')
          }
          thread.assertOk(status)
          const multiReturn = thread.getStackValues(thread.getTop() - base - args.length);

          // const result = thread.getValue(-1)

          thread.pop()
          return multiReturn;
        }

        ext.functionRegistry?.register(jsFunc, func)

        return jsFunc
      };

      ```

      And this is fix for just return
      ```ts
      engine.doString = async (code : string) => {
        const thread = engine.global.newThread()
        const threadIndex = engine.global.getTop()
        try {
            thread.loadString(code);
            const results = await thread.run(0)
            if (results.length > 0) {
              engine.cmodule.lua_xmove(thread.address, engine.global.address, results.length)
            }
            return results;
        } finally {
            // Pop the read on success or failure
            engine.global.remove(threadIndex)
        }
      };

      ```

now need to figure out how to merge this changes without breaking changes. like options for engine
{
  // ....
  alwaysMultiReturn : true // false by default
}

@ceifa seems i need your advice there

around 10 test's breaks if wrap returns in
if(returnsLength > 1)
  return returns;

return returns[0];

const [result] = fn();

Is looks much predicteble when we always mean that lua function returns multiresult.
Or we can use kinda helper method that
const result = fn().singleResult();

where
array.prototype.singleResult = function()
{
  return this.at(0);
}

https://github.com/ParadiseFallen/wasmoon/blob/main/test/engine.test.js#L268
So this test is always failed. seems like top calculated not correctly

https://github.com/ceifa/wasmoon/issues/84
Fix is right there

@ceifa i can change apis to always return LuaReturn or just array. It will be breaking change. I want this feature

@ceifa i can change apis to always return LuaReturn or just array. It will be breaking change. I want this feature

@ParadiseFallen this is not actually a bug, it was designed to work that way. We need to understand now which design would be better for readability/functionality/DX.
What is your use case?

@ceifa use function from lua with multireturn.
So for example if you want to call require in js side from lua it must return module & optionally path to module.
It will also good to use with other methods.
I suggest
Always return an array expecting mulret

It looks like
let [module,path] = luaRequire('abc');

And if it still
function ()
  return 123;
end

on js will looks like
let [result] = luaMethod();


add stubs_with/2 for stubbing whole module

As per https://github.com/edgurgel/mimic/issues/30
Adding stubs_with/2 function which could mock a module with another module

Thanks for sending this PR!  ❤
I gave a quick look and it looks pretty good 👍 but at the moment pretty busy! I hope that I have some time next week to review it properly


fix: fix playground dynamic layout cannot work

fix: #30

Summary by CodeRabbit

New Features

Added the ability to disable the layout for specific pages via route configuration.

虽然和本主题无关，不过现在 playground 有两个 uni-pages.d.ts

刚才可以先不合并，@我一下，我能顺便给删了


Needs more info

[Enter feedback here]
Followed this guide but it does not work. There needs to be more informtion or detail about the nuget section and creating the package. Since I have followed this but nothing is installed when this package installs.
thanks

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 35b3cd51-b365-433b-9149-697582d64e88
Version Independent ID: 1cb1f4a3-5928-223d-592f-f3462fef4e4c
Content: Use Azure Artifacts feeds as a private PowerShell repository - Azure Artifacts
Content Source: docs/artifacts/tutorials/private-powershell-library.md
Service: azure-devops-artifacts
GitHub Login: @ramiMSFT
Microsoft Alias: rabououn

Hey all,
I have a PowerShell module I am continuously writing. It's a Toolkit that when I create scripts I add as functions and can then use them over and over. I would like to share this privately with my colleges using a private Project.
I have an azure DevOps account, created the Repo and uploaded all my code for source control, have created a feed and have been following this guide:
Use Azure Artifacts feeds as a private PowerShell repository - Azure Artifacts | Microsoft Learn
This has uploaded the Toolkit package I created and I can do an install-module and it says its installing but it does not actually seem to install the module at all.
The module has a manifest file and PowerShell files as shown bellow:

r/azuredevops - Nuget packaging and Azure artifcats
My toolkit.nuspec looks like this:

r/azuredevops - Nuget packaging and Azure artifcats
I feel like I must be missing some sort of linking to the files I need but I am not sure what I am missing.
Can anyone help me get this working?
Thanks

I got this working, I ended up adding a source folder and adding the powershell to that. This seemed to fix the issue and its working now.


problem with executing csruntime --host localhost hello.calvin

Hi, I'm trying to execute this code $ csruntime --host localhost hello.calvin in the mini tutorial, but I get this:
2017-09-08 16:51:28,040 ERROR 2286-calvin.calvin.utilities.runtime_credentials: get_domain: error while trying to read domain from Calvin config, err=argument of type 'NoneType' is not iterable
2017-09-08 16:51:28,040 INFO 2286-calvin.calvin.runtime.north.calvincontrol: Control API trying to listening on: localhost:5001
DEPLOY STATUS 200, OK
Deployed application 38835e32-a853-449c-9aeb-1c402e5561c5
2017-09-08 16:51:30,965 INFO 2286-calvin.calvin.runtime.north.calvin_node: All done, exiting
2017-09-08 16:51:30,980 INFO 2286-calvin.calvin.runtime.north.calvin_node: Quitting node "['calvinip://localhost:5000']"
I suppose to get something like this:
Deployed application 922a2096-bfd9-48c8-a5a4-ee900a180ca4
2016-07-11 08:20:34,667 INFO 11202-calvin.Log: Hello, world
2016-07-11 08:20:35,667 INFO 11202-calvin.Log: Hello, world
Any help would be appreciated. Thanks!

2017-09-08 16:51:28,040 ERROR 2286-calvin.calvin.utilities.runtime_credentials: get_domain: error while trying to read domain from Calvin config, err=argument of type 'NoneType' is not iterable

This is actually OK. It just means that Calvin's security isn't configured on your machine and it will fall back to running with security disabled. The log message should really be changed to something more informative.

2017-09-08 16:51:28,040 INFO 2286-calvin.calvin.runtime.north.calvincontrol: Control API trying to listening on: localhost:5001
DEPLOY STATUS 200, OK
Deployed application 38835e32-a853-449c-9aeb-1c402e5561c5

This is actually confirmation that your example is running...

2017-09-08 16:51:30,965 INFO 2286-calvin.calvin.runtime.north.calvin_node: All done, exiting
2017-09-08 16:51:30,980 INFO 2286-calvin.calvin.runtime.north.calvin_node: Quitting node "['calvinip://localhost:5000']"

...and this is the runtime quitting as it is supposed to do when started like this.
However, the default is for the runtime to quit after 3s (!) when started with a file argument. I suspect that your runtime simply didn't have enough time to produce output. Try changing the keep-alive time to 10s or perhaps even longer
csruntime --host localhost --keep-alive 10 hello.calvin
and see if that helps.
The ability to add a script to the csruntime command is just convenience to allow quick testing of a script, the "normal" way to start a runtime is to start it without a script argument, in which case it will keep on running forever and you feed it applications using the cscontrol command.

Thank you! It works now.


Update ReadMe for graphic  location

Please update ReadMe to include file location of where to place the graphic the program updates.  And graphic requirements/size.

Place your graphics here. https://github.com/benyaminahmed/nft-image-generator/tree/main/trait-layers
For size and dimension, different projects use different sizes.


ipa issue

I use altstore to install the ipa, and it tell that "The app is in an invaild format"

It’s somehow a common issue with altstore and it’s not necessary a problem with the ipa itself.
To fix it just extract the ipa in filza and zip it back. Open the new ipa in altstore and should now work


Image shows full outer join instead of inner join

The top image shows a full outer join, instead of an inner join.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 123ac609-3092-d713-ec4b-299dad23310d
Version Independent ID: 42bc7a48-4e9f-6dd5-8c40-d727b328a18f
Content: Inner join
Content Source: powerquery-docs/merge-queries-inner.md
Service: powerquery
GitHub Login: @ptyx507
Microsoft Alias: v-miesco

Thank you for reporting this! We've recently pushed an update to the article to address this issue.
Please keep the feedback coming and let us know if there's anything else that we could do to improve the content.


Last plotly par chart has truncated university names

And why does it have more entries than earlier examples of the following highchart one.

Agreed, add a note to this page and link back to the plotly/highchart comparison page inline.