error in entry template execution: Maximum recursion depth exceeded. See https://argoproj.github.io/argo-workflows/scaling/#maximum-recursion-depth

Pre-requisites

[X] I have double-checked my configuration
[X] I can confirm the issues exists when I tested with :latest
[ ] I'd like to contribute the fix myself (see contributing guide)

What happened/what you expected to happen?
never saw this error in 3.4.11
but after upgrading to 3.5.1
got "error in entry template execution: Maximum recursion depth exceeded. See https://argoproj.github.io/argo-workflows/scaling/#maximum-recursion-depth" on 2 different small (1 step) workflows
Version
3.5.1
Paste a small workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflows that uses private images.
n/a

Logs from the workflow controller
kubectl logs -n argo deploy/workflow-controller | grep ${workflow}

Logs from in your workflow's wait container
kubectl logs -n argo -c wait -l workflows.argoproj.io/workflow=${workflow},workflow.argoproj.io/phase!=Succeeded

never saw this error in 3.4.11

Correct, that's a new feature in 3.5, not a bug. Implemented by #11646.
It's mentioned in the 3.5 blog post as well (search "recursion").

on 2 different small (1 step) workflows

You'll need to provide an example workflow, which is missing from your issue. I can't reproduce this and we've had many contributors and companies using 3.5+
In general, more details are almost always better than few details, which is why they are asked for in issue templates.

There does appear to be an issue with this specifically in 3.5.1 (not in 3.5.0). Our CI is failing on it, and definitely uses fewer than 100 recursions.
I'm working on something small and replicatable.

I'm looking into it, workflows that reproduce it would be helpful though.

Working with @tico24 here, we've not managed to reproduce this on the same installation with the same workflow. He'd disabled the feature with DISABLE_MAX_RECURSION but re-enabling it and re-running the same workloads doesn't cause the error.
@tico24 had updated to 3.5.1 from 3.5.0 this morning and it started happening near immediately. He's left it so it 'should' fail again.
I really need a runnable workflow @tooptoop4. Does restarting the controller alleviate the issue?

In the meantime, should we disable this feature by default instead?

It's a security, HA, multi-tenancy, etc feature, so I do think it should be on by default. But we should ofc fix any bugs there may be

I agree with @agilgur5, it would be bad to try and hide the bugs this way, we should either have it as a working feature or get rid of it. It's still working fine for @tico24 at the moment.

We tried to upgrade to v3.5.2 and faced this error in the controller logs. We are thinking to DISABLE_MAX_RECURSION and then upgrade to v3.5.2. Any suggestion or anything to be considered around this would be helpful. Thanks !

@guruprasathTT can you please provide a Workflow that reproduces this? We asked for one above a few times and without that, we wouldn't be able to find any possible bugs

@agilgur5
Hi, This is not happened with a particular WF or the Workflow using the particular Workflow template. After the upgrade immediately the workflow controller throws this error message for the running workflows (multiple workflows) but after that we don't see this error in the dev env for sometime.

This seems to be the case now with several upgrades to 3.5, in flight upgrades seem to cause this to happen, but afterwards it is fine.

I think the problem that not actually recursion is counted, but rather any "open" call to executeTemplate. So a workflow that doesn't recurse (that much) but does fan out into a lot of parallel steps can also hit this limit. (At least this seems to be the case for us, we've the feature now.)

I don't have a reproducible test case either, but we've been seeing the Maximum recursion depth exceeded. randomly show up in various workflows after recently upgrading to 3.5.2.
I'm not aware that we make use of recursion in the failing workflows. Our workflows are dynamic but have fixed numbers of nodes. Usually they fan out to fixed number of tasks based on configuration. Restarting the workflows seems to "fix" the issue, even though the number of steps remains the same.

@biochimia, can you see if it reoccurs after a workflow-controller restart? All cases I've had reported have been on a fresh upgrade only, but a controller restart has made them go away.

@roelarents - I didn't see your message before today. I'll try to reproduce that.

O, yeah, I have to retract that statement. It's not true. We tried to reproduce it too and a fan out does only increase it by 1. Sorry. Still trying to find out what does cause it. I tried to sneak in an extra log line for the concurrency so far. But I hadn't time to look at this after anymore (just disabled the max recursion check).

I don't recommend anybody setting DISABLE_MAX_RECURSION=true, because that will end up with stack overflow panics when you start hitting the problem in a workflow, and the argo workflow controller will keep restarting until you delete the problematic workflow.
We were still facing this issue in Argo 3.5.2, to debug the problem, I baked an image to do the mentioned log changed above and promoted its level from debug to info. Then in a cluster where we were having this issue, I switched to this image, setting DISABLE_MAX_RECURSION=false at the same time, and I got the following logs:
# while it was `DISABLE_MAX_RECURSION=true`
2024-05-14T00:04:35.20947308Z,"time=""2024-05-14T00:04:35.209Z"" level=info msg=""Processing workflow"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T00:04:35.252261778Z,"time=""2024-05-14T00:04:35.252Z"" level=info msg=""Task-result reconciliation"" namespace=default numObjs=0 workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T00:04:35.252807048Z,"time=""2024-05-14T00:04:35.252Z"" level=info msg=""Pod failed: Error (exit code 1)"" displayName=""run-train(0)"" namespace=default pod=rr-ml-k-module-xxx-74gmh-1-1379393644-3334131527 templateName=run-train workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T00:04:35.252924701Z,"time=""2024-05-14T00:04:35.252Z"" level=warning msg=""workflow uses legacy/insecure pod patch, see https://argoproj.github.io/argo-workflows/workflow-rbac/"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T00:04:35.252990251Z,"time=""2024-05-14T00:04:35.252Z"" level=warning msg=""workflow uses legacy/insecure pod patch, see https://argoproj.github.io/argo-workflows/workflow-rbac/"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T00:04:35.253160097Z,"time=""2024-05-14T00:04:35.253Z"" level=info msg=""node changed"" namespace=default new.message=""Error (exit code 1)"" new.phase=Failed new.progress=0/1 nodeID=rr-ml-k-module-xxx-74gmh-1-1379393644-3334131527 old.message= old.phase=Running old.progress=0/1 workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T00:04:35.253304318Z,"time=""2024-05-14T00:04:35.253Z"" level=info msg=""Workflow pod is missing"" namespace=default nodeName=""rr-ml-k-module-xxx-74gmh-1-1379393644.exit-handler-1.for-loop-5(4...<LONG PARAM LIST>...).run-train(0)"" nodePhase=Running recentlyStarted=false workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T00:04:35.253395075Z,"time=""2024-05-14T00:04:35.253Z"" level=info msg=""node rr-ml-k-module-xxx-74gmh-1-1379393644-794976774 phase Running -> Error"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T00:04:35.253462751Z,"time=""2024-05-14T00:04:35.253Z"" level=info msg=""node rr-ml-k-module-xxx-74gmh-1-1379393644-794976774 message: pod deleted"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T00:04:35.253543399Z,"time=""2024-05-14T00:04:35.253Z"" level=info msg=""node rr-ml-k-module-xxx-74gmh-1-1379393644-794976774 finished: 2024-05-14 00:04:35.253496244 +0000 UTC"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
# ... repeating the same pattern until we set `DISABLE_MAX_RECURSION` from true to false at 2024-05-14T22:12:57, then:
2024-05-14T22:12:57.868454091Z,"time=""2024-05-14T22:12:57.864Z"" level=info msg=""Processing workflow"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:57.905766307Z,"time=""2024-05-14T22:12:57.905Z"" level=info msg=""Task-result reconciliation"" namespace=default numObjs=0 workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:57.90807108Z,"time=""2024-05-14T22:12:57.906Z"" level=info msg=""Pod failed: Error (exit code 1)"" displayName=""run-train(0)"" namespace=default pod=rr-ml-k-module-xxx-74gmh-1-1379393644-3334131527 templateName=run-train workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:57.912209745Z,"time=""2024-05-14T22:12:57.912Z"" level=warning msg=""workflow uses legacy/insecure pod patch, see https://argoproj.github.io/argo-workflows/workflow-rbac/"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:57.912220182Z,"time=""2024-05-14T22:12:57.912Z"" level=warning msg=""workflow uses legacy/insecure pod patch, see https://argoproj.github.io/argo-workflows/workflow-rbac/"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:57.912294118Z,"time=""2024-05-14T22:12:57.912Z"" level=info msg=""node changed"" namespace=default new.message=""Error (exit code 1)"" new.phase=Failed new.progress=0/1 nodeID=rr-ml-k-module-xxx-74gmh-1-1379393644-3334131527 old.message= old.phase=Running old.progress=0/1 workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:57.912446978Z,"time=""2024-05-14T22:12:57.912Z"" level=info msg=""Workflow pod is missing"" namespace=default nodeName=""rr-ml-k-module-xxx-74gmh-1-1379393644.exit-handler-1.for-loop-5(4...<LONG PARAM LIST>...).run-train(0)"" nodePhase=Running recentlyStarted=false workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:57.912465037Z,"time=""2024-05-14T22:12:57.912Z"" level=info msg=""node rr-ml-k-module-xxx-74gmh-1-1379393644-794976774 phase Running -> Error"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:57.912484951Z,"time=""2024-05-14T22:12:57.912Z"" level=info msg=""node rr-ml-k-module-xxx-74gmh-1-1379393644-794976774 message: pod deleted"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:57.912544446Z,"time=""2024-05-14T22:12:57.912Z"" level=info msg=""node rr-ml-k-module-xxx-74gmh-1-1379393644-794976774 finished: 2024-05-14 22:12:57.912498416 +0000 UTC"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:57.912617332Z,"time=""2024-05-14T22:12:57.912Z"" level=info msg=""Evaluating node rr-ml-k-module-xxx-74gmh-1-1379393644: template: *v1alpha1.WorkflowStep (ml-k-module-xxx-ads-categorize-train), boundaryID: , currentStackDepth: 0"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:57.912878499Z,"time=""2024-05-14T22:12:57.912Z"" level=info msg=""Evaluating node rr-ml-k-module-xxx-74gmh-1-1379393644.exit-handler-1: template: *v1alpha1.DAGTask (exit-handler-1), boundaryID: rr-ml-k-module-xxx-74gmh-1-1379393644, currentStackDepth: 1"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:57.913926903Z,"time=""2024-05-14T22:12:57.913Z"" level=warning msg=""was unable to obtain the node for rr-ml-k-module-xxx-74gmh-1-1379393644-2682863345, taskName run-mlflow-log"""
2024-05-14T22:12:57.914737228Z,"time=""2024-05-14T22:12:57.914Z"" level=warning msg=""was unable to obtain the node for rr-ml-k-module-xxx-74gmh-1-1379393644-3044613789, taskName run-finalize"""
2024-05-14T22:12:57.914752884Z,"time=""2024-05-14T22:12:57.914Z"" level=warning msg=""was unable to obtain the node for rr-ml-k-module-xxx-74gmh-1-1379393644-2946699175, taskName list-all-trainings"""
2024-05-14T22:12:57.920401028Z,"time=""2024-05-14T22:12:57.920Z"" level=info msg=""Evaluating node rr-ml-k-module-xxx-74gmh-1-1379393644.exit-handler-1.for-loop-5(0...<LONG PARAM LIST>...): template: *v1alpha1.DAGTask (for-loop-5), boundaryID: rr-ml-k-module-xxx-74gmh-1-1379393644-2150580619, currentStackDepth: 2"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:57.921250032Z,"time=""2024-05-14T22:12:57.921Z"" level=info msg=""Evaluating node rr-ml-k-module-xxx-74gmh-1-1379393644.exit-handler-1.for-loop-5(1...<LONG PARAM LIST>...): template: *v1alpha1.DAGTask (for-loop-5), boundaryID: rr-ml-k-module-xxx-74gmh-1-1379393644-2150580619, currentStackDepth: 2"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:57.922065485Z,"time=""2024-05-14T22:12:57.921Z"" level=info msg=""Evaluating node rr-ml-k-module-xxx-74gmh-1-1379393644.exit-handler-1.for-loop-5(2...<LONG PARAM LIST>...): template: *v1alpha1.DAGTask (for-loop-5), boundaryID: rr-ml-k-module-xxx-74gmh-1-1379393644-2150580619, currentStackDepth: 2"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:57.922882744Z,"time=""2024-05-14T22:12:57.922Z"" level=info msg=""Evaluating node rr-ml-k-module-xxx-74gmh-1-1379393644.exit-handler-1.for-loop-5(3...<LONG PARAM LIST>...): template: *v1alpha1.DAGTask (for-loop-5), boundaryID: rr-ml-k-module-xxx-74gmh-1-1379393644-2150580619, currentStackDepth: 2"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:57.926746957Z,"time=""2024-05-14T22:12:57.923Z"" level=info msg=""Evaluating node rr-ml-k-module-xxx-74gmh-1-1379393644.exit-handler-1.for-loop-5(3...<LONG PARAM LIST>...).run-train: template: *v1alpha1.DAGTask (run-train), boundaryID: rr-ml-k-module-xxx-74gmh-1-1379393644-2035740812, currentStackDepth: 3"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
# ... repeating ~100 times
2024-05-14T22:12:58.46761812Z,"time=""2024-05-14T22:12:58.453Z"" level=info msg=""Evaluating node rr-ml-k-module-xxx-74gmh-1-1379393644.exit-handler-1.for-loop-5(3...<LONG PARAM LIST>...).run-train: template: *v1alpha1.DAGTask (run-train), boundaryID: rr-ml-k-module-xxx-74gmh-1-1379393644-2035740812, currentStackDepth: 99"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:58.472900557Z,"time=""2024-05-14T22:12:58.471Z"" level=error msg=""Mark error node"" error=""Maximum recursion depth exceeded. See https://argoproj.github.io/argo-workflows/scaling/#maximum-recursion-depth"" namespace=default nodeName=""rr-ml-k-module-xxx-74gmh-1-1379393644.exit-handler-1.for-loop-5(3...<LONG PARAM LIST>...).run-train"" workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:58.472931908Z,"time=""2024-05-14T22:12:58.471Z"" level=info msg=""node rr-ml-k-module-xxx-74gmh-1-1379393644-562365516 phase Running -> Error"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:58.472935313Z,"time=""2024-05-14T22:12:58.471Z"" level=info msg=""node rr-ml-k-module-xxx-74gmh-1-1379393644-562365516 message: Maximum recursion depth exceeded. See https://argoproj.github.io/argo-workflows/scaling/#maximum-recursion-depth"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:58.47293766Z,"time=""2024-05-14T22:12:58.471Z"" level=info msg=""node rr-ml-k-module-xxx-74gmh-1-1379393644-562365516 finished: 2024-05-14 22:12:58.471853015 +0000 UTC"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:58.473096379Z,"time=""2024-05-14T22:12:58.471Z"" level=error msg=""Mark error node"" error=""Maximum recursion depth exceeded. See https://argoproj.github.io/argo-workflows/scaling/#maximum-recursion-depth"" namespace=default nodeName=""rr-ml-k-module-xxx-74gmh-1-1379393644.exit-handler-1.for-loop-5(3...<LONG PARAM LIST>...).run-train(0)"" workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:58.473106159Z,"time=""2024-05-14T22:12:58.471Z"" level=error msg=""node is already fulfilled"" fromPhase=Failed namespace=default nodeName=""rr-ml-k-module-xxx-74gmh-1-1379393644.exit-handler-1.for-loop-5(3...<LONG PARAM LIST>...).run-train(0)"" toPhase=Error workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:58.473112838Z,"time=""2024-05-14T22:12:58.472Z"" level=info msg=""node rr-ml-k-module-xxx-74gmh-1-1379393644-3334131527 phase Failed -> Error"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:58.473115521Z,"time=""2024-05-14T22:12:58.472Z"" level=info msg=""node rr-ml-k-module-xxx-74gmh-1-1379393644-3334131527 message: Maximum recursion depth exceeded. See https://argoproj.github.io/argo-workflows/scaling/#maximum-recursion-depth"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:58.473162192Z,"time=""2024-05-14T22:12:58.472Z"" level=error msg=""Mark error node"" error=""Maximum recursion depth exceeded. See https://argoproj.github.io/argo-workflows/scaling/#maximum-recursion-depth"" namespace=default nodeName=""rr-ml-k-module-xxx-74gmh-1-1379393644.exit-handler-1.for-loop-5(3...<LONG PARAM LIST>...).run-train(0)"" workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
# ... repeating ~100 times
2024-05-14T22:12:58.517350212Z,"time=""2024-05-14T22:12:58.517Z"" level=error msg=""Mark error node"" error=""Maximum recursion depth exceeded. See https://argoproj.github.io/argo-workflows/scaling/#maximum-recursion-depth"" namespace=default nodeName=""rr-ml-k-module-xxx-74gmh-1-1379393644.exit-handler-1.for-loop-5(3...<LONG PARAM LIST>...).run-train(0)"" workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:58.517624319Z,"time=""2024-05-14T22:12:58.517Z"" level=info msg=""Outbound nodes of rr-ml-k-module-xxx-74gmh-1-1379393644-2035740812 set to [rr-ml-k-module-xxx-74gmh-1-1379393644-3334131527]"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:58.517658723Z,"time=""2024-05-14T22:12:58.517Z"" level=info msg=""node rr-ml-k-module-xxx-74gmh-1-1379393644-2035740812 phase Running -> Error"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:58.517662915Z,"time=""2024-05-14T22:12:58.517Z"" level=info msg=""node rr-ml-k-module-xxx-74gmh-1-1379393644-2035740812 finished: 2024-05-14 22:12:58.517618731 +0000 UTC"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:58.52270363Z,"time=""2024-05-14T22:12:58.522Z"" level=info msg=""Evaluating node rr-ml-k-module-xxx-74gmh-1-1379393644.exit-handler-1.for-loop-5(4...<LONG PARAM LIST>...): template: *v1alpha1.DAGTask (for-loop-5), boundaryID: rr-ml-k-module-xxx-74gmh-1-1379393644-2150580619, currentStackDepth: 2"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:58.523771669Z,"time=""2024-05-14T22:12:58.523Z"" level=info msg=""Evaluating node rr-ml-k-module-xxx-74gmh-1-1379393644.exit-handler-1.for-loop-5(4...<LONG PARAM LIST>...).run-train: template: *v1alpha1.DAGTask (run-train), boundaryID: rr-ml-k-module-xxx-74gmh-1-1379393644-2396377327, currentStackDepth: 3"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:58.864898413Z,"time=""2024-05-14T22:12:58.864Z"" level=info msg=""Created pod: rr-ml-k-module-xxx-74gmh-1-1379393644.exit-handler-1.for-loop-5(4...<LONG PARAM LIST>...).run-train(0) (rr-ml-k-module-xxx-74gmh-1-1379393644-794976774)"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:58.865015582Z,"time=""2024-05-14T22:12:58.864Z"" level=info msg=""Evaluating node rr-ml-k-module-xxx-74gmh-1-1379393644.exit-handler-1.for-loop-5(4...<LONG PARAM LIST>...).run-train: template: *v1alpha1.DAGTask (run-train), boundaryID: rr-ml-k-module-xxx-74gmh-1-1379393644-2396377327, currentStackDepth: 4"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:59.418885046Z,"time=""2024-05-14T22:12:59.418Z"" level=info msg=""Failed pod rr-ml-k-module-xxx-74gmh-1-1379393644.exit-handler-1.for-loop-5(4...<LONG PARAM LIST>...).run-train(0) (rr-ml-k-module-xxx-74gmh-1-1379393644-794976774) creation: already exists"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:59.41905412Z,"time=""2024-05-14T22:12:59.418Z"" level=info msg=""Evaluating node rr-ml-k-module-xxx-74gmh-1-1379393644.exit-handler-1.for-loop-5(4...<LONG PARAM LIST>...).run-train: template: *v1alpha1.DAGTask (run-train), boundaryID: rr-ml-k-module-xxx-74gmh-1-1379393644-2396377327, currentStackDepth: 5"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:59.671258908Z,"time=""2024-05-14T22:12:59.671Z"" level=info msg=""Evaluating node rr-ml-k-module-xxx-74gmh-1-1379393644.exit-handler-1.for-loop-5(4...<LONG PARAM LIST>...).run-train: template: *v1alpha1.DAGTask (run-train), boundaryID: rr-ml-k-module-xxx-74gmh-1-1379393644-2396377327, currentStackDepth: 99"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
# ... repeating ~100 times
2024-05-14T22:12:59.671361012Z,"time=""2024-05-14T22:12:59.671Z"" level=error msg=""Mark error node"" error=""Maximum recursion depth exceeded. See https://argoproj.github.io/argo-workflows/scaling/#maximum-recursion-depth"" namespace=default nodeName=""rr-ml-k-module-xxx-74gmh-1-1379393644.exit-handler-1.for-loop-5(4...<LONG PARAM LIST>...).run-train"" workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:59.671370981Z,"time=""2024-05-14T22:12:59.671Z"" level=info msg=""node rr-ml-k-module-xxx-74gmh-1-1379393644-4042243083 phase Running -> Error"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:59.671377684Z,"time=""2024-05-14T22:12:59.671Z"" level=info msg=""node rr-ml-k-module-xxx-74gmh-1-1379393644-4042243083 message: Maximum recursion depth exceeded. See https://argoproj.github.io/argo-workflows/scaling/#maximum-recursion-depth"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:59.671380425Z,"time=""2024-05-14T22:12:59.671Z"" level=info msg=""node rr-ml-k-module-xxx-74gmh-1-1379393644-4042243083 finished: 2024-05-14 22:12:59.671315687 +0000 UTC"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:59.671445639Z,"time=""2024-05-14T22:12:59.671Z"" level=error msg=""Mark error node"" error=""Maximum recursion depth exceeded. See https://argoproj.github.io/argo-workflows/scaling/#maximum-recursion-depth"" namespace=default nodeName=""rr-ml-k-module-xxx-74gmh-1-1379393644.exit-handler-1.for-loop-5(4...<LONG PARAM LIST>...).run-train(0)"" workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:59.671451251Z,"time=""2024-05-14T22:12:59.671Z"" level=info msg=""node rr-ml-k-module-xxx-74gmh-1-1379393644-794976774 message: Maximum recursion depth exceeded. See https://argoproj.github.io/argo-workflows/scaling/#maximum-recursion-depth"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:59.671485433Z,"time=""2024-05-14T22:12:59.671Z"" level=error msg=""Mark error node"" error=""Maximum recursion depth exceeded. See https://argoproj.github.io/argo-workflows/scaling/#maximum-recursion-depth"" namespace=default nodeName=""rr-ml-k-module-xxx-74gmh-1-1379393644.exit-handler-1.for-loop-5(4...<LONG PARAM LIST>...).run-train(0)"" workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
# ... repeating ~100 times
024-05-14T22:12:59.693476433Z,"time=""2024-05-14T22:12:59.678Z"" level=error msg=""Mark error node"" error=""Maximum recursion depth exceeded. See https://argoproj.github.io/argo-workflows/scaling/#maximum-recursion-depth"" namespace=default nodeName=""rr-ml-k-module-xxx-74gmh-1-1379393644.exit-handler-1.for-loop-5(4...<LONG PARAM LIST>...).run-train(0)"" workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:59.693480732Z,"time=""2024-05-14T22:12:59.678Z"" level=info msg=""Outbound nodes of rr-ml-k-module-xxx-74gmh-1-1379393644-2396377327 set to [rr-ml-k-module-xxx-74gmh-1-1379393644-794976774]"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:59.693483374Z,"time=""2024-05-14T22:12:59.678Z"" level=info msg=""node rr-ml-k-module-xxx-74gmh-1-1379393644-2396377327 phase Running -> Error"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:59.69348586Z,"time=""2024-05-14T22:12:59.678Z"" level=info msg=""node rr-ml-k-module-xxx-74gmh-1-1379393644-2396377327 finished: 2024-05-14 22:12:59.678392409 +0000 UTC"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:59.693487846Z,"time=""2024-05-14T22:12:59.678Z"" level=info msg=""node rr-ml-k-module-xxx-74gmh-1-1379393644-2704183877 phase Running -> Error"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:59.693489799Z,"time=""2024-05-14T22:12:59.678Z"" level=info msg=""node rr-ml-k-module-xxx-74gmh-1-1379393644-2704183877 finished: 2024-05-14 22:12:59.678511785 +0000 UTC"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:59.693492096Z,"time=""2024-05-14T22:12:59.678Z"" level=warning msg=""was unable to obtain the node for rr-ml-k-module-xxx-74gmh-1-1379393644-2946699175, taskName list-all-trainings"""
2024-05-14T22:12:59.693494139Z,"time=""2024-05-14T22:12:59.678Z"" level=info msg=""Skipped node rr-ml-k-module-xxx-74gmh-1-1379393644-2946699175 initialized Omitted (message: omitted: depends condition not met)"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:59.693496191Z,"time=""2024-05-14T22:12:59.678Z"" level=warning msg=""was unable to obtain the node for rr-ml-k-module-xxx-74gmh-1-1379393644-3044613789, taskName run-finalize"""
2024-05-14T22:12:59.693499875Z,"time=""2024-05-14T22:12:59.678Z"" level=info msg=""Skipped node rr-ml-k-module-xxx-74gmh-1-1379393644-3044613789 initialized Omitted (message: omitted: depends condition not met)"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:59.693502123Z,"time=""2024-05-14T22:12:59.678Z"" level=warning msg=""was unable to obtain the node for rr-ml-k-module-xxx-74gmh-1-1379393644-2682863345, taskName run-mlflow-log"""
2024-05-14T22:12:59.693504133Z,"time=""2024-05-14T22:12:59.678Z"" level=info msg=""Skipped node rr-ml-k-module-xxx-74gmh-1-1379393644-2682863345 initialized Omitted (message: omitted: depends condition not met)"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:59.693507768Z,"time=""2024-05-14T22:12:59.679Z"" level=info msg=""Outbound nodes of rr-ml-k-module-xxx-74gmh-1-1379393644-2150580619 set to [rr-ml-k-module-xxx-74gmh-1-1379393644-2682863345]"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:59.693509795Z,"time=""2024-05-14T22:12:59.679Z"" level=info msg=""node rr-ml-k-module-xxx-74gmh-1-1379393644-2150580619 phase Running -> Error"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:59.693511871Z,"time=""2024-05-14T22:12:59.679Z"" level=info msg=""node rr-ml-k-module-xxx-74gmh-1-1379393644-2150580619 finished: 2024-05-14 22:12:59.679122842 +0000 UTC"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:59.693584338Z,"time=""2024-05-14T22:12:59.679Z"" level=info msg=""Outbound nodes of rr-ml-k-module-xxx-74gmh-1-1379393644 set to [rr-ml-k-module-xxx-74gmh-1-1379393644-2682863345]"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:59.693586783Z,"time=""2024-05-14T22:12:59.689Z"" level=info msg=""node rr-ml-k-module-xxx-74gmh-1-1379393644 phase Running -> Error"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:59.693589171Z,"time=""2024-05-14T22:12:59.689Z"" level=info msg=""node rr-ml-k-module-xxx-74gmh-1-1379393644 finished: 2024-05-14 22:12:59.689573024 +0000 UTC"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:59.693591599Z,"time=""2024-05-14T22:12:59.689Z"" level=info msg=""TaskSet Reconciliation"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:59.693593733Z,"time=""2024-05-14T22:12:59.689Z"" level=info msg=reconcileAgentPod namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:59.693595694Z,"time=""2024-05-14T22:12:59.689Z"" level=info msg=""Running OnExit handler: exit-handlers-train"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:59.693598475Z,"time=""2024-05-14T22:12:59.689Z"" level=info msg=""Evaluating node rr-ml-k-module-xxx-74gmh-1-1379393644.onExit: template: *v1alpha1.WorkflowStep (exit-handlers-train), boundaryID: , currentStackDepth: 0"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:59.693600994Z,"time=""2024-05-14T22:12:59.689Z"" level=warning msg=""Node was nil, will be initialized as type Skipped"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:59.693604096Z,"time=""2024-05-14T22:12:59.691Z"" level=info msg=""was unable to obtain node for , letting display name to be nodeName"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:59.693606295Z,"time=""2024-05-14T22:12:59.691Z"" level=info msg=""Retry node rr-ml-k-module-xxx-74gmh-1-1379393644-1646786213 initialized Running"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:59.693608247Z,"time=""2024-05-14T22:12:59.692Z"" level=info msg=""was unable to obtain node for , letting display name to be nodeName"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:12:59.693610282Z,"time=""2024-05-14T22:12:59.692Z"" level=info msg=""Pod node rr-ml-k-module-xxx-74gmh-1-1379393644-1649482756 initialized Pending"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:13:00.291666651Z,"time=""2024-05-14T22:13:00.291Z"" level=info msg=""Created pod: rr-ml-k-module-xxx-74gmh-1-1379393644.onExit(0) (rr-ml-k-module-xxx-74gmh-1-1379393644-1649482756)"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:13:00.347960727Z,"time=""2024-05-14T22:13:00.347Z"" level=info msg=""Workflow update successful"" namespace=default phase=Running resourceVersion=2131354266 workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:13:00.351118572Z,"time=""2024-05-14T22:13:00Z"" level=info msg=""Processing object (rr-ml-k-module-xxx-74gmh-1-1379393644): object has no owner."" Workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:13:00.358677615Z,"time=""2024-05-14T22:13:00.358Z"" level=info msg=""cleaning up pod"" action=labelPodCompleted key=default/rr-ml-k-module-xxx-74gmh-1-1379393644-3334131527/labelPodCompleted"
2024-05-14T22:13:00.358858972Z,"time=""2024-05-14T22:13:00.358Z"" level=info msg=""cleaning up pod"" action=labelPodCompleted key=default/rr-ml-k-module-xxx-74gmh-1-1379393644-794976774/labelPodCompleted"
2024-05-14T22:13:06.379389483Z,"time=""2024-05-14T22:13:06.377Z"" level=info msg=""Processing workflow"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:13:06.441437157Z,"time=""2024-05-14T22:13:06.441Z"" level=info msg=""Task-result reconciliation"" namespace=default numObjs=0 workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:13:06.441726468Z,"time=""2024-05-14T22:13:06.441Z"" level=info msg=""node changed"" namespace=default new.message=""Unschedulable: 0/33 nodes are available: 1 node(s) had untolerated taint {node.kubernetes.io/role: mem-8xlarge}, 11 node(s) had untolerated taint {example.com/dedicated: cluster-management}, 12 node(s) didn't match Pod's node affinity/selector, 2 node(s) had untolerated taint {example.com/dedicated: ml-mgmt}, 3 node(s) had untolerated taint {example.com/dedicated: airflow-jobs}, 4 node(s) had untolerated taint {example.com/dedicated: bd-mgmt}. preemption: 0/33 nodes are available: 33 Preemption is not helpful for scheduling.."" new.phase=Pending new.progress=0/1 nodeID=rr-ml-k-module-xxx-74gmh-1-1379393644-1649482756 old.message= old.phase=Pending old.progress=0/1 workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:13:06.441856674Z,"time=""2024-05-14T22:13:06.441Z"" level=info msg=""Evaluating node rr-ml-k-module-xxx-74gmh-1-1379393644: template: *v1alpha1.WorkflowStep (ml-k-module-xxx-ads-categorize-train), boundaryID: , currentStackDepth: 0"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:13:06.44220717Z,"time=""2024-05-14T22:13:06.442Z"" level=info msg=""TaskSet Reconciliation"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:13:06.44228502Z,"time=""2024-05-14T22:13:06.442Z"" level=info msg=reconcileAgentPod namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:13:06.442342693Z,"time=""2024-05-14T22:13:06.442Z"" level=info msg=""Running OnExit handler: exit-handlers-train"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:13:06.442414045Z,"time=""2024-05-14T22:13:06.442Z"" level=info msg=""Evaluating node rr-ml-k-module-xxx-74gmh-1-1379393644.onExit: template: *v1alpha1.WorkflowStep (exit-handlers-train), boundaryID: , currentStackDepth: 0"" namespace=default workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"
2024-05-14T22:13:06.557979115Z,"time=""2024-05-14T22:13:06.557Z"" level=info msg=""Workflow update successful"" namespace=default phase=Running resourceVersion=2131355212 workflow=rr-ml-k-module-xxx-74gmh-1-1379393644"

In status.nodes, that was the state for one of the failing retry step:
status:
  # ...
  nodes:
    # ...
    rr-ml-k-module-xxx-74gmh-1-1379393644-4042243083:
      boundaryID: rr-ml-k-module-xxx-74gmh-1-1379393644-2396377327
      children:
      - rr-ml-k-module-xxx-74gmh-1-1379393644-794976774
      displayName: run-train
      finishedAt: "2024-05-14T22:12:59Z"
      id: rr-ml-k-module-xxx-74gmh-1-1379393644-4042243083
      inputs:
        parameters:
        # ...
      message: Maximum recursion depth exceeded. See https://argoproj.github.io/argo-workflows/scaling/#maximum-recursion-depth
      name: rr-ml-k-module-xxx-74gmh-1-1379393644.exit-handler-1.for-loop-5(4...<LONG PARAM LIST>...).run-train
      phase: Error
      progress: 0/1
      startedAt: "2024-05-03T05:30:54Z"
      templateName: run-train
      templateScope: local/rr-ml-k-module-xxx-74gmh-1-1379393644
      type: Retry
    rr-ml-k-module-xxx-74gmh-1-1379393644-794976774:
      boundaryID: rr-ml-k-module-xxx-74gmh-1-1379393644-2396377327
      children:
      - rr-ml-k-module-xxx-74gmh-1-1379393644-2946699175
      displayName: run-train(0)
      finishedAt: "2024-05-14T22:12:57Z"
      hostNodeName: ip-10-0-30-148.eu-west-1.compute.internal
      id: rr-ml-k-module-xxx-74gmh-1-1379393644-794976774
      inputs:
        parameters:
        # ...
      message: Maximum recursion depth exceeded. See https://argoproj.github.io/argo-workflows/scaling/#maximum-recursion-depth
      name: rr-ml-k-module-xxx-74gmh-1-1379393644.exit-handler-1.for-loop-5(4...<LONG PARAM LIST>...).run-train(0)
      phase: Error
      progress: 0/1
      startedAt: "2024-05-03T05:30:54Z"
      templateName: run-train
      templateScope: local/rr-ml-k-module-xxx-74gmh-1-1379393644
      type: Pod

I noticed that we don't have the following expected part in the retried Pod step:
      nodeFlag:
        retried: true

To repeat the problem, I prepared a minimal workflow like that:
# reproduce-max-recursion-depth-exceeded.yaml
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  name: reproduce-max-recursion-depth-exceeded
spec:
  entrypoint: sleep-long-and-exit-peacefully-on-sigint
  templates:
  - name: entry
    dag:
      tasks:
      - name: run-train
        template: sleep-long-and-exit-peacefully-on-sigint
  - name: sleep-long-and-exit-peacefully-on-sigint
    container:
      args:
      - |
        trap 'echo exiting 0; exit 0' INT
        sleep 3600
      command:
      - ash
      - -c
      image: alpine:latest
      name: sleep-and-exit
    retryStrategy:
      limit: 1

then:
kubectl apply -f reproduce-max-recursion-depth-exceeded.yaml
kubectl edit workflow reproduce-max-recursion-depth-exceeded # in status, delete the `nodeFlag` field
kubectl exec -it "$(kubectl get pods | grep reproduce-max-recursion-depth-exceeded | grep Running | awk '{print $1}')" -- ash -c 'kill -INT 1'

and voila:

Unfortunately I couldn't find a natural way yet to repeat the same issue without manually changing the status field, but I believe this is a case that is hit during a coinciding argo workflow controller restart.

@agilgur5 you've added the During upgrade: prefix to the issue, but for us, that was a continuous issue, not just happened during upgrade.
When we faced these errors during upgrade first, we read the comments in this issue and applied the DISABLE_MAX_RECURSION=true as recommended here, but that ended up with more obscure issues, we began to observe workflows stuck in a retry step while the children were completed like the following one:

To overcome that, we tried to increase MAX_OPERATION_TIME from 30s to 120s, but then we started observing restarts due to stack overflow panics in controllers whenever this recursion bug is hit again.
The recursion occurs in this call here, because it returns here in processNodeRetries call.
I believe this is more a During restart bug than a During upgrade one.

I believe this is more a During restart bug than a During upgrade one.

As far as I can tell, per your report and others, it only occurred during the restart for the upgrade, and not future restarts.

but then we started observing restarts due to stack overflow panics in controllers whenever this recursion bug is hit again.

You didn't provide your Workflow, but that sounds like you did have a recursive template? Or are you saying there's some incorrect recursion in the Controller code, not your template?
The latter could be related to this issue, although I didn't really understand how the rest would be related to this issue? Since you don't get the error in the title if you disable the environment variable.

it only occurred during the restart for the upgrade, and not future restarts.

for us, that's not true, it was a repeating issue, not just in the first restart after the upgrade.

You didn't provide your Workflow, but that sounds like you did have a recursive template?

unfortunately I can't share the workflow as it is, it's a big production workflow generated by kubeflow with sensitive parameters values inside, but I can ensure you that we don't have any thing recursive in our template, and you can check the simplifed workflow manifest I shared above where I can repeat the recursion problem.

Or are you saying there's some incorrect recursion in the Controller code, not your template?

yes, exactly, and this is the line where recursion problem occurs, and it seems to be caused by lack of nodeFlag.retried: true mark in the pod node status, but I'm still not sure how that happens.
what I can say is we were hitting this issue in the clusters where argo controller keeps restarting continuously for different reasons, after stabilising the situation in our clusters, restarts stopped and we stopped observing stuck retries issues.

The latter could be related to this issue, although I didn't really understand how the rest would be related to this issue? Since you don't get the error in the title if you disable the environment variable.

maybe this recursion problem was there in previous versions as well, but we were seeing it as a stuck retry step with Deadline exceeded messages, and deadline was getting exceeded because the executeTemplate method was starting to call itself forever.
I ran into a stack overflow when I came across this, I also made some progress on RCA-ing this.
I am able to reproduce this issue. This is now in my todo list.

Was able to RCA this to the exact line of code where this happens, not sure how to fix it yet unfortunately, will follow up.
Looking at the git blame right now, this seems to have been a bug from approx 2018, but this doesn't make sense given reports are coming in after 3.5.0 as @tico24 states.
I can reproduce  this, will git bisect through the different versions and see if I can  still reproduce.

Just to confirm, are people only getting this issue with steps? If anyone is running into this and not using steps could you please comment.
I am unable to reproduce on 3.5.0, my assumption right now is this a bug in the steps.go logic introduced after 3.5.0.

We don't use steps at Pipekit mate :)

3.5 is where the recursion check was introduced. Prior to that infinite recursion would have had other symptoms.

3.5.1 introduced new nodeFlag to the node status block in #11839.
@isubasinghe has proved the recursion is happening in retry nodes. I suspect the 3.5.1+ code is making incorrect assumptions on retry nodes that cause this recursion, because they don't have the Retried nodeFlag set. Workflows started in 3.5.0 or 3.4 and continued in 3.5.1+ with retry nodes are probably exhibiting this issue.

Could https://github.com/argoproj/argo-workflows/pull/13211#issuecomment-2304414296 be related?
Get Outlook for Androidhttps://aka.ms/AAb9ysg

From: Alan Clucas @.>
Sent: Thursday, August 22, 2024 4:44:19 PM
To: argoproj/argo-workflows @.>
Cc: tooptoop4 @.>; Mention @.>
Subject: Re: [argoproj/argo-workflows] During upgrade: Maximum recursion depth exceeded error when not using recursion (Issue #12162)
3.5.1 introduced new nodeFlag to the node status block in #11839https://github.com/argoproj/argo-workflows/pull/11839.
@isubasinghehttps://github.com/isubasinghe has proved the recursion is happening in retry nodes. I suspect the 3.5.1+ code is making incorrect assumptions on retry nodes that cause this recursion, because they don't have the Retried nodeFlag set. Workflows started in 3.5.0 or 3.4 and continued in 3.5.1+ with retry nodes are probably exhibiting this issue.
—
Reply to this email directly, view it on GitHubhttps://github.com/argoproj/argo-workflows/issues/12162#issuecomment-2303999022, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AH553KHFD6ZQ2PQ4IMV3TCLZSWJFHAVCNFSM6AAAAAA7ARYU2CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMBTHE4TSMBSGI.
You are receiving this because you were mentioned.Message ID: @.***>

Shouldn't this be mentioned in the upgrading guide as "breaking change" until 3.5.11 is released? We were able to migrate to 3.5.10 with existing workflows running after using DISABLE_MAX_RECURSION=true (although its not safe, as logs are full with repeating messages).

not sure why title was changed, it affects new workflows post-upgrade, unrelated to 'during upgrade' so this is still an issue

This problem is about workflows running during an upgrade.
If you have non recursive workflows that you can start on 3.5.1 or later that erroneously create this error message please can you show me one that we can use to reproduce it?

Shouldn't this be mentioned in the upgrading guide as "breaking change" until 3.5.11 is released?

The recursion check was only released in 3.5, and Argo does not follow SemVer so minors can have breakage.
This issue is a bug with that check that affects a small percentage of Workflows (only 4 upvotes, very hard to reproduce). A bug is not a breaking change; that's exactly what patch releases are for.

Shouldn't this be mentioned in the upgrading guide as "breaking change" until 3.5.11 is released?

The recursion check was only released in 3.5, and Argo does not follow SemVer so minors can have breakage. This issue is a bug with that check that affects a small percentage of Workflows (only 4 upvotes, very hard to reproduce). A bug is not a breaking change; that's exactly what patch releases are for.

I follow the not follow semVer reason, and I also follow its a bug (I mentioned, until 3.5.11 is released). But I contest its very hard to reproduce, if you have any 'old' workflow running, (which in a 24/7 CI usecase in production, is hard to prevent without maintenance window), you will be facing this bug unless you DISABLE_MAX_RECURSION=true. It is not only for 3.5.1 or later.
We're upgraded now, but I believe with this wf running on e.g. 3.4.8 controller, and then upgrade to 3.5.10, you'll hit the issue:
(disclaimer, a test wf we use to test hostNodeName feature)
`# This example demonstrates the ability to pass and return
parameters and artifacts into and out of nested workflows
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: nested-workflow-
name: nested-wf-test
spec:
entrypoint: nested-workflow-example
templates:

name: nested-workflow-example
steps:

name: runtb
template: sleepabit
continueOn:
failed: true
arguments:
parameters:

name: excluded_node
value: ""

name: checkinput
template: checkinput
when: "{{steps.runtb.status}} != Succeeded"
arguments:
parameters:
- { name: excluded_node, value: "{{steps.runtb.hostNodeName}}" }

name: retryrun
template: sleepabit
continueOn:
failed: true
arguments:
parameters:

name: excluded_node
value: "{{steps.checkinput.outputs.result}}"

name: checkinput
inputs:
parameters:

name: excluded_node
script:
image: python:alpine3.7
command: [ python ]
source: |
if ("steps.runtb.outputs.parameters.nodename" in "{{inputs.parameters.excluded_node}}"):
print("blablabla")
else:
print("{{inputs.parameters.excluded_node}}")

container template which consumes an input parameter and artifact

name: sleepabit
inputs:
parameters:

name: excluded_node
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:

matchExpressions:

key: kubernetes.io/hostname
operator: NotIn
values:

'{{inputs.parameters.excluded_node}}'
timeout: "120s"
container:
image: alpine:3.7
command: [sh, -c]
args: ["echo $NODE_NAME > nodname.txt && sleep 90 && false"]
env:

name: NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
outputs:
parameters:
name: nodename
globalName: nodename
default: "none"
valueFrom:
path: nodename.txt
`

We're upgraded now, but I believe with this wf running on e.g. 3.4.8 controller, and then upgrade to 3.5.10, you'll hit the issue:

yeah its this upgrade that causes an infinite recursion in the controller, this will be  fixed in the latest 3.5 release.

Unfortunately I couldn't find a natural way yet to repeat the same issue without manually changing the status field

Eventually we found the issue causing this problem in our side, just wanted to share it for others who might be still hitting it using an old Argo version, finding themselves in this thread.
We have an in-house kfp-operator which is deleting ownerReferences of workflow manifests to detach them from parent ScheduledWorkflow, to avoid interrupting ongoing workflow runs in case of a delete or update (delete+create) operation.
We were using Update call in controller-runtime client like this:
workflow.SetOwnerReferences(nil)
controllerRuntimeClient.Update(ctx, workflow)

That was ending up with nodeFlag: {} fields in the existing status.nodes which was ending up with Max Recursion Depth Exceeded errors in Argo side.
To fix it, we just started using the Patch call:
patch := client.RawPatch(types.MergePatchType, []byte(`{"metadata": {"ownerReferences": []}}`))
controllerRuntimeClient.Patch(ctx, workflowInstance, patch)

I hope it would be useful for people who were hitting this issue for a similar reason.


Addition of pull_request template and issue template

I think adding ISSUE_TEMPLATE.md and PULL_REQUEST_TEMPLATE.md file will enhance the workflow of the repository.

@SKAUL05 I want to work on this issue

Assign appropriate labels when raising a PR.


MVP for user-contributed document audio

Opening this as a draft to share the code; not ready to merge.

feat(schema): add new mutations and input types
feat(graphql): implement new mutation resolvers
feat(db) added queries and implemented database_sql.rs functions.

TODO

[x] Couldn't get sqlx-types.json to build

@GracefulLemming update from today:
@sammcvicker and I paired ~2 1/2 hours to get recording audio into a MVP/working state. We found two sets of mocks on figma. The second set I had not reviewed and seemed to have some unanswered questions jotted in the mocks. We decided to focus on functionality and getting some good abstractions for upload/record between word and document oriented components. Aside from UX decisions, the last bits of functionality to add to the frontend are:

[ ] Editors' ability to curate (hide/show to readers) contributed audio
[ ] Contributor names and contributed on dates along side the audio player

On the first point, I don't quite see from the mocks any advice on this flow, so I think following in this MVP/functionality first spirit we will replicate the checkbox feeling from the word audio contribution UX.
On the second, we might consider if this valuable metadata and context should also be provided in other places the audio player is used, eg. word audio.

@CharlieMcVicker @sammcvicker thank you for the updates! Here are some thoughts:

I agree we should replicate what we do for word audio. Let's full send this.
Also agree names will go on the base level audio component. To cue you in a little more: @nole2701 is currently working on implementing more info on user profiles. After that, we will be updating the audio component to include names.
You're working on a fork so you don't have access to environment variables needed to pass CI. You should already have the credentials you need on you, just save them in the variables section on your fork. I'm happy to pass more info in other channels if needed.


Search page ADD FAVORITE buttons pulls you back to top of the page on click instead of staying still.

FIxed. I added event.prevent! :D


cmd/ebitenmobile: ebitenmobile doesn't work with a case-sensitive file system

Reported at https://github.com/hajimehoshi/ebiten/discussions/2172#discussioncomment-3111784
If the fix is obvious, I'll backport this to v2.3.

Confirmed the issue was fixed.

Unfortunately a case-sensitive file system is not available on GitHub actions: https://github.com/actions/virtual-environments/issues/865


[Test] Make ninja check-clang work on Linux

Hi @bob-wilson – Can we merge this fix? It'll make ninja check-clang pass on my Linux box. Thanks!

Is it already on upstream-with-swift? If not, it needs to go there first (or at the same time).
In addition, the "stable" branch is a read-only copy of the latest release branch. Can you rebase this PR for swift-5.1-branch?

@bob-wilson – I've changed the base and cherry-picked the fix from upstream-with-swift. Thank you in advance for merging this :-)


Documentation refactor

Eliminates outdated docs sources currently published to telemetry-framework.readthedocs.io
Replaces them with asciidoc + svgs from downstream
Link in top-level README adjusted to reference internal docs

NOTE: I have submitted these for editing to the docs team, but they are targeting dates in October to finalize the edits. My intention is to backport the downstream changes here when they become available.

 SAF Docs refactor


chore: trivial change roundup

Thanks to @just-mitch and @tudorpintea999.

Benchmark results
Metrics with a significant change:

circuit_output_size_in_bytes (root-rollup): 881 (-20%)
note_trial_decrypting_time_in_ms (8): 28.4 (+46%)

Detailed results
All benchmarks are run on txs on the Benchmarking contract on the repository. Each tx consists of a batch call  to create_note and increment_balance, which guarantees that each tx has a private call, a nested private call, a public call, and a nested public call, as well as an emitted private note, an unencrypted log, and public storage read and write.
This benchmark source data is available in JSON format on S3 here.
Values are compared against data from master at commit 91cb369a and shown if the difference exceeds 1%.
L2 block published to L1
Each column represents the number of txs on an L2 block published to L1.

Metric
8 txs
32 txs
128 txs

l1_rollup_calldata_size_in_bytes
45,444
179,588
716,132

l1_rollup_calldata_gas
222,996
867,752
3,448,016

l1_rollup_execution_gas
842,083
3,594,860
22,203,385

l2_block_processing_time_in_ms
2,219
8,445
34,898 (-1%)

note_successful_decrypting_time_in_ms
319 (+3%)
928 (+1%)
3,314 (-1%)

note_trial_decrypting_time_in_ms
:warning: 28.4 (+46%)
53.8 (+9%)
198 (-1%)

l2_block_building_time_in_ms
20,355
81,483
323,517

l2_block_rollup_simulation_time_in_ms
16,802
67,507
267,494

l2_block_public_tx_process_time_in_ms
3,504
13,825
55,465

L2 chain processing
Each column represents the number of blocks on the L2 chain where each block has 16 txs.

Metric
5 blocks
10 blocks

node_history_sync_time_in_ms
24,866 (-1%)
47,597

note_history_successful_decrypting_time_in_ms
2,196 (-1%)
4,192 (-1%)

note_history_trial_decrypting_time_in_ms
164 (-2%)
228

node_database_size_in_bytes
2,793,218
5,025,804

pxe_database_size_in_bytes
29,748
59,307

Circuits stats
Stats on running time and I/O sizes collected for every circuit run across all benchmarks.

Circuit
circuit_simulation_time_in_ms
circuit_input_size_in_bytes
circuit_output_size_in_bytes

private-kernel-init
199
43,109
20,441

private-kernel-ordering
115
25,833
9,689

base-rollup
2,957
664,108
873

root-rollup
86.4
4,072
:warning: 881 (-20%)

private-kernel-inner
260
64,516
20,441

public-kernel-private-input
171 (-1%)
25,203
20,441

public-kernel-non-first-iteration
168
25,245
20,441

merge-rollup
10.9
2,592
873

Miscellaneous
Transaction sizes based on how many contracts are deployed in the tx.

Metric
0 deployed contracts
1 deployed contracts

tx_size_in_bytes
10,323
25,938


Upgrade to datefns v2 ?

date-fns v2 is out and it would be nice to upgrade this lib as well.

Will do.

Date-fns is updated in version v2.0.0.


Update bot.py

After creating your pull request, tick these boxes if they are applicable to you.

[ ] I have tested my changes against the review branch (the latest developmental version), and this pull request is targeting that branch as a base
[ ] I have tested my changes on Python 3.5.3 or higher

Description
Related issues (if applicable)

Please do not make bogus PRs for the sake of hacktoberfest.


Updated the readme

added brief documentation on how to perform Batch jobs.  Let me know if you feel that there should be more details, or changes.

Thanks a bunch for contributing this


fix - Respect space in path when doing ng serve

fixes https://github.com/dotnet/aspnetcore/issues/38328

Thanks for your PR, @ShreyasJejurkar.
@MackinnonBuck do we still need this given your recent changes in this area? If not, can you please close this as well as the associated issue? Thanks!


[Feature] Add Authentication

What does this issue address?
This issue addresses need of authentication in app, so that users can save their shortened links and get analytics in future.

Closes as #3 merged


Keg.change_rpath and non-native binaries

I am trying to bottle some formula that I would like to redistribute. Most of the time bottling works OK, but I am having trouble particularly with libvirt and qemu. They both contain non-native binaries as they are cross-platform by nature. This means that patchelf fails miserably and brew cannot bottle.
@sjackman: do you think there should be a check in Linuxbrew whether binary is native before switching interpreter and rpath?

Yes, it should check that the executable is native before running patchelf.

OK, I should try to find a solution, since the current mechanism uses names like mach_o_binary which do not belong to Linux(brew) world ;)

True. I don't want to refactor or rename inside Linuxbrew though. The patch should be kept as small as possible to ease the merging maintenance.

Seems no longer relevant for new qemu formula.

Non-ELF executables (namely Mac executables) no longer have their executable bit set by the Linuxbrew post-installation cleanup code.

The original problem was with ELF binaries that were compiled for other platform, I believe (qemu has a few of them). But since I cannot reproduce it I consider it fixed.


Create a blueprints example and fix any issues

REF:
http://stackoverflow.com/questions/41604212/how-to-use-flasgger-with-flask-applications-using-blueprints

https://github.com/gangverk/flask-swagger/pull/34/files


Delegated event callbacks should use the Backbone.RelationalModel as attribute

Example:
var Company = Backbone.RelationalModel.extend({
  relations: {
    employees: Backbone.Collection.extend({})
  }
});
var aCompany = new Company();
// Listening to all changes in our company
aCompany.on("change", function() {
  console.log(arguments);
});
aCompany.set("name", "Kinship ltd");
aCompany.set("employees", [{ name: "Martin" }, { name: "Gustaf" }]);

Currently, the arguments outputted from the first change event are:
The aCompany Model
Set options

The arguments from the second and third change events are:
The employee model
The employees collection
Set options

My suggestion is that the arguments from the seconds and third change events should be:
The aCompany model
Set options

This change would keep this in line with how non-relational change events already work. It's also possible that aModel.changed should be updated with the changed employee models to reflect this change in a proper way.
Also, I suggest that delegated add and remove event should follow the same pattern and send the kinship model as first attribute and the changed collection (as JSON?) as the seconds attribute.

This will be a breaking change, so let's keep until 2.0.

@martinlissmyr What do you think about this? I have stumbled upon this problem a couple of times already (where even though I've caught the change event on a kinship model, I have no easy way to actually get hold of that model).

Sounds reasonable.


Make text in event non mandatory

There's a test you might want to fix:
Results :

Failed tests:
  EventTest.fails_without_text Expected exception: java.lang.IllegalStateException

Tests run: 138, Failures: 1, Errors: 0, Skipped: 40

@hush-hush could you please elaborate a little bit on the use-case? Is this for consistency with other clients?

@hush-hush could you please elaborate a little bit on the use-case? Is this for consistency with other clients?

This was requested by product and is already implemented in all other clients.


Made it possible to change gamesettings in the f1-menu on dedicated servers

Accidentally closed while rebaseing my fork, my bad

bump


Update our fork with upstream changes

@Brantone I don't have write access so would leave the merge to you

Added applicable team to "Write"


logic error for remove one element from a set

Hi @fengprofile, thanks for submitting! This was already fixed via https://github.com/OmniLayer/omnicore/pull/593

Thanks a lot for submitting this!
Can you please merge it against the develop branch?

Sorry, I use git not very mture. I will try it late.

I resubmited this pull request via #690 against the develop branch. It's going to be included in the next release.


Restoration bench: increase heap and remove 0.1%-rnd

Issue Number
ADP-846
Overview
Let the nightly benchmark finish successfully within the alotted time.

[x] Increase max heap from 8GB to 16GB
[x] Replace 0.1% and 0.4% rnd and seq benchmarks with 0.2% to prevent timeout.

Comments

Looking at the current progress, removing 0.1%-rnd will most certainly not be enough.
Before removing anything, it started 0.4%-rnd 30 minutes before the timeout. So even just removing 0.4%-rnd will be tight.
I suspect we might need to remove both 0.4%-seq and 0.4%-rnd…

list transactions: 2946 s

From 0.4%-seq src
😳

bors r+

bors r+

bors r+

bors r+

bors r+

bors r+


chore: add email schema changes

:tada: This PR is included in version 0.0.3 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Basic - Create Smart Contract Script (outdated ? )

Hi ,
On your website about Smart contract it is wrote :

Do note that this template is very outdated and should not be used as an actual ICO template.

For a private services i can use it ? or what kind of issues i can encounter for a public ICO ?
Thanks

It is usable but it is not maintained thus it does not contain any bugfixes found.
You should be looking at some of the newer templates if you are planning anything more than a personal project.


Improve example structure to able to use an existing monitoring depoyments the configs as well.

The issue
The example contains elements that would be fine to able to just include the existing kube-prometheus-stack implementation easily. These elements are the PodMonitor, Dashboards and the 'customResourceState' definitions for kube-state-metrics component.
The proposed solution
This commit makes possible the resources can be included to existing monitoring systems as well. The example will work as described originally. Unfortunately I cannot use just the 'configs' subdirectory only for all configuration items because, the the example not worked anymore. the is the reason why the 'pre-requisites' directory introduced. Optionally it is possible to remove every components from a 'pre-requisites' directory to the 'controllers/kube-prometheus-stack', but here will be mixed the 'configs' with the 'controllers'.
Include path for the components:

Component: Dashboard
Path: monitoring/configs/dashboards
Component: customResourceState definitions for 'kube-state-metrics'
Path: monitoring/pre-requisites/custom-resource-state-config
Component: PodMonitor
Path: monitoring/configs/podmonitor

Example for include kube-state-metrics configuration:
---
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: monitoring-kube-state-metrics-configs
  namespace: flux-system
spec:
  interval: 1h
  retryInterval: 2m
  timeout: 5m
  prune: true
  wait: true
  sourceRef:
    kind: GitRepository
    name: flux-system
  path: ./monitoring/pre-requisites/custom-resource-state-config

That will create the ConfigMap object with same name (flux-kube-state-metrics-config) that is in the example.
The need to be add or update the valuesFrom section in kube-prometheus-stack HelmRelease manifest.
Example to add the HelmRelease:
  valuesFrom:
  - kind: ConfigMap
    name: flux-kube-state-metrics-config
    valuesKey: kube-state-metrics-config.yaml

I am very welcome for any ideas and help.

Custom requirement. Will not implemented.


Saga: fix statemachine def cannot store to db, provide StateLogRepository for the query of state logs

Ⅰ. Describe what this PR did
Fix statemachine definition cannot store to db.
Provide API StateLogRepository for the query of state logs such as StateMachineInstances and StateInstances.
Ⅱ. Does this pull request fix one issue?
#1808
#1809

Codecov Report

Merging #1810 into develop will increase coverage by 0.06%.
The diff coverage is n/a.

@@              Coverage Diff              @@
##             develop    #1810      +/-   ##
=============================================
+ Coverage      55.06%   55.13%   +0.06%
- Complexity      2352     2370      +18
=============================================
  Files            421      421
  Lines          14262    14284      +22
  Branches        1710     1714       +4
=============================================
+ Hits            7854     7875      +21
- Misses          5680     5681       +1
  Partials         728      728

Impacted Files
Coverage Δ
Complexity Δ

...va/io/seata/tm/api/transaction/NoRollbackRule.java
86.66% <0%> (+0.95%)
9% <0%> (+5%)
:arrow_up:

...a/io/seata/tm/api/transaction/TransactionInfo.java
97.05% <0%> (+2.05%)
26% <0%> (+13%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8c80c5c...ed6e559. Read the comment docs.

brief commit log

please resolve conflicts


Order history method is missing in Javascript-SDK

Hey there,
I want to get order history with javascript-sdk but can't find any option for it .
I can see that python-sdk for example have an option called: "client.get_closed_orders" ,
But i couldn't find any equivalent in javascript-sdk 😒 help pls 🙏

Ok found the problem.
"binance-chain/javascript-sdk/blob/master/src/client/index.ts" file is missing the closed order api.
Solved it by duplicating every "getOpenOrders" { whole section } var in that file and changing the
values from Open to Closed.
Will add a pull req later  😎

Sooo it did fix the problem but now i cant query open order it look like an issue with the index.ts file
for now i just use the API to fetch closed orders , not ideal but it work.
https://testnet-dex.binance.org/api/v1/orders/closed?address=


"WebSocket is not open: readyState 3" when calling Browser.newPage() on long-running browser instance

Steps to reproduce
Tell us about your environment:

Puppeteer version: 1.8.0
Platform / OS version: Docker image<EMAIL_ADDRESS>(which uses a Node base image node:10.9.0-slim@sha256:b269112ab0cd9dc8760ff406a07cf91f508257b3473031b414817e0c152c0744)
Node.js version: Node 10

I have a service that executes puppeteer.launch({}) with args on setup, and then continues to use the returned instance of the Browser to load HTML on requests using browser.newPage(). Then I use page.pdf() to render a PDF of the page, and then page.close(). The same instance of the Browser is used throughout the life of the service instance.
Recently we've been noticing WebSocket is not open: readyState 3 error logs when calling browser.newPage(). This seems to happen after thousands of requests have succeeded before. Once it springs up, it seems to consistently fail to create a newPage, as in that browser instance is "no good' anymore.
What's the best way to detect that the Browser is prepared for a newPage call? Or would it be better to create the browser instance, and then on subsequent requests connect to it, do my work, and then disconnect, with error handling to relaunch the browser if the connect() call fails? I know we have access to the browser process, but I'm not sure how to determine whether it's "good" or not.
I can provide a simplified version of our code if needed. In the interim I'm going to upgrade to a more recent version.
Thanks!

I also encountered this problem.

Recently we've been noticing WebSocket is not open: readyState 3 error logs when calling browser.newPage(). This seems to happen after thousands of requests have succeeded before. Once it springs up, it seems to consistently fail to create a newPage, as in that browser instance is "no good' anymore.

@hjdarnel This likely means that the browser has crashed and thus socket got closed. In this case, you should've received the browser.on('disconnected') event - do you subscribe to it?

What's the best way to detect that the Browser is prepared for a newPage call?

I'd say to subscribe to the "disconnecteed" event and handle it appropriately - with re-starting the shared browser you use.
Hope this helps!

Thank you, Andrey! Yes, we've fixed this by subscribing to that event. Below is our setup function which calls itself on disconnection (for others that run into this):
const setup = async () => {
    browser = await puppeteer.launch({
        args: ['--disable-gpu', '--no-sandbox', '--disable-dev-shm-usage']
    });

    browser.on('disconnected', setup);

    logger.info(`Started Puppeteer with pid ${browser.process().pid}`);
};```

This has worked well at production load for a few weeks now.


possible bug in m4a parsing

Reproducer
I tried this code:
fn main() {
    // the error seems to occur with strict or relaxed parsing mode
    let parsing_options = ParseOptions::new().parsing_mode(ParsingMode::Relaxed);
    let path = Path::new("sample.m4a");
    let file = Probe::open(&path)
        .expect("can't probe")
        .options(parsing_options)
        .read()
        .expect("can't read");
    let tag = file.primary_tag().expect("no tag");
    let properties = file.properties();
    println!("{:?}", tag.artist());
    println!("{:?}", tag.year());
}

Summary
When trying to read the file I get this panic:
no read: BadAtom("\"covr\" atom has an unknown type")
stack backtrace:
   0: _rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::result::unwrap_failed
   3: core::result::Result<T,E>::expect
   4: rust_lib_spinner::api::tagging::read_tag

Expected behavior
I don't doubt that there is a bad atom here. Could it be skipped in relaxed mode?
ffprobe output:
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x140e06d30] Unknown cover type: 0x15.
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'sample.m4a':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2mp41
    album_artist    : BADBADNOTGOOD
    disc            : 1/24576
    track           : 1/24576
    artist          : BADBADNOTGOOD
    album           : III
    date            : 2014
    title           : Triangle
    encoder         : Lavf55.14.102
    genre           : Hip Hop
  Duration: 00:03:47.00, start: 0.000000, bitrate: 514 kb/s
  Stream #0:0[0x1](und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 511 kb/s (default)
      Metadata:
        handler_name    : SoundHandler
        vendor_id       : [0][0][0][0]

Assets
sample.zip

I don't doubt that there is a bad atom here. Could it be skipped in relaxed mode?

Definitely. Just another case of an error that existed before ParsingMode. :)
I am curious, do you know the source of the file? Or if any player can tell what the cover is supposed to be? I wonder how it got that data type.

@Serial-ATA Good question I'll try to find the source. Apple Music does seem to be able to display the artwork when I load it there.

Apparently this file originated in Apple Music so I suppose it's not surprising that it shows up there.

The cover is just a PNG, so I wonder if Apple just ignores data types or introduced a new one for some reason. I don't have Apple Music, could you see if it load this file? test.zip
I just took one of the benchmark assets and set a wrong data type, the cover is still a PNG.

Thanks for the sample. The artwork for the file in your test.zip shows up in both the finder and in Apple Music. Interestingly, the artwork for the file I attached shows up only in Apple Music and not in the finder.
Apple Music supports both static images and in some cases animated images for artwork. Maybe this non-standard COVR atom has something to do with that?

The artwork for the file in your test.zip shows up in both the finder and in Apple Music.

I guess that means they ignore data types and try to read images anyway

Apple Music supports both static images and in some cases animated images for artwork. Maybe this non-standard COVR atom has something to do with that?

I was thinking it might have to do with that, but this is just a normal static PNG so there shouldn't be a reason to use anything but the data type for PNG.
I haven't found anyone else talking about this so far, but I imagine if Apple Music is producing these files there's probably a reason?

I suppose it could also just be another case of Apple being Apple.

That's true. There are multiple hacks in here to try and support old iTunes bugs.

What do you think is the best way to handle these?

For now I'll just skip it, most others do that anyway. Do you have this issue with other files from Apple Music? If so, I imagine other people will report similar things.

So far this is the only case I've had reported of this issue.
Thanks yet again for such a quick and helpful response!


Not possible to manually dismiss low risk users without P2

The section entitled "Manually dismiss user's risk" covers how to manually dismiss the user's risk in their home tenant.
If the home tenant is licensed for EID P2, an administrator in the home tenant is always able to view and dismiss the user's risk. However, if the home tenant is not licensed for P2 (i.e., is P1 or Free), an administrator can only view and dismiss user risk at the level of Medium or High; Low level risks are not visible, and therefore also not dismissible. A tenant that is not licensed for P2 will still mark users with Low risk in the same way a P2 tenant would; it's just not visible or dismissible by the admin.
The above creates an edge case where manual dismissal of the risk is (to my knowledge) not possible (unless the home tenant upgrades to P2). The only option left in this case appears to be to ask the user to complete SSPR.
I have just encountered this exact scenario. However, it took me quite a while to work out why I was unable to see or dismiss a risk associated with the user. Eventually, I was able to find the appropriate section of the Identity Protection docs that mentions the fact that the Risky users pane does not show users with Low risk if your tenant is not on P2.
Is it worth including a note that calls out this caveat? It would have saved me quite a bit of time, so hopefully it will help others in future.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 6e0625bb-7367-0bb3-7380-55ac91bde11f
Version Independent ID: efa6711d-73b0-c30f-2e13-97f29beebedb
Content: Identity Protection and B2B users - Microsoft Entra
Content Source: articles/active-directory/identity-protection/concept-identity-protection-b2b.md
Service: active-directory
Sub-service: identity-protection
GitHub Login: @MicrosoftGuyJFlo
Microsoft Alias: joflore

@theonlycoder
Thanks for your feedback! We will investigate and update as appropriate.

@MicrosoftGuyJFlo
Could you please review add comments on this, update as appropriate.

@theonlycoder
Thank you for bringing this to our attention.
I've delegated this to content author @MicrosoftGuyJFlo, who will review it and offer their insightful opinions.

ID Protection itself and the configurations around it are a P2 feature. This is called out in https://learn.microsoft.com/azure/active-directory/identity-protection/overview-identity-protection#license-requirements

#please-close

@MicrosoftGuyJFlo Thanks for reviewing my issue. However, I think you have misunderstood my feedback and therefore closed prematurely.
I am aware that Identity Protection is mostly a P2 feature. However, there is some limited IP functionality available without P2, including the ability to view the Risky users report and dismiss risks that are listed. I imagine this is why the article mentions dismissing risks via this report as an option, without mentioning any licensing requirements.
However, there is a caveat to the above: if you do not have P2, the Risky users report will only list users flagged as either Medium or High risk; it will not list users identified as Low risk, despite still flagging them internally in EID. If the user has been flagged as Low risk, the admin of the home tenant will not see the user listed. Nor can they dismiss the risk (unless they upgrade to P2, or the user's risk level is subsequently upgraded to Medium or High).
Unless the admin of the home tenant is already aware of this rather obscure limitation of the risky users report when not on P2, they are left scratching their head, wondering why the user is being blocked by the resource tenant when they do not appear to have any risk associated with them (because they do not appear in the Risky users list). Nowhere in the portal are there any indications that the report omits Low risk level users.
Although the omittion of Low risk users when not on P2 is documented on the page you linked, I believe it would be highly beneficial to call this caveat out on this page. To my mind, it should probably call out all three of the below points:

That the Risky users report can be accessed and risks dismissed when the home tenant is on any EID tier (Free or P1 work).
However, without P2, it is only possible to see and dismiss users flagged as Medium or High risk. Low risk level users cannot be viewed or dismissed, despite still being flagged by EID in the background.
Without P2, the only options to dismiss Low risk users are to either use the SSPR method or upgrade yard yard tenant to P2.

This sort of obscure quirk is exactly the sort of thing that you want to be calling out in documentation as it helps the reader to avoid the head scratching and wasted time working out what is going on.
Thanks in advance.

So what if SSPR is not solving the issue ? Have anybody actually solved this with SSPR?
Kind regards
Michael


Fix deserialization of legacy text

Fixes all components inheriting color/style from the first component in the chain.
For example, currently:
&3> &rpms &7- &rfalse

is parsed to

That switch block tries to optimize things where optimization isn't necessary, because there is only a a single case where we can leave out the extra root component: that is when pos == 0 && parts == 1.
In all other cases the additional root component is required either because it holds the unformatted text at the beginning or to prevent wrong inheritance (like you've noticed).
The old code completely ignores the unformatted text at the beginning. E.g. let's say we have Hello &ckashike. The current code will parse this to kashike (red) and ignore the Hello  entirely.
You correctly fix this for parts >= 2 by adding back pos > 0 ? input.substring(0, pos) : "" but the same needs to be done for parts == 1. At this point you might as well remove the switch block because all branches do the same. It can be just written as:
return TextComponent.builder(pos > 0 ? input.substring(0, pos) : "").append(parts).build();

Thanks @Minecrell. I've removed the switch block and replaced it with the default for 2+ parts.

(Kinda) related to this, but with the following component, what would be the correct legacy version?
TextComponent component = TextComponent.builder("hi").decoration(TextDecoration.BOLD, TextDecoration.State.TRUE)
  .append(TextComponent.of("foo").color(TextColor.GREEN).decoration(TextDecoration.BOLD, TextDecoration.State.FALSE))
  .append(TextComponent.of("bar").color(TextColor.BLUE))
  .append(TextComponent.of("baz"))
  .build();

Currently it spits out: &lhi&afoo&9barbaz
But should it not be: &lhi&afoo&9&lbar&lbaz (where the child components inherit style & color from their parent?)

You're right, the to code is broken:
[Aug 13 10:32:06] <minecrell> Anyway, you need to fix the "to" code :)
[Aug 13 10:34:18] <kashike> meh, I'll fix it later

These two ways would be correct:

&lhi&afoo&9&lbar&r&lbaz (The additionally &r is necessary before baz because text style (bold etc) does not reset the color)
&lhi&r&afoo&r&9&lbar&r&lbaz&r (That's Mojang's implementation, it resets after each component and applies the formatting again)

The first variant is of course much nicer because it avoids unnecessary codes but it is not that easy to implement: You need to track the current style and reset as-needed when iterating over the texts. The Sponge implementation is kind of messy, there are probably better ways...

Thanks, @lucko.


[Investigate] Webpack Set Up

Investigate and come up with a plan for Action Items to set up webpack.

Completed for now. Need to refactor server + corresponding script to typescript.


perf: speed up integration test compilation

Every file in the tests directory is compiled as a separate binary, which is slow. By using a single entry point, we can speed up compilation.
Inspired by this blogpost.

I merged this to expedite my development, but it would still be helpful to have a look at, @agostbiro

Thanks, LGTM


:sparkles: Add boost hana CPM recipe

How do we test the recipes?

With a recipe test, of course. To be written :)


Help wanted: embassy-net-ppp

Hello,
I'm trying to set up a project with a simcom sim868 GSM/GNSS module using PPP. I managed to setup the module using PPP on linux successfully using the following guide as well as talking to the simcom module via serial from an stm32f449 and connecting it to a network and sending data using AT commands but using the ppp network from embassy seems much nicer integrated.
However I can not figure out how to setup embassy ppp from the example to connect using the real modem. I guess what i am missing is the modem and carrier specifics that are setup via the ppp config and the chatscript when using linux and pppd.
Has anyone already setup a real gsm modem using ppp and maybe could provide an example or some cues on the process?

embassy-net-ppp only does the "PPP" part, not the "AT command" part. You can use it with a modem but you have to do the AT commands yourself.
Run the AT commands needed to setup the modem (write it as a string to the uart, then wait the modem to reply with OK). The last command ATD*99# is special: write it as a string, then wait for the modem to reply with CONNECT. After this, the UART is switched to "data mode", speaking PPP. After this is done, you can start forwarding bytes between the serial port and embassy-net-ppp.

Hello, I'm trying to set up a project with a simcom sim868 GSM/GNSS module using PPP. I managed to setup the module using PPP on linux successfully using the following guide as well as talking to the simcom module via serial from an stm32f449 and connecting it to a network and sending data using AT commands but using the ppp network from embassy seems much nicer integrated.
However I can not figure out how to setup embassy ppp from the example to connect using the real modem. I guess what i am missing is the modem and carrier specifics that are setup via the ppp config and the chatscript when using linux and pppd.
Has anyone already setup a real gsm modem using ppp and maybe could provide an example or some cues on the process?

I'm actually working on this and managed to make it work (dirty code, a lot of messing). I can provide you the steps what you should look for (right now I have huge mess in code, doing a lot of this and that and mixed up together). I did exactly what Dirbaio said and worked pretty well.

you need to setup two buffers (rx and tx), like [u8; N_RX/TX];
connect to modem via UART, dirty example taken from my code:

    // somewhere top before main
    bind_interrupts!(struct UartIrqs {
        USART1 => embassy_stm32::usart::BufferedInterruptHandler<peripherals::USART1>;
    });

    // in main
     let mut uart = embassy_stm32::usart::BufferedUart::new(
         p.USART1,
         UartIrqs,
         p.PA10,
         p.PA9,
         tx_buffer,
         rx_buffer,
         Default::default(),
     )
     .unwrap();

send commands to modem to setup and switch to data mode (in my case I send AT+CGDCONT=1,"ip","apn"\r followed by ATD*99#\r. example sending / receiving part:

    // reading & parsing output is up to you, to make sure modem is up, got successful replies on commands, etc.
    uart.write_all("AT\r".as_bytes()).await.unwrap();
    uart.flush().await.unwrap();
    Timer::after_secs(1).await; // wait for a bit for modem to respond
    uart.read(&mut buf).await.unwrap();

now comes the embassy_net_ppp part:

let state = embassy_net_ppp::State::<N_RX, N_TX>::new();
let (device, runner) = embassy_net_ppp::new(state);

let cfg = embassy_net_ppp::Config {
  username: "foo".as_bytes(),
  password: "bar".as_bytes(),
}

runner.run(uart, cfg, |stat| {
  info!("stat: {}", stat);
});

then basically follow the docs, you already have the connection. create network stack, etc.

ah i was only missing the ATD*99#\r part i already figured i probably need to connect the modem first. Thanks a lot to both of you. Maybe i can make a PR to the README of embassy_net_ppp to explain this a little better


Can't install pico-sdk on Mac

I'm unable to build the examples on a MacOS 12.1.
I have cmake installed and clone the repo to my user account, but running cmake .. just results in cmake not being able to find either CMakeLists.txt, even after I move it to the ~/ directory, or the pico-sdk directory, wherever I put it.
I set the path for SDK in .zshrc but it still doesn't work.
Part of the problem seems to be that the downloaded repo isn't even called pico-sdk, but picosystem. The CMakeLists.txt file can't find the locations specified.
I currently have the picosystem repo in my root user directory. Do I need to rewrite CMakeLists.txt to get it to compile the examples? All the dependencies appear to be in place.

the downloaded repo isn't even called pico-sdk, but picosystem.

I'd guess you haven't downloaded the right repo at this point, this should not be the case.

A quick google-search finds https://github.com/pimoroni/picosystem so if you're having trouble getting that set up, it sounds like you ought to be asking Pimoroni for help.

A quick google-search finds https://github.com/pimoroni/picosystem so if you're having trouble getting that set up, it sounds like you ought to be asking Pimoroni for help.

That's the device's operating system, whereas I'm trying to set up the sdk.

whereas I'm trying to set up the sdk.

In that case you probably want to read https://datasheets.raspberrypi.com/pico/getting-started-with-pico.pdf (which is linked to from https://www.raspberrypi.com/documentation/microcontrollers/rp2040.html#documentation )

Any more comments @mroberts1 or can this be closed now?


chart: allow security context to be disabled on OpenShift

A work in progress for changes to make running flagger on OpenShift easier.
Relates To: #542

@richardcase I'm going to merge this an include it in the next release, please open a new PR if other OpenShift changes are needed.


Segmentation Fault Help!

Glibc problem.I changed the glibc version and tried to execute mitmproxy.But all of them were failed.
They looks like bellow:
[localhost software]# LD_LIBRARY_PATH=/home/work/libs/glibc-2.17/lib ./mitmproxy
./mitmproxy: error while loading shared libraries: __vdso_time: invalid mode for dlopen(): Invalid argument
[localhost software]# LD_LIBRARY_PATH=/home/work/libs/glibc-2.15/lib ./mitmproxy
Segmentation fault
[localhost software]# LD_LIBRARY_PATH=/home/work/libs/glibc-2.14/lib ./mitmproxy
Traceback (most recent call last):
File "site-packages/PyInstaller/loader/rthooks/pyi_rth_pkgres.py", line 11, in
File "/home/travis/build/mitmproxy/mitmproxy/.tox/rtool/lib/python3.5/site-packages/PyInstaller/loader/pyimod03_importers.py", line 631, in exec_module
File "site-packages/pkg_resources/init.py", line 33, in
File "/home/travis/build/mitmproxy/mitmproxy/.tox/rtool/lib/python3.5/site-packages/PyInstaller/loader/pyimod03_importers.py", line 631, in exec_module
File "platform.py", line 117, in
File "/home/travis/build/mitmproxy/mitmproxy/.tox/rtool/lib/python3.5/site-packages/PyInstaller/loader/pyimod03_importers.py", line 631, in exec_module
File "subprocess.py", line 438, in
File "/home/travis/build/mitmproxy/mitmproxy/.tox/rtool/lib/python3.5/site-packages/PyInstaller/loader/pyimod03_importers.py", line 714, in load_module
ImportError: /home/work/libs/glibc-2.14/lib/libc.so.6: version `GLIBC_2.15' not found (required by /tmp/_MEICvKEA3/select.so)
[30568] Failed to execute script pyi_rth_pkgres
[localhost software]# LD_LIBRARY_PATH=/home/work/libs/glibc-2.17/lib ./mitmproxy
./mitmproxy: error while loading shared libraries: __vdso_time: invalid mode for dlopen(): Invalid argument

Please provide more information.
What OS are you using?
Which mitmproxy version?
How did you install mitmproxy?
Why are you setting the library path?

Please, reopen this (and give us a little bit more information :) ), if you still have any problems.


Install pure, etc. from brew.

HomeBrew is not good for heavy downloads but replacing the zsh related git installations with brew installs seems to be cleaner.
One concern is that brew might not be downloadable from all locations.

I should enable the k option in curl to allow firewall bypass, I think.

Difficult to integrate with ZDOTDIR.


[Macaroon-Leftovers] root key encrypt pass

Currently password, used for encryption of macaroon root key, is fetched from config as default value.
Create RPC methods to pass password and block calling other RPC methods until user provides key.

I think we shoudl use the same InitWallet so we have a single password for both

This is not possible because the generation of the macaroons is done when starting the daemon. Also, the user would need to use the admin or operator macaroon to be able to call wallet's RPCs like InitWallet.
The root key encrypt must be provided at start time. It could even be an ephemeral random one, but not sure if this is the best choice. I'll take a look at how LND designed this, but I guess they might have used the same password they ask the user for locking/unlocking the db.

but I guess they might have used the same password they ask the user for locking/unlocking the db.

It's exactly how I supposed.
By looking at the ChangePassword RPC (which is something similar to our Wallet/ChangePassword), they use the same password for locking/unlocking the DB and for encrypting the macaroons.
Shal we add an env var at this point? We could add something in the readme to suggest cleaning the var once the daemon is up and running, eventually.

This is not possible because the generation of the macaroons is done when starting the daemon. Also, the user would need to use the admin or operator macaroon to be able to call wallet's RPCs like InitWallet.

We can "delay" the creation of the root macaroon after wallet initialization. genseed and initwallet are indeed unprotected, since are needed to be called the first time anyway.


converted from release notes section in README.md to CHANGELOG.md, ad…

Pull Request Test Coverage Report for Build 258

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 85.903%

Totals

Change from base Build 256:
0.0%

Covered Lines:
1499

Relevant Lines:
1745

💛  - Coveralls


Resize Constraint Glitch When Grid Width Changes

Using this jsfiddle, here are the steps to reproduce:

Find the bottom-most grid item and resize it to its maximum possible width.
Click the "Toggle Grid Width" link at the top.
Find the same grid item (from step 1) and try to resize it. Watch how the grid item instantly reduces in size, and how it cannot be expanded to the full width of the grid anymore.

Thanks for the good reproduction. A similar bug has caused me to unpublish 0.9.2 from npm.
There is some state lying inside react-resizable and react-draggable that can make this difficult to deal with. I would like to make them more stateless or at least have the option to be 100% stateless and rely on the user to properly set left/top and width/height via the parent component.
This will require some reworking of those projects.

Hm. Do you mean that you'd rather have the parent component pass in the mouse coordinates?

Basically, this state from react-draggable and this state from react-resizable causes issues. I've tried some hacks to allow external resetting of this state but it's all bad hacks. I need to rearchitect it.

Do you mean this state from react-resizable?
What can I do, if anything, to help?

Revert back to 0.8.3, which doesn't exhibit this bug.
Unfortunately I will have to unpublish all of 0.9. A fixed version will be 0.10.x.

Great, thanks. Any idea of when 0.10.x will be released?

I have a few other projects that are taking up all of my time right now so RGL can't be a priority (I don't use it in any of my projects yet). If you're brave, you could fork react-draggable and resizable and take a crack at it. Otherwise I would expect to get to it this weekend.

Okay, thanks for letting me know. And thanks again for all of your great work with these libraries!

Any luck with this?

This would be extremely helpful.

Any updates on this bug?

This is fixed in 0.10.


how to plot data already on gpu

suppose I have a million time traces on a gpu computed with pycuda. How would I plot them (or a subset) from the gpu to a notebook cell or screen with pygfx?

earlier discussion on this https://github.com/pygfx/pygfx/issues/510
As far as I understand, this would require sharing between CUDA contexts and WGPU contexts.

thx
I'll see #510. I've seen c++ examples using cuda  and opengl. Seems like
an obvious application with large data generated on gpu. I have a case
where matplotlib rendering and data transfer are both >>> gpu generation
of data
data is like 10^6 x 10^6
looks like Im stuck at a lower level for now
On 2024-07-23 21:37, Kushal Kolar wrote:

earlier discussion on this #510 [1]
As far as I understand, this would require sharing between CUDA
contexts and WGPU contexts.
--
Reply to this email directly, view it on GitHub [2], or unsubscribe
[3].
You are receiving this because you authored the thread.Message ID:
@.***>

--
-john
Principal Research Scientist John Wright
Office<PHONE_NUMBER>
zoom: https://mit.zoom.us/my/jcwright
Links:
[1] https://github.com/pygfx/pygfx/issues/510
[2] https://github.com/pygfx/pygfx/issues/830#issuecomment-2246692889
[3]
https://github.com/notifications/unsubscribe-auth/AB7SLTKKNOAKGHAWLKB5QIDZN4ANNAVCNFSM6AAAAABLLPDZGWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENBWGY4TEOBYHE

I've seen c++ examples using cuda and opengl.

Interesting, if you can post them that could help.

Short answer is that its not possible for the foreseeable future. I wonder though to what extend wgpu compute shaders can replace Cuda. Because in that case it would be easy ...

see the few years old book abstract on cuda+opengl https://www.informit.com/articles/article.aspx?p=2455391&seqNum=2
This article: 3D Terrain Real-time Rendering Method Based on CUDA-OpenGL Interoperability
 also covers some of the basic ideas of common arrays for CUDA-OpenGL

burn seems to be a rust lib that supports a WGPU backend. They seem to be interested in implementing basic linalg which would be amazing! https://github.com/tracel-ai/burn/issues/1538 It could help with figuring out how to use wgpu-py compute shaders to implement basic decompositions and build on from there

Other thought: I wonder how hard it'd be for someone to contribute a wgpu-backend for PyTorch 🤔

Well, tinygrad supports wgpu-py as a backend, so I'm sure it's possible for pytorch. They do have a lot more technical debt to deal with though so there are probably a bunch of unexpected complications.

The burn library has onnx support. I’ve been using onnx to more easily work around some peculiarities of torch and tensorflow.

Seems like tinygrad has discontinued wgpu-py as a backend, but it's still in there? https://github.com/tinygrad/tinygrad/issues/3307
Might still be useful, maybe if they get some help it can continue to be maintained :thinking: , will look into later

Seems like tinygrad has discontinued wgpu-py as a backend, but it's still in there? tinygrad/tinygrad#3307

Might make a return, as there was a bounty added to the spreadsheet and this PR branch opened just the other day: https://github.com/tinygrad/tinygrad/pull/6304

for future reference, they seem to be able to auto-generate compute shaders :thinking:
https://github.com/tinygrad/tinygrad/pull/1077/files#diff-5f6f19d500b2fd4414e87a94669148aedb3af14318f880b7618bc293d0eb3e82
Not sure where this code is in the release, will dig into later


[SPARK-11839] [ML] refactor save/write traits

add "ML" prefix to reader/writer/readable/writable to avoid name collision with java.util.*
define DefaultParamsReadable/Writable and use them to save some code
use super.load instead so people can jump directly to the doc of Readable.load, which documents the Java compatibility issues

@jkbradley

Test build #46282 has started for PR 9827 at commit bdd5a39.

Nice, this LGTM except you'll need to update the JavaDefaultReadWriteSuite

One more comment: Do you want to add Since tags to the new parts of LogisticRegression, Pipeline?

Test build #46282 has finished for PR 9827 at commit bdd5a39.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):\n  * public final class UnsafeSorterSpillReader extends UnsafeSorterIterator implements Closeable \n  * class Pipeline(override val uid: String) extends Estimator[PipelineModel] with MLWritable \n  *  class CountVectorizerModelWriter(instance: CountVectorizerModel) extends MLWriter\n  * final class IDF(override val uid: String) extends Estimator[IDFModel] with IDFBase\n  *  class MinMaxScalerModelWriter(instance: MinMaxScalerModel) extends MLWriter\n  * class SQLTransformer @Since(\"1.6.0\") (override val uid: String) extends Transformer\n  *  class StandardScalerModelWriter(instance: StandardScalerModel) extends MLWriter\n  *  class StringIndexModelWriter(instance: StringIndexerModel) extends MLWriter\n  * class ALS(override val uid: String) extends Estimator[ALSModel] with ALSParams\n  * abstract class MLWriter extends BaseReadWrite with Logging \n  * trait MLWritable \n  * abstract class MLReader[T] extends BaseReadWrite \n  * trait MLReadable[T] \n  * case class EncodeUsingSerializer(child: Expression, kryo: Boolean) extends UnaryExpression \n  * case class DecodeUsingSerializer[T](child: Expression, tag: ClassTag[T], kryo: Boolean)\n

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/46282/
Test PASSed.

LGTM, merging with master and branch-1.6


#20585: Fixed minor typo

[x] I have followed (at least) the PR section of the contributing guide.

You only need to change the English ones (the file you changed in #20585), the others will be synced later. Thanks!


Make the game all-device compatible

Add mobile controls.
Add small, medium, and mobile display compatibility.

Will develop a mobile version later


IPv6 requests

Hello,
Is there any reason why the IP address which gets saved to a form is Cloudflare's when it should be the user's?
Cloudflare's IPv6 address is visible as the form submitter's IP address.
The IP:
2405:8100:8000:5ca1:0:0:98:b201

Since Cloudflare is a proxy, by nature the connecting IP to your server will be a Cloudflare IP when it proxies your traffic. Cloudflare does provide a CF-CONNECTING-IP header that will contain the original end user IP.
The exact answer as to how to get that IP into your server or application logs depends on specifics of your environment. You can find various support articles about restoring visitor IPs here.


Make source map generation optional

This is nice when debugging JSX code

@andreypopp - I've made the change you suggested.

@aldendaniels You've mentioned debugging. Did you encounter the same as I have? The grey lines in Chrome and break points does not work

And if I change the true to false in the node_modules/reactify/index.js, the whole thing works as before and looks good to me.

@hbrls - I hadn't noticed the gray bar issue with Reactify.
I recently switch to babelify - which also transpiles JSX -> JS.
Unlike Reactify, babelify uses acorn-jsx under the hood instead of FBs offical (esprima-based) JSX transpiler.
Anyway, today I noticed that Babel has the same issue with gray bars that you described:

Glad to know that disabling source maps fixes the issue.

@hbrls - saw your comment in http://stackoverflow.com/questions/29650951/what-are-these-grey-lines-in-the-chrome-sources-panel
So you also noticed that the gray line issues occurs both on babelify and reactify.