Default Archive Location Error

I followed the default getting started document on argo website :
https://argoproj.github.io/docs/argo/getting-started.html
Followed until step 7 ( after minio setup )
Error:
Name:                artifact-passing-mwwct
Namespace:           default
ServiceAccount:      default
Status:              Failed
Message:             child 'artifact-passing-mwwct[0].generate-artifact' errored
Created:             Fri Apr 17 20:16:42 +0500 (10 seconds from now)
Started:             Fri Apr 17 20:16:42 +0500 (10 seconds from now)
Finished:            Fri Apr 17 20:16:42 +0500 (10 seconds from now)
Duration:            0 seconds

STEP                       TEMPLATE          PODNAME                            DURATION  MESSAGE
 ✖ artifact-passing-mwwct  artifact-example                                               child 'artifact-passing-mwwct[0].generate-artifact' errored
 └---⚠ generate-artifact   whalesay          artifact-passing-mwwct-3765465502  0s        controller is not configured with a default archive location

I'm using Azure Kuberenetes Service (AKS)
Argo Version:
argo: v2.7.0
  BuildDate: 2020-03-31T23:35:04Z
  GitCommit: 4d1175eb68f6578ed5d599f877be9b4855d33ce9
  GitTreeState: clean
  GitTag: v2.7.0
  GoVersion: go1.13.4
  Compiler: gc
  Platform: linux/amd64

Let me know if more info is needed to look into this issue.

Could you please check the controller logs for errors?

After getting the error, these are logs of the controller:
time="2020-04-20T06:12:46Z" level=info msg="Processing workflow" namespace=default workflow=output-artifact-s3-hjspp
time="2020-04-20T06:12:46Z" level=info msg="Updated phase  -> Running" namespace=default workflow=output-artifact-s3-hjspp
time="2020-04-20T06:12:46Z" level=info msg="Pod node {output-artifact-s3-hjspp output-artifact-s3-hjspp output-artifact-s3-hjspp Pod whalesay nil    Pending   2020-04-20 06:12:46.457068818 +0000 UTC 0001-01-01 00:00:00 +0000 UTC   <nil
> nil nil [] []} initialized Pending" namespace=default workflow=output-artifact-s3-hjspp
time="2020-04-20T06:12:46Z" level=error msg="Mark error node output-artifact-s3-hjspp: controller is not configured with a default archive location\ngithub.com/argoproj/argo/errors.New\n\t/go/src/github.com/argoproj/argo/errors/errors.
go:49\ngithub.com/argoproj/argo/errors.Errorf\n\t/go/src/github.com/argoproj/argo/errors/errors.go:55\ngithub.com/argoproj/argo/workflow/controller.(*wfOperationCtx).addArchiveLocation\n\t/go/src/github.com/argoproj/argo/workflow/contr
oller/workflowpod.go:937\ngithub.com/argoproj/argo/workflow/controller.(*wfOperationCtx).createWorkflowPod\n\t/go/src/github.com/argoproj/argo/workflow/controller/workflowpod.go:165\ngithub.com/argoproj/argo/workflow/controller.(*wfOpe
rationCtx).executeContainer\n\t/go/src/github.com/argoproj/argo/workflow/controller/operator.go:1749\ngithub.com/argoproj/argo/workflow/controller.(*wfOperationCtx).executeTemplate\n\t/go/src/github.com/argoproj/argo/workflow/controlle
r/operator.go:1432\ngithub.com/argoproj/argo/workflow/controller.(*wfOperationCtx).operate\n\t/go/src/github.com/argoproj/argo/workflow/controller/operator.go:281\ngithub.com/argoproj/argo/workflow/controller.(*WorkflowController).proc
essNextItem\n\t/go/src/github.com/argoproj/argo/workflow/controller/controller.go:411\ngithub.com/argoproj/argo/workflow/controller.(*WorkflowController).runWorker\n\t/go/src/github.com/argoproj/argo/workflow/controller/controller.go:3
25\nk8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\t/go/pkg/mod/k8s.io/apimachinery<EMAIL_ADDRESS>il/wait/wait.go:153\nk8s.io/apimachinery/pkg/util/wait.Until\n\t/go/pkg/mod/k8s.io/apimachinery@v0.16.7-beta.0/pkg/util/wait/wait.go:88\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1357" namespace=default workflow=output-ar
tifact-s3-hjspp
time="2020-04-20T06:12:46Z" level=info msg="node &NodeStatus{ID:output-artifact-s3-hjspp,Name:output-artifact-s3-hjspp,DisplayName:output-artifact-s3-hjspp,Type:Pod,TemplateName:whalesay,TemplateRef:nil,Phase:Pending,BoundaryID:,Messag
e:,StartedAt:2020-04-20 06:12:46.457068818 +0000 UTC,FinishedAt:0001-01-01 00:00:00 +0000 UTC,PodIP:,Daemoned:nil,Inputs:nil,Outputs:nil,Children:[],OutboundNodes:[],StoredTemplateID:,WorkflowTemplateName:,TemplateScope:,ResourcesDurat
ion:ResourcesDuration{},} phase Pending -> Error" namespace=default workflow=output-artifact-s3-hjspp
time="2020-04-20T06:12:46Z" level=info msg="node &NodeStatus{ID:output-artifact-s3-hjspp,Name:output-artifact-s3-hjspp,DisplayName:output-artifact-s3-hjspp,Type:Pod,TemplateName:whalesay,TemplateRef:nil,Phase:Error,BoundaryID:,Message:
,StartedAt:2020-04-20 06:12:46.457068818 +0000 UTC,FinishedAt:0001-01-01 00:00:00 +0000 UTC,PodIP:,Daemoned:nil,Inputs:nil,Outputs:nil,Children:[],OutboundNodes:[],StoredTemplateID:,WorkflowTemplateName:,TemplateScope:,ResourcesDuratio
n:ResourcesDuration{},} message: controller is not configured with a default archive location" namespace=default workflow=output-artifact-s3-hjspp
time="2020-04-20T06:12:46Z" level=info msg="node &NodeStatus{ID:output-artifact-s3-hjspp,Name:output-artifact-s3-hjspp,DisplayName:output-artifact-s3-hjspp,Type:Pod,TemplateName:whalesay,TemplateRef:nil,Phase:Error,BoundaryID:,Message:
controller is not configured with a default archive location,StartedAt:2020-04-20 06:12:46.457068818 +0000 UTC,FinishedAt:2020-04-20 06:12:46.457966194 +0000 UTC,PodIP:,Daemoned:nil,Inputs:nil,Outputs:nil,Children:[],OutboundNodes:[],S
toredTemplateID:,WorkflowTemplateName:,TemplateScope:,ResourcesDuration:ResourcesDuration{},} finished: 2020-04-20 06:12:46.457966194 +0000 UTC" namespace=default workflow=output-artifact-s3-hjspp
time="2020-04-20T06:12:46Z" level=error msg="output-artifact-s3-hjspp error in entry template execution: controller is not configured with a default archive location\ngithub.com/argoproj/argo/errors.New\n\t/go/src/github.com/argoproj/a
rgo/errors/errors.go:49\ngithub.com/argoproj/argo/errors.Errorf\n\t/go/src/github.com/argoproj/argo/errors/errors.go:55\ngithub.com/argoproj/argo/workflow/controller.(*wfOperationCtx).addArchiveLocation\n\t/go/src/github.com/argoproj/a
rgo/workflow/controller/workflowpod.go:937\ngithub.com/argoproj/argo/workflow/controller.(*wfOperationCtx).createWorkflowPod\n\t/go/src/github.com/argoproj/argo/workflow/controller/workflowpod.go:165\ngithub.com/argoproj/argo/workflow/
controller.(*wfOperationCtx).executeContainer\n\t/go/src/github.com/argoproj/argo/workflow/controller/operator.go:1749\ngithub.com/argoproj/argo/workflow/controller.(*wfOperationCtx).executeTemplate\n\t/go/src/github.com/argoproj/argo/
workflow/controller/operator.go:1432\ngithub.com/argoproj/argo/workflow/controller.(*wfOperationCtx).operate\n\t/go/src/github.com/argoproj/argo/workflow/controller/operator.go:281\ngithub.com/argoproj/argo/workflow/controller.(*Workfl
owController).processNextItem\n\t/go/src/github.com/argoproj/argo/workflow/controller/controller.go:411\ngithub.com/argoproj/argo/workflow/controller.(*WorkflowController).runWorker\n\t/go/src/github.com/argoproj/argo/workflow/controll
er/controller.go:325\nk8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\t/go/pkg/mod/k8s.io/apimachinery@v0.16.7-beta.0/pkg/util/wait/wait.go:152\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/pkg/mod/k8s.io/apimachinery@v0.
16.7-beta.0/pkg/util/wait/wait.go:153\nk8s.io/apimachinery/pkg/util/wait.Until\n\t/go/pkg/mod/k8s.io/apimachinery@v0.16.7-beta.0/pkg/util/wait/wait.go:88\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1357" namespace=default
workflow=output-artifact-s3-hjspp
time="2020-04-20T06:12:46Z" level=info msg="Workflow update successful" namespace=default phase=Running resourceVersion=281296 workflow=output-artifact-s3-hjspp
time="2020-04-20T06:12:47Z" level=info msg="Processing workflow" namespace=default workflow=output-artifact-s3-hjspp
time="2020-04-20T06:12:47Z" level=info msg="Updated phase Running -> Error" namespace=default workflow=output-artifact-s3-hjspp
time="2020-04-20T06:12:47Z" level=info msg="Updated message  -> controller is not configured with a default archive location" namespace=default workflow=output-artifact-s3-hjspp
time="2020-04-20T06:12:47Z" level=info msg="Marking workflow completed" namespace=default workflow=output-artifact-s3-hjspp
time="2020-04-20T06:12:47Z" level=info msg="Checking daemoned children of " namespace=default workflow=output-artifact-s3-hjspp
time="2020-04-20T06:12:47Z" level=info msg="Workflow update successful" namespace=default phase=Error resourceVersion=281300 workflow=output-artifact-s3-hjspp

OK. You need to configure the default location in the workflow-controller-configmap:
https://github.com/argoproj/argo/blob/master/docs/workflow-controller-configmap.yaml

I did, as described on step 6, and this is my configmap:
# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: v1
data:
  config: |
    artifactRepository: |
      s3:
        bucket: my-bucket
        endpoint: argo-artifacts.default:9000
        insecure: true
        # accessKeySecret and secretKeySecret are secret selectors.
        # It references the k8s secret named 'argo-artifacts'
        # which was created during the minio helm install. The keys,
        # 'accesskey' and 'secretkey', inside that secret are where the
        # actual minio credentials are stored.
        accessKeySecret:
          name: argo-artifacts
          key: accesskey
        secretKeySecret:
          name: argo-artifacts
          key: secretkey
kind: ConfigMap
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"v1","kind":"ConfigMap","metadata":{"annotations":{},"name":"workflow-controller-configmap","namespace":"argo"}}
  creationTimestamp: "2020-04-17T15:03:15Z"
  name: workflow-controller-configmap
  namespace: argo
  resourceVersion: "2250"
  selfLink: /api/v1/namespaces/argo/configmaps/workflow-controller-configmap
  uid: 2aa67b8c-9ef5-4ebc-9b5a-a8342b40acd0

The error is pretty explicit, so I'm a bit confused. Can you check that the controller logs show the correct configuration please?

Ok so now I copied the secret in both default and argo namespace and tried to run artifact sample and error remains same, here are the controller logs:
time="2020-04-20T16:07:50Z" level=info msg="Processing workflow" namespace=default workflow=artifact-passing-8l5dt
time="2020-04-20T16:07:50Z" level=info msg="Updated phase  -> Running" namespace=default workflow=artifact-passing-8l5dt
time="2020-04-20T16:07:50Z" level=info msg="Steps node {artifact-passing-8l5dt artifact-passing-8l5dt artifact-passing-8l5dt Steps artifact-example nil    Running   2020-04-20 16:07:50.689468384 +0000 UTC 0001-01-01 00:00:00 +0000 UTC
  <nil> nil nil [] []} initialized Running" namespace=default workflow=artifact-passing-8l5dt
time="2020-04-20T16:07:50Z" level=info msg="StepGroup node {artifact-passing-8l5dt-134184679 artifact-passing-8l5dt[0] [0] StepGroup artifact-example nil    Running artifact-passing-8l5dt  2020-04-20 16:07:50.689575493 +0000 UTC 0001-0
1-01 00:00:00 +0000 UTC   <nil> nil nil [] []} initialized Running" namespace=default workflow=artifact-passing-8l5dt
time="2020-04-20T16:07:50Z" level=info msg="Pod node {artifact-passing-8l5dt-1146599573 artifact-passing-8l5dt[0].generate-artifact generate-artifact Pod whalesay nil    Pending artifact-passing-8l5dt  2020-04-20 16:07:50.690059235 +00
00 UTC 0001-01-01 00:00:00 +0000 UTC   <nil> nil nil [] []} initialized Pending" namespace=default workflow=artifact-passing-8l5dt
time="2020-04-20T16:07:50Z" level=error msg="Mark error node artifact-passing-8l5dt[0].generate-artifact: controller is not configured with a default archive location\ngithub.com/argoproj/argo/errors.New\n\t/go/src/github.com/argoproj/
argo/errors/errors.go:49\ngithub.com/argoproj/argo/errors.Errorf\n\t/go/src/github.com/argoproj/argo/errors/errors.go:55\ngithub.com/argoproj/argo/workflow/controller.(*wfOperationCtx).addArchiveLocation\n\t/go/src/github.com/argoproj/
argo/workflow/controller/workflowpod.go:937\ngithub.com/argoproj/argo/workflow/controller.(*wfOperationCtx).createWorkflowPod\n\t/go/src/github.com/argoproj/argo/workflow/controller/workflowpod.go:165\ngithub.com/argoproj/argo/workflow
/controller.(*wfOperationCtx).executeContainer\n\t/go/src/github.com/argoproj/argo/workflow/controller/operator.go:1749\ngithub.com/argoproj/argo/workflow/controller.(*wfOperationCtx).executeTemplate\n\t/go/src/github.com/argoproj/argo
/workflow/controller/operator.go:1432\ngithub.com/argoproj/argo/workflow/controller.(*wfOperationCtx).executeStepGroup\n\t/go/src/github.com/argoproj/argo/workflow/controller/steps.go:223\ngithub.com/argoproj/argo/workflow/controller.(
*wfOperationCtx).executeSteps\n\t/go/src/github.com/argoproj/argo/workflow/controller/steps.go:86\ngithub.com/argoproj/argo/workflow/controller.(*wfOperationCtx).executeTemplate\n\t/go/src/github.com/argoproj/argo/workflow/controller/o
perator.go:1434\ngithub.com/argoproj/argo/workflow/controller.(*wfOperationCtx).operate\n\t/go/src/github.com/argoproj/argo/workflow/controller/operator.go:281\ngithub.com/argoproj/argo/workflow/controller.(*WorkflowController).process
NextItem\n\t/go/src/github.com/argoproj/argo/workflow/controller/controller.go:411\ngithub.com/argoproj/argo/workflow/controller.(*WorkflowController).runWorker\n\t/go/src/github.com/argoproj/argo/workflow/controller/controller.go:325\
nk8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\t/go/pkg/mod/k8s.io/apimachinery<EMAIL_ADDRESS>wait/wait.go:153\nk8s.io/apimachinery/pkg/util/wait.Until\n\t/go/pkg/mod/k8s.io/apimachinery@v0.16.7-beta.0/pkg/util/wait/wait.go:88\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1357" namespace=default workflow=artifact-pas
sing-8l5dt
time="2020-04-20T16:07:50Z" level=info msg="node &NodeStatus{ID:artifact-passing-8l5dt-1146599573,Name:artifact-passing-8l5dt[0].generate-artifact,DisplayName:generate-artifact,Type:Pod,TemplateName:whalesay,TemplateRef:nil,Phase:Pendi
ng,BoundaryID:artifact-passing-8l5dt,Message:,StartedAt:2020-04-20 16:07:50.690059235 +0000 UTC,FinishedAt:0001-01-01 00:00:00 +0000 UTC,PodIP:,Daemoned:nil,Inputs:nil,Outputs:nil,Children:[],OutboundNodes:[],StoredTemplateID:,Workflow
TemplateName:,TemplateScope:,ResourcesDuration:ResourcesDuration{},} phase Pending -> Error" namespace=default workflow=artifact-passing-8l5dt
time="2020-04-20T16:07:50Z" level=info msg="node &NodeStatus{ID:artifact-passing-8l5dt-1146599573,Name:artifact-passing-8l5dt[0].generate-artifact,DisplayName:generate-artifact,Type:Pod,TemplateName:whalesay,TemplateRef:nil,Phase:Error
,BoundaryID:artifact-passing-8l5dt,Message:,StartedAt:2020-04-20 16:07:50.690059235 +0000 UTC,FinishedAt:0001-01-01 00:00:00 +0000 UTC,PodIP:,Daemoned:nil,Inputs:nil,Outputs:nil,Children:[],OutboundNodes:[],StoredTemplateID:,WorkflowTe
mplateName:,TemplateScope:,ResourcesDuration:ResourcesDuration{},} message: controller is not configured with a default archive location" namespace=default workflow=artifact-passing-8l5dt
time="2020-04-20T16:07:50Z" level=info msg="node &NodeStatus{ID:artifact-passing-8l5dt-1146599573,Name:artifact-passing-8l5dt[0].generate-artifact,DisplayName:generate-artifact,Type:Pod,TemplateName:whalesay,TemplateRef:nil,Phase:Error
,BoundaryID:artifact-passing-8l5dt,Message:controller is not configured with a default archive location,StartedAt:2020-04-20 16:07:50.690059235 +0000 UTC,FinishedAt:2020-04-20 16:07:50.690536476 +0000 UTC,PodIP:,Daemoned:nil,Inputs:nil
,Outputs:nil,Children:[],OutboundNodes:[],StoredTemplateID:,WorkflowTemplateName:,TemplateScope:,ResourcesDuration:ResourcesDuration{},} finished: 2020-04-20 16:07:50.690536476 +0000 UTC" namespace=default workflow=artifact-passing-8l5
dt
time="2020-04-20T16:07:50Z" level=info msg="Step group node &NodeStatus{ID:artifact-passing-8l5dt-134184679,Name:artifact-passing-8l5dt[0],DisplayName:[0],Type:StepGroup,TemplateName:artifact-example,TemplateRef:nil,Phase:Running,Bound
aryID:artifact-passing-8l5dt,Message:,StartedAt:2020-04-20 16:07:50.689575493 +0000 UTC,FinishedAt:0001-01-01 00:00:00 +0000 UTC,PodIP:,Daemoned:nil,Inputs:nil,Outputs:nil,Children:[],OutboundNodes:[],StoredTemplateID:,WorkflowTemplate
Name:,TemplateScope:,ResourcesDuration:ResourcesDuration{},} deemed errored due to child artifact-passing-8l5dt[0].generate-artifact error: controller is not configured with a default archive location" namespace=default workflow=artifa
ct-passing-8l5dt
time="2020-04-20T16:07:50Z" level=info msg="node &NodeStatus{ID:artifact-passing-8l5dt-134184679,Name:artifact-passing-8l5dt[0],DisplayName:[0],Type:StepGroup,TemplateName:artifact-example,TemplateRef:nil,Phase:Running,BoundaryID:artif
act-passing-8l5dt,Message:,StartedAt:2020-04-20 16:07:50.689575493 +0000 UTC,FinishedAt:0001-01-01 00:00:00 +0000 UTC,PodIP:,Daemoned:nil,Inputs:nil,Outputs:nil,Children:[artifact-passing-8l5dt-1146599573],OutboundNodes:[],StoredTempla
teID:,WorkflowTemplateName:,TemplateScope:,ResourcesDuration:ResourcesDuration{},} phase Running -> Error" namespace=default workflow=artifact-passing-8l5dt
time="2020-04-20T16:07:50Z" level=info msg="node &NodeStatus{ID:artifact-passing-8l5dt-134184679,Name:artifact-passing-8l5dt[0],DisplayName:[0],Type:StepGroup,TemplateName:artifact-example,TemplateRef:nil,Phase:Error,BoundaryID:artifac
t-passing-8l5dt,Message:,StartedAt:2020-04-20 16:07:50.689575493 +0000 UTC,FinishedAt:0001-01-01 00:00:00 +0000 UTC,PodIP:,Daemoned:nil,Inputs:nil,Outputs:nil,Children:[artifact-passing-8l5dt-1146599573],OutboundNodes:[],StoredTemplate
ID:,WorkflowTemplateName:,TemplateScope:,ResourcesDuration:ResourcesDuration{},} message: child 'artifact-passing-8l5dt[0].generate-artifact' errored" namespace=default workflow=artifact-passing-8l5dt
time="2020-04-20T16:07:50Z" level=info msg="node &NodeStatus{ID:artifact-passing-8l5dt-134184679,Name:artifact-passing-8l5dt[0],DisplayName:[0],Type:StepGroup,TemplateName:artifact-example,TemplateRef:nil,Phase:Error,BoundaryID:artifac
t-passing-8l5dt,Message:child 'artifact-passing-8l5dt[0].generate-artifact' errored,StartedAt:2020-04-20 16:07:50.689575493 +0000 UTC,FinishedAt:2020-04-20 16:07:50.691062221 +0000 UTC,PodIP:,Daemoned:nil,Inputs:nil,Outputs:nil,Childre
n:[artifact-passing-8l5dt-1146599573],OutboundNodes:[],StoredTemplateID:,WorkflowTemplateName:,TemplateScope:,ResourcesDuration:ResourcesDuration{},} finished: 2020-04-20 16:07:50.691062221 +0000 UTC" namespace=default workflow=artifac
t-passing-8l5dt
time="2020-04-20T16:07:50Z" level=info msg="step group artifact-passing-8l5dt-134184679 was unsuccessful: child 'artifact-passing-8l5dt[0].generate-artifact' errored" namespace=default workflow=artifact-passing-8l5dt
time="2020-04-20T16:07:50Z" level=info msg="Outbound nodes of artifact-passing-8l5dt-1146599573 is [artifact-passing-8l5dt-1146599573]" namespace=default workflow=artifact-passing-8l5dt
time="2020-04-20T16:07:50Z" level=info msg="Outbound nodes of artifact-passing-8l5dt is [artifact-passing-8l5dt-1146599573]" namespace=default workflow=artifact-passing-8l5dt
time="2020-04-20T16:07:50Z" level=info msg="node &NodeStatus{ID:artifact-passing-8l5dt,Name:artifact-passing-8l5dt,DisplayName:artifact-passing-8l5dt,Type:Steps,TemplateName:artifact-example,TemplateRef:nil,Phase:Running,BoundaryID:,Me
ssage:,StartedAt:2020-04-20 16:07:50.689468384 +0000 UTC,FinishedAt:0001-01-01 00:00:00 +0000 UTC,PodIP:,Daemoned:nil,Inputs:nil,Outputs:nil,Children:[artifact-passing-8l5dt-134184679],OutboundNodes:[artifact-passing-8l5dt-1146599573],
StoredTemplateID:,WorkflowTemplateName:,TemplateScope:,ResourcesDuration:ResourcesDuration{},} phase Running -> Failed" namespace=default workflow=artifact-passing-8l5dt
time="2020-04-20T16:07:50Z" level=info msg="node &NodeStatus{ID:artifact-passing-8l5dt,Name:artifact-passing-8l5dt,DisplayName:artifact-passing-8l5dt,Type:Steps,TemplateName:artifact-example,TemplateRef:nil,Phase:Failed,BoundaryID:,Mes
sage:,StartedAt:2020-04-20 16:07:50.689468384 +0000 UTC,FinishedAt:0001-01-01 00:00:00 +0000 UTC,PodIP:,Daemoned:nil,Inputs:nil,Outputs:nil,Children:[artifact-passing-8l5dt-134184679],OutboundNodes:[artifact-passing-8l5dt-1146599573],S
toredTemplateID:,WorkflowTemplateName:,TemplateScope:,ResourcesDuration:ResourcesDuration{},} message: child 'artifact-passing-8l5dt[0].generate-artifact' errored" namespace=default workflow=artifact-passing-8l5dt
time="2020-04-20T16:07:50Z" level=info msg="node &NodeStatus{ID:artifact-passing-8l5dt,Name:artifact-passing-8l5dt,DisplayName:artifact-passing-8l5dt,Type:Steps,TemplateName:artifact-example,TemplateRef:nil,Phase:Failed,BoundaryID:,Mes
sage:child 'artifact-passing-8l5dt[0].generate-artifact' errored,StartedAt:2020-04-20 16:07:50.689468384 +0000 UTC,FinishedAt:2020-04-20 16:07:50.691537362 +0000 UTC,PodIP:,Daemoned:nil,Inputs:nil,Outputs:nil,Children:[artifact-passing
-8l5dt-134184679],OutboundNodes:[artifact-passing-8l5dt-1146599573],StoredTemplateID:,WorkflowTemplateName:,TemplateScope:,ResourcesDuration:ResourcesDuration{},} finished: 2020-04-20 16:07:50.691537362 +0000 UTC" namespace=default wor
kflow=artifact-passing-8l5dt
time="2020-04-20T16:07:50Z" level=info msg="Checking daemoned children of artifact-passing-8l5dt" namespace=default workflow=artifact-passing-8l5dt
time="2020-04-20T16:07:50Z" level=info msg="Updated phase Running -> Failed" namespace=default workflow=artifact-passing-8l5dt
time="2020-04-20T16:07:50Z" level=info msg="Updated message  -> child 'artifact-passing-8l5dt[0].generate-artifact' errored" namespace=default workflow=artifact-passing-8l5dt
time="2020-04-20T16:07:50Z" level=info msg="Marking workflow completed" namespace=default workflow=artifact-passing-8l5dt
time="2020-04-20T16:07:50Z" level=info msg="Checking daemoned children of " namespace=default workflow=artifact-passing-8l5dt
time="2020-04-20T16:07:50Z" level=info msg="Workflow update successful" namespace=default phase=Failed resourceVersion=325259 workflow=artifact-passing-8l5dt
time="2020-04-20T16:08:29Z" level=info msg="Alloc=4534 TotalAlloc=179789 Sys=70080 NumGC=2195 Goroutines=101"

The workflow controller prints it's configuration in the first few log lines - can you please attach those?

These are logs from the start:
time="2020-04-17T15:03:29Z" level=info msg="config map" name=workflow-controller-configmap
time="2020-04-17T15:03:29Z" level=info msg="Configuration:\nartifactRepository: {}\nfeatureFlags: {}\nmetricsConfig:\n  disableLegacy: false\npodSpecLogStrategy: {}\ntelemetryConfig:\n  disableLegacy: false\n"
time="2020-04-17T15:03:29Z" level=info msg="Persistence configuration disabled"
time="2020-04-17T15:03:29Z" level=info msg="Starting CronWorkflow controller"
time="2020-04-17T15:03:29Z" level=info msg="Starting Workflow Controller" version=vv2.7.4+50b209c.dirty
time="2020-04-17T15:03:29Z" level=info msg="Workers: workflow: 8, pod: 8"
time="2020-04-17T15:03:29Z" level=info msg="Starting workflow TTL controller (resync 20m0s)"
time="2020-04-17T15:03:29Z" level=info msg="Performing periodic GC every 5m0s"
time="2020-04-17T15:03:29Z" level=info msg="Started workflow TTL worker"
time="2020-04-17T15:04:52Z" level=info msg="Processing workflow" namespace=default workflow=loops-maps-9cv79
time="2020-04-17T15:04:52Z" level=info msg="Updated phase  -> Running" namespace=default workflow=loops-maps-9cv79
time="2020-04-17T15:04:52Z" level=info msg="Steps node {loops-maps-9cv79 loops-maps-9cv79 loops-maps-9cv79 Steps loop-map-example nil    Running   2020-04-17 15:04:52.684566096 +0000 UTC 0001-01-01 00:00:00 +0000 UTC   <nil> nil nil []
 []} initialized Running" namespace=default workflow=loops-maps-9cv79
time="2020-04-17T15:04:52Z" level=info msg="StepGroup node {loops-maps-9cv79-2560827875 loops-maps-9cv79[0] [0] StepGroup loop-map-example nil    Running loops-maps-9cv79  2020-04-17 15:04:52.684628 +0000 UTC 0001-01-01 00:00:00 +0000
UTC   <nil> nil nil [] []} initialized Running" namespace=default workflow=loops-maps-9cv79
time="2020-04-17T15:04:52Z" level=info msg="Pod node {loops-maps-9cv79-2603470149 loops-maps-9cv79[0].test-linux(0:image:debian,tag:9.1) test-linux(0:image:debian,tag:9.1) Pod cat-os-release nil    Pending loops-maps-9cv79  2020-04-17
15:04:52.685107735 +0000 UTC 0001-01-01 00:00:00 +0000 UTC   <nil> nil nil [] []} initialized Pending" namespace=default workflow=loops-maps-9cv79
time="2020-04-17T15:04:52Z" level=info msg="Created pod: loops-maps-9cv79[0].test-linux(0:image:debian,tag:9.1) (loops-maps-9cv79-2603470149)" namespace=default workflow=loops-maps-9cv79
time="2020-04-17T15:04:52Z" level=info msg="Pod node {loops-maps-9cv79-4146441795 loops-maps-9cv79[0].test-linux(1:image:debian,tag:8.9) test-linux(1:image:debian,tag:8.9) Pod cat-os-release nil    Pending loops-maps-9cv79  2020-04-17
15:04:52.70311084 +0000 UTC 0001-01-01 00:00:00 +0000 UTC   <nil> nil nil [] []} initialized Pending" namespace=default workflow=loops-maps-9cv79
time="2020-04-17T15:04:52Z" level=info msg="Created pod: loops-maps-9cv79[0].test-linux(1:image:debian,tag:8.9) (loops-maps-9cv79-4146441795)" namespace=default workflow=loops-maps-9cv79
time="2020-04-17T15:04:52Z" level=info msg="Pod node {loops-maps-9cv79-2502400288 loops-maps-9cv79[0].test-linux(2:image:alpine,tag:3.6) test-linux(2:image:alpine,tag:3.6) Pod cat-os-release nil    Pending loops-maps-9cv79  2020-04-17
15:04:52.711068117 +0000 UTC 0001-01-01 00:00:00 +0000 UTC   <nil> nil nil [] []} initialized Pending" namespace=default workflow=loops-maps-9cv79
time="2020-04-17T15:04:52Z" level=info msg="Created pod: loops-maps-9cv79[0].test-linux(2:image:alpine,tag:3.6) (loops-maps-9cv79-2502400288)" namespace=default workflow=loops-maps-9cv79
time="2020-04-17T15:04:52Z" level=info msg="Pod node {loops-maps-9cv79-3240014489 loops-maps-9cv79[0].test-linux(3:image:ubuntu,tag:17.10) test-linux(3:image:ubuntu,tag:17.10) Pod cat-os-release nil    Pending loops-maps-9cv79  2020-04
-17 15:04:52.724372382 +0000 UTC 0001-01-01 00:00:00 +0000 UTC   <nil> nil nil [] []} initialized Pending" namespace=default workflow=loops-maps-9cv79
time="2020-04-17T15:04:52Z" level=info msg="Created pod: loops-maps-9cv79[0].test-linux(3:image:ubuntu,tag:17.10) (loops-maps-9cv79-3240014489)" namespace=default workflow=loops-maps-9cv79
time="2020-04-17T15:04:52Z" level=info msg="Workflow step group node &NodeStatus{ID:loops-maps-9cv79-2560827875,Name:loops-maps-9cv79[0],DisplayName:[0],Type:StepGroup,TemplateName:loop-map-example,TemplateRef:nil,Phase:Running,Boundar
yID:loops-maps-9cv79,Message:,StartedAt:2020-04-17 15:04:52.684628 +0000 UTC,FinishedAt:0001-01-01 00:00:00 +0000 UTC,PodIP:,Daemoned:nil,Inputs:nil,Outputs:nil,Children:[loops-maps-9cv79-2603470149 loops-maps-9cv79-4146441795 loops-ma
ps-9cv79-2502400288 loops-maps-9cv79-3240014489],OutboundNodes:[],StoredTemplateID:,WorkflowTemplateName:,TemplateScope:,ResourcesDuration:ResourcesDuration{},} not yet completed" namespace=default workflow=loops-maps-9cv79
time="2020-04-17T15:04:52Z" level=info msg="Workflow update successful" namespace=default phase=Running resourceVersion=1302 workflow=loops-maps-9cv79
time="2020-04-17T15:04:53Z" level=info msg="Processing workflow" namespace=default workflow=loops-maps-9cv79

Configuration:\nartifactRepository: {}\nfeatureFlags: {}\nmetricsConfig:\n  disableLegacy: false\npodSpecLogStrategy: {}\ntelemetryConfig:\n  disableLegacy: false\n"
It is not loading the config. This is a problem with your installation, but I suspect your controller is not running in the correct namespace. If it is, are you running latest? If so, try changing your config:
# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: v1
data:
  config: |
    # DELETE THE TRAILING "|" OFF THE NEXT LINE
    artifactRepository:
      s3:
        bucket: my-bucket
        endpoint: argo-artifacts.default:9000
        insecure: true
        # accessKeySecret and secretKeySecret are secret selectors.
        # It references the k8s secret named 'argo-artifacts'
        # which was created during the minio helm install. The keys,
        # 'accesskey' and 'secretkey', inside that secret are where the
        # actual minio credentials are stored.
        accessKeySecret:
          name: argo-artifacts
          key: accesskey
        secretKeySecret:
          name: argo-artifacts
          key: secretkey
kind: ConfigMap
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"v1","kind":"ConfigMap","metadata":{"annotations":{},"name":"workflow-controller-configmap","namespace":"argo"}}
  creationTimestamp: "2020-04-17T15:03:15Z"
  name: workflow-controller-configmap
  namespace: argo
  resourceVersion: "2250"
  selfLink: /api/v1/namespaces/argo/configmaps/workflow-controller-configmap
  uid: 2aa67b8c-9ef5-4ebc-9b5a-a8342b40acd0

I just followed this getting start doc, and I suppose it's latest. I went through it like 3,4 times and followed to the exact.
https://argoproj.github.io/docs/argo/getting-started.html

ok I just restarted the controller pod and it crashed, Gives this error:
time="2020-04-20T16:19:28Z" level=info msg="config map" name=workflow-controller-configmap
time="2020-04-20T16:19:28Z" level=fatal msg="Failed to register watch for controller config map: error unmarshaling JSON: while decoding JSON: json: cannot unmarshal string into Go struct field Config.artifactRepository of type config.
ArtifactRepository"

Okay, removing the keyword config: and restarting the controller pod did the trick.
It works fine now. Thanks.

This is the 3rd exact issue for this and yet it doesn't work, as shown on the Getting Started doc. Maybe someone should actually update the source documentation instead of answering the same question (at least) 3 times in issues? Just a thought...

thanks alot, I got the same issue,for the doc use config:
https://argoproj.github.io/docs/argo/configure-artifact-repository.html#s3-compatible-artifact-repository-bucket-such-as-aws-gcs-and-minio
@talhairfanbentley

we should get rid of those docs

@alexmt can you help me? Can we get rid of those docs ?

Hi, I just started to learn and use Argo and K8s. I also hit this issue and the answers here (# DELETE config:\) helps. Thank you!
I also hit two other issues

The argo namespace is not used consistently through this getting-started tutorial. This is potentially confusing. For example, I couldn't see workflows I submitted, hello-world for example, after launching Argo UI (which is with argo namespace) since the instruction submitted it with default namespace.
I couldn't get minio artifacts to work with the instruction, endpoint: argo-artifacts.default:9000 specifically. I got Failed to put file: Get http://my-minio-endpoint.default:9000/my-bucket/?location=: dial tcp: lookup my-minio-endpoint.default on <IP_ADDRESS>:53: no such host. I got it work by doing endpoint: argo-artifacts:9000.

I'm wondering if I could help with the getting-started documentation? Or do you have other big changes planned for this tutorial? @alexec

I'm wondering if I could help with the getting-started documentation? Or do you have other big changes planned for this tutorial? @alexec

Yes please! We'd love to see someone refresh these!

I'm wondering if I could help with the getting-started documentation? Or do you have other big changes planned for this tutorial? @alexec

Yes please! We'd love to see someone refresh these!

Thank you @alexec . Here is my PR: https://github.com/argoproj/argo/pull/3099

I couldn't get minio artifacts to work with the instruction, endpoint: argo-artifacts.default:9000 specifically. I got Failed to put file: Get http://my-minio-endpoint.default:9000/my-bucket/?location=: dial tcp: lookup my-minio-endpoint.default on <IP_ADDRESS>:53: no such host. I got it work by doing endpoint: argo-artifacts:9000.

The docs assume you deploy Minio in the default namespace, so we should probably add some context to what the command is doing and not presume that a new Argo user understands the nuance of Kubernetes networking. I made a comment on the PR @yunhailuo

@yunhailuo
Any reason this error might pop up with a gcs configuration block but not a s3 block?

@yunhailuo
Any reason this error might pop up with a gcs configuration block but not a s3 block?

I'm not quite sure how you got this error. And I'm not sure the difference between your gcs and s3 block. So I can't really say anything.
The specific reason I got this before is namespace mismatch. I was following the previous version and used argo namespace while the artifact configure written there assume it's default. Fixing that solved my problem

we should get rid of those docs

@alexec not just the docs have to change.. the official helm chart is also broken in that regard:
https://github.com/argoproj/argo-helm/blob/master/charts/argo/templates/workflow-controller-config-map.yaml

we should get rid of those docs

@alexec not just the docs have to change.. the official helm chart is also broken in that regard:
https://github.com/argoproj/argo-helm/blob/master/charts/argo/templates/workflow-controller-config-map.yaml

How does that affect the helm chart?

we should get rid of those docs

@alexec not just the docs have to change.. the official helm chart is also broken in that regard:
https://github.com/argoproj/argo-helm/blob/master/charts/argo/templates/workflow-controller-config-map.yaml

How does that affect the helm chart?

It automatically adds the artifactRepository key inside config, which if I understand correctly is an error.

It automatically adds the artifactRepository key inside config, which if I understand correctly is an error.

Fair. But I guess my point was, what the helm chart had to do with deleting the docs? Or, more succinctly, how deleting those docs fixes the helm chart. Not trying to be argumentative, I just don't see the correlation. If the docs and the chart is wrong, it seems like they should be fixed and/or removed.

@ntwrkguru my rhetorical point was that it wasn't the only "leftover" regarding this breaking change.
On one hand you are right that this should have been opened as an issue on the relevant repository and it's somehow unrelated here, and on the other I was trying to enlighten the fact that breaking changes should be communicated better and pushed across the board on all side repositories as well.
I was using "the docs" as a mere example of something that was accidentally overlooked during a breaking change, and since this isn't the first time I'm encountering breaking differences between argo products I do think that it's worth mentioning out load.

@iMoses Thanks for the explanation! I didn't catch the rhetorical bit, so my bad on that one. Agreed on your other points.

If the docs and the chart is wrong, it seems like they should both be fixed and/or removed.

it's a little unclear to me if the Helm chart was ever fixed ? I think I am experiencing this issue but it may be another subtle mis-configuration I have yet to sniff out.
https://github.com/argoproj/argo-helm/blob/master/charts/argo-workflows/templates/controller/workflow-controller-config-map.yaml#L7-L8

upon yet a further examination, it seems like artifactRepository: {} gets set at controller start time, even when ConfigMap workflow-controller-configmap is effectively empty and ConfigMap artifact-repositories exists...
apiVersion: v1
data:
  config: |
    containerRuntimeExecutor: docker
kind: ConfigMap

once I learned about ConfigMap artifact-repositories, I wanted to go that route entirely and de-couple my configs from potentially-buggy Helm Charts.
from reading thru the golang source yesterday, I have a feeling something is still reading artifactRepository: {} and considering that a Fatal misconfig before getting all the way to the ConfigMap artifact-repositories.
it almost seems like my complaints are growing into a new, fresh Issue. but I generally assume I'm making a mistake and not "upstream is broken".

@rkennedy-tpl I can't tell if you think there is a bug or not. If there is, could you raise a new issue?

I cannot tell and I've left copious notes in case someone with more domain expertise can assist in pointing me further.
I've read all the docs, copious Github issues, and the golang source code itself. controller is not configured with a default archive location seems unavoidable in my configuration and there seems to be insufficient data / logging to move forward.
I almost suspect something is mis-configured in k8s RBAC or even AWS Permissions, but there's simply "no logs" indicating that. I can't even find reference to controller is not configured with a default archive location in the current master branch.
I am utterly stumped as to even the nature of a new issue at this point, unless it's related to:
time="2021-06-04T15:01:20.892Z" level=info msg="Update workflows 409"
time="2021-06-04T15:01:20.892Z" level=warning msg="Error updating workflow: Operation cannot be fulfilled on workflows.argoproj.io \"artifact-passing-8z28f\": the object has been modified; please apply your changes to the latest version and try again Conflict" namespace=projectname-development workflow=artifact-passing-8z28f

"Conflict / 409" in kubernetes seems to be a recurring issue across the ecosystem...
https://github.com/kubernetes/kubernetes/issues/67761


Some sort of feedback

I was looking at the current implementation of PollingConsumerProperties, and I thought I could do a port, primarily for my own interest, to Hedgehog, in order to remove the noise from <!>, <*>, Arb.fromGen and Prop.forAll, and fiddle with the automatic/integrated shrinking.
Of course, this isn't perfect either; from 136 lines of code, it went to 155, and I'm positive that it might be introducing False Positives and/or False Negatives.
But I thought I'd share it with you anyway, in the form of a pull request, just in case you find it useful, and to provide some sort of feedback.
(Closing the pull request right away since the goal isn't to get this merged into master.)

Since all NuGet Packages are checked in, I didn't want to run paket install, as this would produce a large diff. It should be done fairly easily though with something like:
# Assuming that `git clean -xdf` has run first.
$ .paket/paket.bootstrapper.exe
$ .paket/paket.exe install
$ ./build.sh

I half expected you to do this 😉
It's interesting that, despite the cleaner API, Hedgehog-based test code bases are bigger than their FsCheck counterparts. It looks to me that this is because of its lack of reflection-based generators. Is that correct?

despite the cleaner API, Hedgehog-based test code bases are bigger than their FsCheck counterparts.

It depends. If not writing shrinking functions for the (QuickCheck/FsCheck) arbitraries, then yes, Hedgehog-based test code bases can be bigger.
On the other hand, Hedgehog's generators, and the Range DSL, provide fine-grained control over the scope and shrinking of generated values. In this case, the code bases can be smaller than their counterparts:
/// A generator for random instants in time, expressed as
/// a date and time of day that shrinks towards year 2000.
let dateTime : Gen<DateTime> =
    let yMin = DateTime.MinValue.Year
    let yMax = DateTime.MaxValue.Year
    gen {
        let! y = Gen.integral <| Range.linearFrom 2015 yMin yMax
        let! m = Gen.integral <| Range.constant 1 12
        let! d = Gen.integral <| Range.constant 1 (DateTime.DaysInMonth (y, m))
        let! h = Gen.integral <| Range.constant 0 23
        let! n = Gen.integral <| Range.constant 0 59
        let! s = Gen.integral <| Range.constant 0 59

        return DateTime (y, m, d, h, n, s)
    }

That's a Gen<DateTime> generator with integrated shrinking towards the year 2015. Here's a similar for FsCheck, and perhaps one for QuickCheck can be found here.

It looks to me that this is because of its lack of reflection-based generators.

There is no reflection-based generator, or even default generators (without taking a Range type), because I don't know how to make high-quality ones.
@jystic shows how a bug slipped in bos/text library by using default arbitraries. It's on this video at 11:50 (up to 13:09).
However, it is possible to write a reflection-based generator for F# Hedgehog using TypeShape, similar to this one, and it'll have shrinking integrated.
(I'm not aware of something similar for the Haskell version of Hedgehog, but for QuickCheck, a high-quality generic arbitrary implementation is Li-yao Xia's generic-random library, from Brent Yorgey's post.)

You could certainly build reflection-based generators on top of Hedgehog, as @moodmosaic points out, it's not clear what the defaults should be. Personally I always like to be explicit about the range and distribution of my test inputs, and that has come across in Hedgehog's design, but I can understand why people would also want something automatic that isn't as precise but is quicker / easy to use.

I support the idea that explicit is better than implicit, so I'm not trying to convince anyone that it should be different. I do think, though, that Hedgehog could benefit from some sort of low-friction option for user data types.
I'm aware of the TypeShape option for F#, although I haven't had time to familiarise myself with it. Eventually, I'll get to it, I hope.

It may be easy to modify this example so that it works with Hedgehog's Gen type. Also, as an alternative to TypeShape, @AnthonyLloyd mentioned Infers in https://github.com/hedgehogqa/fsharp-hedgehog/issues/93.

It may be easy to modify this example so that it works with Hedgehog's Gen type.

@ploeh, we've made some experiments with this here. We used TypeShape, so it's not reflection-based. This is not part of the F# Hedgehog library, but we may release it separately on NuGet to get feedback.
(Development around this has been slow, as I'm more working with Haskell and C than .NET nowadays.)


Connect time out when install spark history server with s3 storage

Describe the bug
when I tried to install the the spark history server with s3 storage, the pod failed to start because of connect timed out
the following is the log from the pod
Is it related to the hadoop-aws version?
2019-11-12 19:59:52 INFO  HistoryServer:2566 - Started daemon with process name: 10@old-buffoon-spark-history-server-84664f8f79-pbtm8
2019-11-12 19:59:52 INFO  SignalUtils:54 - Registered signal handler for TERM
2019-11-12 19:59:52 INFO  SignalUtils:54 - Registered signal handler for HUP
2019-11-12 19:59:52 INFO  SignalUtils:54 - Registered signal handler for INT
2019-11-12 19:59:53 WARN  NativeCodeLoader:62 - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2019-11-12 19:59:53 INFO  SecurityManager:54 - Changing view acls to: root
2019-11-12 19:59:53 INFO  SecurityManager:54 - Changing modify acls to: root
2019-11-12 19:59:53 INFO  SecurityManager:54 - Changing view acls groups to:
2019-11-12 19:59:53 INFO  SecurityManager:54 - Changing modify acls groups to:
2019-11-12 19:59:53 INFO  SecurityManager:54 - SecurityManager: authentication disabled; ui acls disabled; users  with view permissions: Set(root); groups with view permissions: Set(); users  with modify permissions: Set(root); groups with modify permissions: Set()
2019-11-12 19:59:53 INFO  FsHistoryProvider:54 - History server ui acls disabled; users with admin permissions: ; groups with admin permissions
2019-11-12 20:00:01 INFO  AmazonHttpClient:448 - Unable to execute HTTP request: connect timed out
java.net.SocketTimeoutException: connect timed out
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:589)
	at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:542)
	at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:414)
	at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:326)
	at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:610)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:445)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:835)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
	at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:384)
	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:232)
	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3528)
	at com.amazonaws.services.s3.AmazonS3Client.headBucket(AmazonS3Client.java:1031)
	at com.amazonaws.services.s3.AmazonS3Client.doesBucketExist(AmazonS3Client.java:994)
	at org.apache.hadoop.fs.s3a.S3AFileSystem.initialize(S3AFileSystem.java:297)
	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2669)
	at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:94)
	at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2703)
	at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2685)
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:373)
	at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
	at org.apache.spark.deploy.history.FsHistoryProvider.<init>(FsHistoryProvider.scala:117)
	at org.apache.spark.deploy.history.FsHistoryProvider.<init>(FsHistoryProvider.scala:86)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at org.apache.spark.deploy.history.HistoryServer$.main(HistoryServer.scala:280)
	at org.apache.spark.deploy.history.HistoryServer.main(HistoryServer.scala)

Version of Helm and Kubernetes:
helm version:
Client: &version.Version{SemVer:"v2.15.2", GitCommit:"8dce272473e5f2a7bf58ce79bb5c3691db54c96b", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.15.2", GitCommit:"8dce272473e5f2a7bf58ce79bb5c3691db54c96b", GitTreeState:"clean"}

kubectl version (minikube with 8 cpus and 8096m Memeory):
Client Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.7", GitCommit:"8fca2ec50a6133511b771a11559e24191b1aa2b4", GitTreeState:"clean", BuildDate:"2019-09-18T14:47:22Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.7", GitCommit:"8fca2ec50a6133511b771a11559e24191b1aa2b4", GitTreeState:"clean", BuildDate:"2019-09-18T14:39:02Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}

Which chart:
stable/spark-history-server
What happened:
What you expected to happen:
How to reproduce it (as minimally and precisely as possible):

aws-access-key / aws-secret-key have already been set on default namespace.
helm package .
helm install .

values.yaml:
replicaCount: 1
nameOverride: ""
fullnameOverride: ""

rbac:
  create: true

serviceAccount:
  create: true
  name:

image:
  repository: lightbend/spark-history-server
  tag: 2.4.0
  pullPolicy: IfNotPresent

service:
  type: LoadBalancer
  port: 18080
  annotations: {}

environment:
# Note: do not configure Spark history events directory using SPARK_HISTORY_OPTS. It will be
# configured by this chart based on the values in "pvc", "gcs" or "hdfs" attribute.
  # SPARK_HISTORY_OPTS: ...
  # SPARK_DAEMON_MEMORY: 1g
  # SPARK_DAEMON_JAVA_OPTS: ...
  # SPARK_DAEMON_CLASSPATH: ...
  # SPARK_PUBLIC_DNS: ...

podAnnotations: {}

resources:
  # We usually recommend not to specify default resources and to leave this as a conscious
  # choice for the user. This also increases chances charts run on environments with little
  # resources, such as Minikube. If you do want to specify resources, uncomment the following
  # lines, adjust them as necessary, and remove the curly braces after 'resources:'.
  #
  # To let the application start up quickly give it a big limit
  limits:
    cpu: 1000m
    memory: 1Gi
  requests:
    cpu: 100m
    memory: 512Mi

ingress:
  enabled: false
  annotations: {}
  # kubernetes.io/ingress.class: nginx
  # kubernetes.io/tls-acme: "true"
  path: /
  hosts:
    - spark-history-server.example.com
  tls: []
  #  - secretName:spark-history-server.example.com
  #    hosts:
  #      - spark-history-server.example.com

pvc:
  # to use a file system path for Spark events dir, set 'enablePVC' to true and mention the
  # name of an already created persistent volume claim in existingClaimName.
  # The volume will be mounted on /data in the pod
  enablePVC: false
  existingClaimName: nfs-pvc
  eventsDir: "/"

# Settings for the sub-chart
# When pvc.enablePVC is true, make sure:
# pvc.existingClaimName == nfs.pvcName
nfs:
  enableExampleNFS: false
  pvName: nfs-pv
  pvcName: nfs-pvc

gcs:
  enableGCS: false
  secret: history-secrets
  key: sparkonk8s.json
  logDirectory: gs://spark-hs/

hdfs:
  hdfsSiteConfigMap: hdfs-site
  coreSiteConfigMap: core-site
  logDirectory: hdfs://hdfs/history/
  HADOOP_CONF_DIR: /etc/hadoop

s3:
  enableS3: true
  enableIAM: false
  # Omit for IAM role-based or provider-based authentication.
  secret: aws-secrets
  # accessKeyName is a is AWS access key ID. Omit for IAM role-based or provider-based authentication.
  accessKeyName: aws-access-key
  # secretKey is AWS secret key. Omit for IAM role-based or provider-based authentication.
  secretKeyName: aws-secret-key
  logDirectory: s3a://path/to/my/bucket/

wasbs:
  enableWASBS: false
  sasKeyMode: true
  secret: azure-secrets
  sasKeyName: azure-blob-sas-key
  storageAccountKeyName: azure-storage-account-key
  storageAccountNameKeyName: azure-storage-account-name
  containerKeyName: azure-blob-container-name
  logDirectory: wasbs:///spark-hs

imagePullSecrets: []

nodeSelector: {}

tolerations: []

affinity: {}

Anything else we need to know:

Thanks for reporting the issue. Unfortunately we haven't had bandwidth to maintain and update this project for a while.  I'm not sure if the client version is to blame. If you figure it out, feel free to make a PR and it will be much appreciated. Thanks!


Remove The Last Vestiges of isInvalid from Parse Harder This Time

Another go at #27820

@swift-ci please smoke test

And because I ain't about to make the same mistake twice
@swift-ci please test source compatibility

Release compatibility failure isn't mine.

@swift-ci please smoke test and merge

@swift-ci please smoke test and merge

@swift-ci please smoke test

@swift-ci please smoke test

@swift-ci please smoke test

@swift-ci please smoke test

'Long as we're waiting on the bots to change
@swift-ci please test source compatibility

@swift-ci please smoke test macOS platform

@swift-ci please smoke test macOS platform

⛵

⛵


spaCy >= 2.0 support

Hi Chris, thanks for the big 2.0 updates!
This is regarding the following section of the README

Note: ERRANT does not support spaCy 2 at this time. spaCy 2 POS tags are slightly different from spaCy 1 POS tags and so ERRANT rules, which were designed for spaCy 1, may not always work with spaCy 2.

Since Python can't handle having multiple versions of a given library in a single project, and we need to use features that were introduced post spacy 2.0, we currently have to keep ERRANT isolated in a separate service which we talk to over HTTP. This is not ideal. Since ERRANT now supports passing in an nlp spacy object, it seems like adding support for spacy >= 2.0 would not be bad.
Specifically, I think we could check nlp._meta['spacy_version']. If the spacy version is less than 2.0, nlp._meta doesn't exist, above 2.0, this gives us the exact spacy version. For this current purpose, just testing is_spacy_2_or_above = bool(getattr(nlp, "_meta", False)) should be enough. Then the quickest fix would be to just map the 2.0 tags to 1.9 tags if is_spacy_2_or_above.
Is this acceptable? If not, is there some other path to supporting spacy 2.0+? Thank you!
EDIT: we are happy to work on this, we'd just like to find an approach that you would approve.

Hey Sam,
Yes, Spacy 2 support is definitely on the to-do list. I mainly wanted the first pip version to be compatible with the BEA shared task, but newer versions will change the results slightly.
Some good news: Spacy finally updated their English tag map to the same one that I use, so as long as you use spacy >= 2.2.2, rule compatibility shouldn't be a problem. I'm in the process of testing ERRANT with this version of spacy too, so hopefully ERRANT 2.1 will come out soon!

Quick update:
I tried using ERRANT with the latest version of spacy (2.2), and the only thing that broke is a call to an old lemmatiser in the classifier. For a quick fix, you can change the same_lemma function to:
if o_tok.lemma == c_tok.lemma: return True
return False

Otherwise, it looked as if annotation performance decreased by about 1% and processing time took about 3 times longer. I'll need to debug the accuracy loss (and have some ideas already), but there's not really anything I can do about the speed loss...

Thanks for the update.
The performance thing is interesting, since spacy2.0 was supposed to be faster... In both cases, is this using en_core_web_sm?

Yes, that's using en_core_web_sm in both cases.
From what I've read, it was never supposed to be faster, just slightly more accurate and memory efficient. Check the "Model Comparison" table on this page. It shows a large speed drop for a relatively small performance gain. It's true that the parser and NER components are >5% more accurate, but ERRANT mainly relies on the POS tagger, so the ~0.5% POS improvement isn't really significant.
There's also a long issue thread about it here when it went from v1 to v2 , and it seemed to me that the conclusion was that it'll never be as fast as v1.


2 RM but 1 only one discovered

I need 2 Broadlink, because my AVR is hidden in a lowboard. So I have a Broadlink in the lowboard and the other so that he has the DVR in view.
But my Homebridge always only recognizes a Broadlink.
My Config:
{
   	"platform": "BroadlinkRM",
   	"name": "Broadlink",
   	"hideScanFrequencyButton": true,
   	"hideLearnButton": false,
   	"accessories": [
   		{
   		"name":"AVR Learn",
   		"type":"learn-code",
   		"host":"<IP_ADDRESS>",
   		"disableAutomaticOff": true,
   		"scanFrequency": false
   		},
   		{
   			"name": "Lauter",
   			"type": "switch-repeat",
   			"sendCount": 5,
   			"interval": 0.3,
   			"host": "<IP_ADDRESS>",
   			"data": "26004800000127941213111411131213111411131213123811131411121311141138123712381213113812381138121313121113123811381213121311141138123712381114111312000d05"
   		},
   		{
   			"name": "Leiser",
   			"type": "switch-repeat",
   			"sendCount": 5,
   			"interval": 0.3,
   			"host": "<IP_ADDRESS>",
   			"data": "26004800000129931311131213121312121213121312123713121312121312121337133613371311131213121312123713121213123713361436133614361312123713371311131213000d05"
   		},
   		{
   			"name": "Lautlos",
   			"type": "switch",
   			"host": "<IP_ADDRESS>",
   			"data": {
   				"on": "260050000001289414111213131213121313131213131336121312131411131213371435153514111337131312121412131212131336153513121337133614361237143612131411130005bc0001284a12000d050000000000000000",
   				"off": "260050000001289414111213131213121313131213131336121312131411131213371435153514111337131312121412131212131336153513121337133614361237143612131411130005bc0001284a12000d050000000000000000"
   			}
   		},
   		{
   		"name":"DVR Learn",
   		"type":"learn-code",
   		"host":"<IP_ADDRESS>",
   		"disableAutomaticOff": true,
   		"scanFrequency": false
   		},
   		{
   			"name": "Sender vor",
   			"type": "switch-repeat",
   			"sendCount": 1,
   			"host": "<IP_ADDRESS>",
   			"data": "2600b4000e091513050f0515050f051a050a0509050a0514060f0514050a0514050a050905000b870e091414050f0515050f051a0509050a05090515050f0515050905150509050a05000b860f08050a0514060f0514050f051a050a0509050a0515050f0514050a05140609050a05000b860e0905090515080c0515050f051a0509050a050a0415050f051505090515050a040a05000b870e081514070d0514060e051a0707070808120515050f05141514051805000d0500000000"
   		},
   		{
   			"name": "Sender zurück",
   			"type": "switch-repeat",
   			"sendCount": 1,
   			"host": "<IP_ADDRESS>",
   			"data": "260092000e091414050f0515050f051a050a0509050a0514050f0515050a0514050a050f05000b7e0e0905090614050f0515050f051a05090609050a0514050f0515050a0415050a050f05000b7e0e091414050f0515050f051a0509050a050a0415050f051505090515050a041005000b7e0e081514050f0514060e06190609050a05140515050f0515050905150509050f05000d05000000000000"
   		}
   	]
   }

Homebridge discovered only 1 Broadlink:
Discovered Broadlink RM device at <IP_ADDRESS> (34:ea:34:43:a5:d8)

Odd that the second one isn't discovered automatically.
Specifying a "host" by-passes the discovery though. If the switches don't work on the <IP_ADDRESS> host then it sounds like there's a connectivity issue - can you ping <IP_ADDRESS> ?

I have the same problem.
2 RM mini
ping ok
random discovered only 1
v2.6.3
node /node_modules/homebridge-broadlink-rm/helpers/getDevice.js
Discovered Broadlink RM device at <IP_ADDRESS> (34:ea:34:42:bb:bb)
^C
node /node_modules/homebridge-broadlink-rm/helpers/getDevice.js
Discovered Broadlink RM device at <IP_ADDRESS> (34:ea:34:42:b5:e7)
^C
Thanks

I have the same problem.
Although python-broadlink discovers all RMs in a local network,  getDevice.js does not.
I hope you will find the function discover of python-broadlink helpful.
I really appreciate all your work on homebridge-broadlink-rm.

Yes, I can ping my RM Minis.
On my router can I See the RM Minis, too.

Solved!  Please check a PR: https://github.com/lprhodes/broadlinkjs-rm/pull/6 .

Fixed in 2.6.4 - due out in the next hour or so

Thx now works fine

In case you have further issues there's a new method of adding devices manually to your config in case auto-discovery isn't working for whatever reason.
Check out the docs on it.


Redis support as a datastore

Hey! It would be a good idea to add the support capabilities of redis as a database. Redis is much faster than MySQL etc. and I use it in my applications.

Redis is not designed for (and in my opinion should not be used for) persistent data storage.
Very unlikely to ever be implemented as a storage provider in LuckPerms for this reason - sorry! :/


Add sink operator

fixes #376

Coverage decreased (-0.02%) when pulling e71a807060e5f5a00547a2397d5cd572e294722b on shumo_add_sink_operator into f17f9f6a724b9cac500aa43089afa80c43453878 on master.

bump this PR :)

Last I checked, I still did not like the way this was tested. I'll look again.


Feature option - Calendar weeks

It will be great if there will be option to add on the left side of calendar number of year week.

I even have tested and worked exmaple, but just have for now no idea how to implement it:
// This script is released to the public domain and may be used, modified and
// distributed without restrictions. Attribution not necessary but appreciated.
// Source: http://weeknumber.net/how-to/javascript

// Returns the ISO week of the date.
Date.prototype.getWeek = function() {
  var date = new Date(this.getTime());
   date.setHours(0, 0, 0, 0);
  // Thursday in current week decides the year.
  date.setDate(date.getDate() + 3 - (date.getDay() + 6) % 7);
  // January 4 is always in week 1.
  var week1 = new Date(date.getFullYear(), 0, 4);
  // Adjust to Thursday in week 1 and count number of weeks from date to week1.
  return 1 + Math.round(((date.getTime() - week1.getTime()) / 86400000
                        - 3 + (week1.getDay() + 6) % 7) / 7);
}


[Bug]: withFileSystemBind does not mount any files

Module
Core
Testcontainers version
1.17.6
Using the latest Testcontainers version?

Steps to Reproduce:

Crate a new Container
Add a withFileSystemBind of any folder
Start the container

Expected:

Folder is successfully bound to container and files are available

Actual:

The folder was not mounted.

Additional Information:
The argument "--pre-archive" needs to be set for kubedock. Otherwise no copy/mount did work at all.
We tried mounting and copying files to the container with all available possibilities.
None of them is working on it's own. Meaning if you use any of the following methods
no file is ever mounted or copied.
.withFileSystemBind(...)
.withCopyToContainer(...)
.withCopyFileToContainer(...)
.withClasspathResourceMapping(...)
However when combining multiple of these Methods they partly work.
Whenever the method ".withFileSystemBind(...)" is present for the container, all
other copies work as expected. But the filesystembind itself does not. In this case
there is a config map as well as the mounted files present in the container.
Following shows the describe for the started alpinecontainer.
With all 4 methods called:
	Mounts:
		/config2 from pfiles (rw,path="fbc1ac88b427356b61892951faa93e54")
		/config3/redis.conf from pfiles (rw,path="c7f85a5709e2ec3a35488d25ef0bdde7")
		/config4/redis.conf from pfiles (rw,path="377f7586930fe9b2b041e64a12d5e76e")
		/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-r4f4t (ro)
	Volumes:
		pfiles:
			Type: ConfigMap (a volume populated by a ConfigMap)
			Name: e363d58d2cf8
			Optional:  false

The same output without the .withFileSystemBind BUT still we the other 3 copies (no volume present afterwards):
	Mounts:
		/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-jn4c8 (ro)

Relevant log output
No response
Additional Information
@Test
@SuppressWarnings("unchecked")
void shouldBindDirectory() {
    String configPath = Paths.get("").toAbsolutePath() + "/build/it-config/data";
    File file = new File(configPath + "/redis.conf");
    GenericContainer test = new GenericContainer(DockerImageName.parse("alpine:3.15"))
        .withLogConsumer(new Slf4jLogConsumer(getLogger(this.getClass().getName())))
        .withFileSystemBind(configPath, "/config", BindMode.READ_ONLY)
        .withCopyToContainer(Transferable.of(file.getAbsolutePath()), "/config2/redis.conf")
        .withCopyFileToContainer(MountableFile.forHostPath(file.getAbsolutePath()), "/config3/redis.conf")
        .withClasspathResourceMapping("redis.conf", "/config4/redis.conf", BindMode.READ_ONLY)
        .withCommand("sh", "-c", "sleep 5 && echo started && tail -f /dev/null")
        .waitingFor(Wait.forLogMessage(".*started.*", 1).withStartupTimeout(Durations.ONE_MINUTE));
    test.start();
    try {
        String basePath = test.execInContainer("ls", "/").getStdout();
        log.info(basePath);
        String foo1 = test.execInContainer("ls", "/config").getStdout();
        String foo2 = test.execInContainer("ls", "/config2/").getStdout();
        String foo3 = test.execInContainer("ls", "/config3").getStdout();
        String foo4 = test.execInContainer("ls", "/config4").getStdout
        log.info("TEST withFileSystemBind: " + foo1);
        log.info("TEST withCopyToContainer: " + foo2);
        log.info("TEST withCopyFileToContainer: " + foo3);
        log.info("TEST withClasspathResourceMapping: " + foo4);
    } catch (IOException | InterruptedException e) {
        throw new RuntimeException(e);
    }
}

created in the wrong repo


Query - Does the /workday API support a comma separated list of countries?

In the developer docs, it states that /holidays supports up to 10 countries in a comma-separated list.
Does the /workday API also support multiple countries? If not, can it?
Thanks.

Hey @recipher
Currently only our /v1/holidays endpoint supports multiple countries. Hasn't actually come up for our other endpoints so it hasn't been something we've considered at all.
Since /v1/workday doesn't return an array of data currently, we'd have to redo the payload for the endpoint, which wouldn't happen until we version from v1 to v2 of the API (which is probably a ways off).
I'm curious about what the use case would be for needing to calculate the next business day for a bulk of countries at one time. Feel free to drop us a line at<EMAIL_ADDRESS>


dark mode error bar

error bar in dark mode should be "white" or lightgray?
the standard color is "invisible"

You mean this error bar? I think this looks ok, does it look different for you or do you not like it?

Yes I don't like it. Tthe background of the bar is too dark.

default is fine for dark-mode or even darker than that?

I've adjusted the color... is this ok?


cannot copy ANYTHING from VS Code Stable and paste into VS Code Insider

I cannot copy ANYTHING from VS Code Stable and paste into VS Code Insider.
So what I am having to do is first copy from VSCode Stable to any other Text Editor, then copy from there, and paste it into VSCode Insider
I have tried opening inside with code-insiders --disable-extensions - BUT that did not help
Also manually deleted C:\Users\user_name\AppData\Roaming\Code - Insiders\Local Storage - Did not help though.
Stable Version
Version: 1.78.2 (user setup)
Commit: b3e4e68a0bc097f0ae7907b217c1119af9e03435
Date: 2023-05-10T14:39:26.248Z
Electron: 22.5.2
Chromium: 108.0.5359.215
Node.js: 16.17.1
V8: <IP_ADDRESS>-electron.0
OS: Windows_NT x64 10.0.22621
Sandboxed: No

Insider Version
Version: 1.79.0-insider (user setup)
Commit: f1258a5fbed44b8f84a097c73c1799c13c5c3646
Date: 2023-05-17T05:27:38.052Z
Electron: 22.5.2
Chromium: 108.0.5359.215
Node.js: 16.17.1
V8: <IP_ADDRESS>-electron.0
OS: Windows_NT x64 10.0.22621

has this always been the case? I'm confident this is an issue with your setup as I cannot reproduce.

has this always been the case? I'm confident this is an issue with your setup as I cannot reproduce. maybe electron?

It has always worked fine, this problem started pretty recently. Today I completely uninstalled Insiders and re-installed it. But the problem remains.

I can repro for any type of file in stable to anything in insiders
with keyboard shortcut troubleshooting:
2023-05-19 09:26:06.649 [info] [KeybindingService]: + Invoking command editor.action.clipboardPasteAction.
2023-05-19 09:26:06.665 [error] Cannot read properties of undefined (reading 'pasteOnNewLine'): TypeError: Cannot read properties of undefined (reading 'pasteOnNewLine')
    at al.F (vscode-file://vscode-app/c:/Users/aamunger/AppData/Local/Programs/Microsoft%20VS%20Code%20Insiders/resources/app/out/vs/workbench/workbench.desktop.main.js:750:36052)
    at async vscode-file://vscode-app/c:/Users/aamunger/AppData/Local/Programs/Microsoft%20VS%20Code%20Insiders/resources/app/out/vs/workbench/workbench.desktop.main.js:750:33983
2023-05-19 09:26:06.843 [info] [KeybindingService]: + Ignoring single modifier ctrl due to it being pressed together with other keys.

@amunger Can you test this again? I think it may have already been fixed but am not sure I am following the correct steps

can't repro anymore

Thanks for confirming. Closing as fixed


School Backpack - Check what happens when used with Vex's Artificer

Ok, School Backpack ends doing nothing since the Artificer does not have a deck.


Certificate-Key Match Implementation in FileWatcherCertificateProvider

@ZhenLian @ashithasantosh

Closed in favor of another Pull Request: https://github.com/grpc/grpc/pull/26709


Some issues

Make Server update game logic 50 times per second (instead of 25):
I dont think it can affect game physics much but we need to try (it will still send updates at 25pps)
Food Mass Grow: in official servers food is growing slow, MO-E spawns food at random Radius between 10-20
Make better algorithm for getting safe spawn position: Sometimes i cant spawn even if there is free space in game server

I can make first 2 points myself, but i cant guarantee that my code would be readable, lol

Make a PR about the 2 points and people will try to make it better I guess.

The second option has been added to many other repos and it's been known to cause lag, I don't know if it can be done in Node.js without impacting performance eventually. Other 2 can be tried out :)

The three option can't be possible.
Can be, but, if the random position function get modify and update to any place do not let the players, are going to get possible bugs or lags in the spawn, bugs, if the system dont search any safe position.... what happend?, and, if the map are small because gamemode, etc, where is going to indicate that spawn?.
One Better suggestion are:
Add a phantom fuction in the physics, to: if one player spawn, get invincible for 3 seconds, 3 seconds to split/get in safe position, but this too can get angry to player's, if get invincible, and other player are in front of him, can follow and if he split, get in and the player can made better random spawn.
Other better suggestion are:
Same invincible spawn with 3 seconds, but he can collapse with others players.

I know this is a very old issue, but it is possible @pabloalber84, it's already happened

@NuclearC I've taken a look at the things you suggested and i have concluded that:

Changing the update interval to the one you suggested will result in a severe performance drop (high drop in fps). Plus vanilla
Making the algorithm more lenient will frequently result in it being ineffective and most likely inefficient.

So unless you implement said features yourself in a way that doesn't affect the performance negatively, they are pretty impractical.

@B0RYS this was 2 years ago, come on

Someone had to do it, sorry @NuclearC.


RuntimeError: CUDA error: device-side assert triggered. indexSelectLargeIndex: block: [58,0,0], thread: [24,0,0] Assertion srcIndex < srcSelectDimSize failed.

Traceback (most recent call last):
File "/home/ec2-user/SageMaker/MedicalGPT/supervised_finetuning.py", line 1346, in
main()
File "/home/ec2-user/SageMaker/MedicalGPT/supervised_finetuning.py", line 1307, in main
train_result = trainer.train(resume_from_checkpoint=checkpoint)
File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/transformers/trainer.py", line 1555, in train
return inner_training_loop(
File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/transformers/trainer.py", line 1860, in _inner_training_loop
tr_loss_step = self.training_step(model, inputs)
File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/transformers/trainer.py", line 2725, in training_step
loss = self.compute_loss(model, inputs)
File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/transformers/trainer.py", line 2748, in compute_loss
outputs = model(**inputs)
File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/accelerate/utils/operations.py", line 659, in forward
return model_forward(*args, **kwargs)
File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/accelerate/utils/operations.py", line 647, in call
return convert_to_fp32(self.model_forward(*args, **kwargs))
File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/torch/amp/autocast_mode.py", line 14, in decorate_autocast
return func(*args, **kwargs)
File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/peft/peft_model.py", line 1003, in forward
return self.base_model(
File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/peft/tuners/tuners_utils.py", line 107, in forward
return self.model.forward(*args, **kwargs)
File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 1034, in forward
outputs = self.model(
File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 886, in forward
attention_mask = _prepare_4d_causal_attention_mask(
File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/transformers/modeling_attn_mask_utils.py", line 193, in _prepare_4d_causal_attention_mask
attention_mask = attn_mask_converter.to_4d(
File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/transformers/modeling_attn_mask_utils.py", line 101, in to_4d
causal_4d_mask = self._make_causal_mask(
File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/transformers/modeling_attn_mask_utils.py", line 131, in _make_causal_mask
mask = torch.full((tgt_len, tgt_len), torch.finfo(dtype).min, device=device)
RuntimeError: CUDA error: device-side assert triggered
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.
0%|                                                   | 0/993 [00:03<?, ?it/s]

补充说明：我增量训练了一个在llama 2 7b上使用医学数据预训练过的模型，meditron 7b （https://github.com/epfLLM/meditron）。在这个过程中增加了几个新的token，在added_tokens.json里。继续在这个基础上做sft的时候，就会报上面的错误 /opt/conda/conda-bld/pytorch_1686274778240/work/aten/src/ATen/native/cuda/Indexing.cu:1146: indexSelectLargeIndex: block: [58,0,0], thread: [21,0,0] Assertion srcIndex < srcSelectDimSize failed. 请问这个还是因为第一步pre-train的时候tokenizer有问题吗？由于是在调试，我使用的都是data文件夹里默认的数据做的训练。

谢谢。是要加这句参数吗 --modules_to_save embed_tokens,lm_head
pretraining.py:
modules_to_save = script_args.modules_to_save
if modules_to_save is not None:
modules_to_save = modules_to_save.split(',')
# Resize the embedding layer to match the new tokenizer
embedding_size = model.get_input_embeddings().weight.shape[0]
if len(tokenizer) > embedding_size:
model.resize_token_embeddings(len(tokenizer))

2023-11-30 04:40:23.107 | INFO     | main:main:377 - Model args: ModelArguments(model_type='llama', model_name_or_path='epfl-llm/meditron-7b', tokenizer_name_or_path=None, load_in_8bit=False, load_in_4bit=False, cache_dir=None, use_fast_tokenizer=False, torch_dtype='float16', device_map='auto', trust_remote_code=True, HUB_TOKEN='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')
2023-11-30 04:40:23.107 | INFO     | main:main:378 - Data args: DataArguments(dataset_name=None, dataset_config_name=None, train_file_dir='./data/pretrain', validation_file_dir='./data/pretrain', max_train_samples=10000, max_eval_samples=10, streaming=False, block_size=1024, overwrite_cache=False, validation_split_percentage=1, preprocessing_num_workers=1, keep_linebreaks=True)
2023-11-30 04:40:23.108 | INFO     | main:main:379 - Training args: Seq2SeqTrainingArguments(
n_gpu=1,
adafactor=False,
adam_beta1=0.9,
adam_beta2=0.999,
adam_epsilon=1e-08,
auto_find_batch_size=False,
bf16=False,
bf16_full_eval=False,
data_seed=None,
dataloader_drop_last=False,
dataloader_num_workers=0,
dataloader_pin_memory=True,
ddp_backend=None,
ddp_broadcast_buffers=None,
ddp_bucket_cap_mb=None,
ddp_find_unused_parameters=False,
ddp_timeout=30000,
debug=[],
deepspeed=None,
disable_tqdm=False,
dispatch_batches=None,
do_eval=True,
do_predict=False,
do_train=True,
eval_accumulation_steps=None,
eval_delay=0,
eval_steps=50,
evaluation_strategy=steps,
fp16=True,
fp16_backend=auto,
fp16_full_eval=False,
fp16_opt_level=O1,
fsdp=[],
fsdp_config={'min_num_params': 0, 'xla': False, 'xla_fsdp_grad_ckpt': False},
fsdp_min_num_params=0,
fsdp_transformer_layer_cls_to_wrap=None,
full_determinism=False,
generation_config=None,
generation_max_length=None,
generation_num_beams=None,
gradient_accumulation_steps=1,
gradient_checkpointing=True,
gradient_checkpointing_kwargs=None,
greater_is_better=None,
group_by_length=False,
half_precision_backend=auto,
hub_always_push=False,
hub_model_id=None,
hub_private_repo=False,
hub_strategy=every_save,
hub_token=<HUB_TOKEN>,
ignore_data_skip=False,
include_inputs_for_metrics=False,
include_tokens_per_second=False,
jit_mode_eval=False,
label_names=None,
label_smoothing_factor=0.0,
learning_rate=0.0002,
length_column_name=length,
load_best_model_at_end=False,
local_rank=0,
log_level=passive,
log_level_replica=warning,
log_on_each_node=True,
logging_dir=outputs-pt-v1/runs/Nov30_04-40-23_ip-172-16-91-101.us-east-2.compute.internal,
logging_first_step=True,
logging_nan_inf_filter=True,
logging_steps=10,
logging_strategy=steps,
lr_scheduler_type=linear,
max_grad_norm=1.0,
max_steps=-1,
metric_for_best_model=None,
mp_parameters=,
neftune_noise_alpha=None,
no_cuda=False,
num_train_epochs=1.0,
optim=adamw_torch,
optim_args=None,
output_dir=outputs-pt-v1,
overwrite_output_dir=True,
past_index=-1,
per_device_eval_batch_size=1,
per_device_train_batch_size=1,
predict_with_generate=False,
prediction_loss_only=False,
push_to_hub=False,
push_to_hub_model_id=None,
push_to_hub_organization=None,
push_to_hub_token=<PUSH_TO_HUB_TOKEN>,
ray_scope=last,
remove_unused_columns=True,
report_to=['tensorboard'],
resume_from_checkpoint=None,
run_name=outputs-pt-v1,
save_on_each_node=False,
save_safetensors=True,
save_steps=500,
save_strategy=steps,
save_total_limit=3,
seed=42,
skip_memory_metrics=True,
sortish_sampler=False,
split_batches=False,
tf32=None,
torch_compile=False,
torch_compile_backend=None,
torch_compile_mode=None,
torchdynamo=None,
tpu_metrics_debug=False,
tpu_num_cores=None,
use_cpu=False,
use_ipex=False,
use_legacy_prediction_loop=False,
use_mps_device=False,
warmup_ratio=0.05,
warmup_steps=0,
weight_decay=0.01,
)
2023-11-30 04:40:23.108 | INFO     | main:main:380 - Script args: ScriptArguments(use_peft=True, target_modules='all', lora_rank=8, lora_dropout=0.05, lora_alpha=16.0, modules_to_save='embed_tokens,lm_head', peft_path=None, qlora=False)
2023-11-30 04:40:23.108 | INFO     | main:main:381 - Process rank: 0, device: cuda:0, n_gpu: 1 distributed training: True, 16-bits training: True
2023-11-30 04:40:23.289 | INFO     | main:main:479 - train files: ['./data/pretrain/en_article_tail500.txt', './data/pretrain/tianlongbabu.txt', './data/pretrain/fever.txt']
2023-11-30 04:40:23.289 | INFO     | main:main:489 - eval files: ['./data/pretrain/en_article_tail500.txt', './data/pretrain/tianlongbabu.txt', './data/pretrain/fever.txt']
2023-11-30 04:40:23.399 | INFO     | main:main:521 - Raw datasets: DatasetDict({
train: Dataset({
features: ['text'],
num_rows: 3876
})
validation: Dataset({
features: ['text'],
num_rows: 3876
})
})
2023-11-30 04:40:23.424 | DEBUG    | main:main:567 - Num train_samples: 656
2023-11-30 04:40:23.424 | DEBUG    | main:main:568 - Tokenized training example:
2023-11-30 04:40:23.451 | DEBUG    | main:main:569 -  contract to work in specified mines and mills. There seemed to be no
 limit to the factories, forges, refineries, and railways that could be
 built, to the multitudes that could be employed in conquering a
 continent. As for the future, that was in the hands of Providence!

 =Business Theories of Politics.=--As the statesmen of Hamilton's school
 and the planters of Calhoun's had their theories of government and
 politics, so the leaders in business enterprise had theirs. It was
 simple and easily stated. "It is the duty of the government," they
 urged, "to protect American industry against foreign competition by
 means of high tariffs on imported goods, to aid railways by generous
 grants of land, to sell mineral and timber lands at low prices to
 energetic men ready to develop them, and then to leave the rest to the
 initiative and drive of individuals and companies." All government
 interference with the management, prices, rates, charges, and conduct of
 private business they held to be either wholly pernicious or intolerably
 impertinent. Judging from their speeches and writings, they conceived
 the nation as a great collection of individuals, companies, and labor
 unions all struggling for profits or high wages and held together by a
 government whose principal duty was to keep the peace among them and
 protect industry against the foreign manufacturer. Such was the
 political theory of business during the generation that followed the
 Civil War.

 THE SUPREMACY OF THE REPUBLICAN PARTY (1861-85)

 =Business Men and Republican Policies.=--Most of the leaders in industry
 gravitated to the Republican ranks. They worked in the North and the
 Republican party was essentially Northern. It was moreover--at least so
 far as the majority of its members were concerned--committed to
 protective tariffs, a sound monetary and banking system, the promotion
 of railways and industry by land grants, and the development of internal
 improvements. It was furthermore generous in its immigration policy. It
 proclaimed America to be an asylum for the oppressed of all countries
 and flung wide the doors for immigrants eager to fill the factories, man
 the mines, and settle upon Western lands. In a word the Republicans
 stood for all those specific measures which favored the enlargement and
 prosperity of business. At the same time they resisted government
 interference with private enterprise. They did not regulate railway
 rates, prosecute trusts for forming combinations, or prevent railway
 companies from giving lower rates to some shippers than to others. To
 sum it up, the political theories of the Republican party for three
 decades after the Civil War were the theories of American
 business--prosperous and profitable industries for the owners and "the
 full dinner pail" for the workmen. Naturally a large portion of those
 who flourished under its policies gave their support to it, voted for
 its candidates, and subscribed to its campaign funds.

 =Sources of Republican Strength in the North.=--The Republican party was
 in fact a political organization of singular power. It originated in a
 wave of moral enthusiasm, having attracted to itself, if not the
 abolitionists, certainly all those idealists, like James Russell Lowell
 and George William Curtis, who had opposed slavery when opposition was
 neither safe nor popular. To moral principles it added practical
 considerations. Business men had confidence in it. Workingmen, who
 longed for the independence of the farmer, owed to its indulgent land
 policy the opportunity of securing free homesteads in the West. The
 immigrant, landing penniless on these shores, as a result of the same
 beneficent system, often found himself in a little while with an estate
 as large as many a baronial domain in the Old World. Under a Republican
 administration, the union had been saved. To it the veterans of the war
 could turn with confidence for those rewards of service which the
 government could bestow: pensions surpassing in liberality anything that
the world had ever seen. Under a Republican administration
2023-11-30 04:40:23.453 | DEBUG    | main:main:581 - Num eval_samples: 10
2023-11-30 04:40:23.453 | DEBUG    | main:main:582 - Tokenized eval example:
2023-11-30 04:40:23.478 | DEBUG    | main:main:583 -  contract to work in specified mines and mills. There seemed to be no
 limit to the factories, forges, refineries, and railways that could be
 built, to the multitudes that could be employed in conquering a
 continent. As for the future, that was in the hands of Providence!

 =Business Theories of Politics.=--As the statesmen of Hamilton's school
 and the planters of Calhoun's had their theories of government and
 politics, so the leaders in business enterprise had theirs. It was
 simple and easily stated. "It is the duty of the government," they
 urged, "to protect American industry against foreign competition by
 means of high tariffs on imported goods, to aid railways by generous
 grants of land, to sell mineral and timber lands at low prices to
 energetic men ready to develop them, and then to leave the rest to the
 initiative and drive of individuals and companies." All government
 interference with the management, prices, rates, charges, and conduct of
 private business they held to be either wholly pernicious or intolerably
 impertinent. Judging from their speeches and writings, they conceived
 the nation as a great collection of individuals, companies, and labor
 unions all struggling for profits or high wages and held together by a
 government whose principal duty was to keep the peace among them and
 protect industry against the foreign manufacturer. Such was the
 political theory of business during the generation that followed the
 Civil War.

 THE SUPREMACY OF THE REPUBLICAN PARTY (1861-85)

 =Business Men and Republican Policies.=--Most of the leaders in industry
 gravitated to the Republican ranks. They worked in the North and the
 Republican party was essentially Northern. It was moreover--at least so
 far as the majority of its members were concerned--committed to
 protective tariffs, a sound monetary and banking system, the promotion
 of railways and industry by land grants, and the development of internal
 improvements. It was furthermore generous in its immigration policy. It
 proclaimed America to be an asylum for the oppressed of all countries
 and flung wide the doors for immigrants eager to fill the factories, man
 the mines, and settle upon Western lands. In a word the Republicans
 stood for all those specific measures which favored the enlargement and
 prosperity of business. At the same time they resisted government
 interference with private enterprise. They did not regulate railway
 rates, prosecute trusts for forming combinations, or prevent railway
 companies from giving lower rates to some shippers than to others. To
 sum it up, the political theories of the Republican party for three
 decades after the Civil War were the theories of American
 business--prosperous and profitable industries for the owners and "the
 full dinner pail" for the workmen. Naturally a large portion of those
 who flourished under its policies gave their support to it, voted for
 its candidates, and subscribed to its campaign funds.

 =Sources of Republican Strength in the North.=--The Republican party was
 in fact a political organization of singular power. It originated in a
 wave of moral enthusiasm, having attracted to itself, if not the
 abolitionists, certainly all those idealists, like James Russell Lowell
 and George William Curtis, who had opposed slavery when opposition was
 neither safe nor popular. To moral principles it added practical
 considerations. Business men had confidence in it. Workingmen, who
 longed for the independence of the farmer, owed to its indulgent land
 policy the opportunity of securing free homesteads in the West. The
 immigrant, landing penniless on these shores, as a result of the same
 beneficent system, often found himself in a little while with an estate
 as large as many a baronial domain in the Old World. Under a Republican
 administration, the union had been saved. To it the veterans of the war
 could turn with confidence for those rewards of service which the
 government could bestow: pensions surpassing in liberality anything that
the world had ever seen. Under a Republican administration
The argument trust_remote_code is to be used with Auto classes. It has no effect here and is ignored.
Loading checkpoint shards: 100%|██████████████████| 8/8 [00:19<00:00,  2.43s/it]
/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/transformers/utils/hub.py:374: FutureWarning: The use_auth_token argument is deprecated and will be removed in v5 of Transformers. Please use token instead.
warnings.warn(
2023-11-30 04:40:43.546 | INFO     | main:main:636 - Fine-tuning method: LoRA(PEFT)
2023-11-30 04:40:43.546 | INFO     | main:main:641 - Init new peft model
2023-11-30 04:40:43.651 | INFO     | main:main:654 - Peft target_modules: ['down_proj', 'gate_proj', 'k_proj', 'o_proj', 'q_proj', 'up_proj', 'v_proj']
2023-11-30 04:40:43.651 | INFO     | main:main:655 - Peft lora_rank: 8
trainable params: 282,288,128 || all params: 7,020,859,392 || trainable%: 4.020706187645012
2023-11-30 04:40:44.321 | INFO     | main:main:698 - *** Train ***
2023-11-30 04:40:44.335 | DEBUG    | main:main:699 - Train dataloader example: {'input_ids': tensor([[30413, 31347,   231,  ..., 30267,   232,   151]], device='cuda:0'), 'attention_mask': tensor([[1, 1, 1,  ..., 1, 1, 1]], device='cuda:0'), 'labels': tensor([[30413, 31347,   231,  ..., 30267,   232,   151]], device='cuda:0')}
0%|                                                   | 0/656 [00:00<?, ?it/s]Traceback (most recent call last):
File "/home/ec2-user/SageMaker/MedicalGPT/pretraining.py", line 742, in
main()
File "/home/ec2-user/SageMaker/MedicalGPT/pretraining.py", line 703, in main
train_result = trainer.train(resume_from_checkpoint=checkpoint)
File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/transformers/trainer.py", line 1555, in train
return inner_training_loop(
File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/transformers/trainer.py", line 1904, in inner_training_loop
self.accelerator.clip_grad_norm(
File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/accelerate/accelerator.py", line 2124, in clip_grad_norm
self.unscale_gradients()
File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/accelerate/accelerator.py", line 2087, in unscale_gradients
self.scaler.unscale_(opt)
File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/torch/cuda/amp/grad_scaler.py", line 284, in unscale_
optimizer_state["found_inf_per_device"] = self.unscale_grads(optimizer, inv_scale, found_inf, False)
File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/torch/cuda/amp/grad_scaler.py", line 212, in unscale_grads
raise ValueError("Attempting to unscale FP16 gradients.")
ValueError: Attempting to unscale FP16 gradients.
0%|                                                   | 0/656 [00:03<?, ?it/s]
加这句参数 --modules_to_save embed_tokens,lm_head 之后会报错

0%|                                                   | 0/656 [00:00<?, ?it/s]Traceback (most recent call last):
File "/home/ec2-user/SageMaker/MedicalGPT/pretraining.py", line 742, in
main()
File "/home/ec2-user/SageMaker/MedicalGPT/pretraining.py", line 703, in main
train_result = trainer.train(resume_from_checkpoint=checkpoint)
File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/transformers/trainer.py", line 1555, in train
return inner_training_loop(
File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/transformers/trainer.py", line 1904, in inner_training_loop
self.accelerator.clip_grad_norm(
File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/accelerate/accelerator.py", line 2124, in clip_grad_norm_
self.unscale_gradients()
File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/accelerate/accelerator.py", line 2087, in unscale_gradients
self.scaler.unscale_(opt)
File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/torch/cuda/amp/grad_scaler.py", line 284, in unscale_
optimizer_state["found_inf_per_device"] = self.unscale_grads(optimizer, inv_scale, found_inf, False)
File "/home/ec2-user/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/torch/cuda/amp/grad_scaler.py", line 212, in unscale_grads
raise ValueError("Attempting to unscale FP16 gradients.")
ValueError: Attempting to unscale FP16 gradients.
加这句参数 --modules_to_save embed_tokens,lm_head 会报错


weird track data/bios to look at

Magazine - Shot by Both Sides:
Shot by Both Sides by Magazine. This song was on the album Only After Dark: Nick Rhodes & John Taylor Present... It was published by EMI. The date is listed as 2006-05-08, but that might be the date of the media.

Scritti Politti - Wood Beez
Wood Beez by Scritti Politti. It is track #08 . This song was on the album Cupid & Psyche 85 (2022 Remaster) . on disc #1 . The album artist is listed as Scritti Politti. It is classified as Alternative by some people. The BPM is approximately 113.24. Performed in the key of E. It was published by Rough Trade. The date is listed as 10-06-1985, but that might be the date of the media.
.. but no bio!

Oliver Heldens feat Devin & Nile Rogers - Summer Lover  - no bio

Run DMC - Hit It Run ... data exists for Run-D.M.C. and "RUN DMC" is listed as an alias. So something went wrong with alt name matching.

The Wrens - Everyone Chooses Sides  -- no info

Kazaky feat Gaspar - Push -- no info

Tiga feat Jake Shears - You Gonna Want Me


Text needing more clarification

https://github.com/coderefinery/git-intro/blob/gh-pages/_episodes/06-branches.md
We have three commits (we use the first two characters of the commits) and only one development line (branch) and this branch is called master.  May be "we use the first two characters of the commits " referring to the labels in the diagram

That sentence seems to be a leftover. It sounds to me like at some point there was an associated graph.


Add a super quick indeterminite progress meter to noms-merge

This breaks tests because it puts stuff on stdout when the test isn't expecting :-/
Since this isn't really a permanent thing, I'm fine with any number of kinda-hacky ways to silence the output during testing. You could make the updateProgress closure a thing that's passed in, or have merge.ThreeWay() take some kind of progress channel...whatever you want.

Shoot, forgot to run tests. I'll add a --silent flag.

ptal

lgtm with nit


How to Extract maps which can be used in UE?

Great job. I have seen that it is possible to export mesh, but I would like to know how to export texture maps so that they can be used in UE rendering engines effectively. Thanks a lot.

Thank you for your interest in this work!
To answer your question, I have not yet implemented this feature now.
It would require querying the BRDF and normals for each mesh vertex in 3D space, which involves first obtaining the intrinsic feature of the mesh vertex and then querying the corresponding MLPs. After obtaining the BRDF and normals, they would need to be mapped into a material textures map and normal map using some graphics algorithm.
If you tried it by yourself, please feel free to tell me if you have further questions.

It would be helpful to implement texture export function.


Fixes for upcoming pcs and pacemaker release

[citest]
ok - are these settings backwards compatible with older versions of pacemaker?

Yes, the changes are backwards compatible. New pcs packages impose stricter validation on user input to prevent certain cluster resources misconfiguration. The role tests happened to be creating resources in the way which is now prevented by pcs. I fixed the tests so that they pass with both new and old pcs.

@tomjelinek looks like something is up with github actions - can you make the following changes in this PR to .github/workflows/tox.yml:
-         uses: actions/checkout@v2
+        uses: actions/checkout@v3

and
-         uses: actions/setup-python@v2
+        uses: actions/setup-python@v4

and see if that helps?  @spetrosi fyi

@tomjelinek thanks - but that didn't help :-( - @spetrosi can you investigate this?

[citest]

STDOUT:

Warning: Agent 'pacemaker-fenced' is not installed or does not provide valid metadata: Element content failed to validate content, line 30

STDERR:

Error: invalid stonith option 'pcmk_host_list', allowed options are: 'action', 'family', 'ipport', 'nodename', 'timeout', 'usage', 'verbose', use --force to override
Error: Errors have occurred, therefore pcs is unable to continue

[citest]

I had to make some fixes for ansible-core 2.14 (get rid of command warn argument) please rebase on top of the latest master branch

STDOUT:

Warning: Agent 'pacemaker-fenced' is not installed or does not provide valid metadata: Element content failed to validate content, line 30

STDERR:

Error: invalid stonith option 'pcmk_host_list', allowed options are: 'action', 'family', 'ipport', 'nodename', 'timeout', 'usage', 'verbose', use --force to override
Error: Errors have occurred, therefore pcs is unable to continue

Ah, that's a know pacemaker issue. It's already fixed in pcs-0.10.15 in RHEL 8 / CentOS 8. Hopefully, the packages will be available in repos soon. Fixed Fedora packages will be built in a week or two.
Let's wait for the fixed pcs packages and then run the tests once more.

Looks like rhel 8.8 has the correct packages - the tests are working.  But centos stream 8 does not - tests fail - but I can live with that.  rhel 8.7 has pcs-0.10.14-5.el8.x86_64 but it works?
Fedora-36 and 37 have pcs-0.11.3 and are broken.  Are we waiting for package updates of pcs-0.11.x in Fedora?

centos 8 has pcs-0.10.14-6.el8.x86_64 but rhel 8.7 has pcs-0.10.14-5.el8.x86_64 - is this some problem that was not present in -5, introduced in -6, and fixed in a later version?

Looks like rhel 8.8 has the correct packages - the tests are working. But centos stream 8 does not - tests fail - but I can live with that. rhel 8.7 has pcs-0.10.14-5.el8.x86_64 but it works?

I'm not sure what's going on with CentOS stream. Maybe it doesn't pull from RHEL that often?

Fedora-36 and 37 have pcs-0.11.3 and are broken. Are we waiting for package updates of pcs-0.11.x in Fedora?

New pcs packages are finally being built and should be available in Fedora in a week or two. I'm fine with not waiting for it and proceed with merging the PR to speed things up. If that works for you.

centos 8 has pcs-0.10.14-6.el8.x86_64 but rhel 8.7 has pcs-0.10.14-5.el8.x86_64 - is this some problem that was not present in -5, introduced in -6, and fixed in a later version?

To trigger the bug, you need a new enough version of pacemaker and an old enough version of pcs. In RHEL 8.7, pacemaker is old enough that the issue isn't there.


Display more repository type on admin repository management

As title.

Assuming .Owner is definitely loaded lgtm

line 54 have reference .Owner so I think it's safe.

:rocket: