Controls are not executed

Describe the bug
I followed all the instructions but when loading the interface I don't see any control failling but there should be some failing (for example darkbit-aws-61).
To Reproduce
No sure what the problem is since the output.json appear to contain all the necessary data and the the UI says that it loaded my custom data set.
Expected behavior
There should be multiple control failing.
Screenshots
Client: Docker Engine - Community
 Cloud integration: 1.0.3
 Version:           20.10.0-rc1
 API version:       1.41
 Go version:        go1.13.15
 Git commit:        5cc2396
 Built:             Tue Nov 17 22:49:30 2020
 OS/Arch:           darwin/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.0-rc1
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       131bf7e
  Built:            Tue Nov 17 22:52:57 2020
  OS/Arch:          linux/amd64
  Experimental:     true
 containerd:
  Version:          v1.4.1
  GitCommit:        c623d1b36f09f8ef6536a057bd658b3aa8632828
 runc:
  Version:          1.0.0-rc92
  GitCommit:        ff819c7e9184c13b7c2607fe6c30ae19403a7aff
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

docker-compose version 1.27.4, build 40524192
docker-py version: 4.3.1
CPython version: 3.7.7
OpenSSL version: OpenSSL 1.1.1g  21 Apr 2020

Cloud provider (if applicable):
AWS
Additional context
I'm not sure what info would be relevant to troubleshoot this issue.
FYI: The link to join the slack channel is no longer valid. Maybe you should consider using Discord :)

Hi @Techbrunch
Thanks for reaching out, and I hope you had a nice New Year's.  To help understand where things are potentially going wrong, can you try the following:

git pull again from main in the control repos you cloned
repull the opencspm repo
double-check the config.yaml lists only to the /app/data/custom directory
the /app/data/custom directory contains the output.json and manifest.txt (which has one line of content being "output.json")
docker-compose up again and validate that the controls appear to load without error, the data import appears to succeed without error, and the runnerjob completes successfully.
FYI: the loader/analysis job should run on the 5th minute of every hour automatically.

If the loading/importing process has any errors, please feel free to share them here.  You can grab them with docker-compose logs.  Thanks for bringing this to our attention!

Hi @Techbrunch
Thanks for reaching out, and I hope you had a nice New Year's.  To help understand where things are potentially going wrong, can you try the following:

git pull again from main in the control repos you cloned
repull the opencspm repo
double-check the config.yaml lists only to the /app/data/custom directory
the /app/data/custom directory contains the output.json and manifest.txt (which has one line of content being "output.json")
docker-compose up again and validate that the controls appear to load without error, the data import appears to succeed without error, and the runnerjob completes successfully.
FYI: the loader/analysis job should run on the 5th minute of every hour automatically.

If the loading/importing process has any errors, please feel free to share them here.  You can grab them with docker-compose logs.  Thanks for bringing this to our attention!

Hi @bgeesaman,
Ok I think I found the issue using docker-compose logs:
worker_1  | I, [2021-01-05T19:55:13.113580 #1]  INFO -- : [ActiveJob] [RunnerJob] [bcea1727-d778-4100-9ba9-560c7edd3ff5] 3ba25700-c2f5-527e-bfba-b45d9ebd50bc Loader job finished
worker_1  | I, [2021-01-05T19:55:13.119757 #1]  INFO -- : [ActiveJob] [RunnerJob] [bcea1727-d778-4100-9ba9-560c7edd3ff5] 3ba25700-c2f5-527e-bfba-b45d9ebd50bc Analysis job started
worker_1  |
worker_1  | An error occurred while loading ./controls/opencspm-darkbit-community-controls/controls_spec.rb.
worker_1  | Failure/Error: RSpec::Core::Runner.run(options)
worker_1  |
worker_1  | Psych::SyntaxError:
worker_1  |   (<unknown>): could not find expected ':' while scanning a simple key at line 2113 column 1
worker_1  | # ./controls/opencspm-darkbit-community-controls/controls_spec.rb:4:in `<main>'
worker_1  | # /gems/gems/bootsnap-1.5.1/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:59:in `load'
worker_1  | # /gems/gems/bootsnap-1.5.1/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:59:in `load'
worker_1  | # ./app/jobs/analysis_job.rb:44:in `perform'
worker_1  | # ./app/jobs/runner_job.rb:40:in `perform'
worker_1  | # /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/execution.rb:40:in `block in perform_now'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:112:in `block in run_callbacks'
worker_1  | # /gems/gems/i18n-1.8.5/lib/i18n.rb:313:in `with_locale'
worker_1  | # /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/translation.rb:9:in `block (2 levels) in <module:Translation>'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `instance_exec'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/core_ext/time/zones.rb:66:in `use_zone'
worker_1  | # /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/timezones.rb:9:in `block (2 levels) in <module:Timezones>'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `instance_exec'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
worker_1  | # /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:25:in `block (4 levels) in <module:Logging>'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/notifications.rb:180:in `block in instrument'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/notifications/instrumenter.rb:24:in `instrument'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/notifications.rb:180:in `instrument'
worker_1  | # /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:24:in `block (3 levels) in <module:Logging>'
worker_1  | # /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:45:in `block in tag_logger'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/tagged_logging.rb:80:in `block in tagged'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/tagged_logging.rb:28:in `tagged'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/tagged_logging.rb:80:in `tagged'
worker_1  | # /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:45:in `tag_logger'
worker_1  | # /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:21:in `block (2 levels) in <module:Logging>'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `instance_exec'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:139:in `run_callbacks'
worker_1  | # /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/execution.rb:39:in `perform_now'
worker_1  | # /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/execution.rb:25:in `block in execute'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:112:in `block in run_callbacks'
worker_1  | # /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/railtie.rb:43:in `block (4 levels) in <class:Railtie>'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/execution_wrapper.rb:88:in `wrap'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/reloader.rb:72:in `block in wrap'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/execution_wrapper.rb:84:in `wrap'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/reloader.rb:71:in `wrap'
worker_1  | # /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/railtie.rb:42:in `block (3 levels) in <class:Railtie>'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `instance_exec'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:139:in `run_callbacks'
worker_1  | # /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/execution.rb:23:in `execute'
worker_1  | # /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/queue_adapters/sidekiq_adapter.rb:42:in `perform'
worker_1  | # /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:196:in `execute_job'
worker_1  | # /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:164:in `block (2 levels) in process'
worker_1  | # /gems/gems/sidekiq-6.1.2/lib/sidekiq/middleware/chain.rb:133:in `invoke'
worker_1  | # /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:163:in `block in process'
worker_1  | # /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:136:in `block (6 levels) in dispatch'
worker_1  | # /gems/gems/sidekiq-6.1.2/lib/sidekiq/job_retry.rb:111:in `local'
worker_1  | # /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:135:in `block (5 levels) in dispatch'
worker_1  | # /gems/gems/sidekiq-6.1.2/lib/sidekiq/rails.rb:14:in `block in call'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/execution_wrapper.rb:88:in `wrap'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/reloader.rb:72:in `block in wrap'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/execution_wrapper.rb:88:in `wrap'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/reloader.rb:71:in `wrap'
worker_1  | # /gems/gems/sidekiq-6.1.2/lib/sidekiq/rails.rb:13:in `call'
worker_1  | # /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:131:in `block (4 levels) in dispatch'
worker_1  | # /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:257:in `stats'
worker_1  | # /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:126:in `block (3 levels) in dispatch'
worker_1  | # /gems/gems/sidekiq-6.1.2/lib/sidekiq/job_logger.rb:13:in `call'
worker_1  | # /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:125:in `block (2 levels) in dispatch'
worker_1  | # /gems/gems/sidekiq-6.1.2/lib/sidekiq/job_retry.rb:78:in `global'
worker_1  | # /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:124:in `block in dispatch'
worker_1  | # /gems/gems/sidekiq-6.1.2/lib/sidekiq/logger.rb:10:in `with'
worker_1  | # /gems/gems/sidekiq-6.1.2/lib/sidekiq/job_logger.rb:33:in `prepare'
worker_1  | # /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:123:in `dispatch'
worker_1  | # /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:162:in `process'
worker_1  | # /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:78:in `process_one'
worker_1  | # /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:68:in `run'
worker_1  | # /gems/gems/sidekiq-6.1.2/lib/sidekiq/util.rb:15:in `watchdog'
worker_1  | # /gems/gems/sidekiq-6.1.2/lib/sidekiq/util.rb:24:in `block in safe_thread'
worker_1  | No examples found.

It doesn't look that helpful though.

Hi @bgeesaman,
Ok I think I found the issue using docker-compose logs:
worker_1  | I, [2021-01-05T19:55:13.113580 #1]  INFO -- : [ActiveJob] [RunnerJob] [bcea1727-d778-4100-9ba9-560c7edd3ff5] 3ba25700-c2f5-527e-bfba-b45d9ebd50bc Loader job finished
worker_1  | I, [2021-01-05T19:55:13.119757 #1]  INFO -- : [ActiveJob] [RunnerJob] [bcea1727-d778-4100-9ba9-560c7edd3ff5] 3ba25700-c2f5-527e-bfba-b45d9ebd50bc Analysis job started
worker_1  |
worker_1  | An error occurred while loading ./controls/opencspm-darkbit-community-controls/controls_spec.rb.
worker_1  | Failure/Error: RSpec::Core::Runner.run(options)
worker_1  |
worker_1  | Psych::SyntaxError:
worker_1  |   (<unknown>): could not find expected ':' while scanning a simple key at line 2113 column 1
worker_1  | # ./controls/opencspm-darkbit-community-controls/controls_spec.rb:4:in `<main>'
worker_1  | # /gems/gems/bootsnap-1.5.1/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:59:in `load'
worker_1  | # /gems/gems/bootsnap-1.5.1/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:59:in `load'
worker_1  | # ./app/jobs/analysis_job.rb:44:in `perform'
worker_1  | # ./app/jobs/runner_job.rb:40:in `perform'
worker_1  | # /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/execution.rb:40:in `block in perform_now'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:112:in `block in run_callbacks'
worker_1  | # /gems/gems/i18n-1.8.5/lib/i18n.rb:313:in `with_locale'
worker_1  | # /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/translation.rb:9:in `block (2 levels) in <module:Translation>'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `instance_exec'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/core_ext/time/zones.rb:66:in `use_zone'
worker_1  | # /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/timezones.rb:9:in `block (2 levels) in <module:Timezones>'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `instance_exec'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
worker_1  | # /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:25:in `block (4 levels) in <module:Logging>'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/notifications.rb:180:in `block in instrument'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/notifications/instrumenter.rb:24:in `instrument'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/notifications.rb:180:in `instrument'
worker_1  | # /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:24:in `block (3 levels) in <module:Logging>'
worker_1  | # /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:45:in `block in tag_logger'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/tagged_logging.rb:80:in `block in tagged'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/tagged_logging.rb:28:in `tagged'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/tagged_logging.rb:80:in `tagged'
worker_1  | # /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:45:in `tag_logger'
worker_1  | # /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:21:in `block (2 levels) in <module:Logging>'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `instance_exec'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:139:in `run_callbacks'
worker_1  | # /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/execution.rb:39:in `perform_now'
worker_1  | # /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/execution.rb:25:in `block in execute'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:112:in `block in run_callbacks'
worker_1  | # /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/railtie.rb:43:in `block (4 levels) in <class:Railtie>'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/execution_wrapper.rb:88:in `wrap'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/reloader.rb:72:in `block in wrap'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/execution_wrapper.rb:84:in `wrap'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/reloader.rb:71:in `wrap'
worker_1  | # /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/railtie.rb:42:in `block (3 levels) in <class:Railtie>'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `instance_exec'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:139:in `run_callbacks'
worker_1  | # /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/execution.rb:23:in `execute'
worker_1  | # /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/queue_adapters/sidekiq_adapter.rb:42:in `perform'
worker_1  | # /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:196:in `execute_job'
worker_1  | # /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:164:in `block (2 levels) in process'
worker_1  | # /gems/gems/sidekiq-6.1.2/lib/sidekiq/middleware/chain.rb:133:in `invoke'
worker_1  | # /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:163:in `block in process'
worker_1  | # /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:136:in `block (6 levels) in dispatch'
worker_1  | # /gems/gems/sidekiq-6.1.2/lib/sidekiq/job_retry.rb:111:in `local'
worker_1  | # /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:135:in `block (5 levels) in dispatch'
worker_1  | # /gems/gems/sidekiq-6.1.2/lib/sidekiq/rails.rb:14:in `block in call'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/execution_wrapper.rb:88:in `wrap'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/reloader.rb:72:in `block in wrap'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/execution_wrapper.rb:88:in `wrap'
worker_1  | # /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/reloader.rb:71:in `wrap'
worker_1  | # /gems/gems/sidekiq-6.1.2/lib/sidekiq/rails.rb:13:in `call'
worker_1  | # /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:131:in `block (4 levels) in dispatch'
worker_1  | # /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:257:in `stats'
worker_1  | # /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:126:in `block (3 levels) in dispatch'
worker_1  | # /gems/gems/sidekiq-6.1.2/lib/sidekiq/job_logger.rb:13:in `call'
worker_1  | # /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:125:in `block (2 levels) in dispatch'
worker_1  | # /gems/gems/sidekiq-6.1.2/lib/sidekiq/job_retry.rb:78:in `global'
worker_1  | # /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:124:in `block in dispatch'
worker_1  | # /gems/gems/sidekiq-6.1.2/lib/sidekiq/logger.rb:10:in `with'
worker_1  | # /gems/gems/sidekiq-6.1.2/lib/sidekiq/job_logger.rb:33:in `prepare'
worker_1  | # /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:123:in `dispatch'
worker_1  | # /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:162:in `process'
worker_1  | # /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:78:in `process_one'
worker_1  | # /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:68:in `run'
worker_1  | # /gems/gems/sidekiq-6.1.2/lib/sidekiq/util.rb:15:in `watchdog'
worker_1  | # /gems/gems/sidekiq-6.1.2/lib/sidekiq/util.rb:24:in `block in safe_thread'
worker_1  | No examples found.

It doesn't look that helpful though.

That was very helpful, thanks.  We just pushed a fix to that community controls repo.  Please re-pull the main branch of opencspm-darkbit-community-controls and retry when you can.

That was very helpful, thanks.  We just pushed a fix to that community controls repo.  Please re-pull the main branch of opencspm-darkbit-community-controls and retry when you can.

@Techbrunch We have another fix in the pipeline and we'll recut a new image here shortly.  In the meantime, a quick workaround to get you going.  Change:
.
├── opencspm
│   ├── LICENSE
│   ├── README.md
│   ├── assets
│   │   ├── custom
│   │   │   ├── manifest.txt
│   │   │   └── output.json

to
.
├── opencspm
│   ├── LICENSE
│   ├── README.md
│   ├── assets
│   │   ├── custom
│   │   │   ├── aws
│   │   │   │   ├── manifest.txt
│   │   │   │   └── output.json

And "down" and "up" again.

@Techbrunch please pull the latest from main, which will bump the Docker image to gcr.io/opencspm/engine:0.1.15.
We also updated the Slack invitation link 😄 .

@Techbrunch please pull the latest from main, which will bump the Docker image to gcr.io/opencspm/engine:0.1.15.
We also updated the Slack invitation link 😄 .

Looks like there is another issue:
worker_1  | Cleaning /app/load_dir
worker_1  | /app/data/custom
worker_1  | Appending /app/data/custom/aws/output.json to /app/load_dir/combined_for_load.json
worker_1  | Loading /app/load_dir/combined_for_load.json
worker_1  | E, [2021-01-06T09:12:14.822430 #1] ERROR -- : [ActiveJob] [RunnerJob] [4da09f6c-f882-45aa-aab4-0bfac50f460a] Error performing RunnerJob (Job ID: 4da09f6c-f882-45aa-aab4-0bfac50f460a) from Sidekiq(default) in 2116.19ms: NoMethodError (undefined method `keys' for #<Array:0x000055c6aac7faf8>):
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:48:in `block (3 levels) in loop_over_asset_lines'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:509:in `call_with_index'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:477:in `process_incoming_jobs'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:459:in `block in worker'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:450:in `fork'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:450:in `worker'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:441:in `block in create_workers'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:440:in `each'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:440:in `each_with_index'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:440:in `create_workers'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:380:in `work_in_processes'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:283:in `map'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:232:in `each'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:46:in `block (2 levels) in loop_over_asset_lines'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:45:in `each_slice'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:45:in `block in loop_over_asset_lines'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:44:in `open'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:44:in `loop_over_asset_lines'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:35:in `block in loop_over_files'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:34:in `each'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:34:in `loop_over_files'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:27:in `load'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:23:in `initialize'
worker_1  | /app/app/jobs/lib/batch_importer.rb:26:in `new'
worker_1  | /app/app/jobs/lib/batch_importer.rb:26:in `import'
worker_1  | /app/app/jobs/loader_job.rb:25:in `perform'
worker_1  | /app/app/jobs/runner_job.rb:32:in `perform'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/execution.rb:40:in `block in perform_now'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:112:in `block in run_callbacks'
worker_1  | /gems/gems/i18n-1.8.7/lib/i18n.rb:313:in `with_locale'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/translation.rb:9:in `block (2 levels) in <module:Translation>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `instance_exec'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/core_ext/time/zones.rb:66:in `use_zone'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/timezones.rb:9:in `block (2 levels) in <module:Timezones>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `instance_exec'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:25:in `block (4 levels) in <module:Logging>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/notifications.rb:180:in `block in instrument'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/notifications/instrumenter.rb:24:in `instrument'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/notifications.rb:180:in `instrument'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:24:in `block (3 levels) in <module:Logging>'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:45:in `block in tag_logger'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/tagged_logging.rb:80:in `block in tagged'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/tagged_logging.rb:28:in `tagged'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/tagged_logging.rb:80:in `tagged'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:45:in `tag_logger'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:21:in `block (2 levels) in <module:Logging>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `instance_exec'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:139:in `run_callbacks'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/execution.rb:39:in `perform_now'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/execution.rb:25:in `block in execute'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:112:in `block in run_callbacks'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/railtie.rb:43:in `block (4 levels) in <class:Railtie>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/execution_wrapper.rb:88:in `wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/reloader.rb:72:in `block in wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/execution_wrapper.rb:84:in `wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/reloader.rb:71:in `wrap'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/railtie.rb:42:in `block (3 levels) in <class:Railtie>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `instance_exec'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:139:in `run_callbacks'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/execution.rb:23:in `execute'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/queue_adapters/sidekiq_adapter.rb:42:in `perform'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:196:in `execute_job'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:164:in `block (2 levels) in process'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/middleware/chain.rb:133:in `invoke'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:163:in `block in process'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:136:in `block (6 levels) in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/job_retry.rb:111:in `local'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:135:in `block (5 levels) in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/rails.rb:14:in `block in call'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/execution_wrapper.rb:88:in `wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/reloader.rb:72:in `block in wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/execution_wrapper.rb:88:in `wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/reloader.rb:71:in `wrap'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/rails.rb:13:in `call'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:131:in `block (4 levels) in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:257:in `stats'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:126:in `block (3 levels) in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/job_logger.rb:13:in `call'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:125:in `block (2 levels) in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/job_retry.rb:78:in `global'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:124:in `block in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/logger.rb:10:in `with'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/job_logger.rb:33:in `prepare'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:123:in `dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:162:in `process'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:78:in `process_one'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:68:in `run'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/util.rb:15:in `watchdog'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/util.rb:24:in `block in safe_thread'
worker_1  | 2021-01-06T09:12:14.828Z pid=1 tid=gpr3yri2t class=RunnerJob jid=4c476cc6bd7499c3fd68e634 elapsed=2.123 INFO: fail
worker_1  | 2021-01-06T09:12:14.828Z pid=1 tid=gpr3yri2t WARN: {"context":"Job raised exception","job":{"retry":true,"queue":"default","class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","wrapped":"RunnerJob","args":[{"job_class":"RunnerJob","job_id":"4da09f6c-f882-45aa-aab4-0bfac50f460a","provider_job_id":null,"queue_name":"default","priority":null,"arguments":[],"executions":0,"exception_executions":{},"locale":"en","timezone":"UTC","enqueued_at":"2021-01-06T09:12:12Z"}],"jid":"4c476cc6bd7499c3fd68e634","created_at":1609924332.7043273,"enqueued_at":1609924332.7043912},"jobstr":"{\"retry\":true,\"queue\":\"default\",\"class\":\"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper\",\"wrapped\":\"RunnerJob\",\"args\":[{\"job_class\":\"RunnerJob\",\"job_id\":\"4da09f6c-f882-45aa-aab4-0bfac50f460a\",\"provider_job_id\":null,\"queue_name\":\"default\",\"priority\":null,\"arguments\":[],\"executions\":0,\"exception_executions\":{},\"locale\":\"en\",\"timezone\":\"UTC\",\"enqueued_at\":\"2021-01-06T09:12:12Z\"}],\"jid\":\"4c476cc6bd7499c3fd68e634\",\"created_at\":1609924332.7043273,\"enqueued_at\":1609924332.7043912}"}
worker_1  | 2021-01-06T09:12:14.828Z pid=1 tid=gpr3yri2t WARN: NoMethodError: undefined method `keys' for #<Array:0x000055c6aac7faf8>
worker_1  | 2021-01-06T09:12:14.829Z pid=1 tid=gpr3yri2t WARN: /app/app/jobs/lib/loader/file_loader.rb:48:in `block (3 levels) in loop_over_asset_lines'

Looks like there is another issue:
worker_1  | Cleaning /app/load_dir
worker_1  | /app/data/custom
worker_1  | Appending /app/data/custom/aws/output.json to /app/load_dir/combined_for_load.json
worker_1  | Loading /app/load_dir/combined_for_load.json
worker_1  | E, [2021-01-06T09:12:14.822430 #1] ERROR -- : [ActiveJob] [RunnerJob] [4da09f6c-f882-45aa-aab4-0bfac50f460a] Error performing RunnerJob (Job ID: 4da09f6c-f882-45aa-aab4-0bfac50f460a) from Sidekiq(default) in 2116.19ms: NoMethodError (undefined method `keys' for #<Array:0x000055c6aac7faf8>):
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:48:in `block (3 levels) in loop_over_asset_lines'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:509:in `call_with_index'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:477:in `process_incoming_jobs'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:459:in `block in worker'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:450:in `fork'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:450:in `worker'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:441:in `block in create_workers'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:440:in `each'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:440:in `each_with_index'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:440:in `create_workers'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:380:in `work_in_processes'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:283:in `map'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:232:in `each'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:46:in `block (2 levels) in loop_over_asset_lines'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:45:in `each_slice'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:45:in `block in loop_over_asset_lines'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:44:in `open'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:44:in `loop_over_asset_lines'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:35:in `block in loop_over_files'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:34:in `each'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:34:in `loop_over_files'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:27:in `load'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:23:in `initialize'
worker_1  | /app/app/jobs/lib/batch_importer.rb:26:in `new'
worker_1  | /app/app/jobs/lib/batch_importer.rb:26:in `import'
worker_1  | /app/app/jobs/loader_job.rb:25:in `perform'
worker_1  | /app/app/jobs/runner_job.rb:32:in `perform'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/execution.rb:40:in `block in perform_now'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:112:in `block in run_callbacks'
worker_1  | /gems/gems/i18n-1.8.7/lib/i18n.rb:313:in `with_locale'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/translation.rb:9:in `block (2 levels) in <module:Translation>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `instance_exec'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/core_ext/time/zones.rb:66:in `use_zone'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/timezones.rb:9:in `block (2 levels) in <module:Timezones>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `instance_exec'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:25:in `block (4 levels) in <module:Logging>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/notifications.rb:180:in `block in instrument'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/notifications/instrumenter.rb:24:in `instrument'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/notifications.rb:180:in `instrument'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:24:in `block (3 levels) in <module:Logging>'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:45:in `block in tag_logger'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/tagged_logging.rb:80:in `block in tagged'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/tagged_logging.rb:28:in `tagged'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/tagged_logging.rb:80:in `tagged'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:45:in `tag_logger'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:21:in `block (2 levels) in <module:Logging>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `instance_exec'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:139:in `run_callbacks'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/execution.rb:39:in `perform_now'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/execution.rb:25:in `block in execute'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:112:in `block in run_callbacks'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/railtie.rb:43:in `block (4 levels) in <class:Railtie>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/execution_wrapper.rb:88:in `wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/reloader.rb:72:in `block in wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/execution_wrapper.rb:84:in `wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/reloader.rb:71:in `wrap'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/railtie.rb:42:in `block (3 levels) in <class:Railtie>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `instance_exec'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:139:in `run_callbacks'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/execution.rb:23:in `execute'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/queue_adapters/sidekiq_adapter.rb:42:in `perform'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:196:in `execute_job'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:164:in `block (2 levels) in process'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/middleware/chain.rb:133:in `invoke'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:163:in `block in process'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:136:in `block (6 levels) in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/job_retry.rb:111:in `local'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:135:in `block (5 levels) in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/rails.rb:14:in `block in call'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/execution_wrapper.rb:88:in `wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/reloader.rb:72:in `block in wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/execution_wrapper.rb:88:in `wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/reloader.rb:71:in `wrap'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/rails.rb:13:in `call'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:131:in `block (4 levels) in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:257:in `stats'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:126:in `block (3 levels) in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/job_logger.rb:13:in `call'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:125:in `block (2 levels) in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/job_retry.rb:78:in `global'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:124:in `block in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/logger.rb:10:in `with'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/job_logger.rb:33:in `prepare'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:123:in `dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:162:in `process'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:78:in `process_one'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:68:in `run'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/util.rb:15:in `watchdog'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/util.rb:24:in `block in safe_thread'
worker_1  | 2021-01-06T09:12:14.828Z pid=1 tid=gpr3yri2t class=RunnerJob jid=4c476cc6bd7499c3fd68e634 elapsed=2.123 INFO: fail
worker_1  | 2021-01-06T09:12:14.828Z pid=1 tid=gpr3yri2t WARN: {"context":"Job raised exception","job":{"retry":true,"queue":"default","class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","wrapped":"RunnerJob","args":[{"job_class":"RunnerJob","job_id":"4da09f6c-f882-45aa-aab4-0bfac50f460a","provider_job_id":null,"queue_name":"default","priority":null,"arguments":[],"executions":0,"exception_executions":{},"locale":"en","timezone":"UTC","enqueued_at":"2021-01-06T09:12:12Z"}],"jid":"4c476cc6bd7499c3fd68e634","created_at":1609924332.7043273,"enqueued_at":1609924332.7043912},"jobstr":"{\"retry\":true,\"queue\":\"default\",\"class\":\"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper\",\"wrapped\":\"RunnerJob\",\"args\":[{\"job_class\":\"RunnerJob\",\"job_id\":\"4da09f6c-f882-45aa-aab4-0bfac50f460a\",\"provider_job_id\":null,\"queue_name\":\"default\",\"priority\":null,\"arguments\":[],\"executions\":0,\"exception_executions\":{},\"locale\":\"en\",\"timezone\":\"UTC\",\"enqueued_at\":\"2021-01-06T09:12:12Z\"}],\"jid\":\"4c476cc6bd7499c3fd68e634\",\"created_at\":1609924332.7043273,\"enqueued_at\":1609924332.7043912}"}
worker_1  | 2021-01-06T09:12:14.828Z pid=1 tid=gpr3yri2t WARN: NoMethodError: undefined method `keys' for #<Array:0x000055c6aac7faf8>
worker_1  | 2021-01-06T09:12:14.829Z pid=1 tid=gpr3yri2t WARN: /app/app/jobs/lib/loader/file_loader.rb:48:in `block (3 levels) in loop_over_asset_lines'

@Techbrunch it looks like output.json might be standard JSON instead of NDJSON. Can you confirm?

@Techbrunch it looks like output.json might be standard JSON instead of NDJSON. Can you confirm?

@joshlarsen This is indeed the issue but for some reasons I can't get the output to be ndjson:
docker run -t --rm \
  -e AWS_REGION=eu-west-1 \
  -e AWS_ACCESS_KEY_ID=... \
  -e AWS_SECRET_ACCESS_KEY=... \
  -v $(pwd)/output.json:/recon/output.json \
  darkbitio/aws_recon:latest \
  aws_recon -v -r global,eu-west-1,eu-west-3 -f custom

I'll try again later today

@joshlarsen This is indeed the issue but for some reasons I can't get the output to be ndjson:
docker run -t --rm \
  -e AWS_REGION=eu-west-1 \
  -e AWS_ACCESS_KEY_ID=... \
  -e AWS_SECRET_ACCESS_KEY=... \
  -v $(pwd)/output.json:/recon/output.json \
  darkbitio/aws_recon:latest \
  aws_recon -v -r global,eu-west-1,eu-west-3 -f custom

I'll try again later today

Give cat output.json | jq -c > newoutput.json and then echo 'newoutput.json' > manifest.txt a shot in the meantime

Give cat output.json | jq -c > newoutput.json and then echo 'newoutput.json' > manifest.txt a shot in the meantime

@Techbrunch the AWS Recon instructions were missing an important detail: you need to use the -j flag to stream NDJSON output. Thanks for catching this as well. I've updated the AWS Recon readme to reflect that.
For your case, please adjust your command as follows:
docker run -t --rm \
  -e AWS_REGION=eu-west-1 \
  -e AWS_ACCESS_KEY_ID=... \
  -e AWS_SECRET_ACCESS_KEY=... \
  darkbitio/aws_recon:latest \
  aws_recon -r global,eu-west-1,eu-west-3 -f custom -j > output.json

Note: this removes the -v flag and adds the -j flag. Also, the docker volume is not necessary in this case. Let us know how it goes. Thx.

@Techbrunch the AWS Recon instructions were missing an important detail: you need to use the -j flag to stream NDJSON output. Thanks for catching this as well. I've updated the AWS Recon readme to reflect that.
For your case, please adjust your command as follows:
docker run -t --rm \
  -e AWS_REGION=eu-west-1 \
  -e AWS_ACCESS_KEY_ID=... \
  -e AWS_SECRET_ACCESS_KEY=... \
  darkbitio/aws_recon:latest \
  aws_recon -r global,eu-west-1,eu-west-3 -f custom -j > output.json

Note: this removes the -v flag and adds the -j flag. Also, the docker volume is not necessary in this case. Let us know how it goes. Thx.

Thanks @joshlarsen, I have a new error:
worker_1  | I, [2021-01-06T16:09:53.771858 #1]  INFO -- : [ActiveJob] [RunnerJob] [00f4da8e-0805-4ea3-8958-110ee06ea764] Loading data
worker_1  | Loading config file: load_config/config.yaml
worker_1  | Fetching files from remote buckets and local directories
worker_1  | Cleaning /app/load_dir
worker_1  | /app/data/custom
worker_1  | Appending /app/data/custom/aws/output.json to /app/load_dir/combined_for_load.json
worker_1  | Loading /app/load_dir/combined_for_load.json
worker_1  |
worker_1  | Warning: no Organizations loader defined for asset type: service_control_policy
worker_1  | ...............................E, [2021-01-06T16:09:54.799352 #1] ERROR -- : [ActiveJob] [RunnerJob] [00f4da8e-0805-4ea3-8958-110ee06ea764] Error performing RunnerJob (Job ID: 00f4da8e-0805-4ea3-8958-110ee06ea764) from Sidekiq(default) in 1278.19ms: ApplicationError (Missing parameters):
worker_1  | /app/app/jobs/lib/loader/asset/graph_db_loader.rb:56:in `_upsert'
worker_1  | /app/app/jobs/lib/loader/asset/loaders/aws_loader/ec2.rb:18:in `account'
worker_1  | /app/app/jobs/lib/loader/asset/loaders/aws_loader/_resource_loader.rb:31:in `load'
worker_1  | /app/app/jobs/lib/loader/asset/loaders/aws_loader/_resource_loader.rb:8:in `initialize'
worker_1  | /app/app/jobs/lib/loader/asset/asset_router.rb:49:in `new'
worker_1  | /app/app/jobs/lib/loader/asset/asset_router.rb:49:in `aws_load'
worker_1  | /app/app/jobs/lib/loader/asset/asset_router.rb:29:in `route'
worker_1  | /app/app/jobs/lib/loader/asset/asset_router.rb:24:in `initialize'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:49:in `new'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:49:in `block (3 levels) in loop_over_asset_lines'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:509:in `call_with_index'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:477:in `process_incoming_jobs'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:459:in `block in worker'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:450:in `fork'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:450:in `worker'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:441:in `block in create_workers'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:440:in `each'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:440:in `each_with_index'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:440:in `create_workers'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:380:in `work_in_processes'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:283:in `map'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:232:in `each'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:46:in `block (2 levels) in loop_over_asset_lines'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:45:in `each_slice'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:45:in `block in loop_over_asset_lines'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:44:in `open'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:44:in `loop_over_asset_lines'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:35:in `block in loop_over_files'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:34:in `each'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:34:in `loop_over_files'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:27:in `load'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:23:in `initialize'
worker_1  | /app/app/jobs/lib/batch_importer.rb:26:in `new'
worker_1  | /app/app/jobs/lib/batch_importer.rb:26:in `import'
worker_1  | /app/app/jobs/loader_job.rb:25:in `perform'
worker_1  | /app/app/jobs/runner_job.rb:32:in `perform'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/execution.rb:40:in `block in perform_now'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:112:in `block in run_callbacks'
worker_1  | /gems/gems/i18n-1.8.7/lib/i18n.rb:313:in `with_locale'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/translation.rb:9:in `block (2 levels) in <module:Translation>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `instance_exec'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/core_ext/time/zones.rb:66:in `use_zone'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/timezones.rb:9:in `block (2 levels) in <module:Timezones>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `instance_exec'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:25:in `block (4 levels) in <module:Logging>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/notifications.rb:180:in `block in instrument'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/notifications/instrumenter.rb:24:in `instrument'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/notifications.rb:180:in `instrument'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:24:in `block (3 levels) in <module:Logging>'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:45:in `block in tag_logger'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/tagged_logging.rb:80:in `block in tagged'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/tagged_logging.rb:28:in `tagged'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/tagged_logging.rb:80:in `tagged'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:45:in `tag_logger'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:21:in `block (2 levels) in <module:Logging>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `instance_exec'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:139:in `run_callbacks'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/execution.rb:39:in `perform_now'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/execution.rb:25:in `block in execute'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:112:in `block in run_callbacks'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/railtie.rb:43:in `block (4 levels) in <class:Railtie>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/execution_wrapper.rb:88:in `wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/reloader.rb:72:in `block in wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/execution_wrapper.rb:84:in `wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/reloader.rb:71:in `wrap'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/railtie.rb:42:in `block (3 levels) in <class:Railtie>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `instance_exec'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:139:in `run_callbacks'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/execution.rb:23:in `execute'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/queue_adapters/sidekiq_adapter.rb:42:in `perform'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:196:in `execute_job'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:164:in `block (2 levels) in process'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/middleware/chain.rb:133:in `invoke'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:163:in `block in process'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:136:in `block (6 levels) in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/job_retry.rb:111:in `local'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:135:in `block (5 levels) in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/rails.rb:14:in `block in call'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/execution_wrapper.rb:88:in `wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/reloader.rb:72:in `block in wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/execution_wrapper.rb:88:in `wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/reloader.rb:71:in `wrap'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/rails.rb:13:in `call'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:131:in `block (4 levels) in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:257:in `stats'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:126:in `block (3 levels) in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/job_logger.rb:13:in `call'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:125:in `block (2 levels) in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/job_retry.rb:78:in `global'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:124:in `block in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/logger.rb:10:in `with'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/job_logger.rb:33:in `prepare'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:123:in `dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:162:in `process'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:78:in `process_one'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:68:in `run'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/util.rb:15:in `watchdog'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/util.rb:24:in `block in safe_thread'
worker_1  | 2021-01-06T16:09:54.833Z pid=1 tid=goz3jnusd class=RunnerJob jid=b8459010d7bb2e49f7617ae5 elapsed=1.321 INFO: fail
worker_1  | 2021-01-06T16:09:54.834Z pid=1 tid=goz3jnusd WARN: {"context":"Job raised exception","job":{"retry":true,"queue":"default","class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","wrapped":"RunnerJob","args":[{"job_class":"RunnerJob","job_id":"00f4da8e-0805-4ea3-8958-110ee06ea764","provider_job_id":null,"queue_name":"default","priority":null,"arguments":[],"executions":0,"exception_executions":{},"locale":"en","timezone":"UTC","enqueued_at":"2021-01-06T16:09:53Z"}],"jid":"b8459010d7bb2e49f7617ae5","created_at":1609949393.5121436,"enqueued_at":1609949393.512209},"jobstr":"{\"retry\":true,\"queue\":\"default\",\"class\":\"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper\",\"wrapped\":\"RunnerJob\",\"args\":[{\"job_class\":\"RunnerJob\",\"job_id\":\"00f4da8e-0805-4ea3-8958-110ee06ea764\",\"provider_job_id\":null,\"queue_name\":\"default\",\"priority\":null,\"arguments\":[],\"executions\":0,\"exception_executions\":{},\"locale\":\"en\",\"timezone\":\"UTC\",\"enqueued_at\":\"2021-01-06T16:09:53Z\"}],\"jid\":\"b8459010d7bb2e49f7617ae5\",\"created_at\":1609949393.5121436,\"enqueued_at\":1609949393.512209}"}
worker_1  | 2021-01-06T16:09:54.834Z pid=1 tid=goz3jnusd WARN: ApplicationError: Missing parameters
worker_1  | 2021-01-06T16:09:54.834Z pid=1 tid=goz3jnusd WARN: /app/app/jobs/lib/loader/asset/graph_db_loader.rb:56:in `_upsert'
worker_1  | /app/app/jobs/lib/loader/asset/loaders/aws_loader/ec2.rb:18:in `account'
worker_1  | /app/app/jobs/lib/loader/asset/loaders/aws_loader/_resource_loader.rb:31:in `load'
worker_1  | /app/app/jobs/lib/loader/asset/loaders/aws_loader/_resource_loader.rb:8:in `initialize'
worker_1  | /app/app/jobs/lib/loader/asset/asset_router.rb:49:in `new'
worker_1  | /app/app/jobs/lib/loader/asset/asset_router.rb:49:in `aws_load'
worker_1  | /app/app/jobs/lib/loader/asset/asset_router.rb:29:in `route'
worker_1  | /app/app/jobs/lib/loader/asset/asset_router.rb:24:in `initialize'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:49:in `new'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:49:in `block (3 levels) in loop_over_asset_lines'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:509:in `call_with_index'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:477:in `process_incoming_jobs'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:459:in `block in worker'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:450:in `fork'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:450:in `worker'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:441:in `block in create_workers'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:440:in `each'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:440:in `each_with_index'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:440:in `create_workers'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:380:in `work_in_processes'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:283:in `map'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:232:in `each'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:46:in `block (2 levels) in loop_over_asset_lines'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:45:in `each_slice'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:45:in `block in loop_over_asset_lines'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:44:in `open'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:44:in `loop_over_asset_lines'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:35:in `block in loop_over_files'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:34:in `each'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:34:in `loop_over_files'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:27:in `load'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:23:in `initialize'
worker_1  | /app/app/jobs/lib/batch_importer.rb:26:in `new'
worker_1  | /app/app/jobs/lib/batch_importer.rb:26:in `import'
worker_1  | /app/app/jobs/loader_job.rb:25:in `perform'
worker_1  | /app/app/jobs/runner_job.rb:32:in `perform'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/execution.rb:40:in `block in perform_now'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:112:in `block in run_callbacks'
worker_1  | /gems/gems/i18n-1.8.7/lib/i18n.rb:313:in `with_locale'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/translation.rb:9:in `block (2 levels) in <module:Translation>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `instance_exec'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/core_ext/time/zones.rb:66:in `use_zone'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/timezones.rb:9:in `block (2 levels) in <module:Timezones>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `instance_exec'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:25:in `block (4 levels) in <module:Logging>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/notifications.rb:180:in `block in instrument'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/notifications/instrumenter.rb:24:in `instrument'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/notifications.rb:180:in `instrument'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:24:in `block (3 levels) in <module:Logging>'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:45:in `block in tag_logger'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/tagged_logging.rb:80:in `block in tagged'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/tagged_logging.rb:28:in `tagged'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/tagged_logging.rb:80:in `tagged'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:45:in `tag_logger'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:21:in `block (2 levels) in <module:Logging>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `instance_exec'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:139:in `run_callbacks'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/execution.rb:39:in `perform_now'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/execution.rb:25:in `block in execute'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:112:in `block in run_callbacks'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/railtie.rb:43:in `block (4 levels) in <class:Railtie>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/execution_wrapper.rb:88:in `wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/reloader.rb:72:in `block in wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/execution_wrapper.rb:84:in `wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/reloader.rb:71:in `wrap'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/railtie.rb:42:in `block (3 levels) in <class:Railtie>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `instance_exec'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:139:in `run_callbacks'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/execution.rb:23:in `execute'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/queue_adapters/sidekiq_adapter.rb:42:in `perform'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:196:in `execute_job'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:164:in `block (2 levels) in process'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/middleware/chain.rb:133:in `invoke'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:163:in `block in process'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:136:in `block (6 levels) in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/job_retry.rb:111:in `local'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:135:in `block (5 levels) in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/rails.rb:14:in `block in call'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/execution_wrapper.rb:88:in `wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/reloader.rb:72:in `block in wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/execution_wrapper.rb:88:in `wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/reloader.rb:71:in `wrap'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/rails.rb:13:in `call'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:131:in `block (4 levels) in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:257:in `stats'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:126:in `block (3 levels) in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/job_logger.rb:13:in `call'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:125:in `block (2 levels) in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/job_retry.rb:78:in `global'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:124:in `block in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/logger.rb:10:in `with'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/job_logger.rb:33:in `prepare'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:123:in `dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:162:in `process'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:78:in `process_one'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:68:in `run'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/util.rb:15:in `watchdog'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/util.rb:24:in `block in safe_thread'
worker_1  | 2021-01-06T16:10:42.474Z pid=1 tid=goz3l1ipx class=RunnerJob jid=b8459010d7bb2e49f7617ae5 INFO: start
worker_1  | I, [2021-

Thanks @joshlarsen, I have a new error:
worker_1  | I, [2021-01-06T16:09:53.771858 #1]  INFO -- : [ActiveJob] [RunnerJob] [00f4da8e-0805-4ea3-8958-110ee06ea764] Loading data
worker_1  | Loading config file: load_config/config.yaml
worker_1  | Fetching files from remote buckets and local directories
worker_1  | Cleaning /app/load_dir
worker_1  | /app/data/custom
worker_1  | Appending /app/data/custom/aws/output.json to /app/load_dir/combined_for_load.json
worker_1  | Loading /app/load_dir/combined_for_load.json
worker_1  |
worker_1  | Warning: no Organizations loader defined for asset type: service_control_policy
worker_1  | ...............................E, [2021-01-06T16:09:54.799352 #1] ERROR -- : [ActiveJob] [RunnerJob] [00f4da8e-0805-4ea3-8958-110ee06ea764] Error performing RunnerJob (Job ID: 00f4da8e-0805-4ea3-8958-110ee06ea764) from Sidekiq(default) in 1278.19ms: ApplicationError (Missing parameters):
worker_1  | /app/app/jobs/lib/loader/asset/graph_db_loader.rb:56:in `_upsert'
worker_1  | /app/app/jobs/lib/loader/asset/loaders/aws_loader/ec2.rb:18:in `account'
worker_1  | /app/app/jobs/lib/loader/asset/loaders/aws_loader/_resource_loader.rb:31:in `load'
worker_1  | /app/app/jobs/lib/loader/asset/loaders/aws_loader/_resource_loader.rb:8:in `initialize'
worker_1  | /app/app/jobs/lib/loader/asset/asset_router.rb:49:in `new'
worker_1  | /app/app/jobs/lib/loader/asset/asset_router.rb:49:in `aws_load'
worker_1  | /app/app/jobs/lib/loader/asset/asset_router.rb:29:in `route'
worker_1  | /app/app/jobs/lib/loader/asset/asset_router.rb:24:in `initialize'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:49:in `new'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:49:in `block (3 levels) in loop_over_asset_lines'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:509:in `call_with_index'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:477:in `process_incoming_jobs'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:459:in `block in worker'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:450:in `fork'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:450:in `worker'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:441:in `block in create_workers'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:440:in `each'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:440:in `each_with_index'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:440:in `create_workers'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:380:in `work_in_processes'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:283:in `map'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:232:in `each'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:46:in `block (2 levels) in loop_over_asset_lines'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:45:in `each_slice'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:45:in `block in loop_over_asset_lines'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:44:in `open'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:44:in `loop_over_asset_lines'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:35:in `block in loop_over_files'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:34:in `each'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:34:in `loop_over_files'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:27:in `load'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:23:in `initialize'
worker_1  | /app/app/jobs/lib/batch_importer.rb:26:in `new'
worker_1  | /app/app/jobs/lib/batch_importer.rb:26:in `import'
worker_1  | /app/app/jobs/loader_job.rb:25:in `perform'
worker_1  | /app/app/jobs/runner_job.rb:32:in `perform'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/execution.rb:40:in `block in perform_now'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:112:in `block in run_callbacks'
worker_1  | /gems/gems/i18n-1.8.7/lib/i18n.rb:313:in `with_locale'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/translation.rb:9:in `block (2 levels) in <module:Translation>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `instance_exec'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/core_ext/time/zones.rb:66:in `use_zone'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/timezones.rb:9:in `block (2 levels) in <module:Timezones>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `instance_exec'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:25:in `block (4 levels) in <module:Logging>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/notifications.rb:180:in `block in instrument'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/notifications/instrumenter.rb:24:in `instrument'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/notifications.rb:180:in `instrument'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:24:in `block (3 levels) in <module:Logging>'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:45:in `block in tag_logger'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/tagged_logging.rb:80:in `block in tagged'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/tagged_logging.rb:28:in `tagged'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/tagged_logging.rb:80:in `tagged'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:45:in `tag_logger'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:21:in `block (2 levels) in <module:Logging>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `instance_exec'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:139:in `run_callbacks'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/execution.rb:39:in `perform_now'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/execution.rb:25:in `block in execute'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:112:in `block in run_callbacks'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/railtie.rb:43:in `block (4 levels) in <class:Railtie>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/execution_wrapper.rb:88:in `wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/reloader.rb:72:in `block in wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/execution_wrapper.rb:84:in `wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/reloader.rb:71:in `wrap'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/railtie.rb:42:in `block (3 levels) in <class:Railtie>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `instance_exec'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:139:in `run_callbacks'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/execution.rb:23:in `execute'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/queue_adapters/sidekiq_adapter.rb:42:in `perform'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:196:in `execute_job'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:164:in `block (2 levels) in process'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/middleware/chain.rb:133:in `invoke'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:163:in `block in process'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:136:in `block (6 levels) in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/job_retry.rb:111:in `local'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:135:in `block (5 levels) in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/rails.rb:14:in `block in call'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/execution_wrapper.rb:88:in `wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/reloader.rb:72:in `block in wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/execution_wrapper.rb:88:in `wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/reloader.rb:71:in `wrap'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/rails.rb:13:in `call'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:131:in `block (4 levels) in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:257:in `stats'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:126:in `block (3 levels) in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/job_logger.rb:13:in `call'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:125:in `block (2 levels) in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/job_retry.rb:78:in `global'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:124:in `block in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/logger.rb:10:in `with'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/job_logger.rb:33:in `prepare'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:123:in `dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:162:in `process'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:78:in `process_one'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:68:in `run'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/util.rb:15:in `watchdog'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/util.rb:24:in `block in safe_thread'
worker_1  | 2021-01-06T16:09:54.833Z pid=1 tid=goz3jnusd class=RunnerJob jid=b8459010d7bb2e49f7617ae5 elapsed=1.321 INFO: fail
worker_1  | 2021-01-06T16:09:54.834Z pid=1 tid=goz3jnusd WARN: {"context":"Job raised exception","job":{"retry":true,"queue":"default","class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","wrapped":"RunnerJob","args":[{"job_class":"RunnerJob","job_id":"00f4da8e-0805-4ea3-8958-110ee06ea764","provider_job_id":null,"queue_name":"default","priority":null,"arguments":[],"executions":0,"exception_executions":{},"locale":"en","timezone":"UTC","enqueued_at":"2021-01-06T16:09:53Z"}],"jid":"b8459010d7bb2e49f7617ae5","created_at":1609949393.5121436,"enqueued_at":1609949393.512209},"jobstr":"{\"retry\":true,\"queue\":\"default\",\"class\":\"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper\",\"wrapped\":\"RunnerJob\",\"args\":[{\"job_class\":\"RunnerJob\",\"job_id\":\"00f4da8e-0805-4ea3-8958-110ee06ea764\",\"provider_job_id\":null,\"queue_name\":\"default\",\"priority\":null,\"arguments\":[],\"executions\":0,\"exception_executions\":{},\"locale\":\"en\",\"timezone\":\"UTC\",\"enqueued_at\":\"2021-01-06T16:09:53Z\"}],\"jid\":\"b8459010d7bb2e49f7617ae5\",\"created_at\":1609949393.5121436,\"enqueued_at\":1609949393.512209}"}
worker_1  | 2021-01-06T16:09:54.834Z pid=1 tid=goz3jnusd WARN: ApplicationError: Missing parameters
worker_1  | 2021-01-06T16:09:54.834Z pid=1 tid=goz3jnusd WARN: /app/app/jobs/lib/loader/asset/graph_db_loader.rb:56:in `_upsert'
worker_1  | /app/app/jobs/lib/loader/asset/loaders/aws_loader/ec2.rb:18:in `account'
worker_1  | /app/app/jobs/lib/loader/asset/loaders/aws_loader/_resource_loader.rb:31:in `load'
worker_1  | /app/app/jobs/lib/loader/asset/loaders/aws_loader/_resource_loader.rb:8:in `initialize'
worker_1  | /app/app/jobs/lib/loader/asset/asset_router.rb:49:in `new'
worker_1  | /app/app/jobs/lib/loader/asset/asset_router.rb:49:in `aws_load'
worker_1  | /app/app/jobs/lib/loader/asset/asset_router.rb:29:in `route'
worker_1  | /app/app/jobs/lib/loader/asset/asset_router.rb:24:in `initialize'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:49:in `new'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:49:in `block (3 levels) in loop_over_asset_lines'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:509:in `call_with_index'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:477:in `process_incoming_jobs'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:459:in `block in worker'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:450:in `fork'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:450:in `worker'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:441:in `block in create_workers'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:440:in `each'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:440:in `each_with_index'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:440:in `create_workers'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:380:in `work_in_processes'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:283:in `map'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:232:in `each'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:46:in `block (2 levels) in loop_over_asset_lines'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:45:in `each_slice'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:45:in `block in loop_over_asset_lines'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:44:in `open'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:44:in `loop_over_asset_lines'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:35:in `block in loop_over_files'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:34:in `each'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:34:in `loop_over_files'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:27:in `load'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:23:in `initialize'
worker_1  | /app/app/jobs/lib/batch_importer.rb:26:in `new'
worker_1  | /app/app/jobs/lib/batch_importer.rb:26:in `import'
worker_1  | /app/app/jobs/loader_job.rb:25:in `perform'
worker_1  | /app/app/jobs/runner_job.rb:32:in `perform'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/execution.rb:40:in `block in perform_now'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:112:in `block in run_callbacks'
worker_1  | /gems/gems/i18n-1.8.7/lib/i18n.rb:313:in `with_locale'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/translation.rb:9:in `block (2 levels) in <module:Translation>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `instance_exec'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/core_ext/time/zones.rb:66:in `use_zone'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/timezones.rb:9:in `block (2 levels) in <module:Timezones>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `instance_exec'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:25:in `block (4 levels) in <module:Logging>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/notifications.rb:180:in `block in instrument'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/notifications/instrumenter.rb:24:in `instrument'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/notifications.rb:180:in `instrument'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:24:in `block (3 levels) in <module:Logging>'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:45:in `block in tag_logger'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/tagged_logging.rb:80:in `block in tagged'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/tagged_logging.rb:28:in `tagged'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/tagged_logging.rb:80:in `tagged'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:45:in `tag_logger'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:21:in `block (2 levels) in <module:Logging>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `instance_exec'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:139:in `run_callbacks'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/execution.rb:39:in `perform_now'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/execution.rb:25:in `block in execute'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:112:in `block in run_callbacks'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/railtie.rb:43:in `block (4 levels) in <class:Railtie>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/execution_wrapper.rb:88:in `wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/reloader.rb:72:in `block in wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/execution_wrapper.rb:84:in `wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/reloader.rb:71:in `wrap'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/railtie.rb:42:in `block (3 levels) in <class:Railtie>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `instance_exec'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:139:in `run_callbacks'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/execution.rb:23:in `execute'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/queue_adapters/sidekiq_adapter.rb:42:in `perform'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:196:in `execute_job'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:164:in `block (2 levels) in process'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/middleware/chain.rb:133:in `invoke'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:163:in `block in process'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:136:in `block (6 levels) in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/job_retry.rb:111:in `local'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:135:in `block (5 levels) in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/rails.rb:14:in `block in call'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/execution_wrapper.rb:88:in `wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/reloader.rb:72:in `block in wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/execution_wrapper.rb:88:in `wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/reloader.rb:71:in `wrap'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/rails.rb:13:in `call'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:131:in `block (4 levels) in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:257:in `stats'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:126:in `block (3 levels) in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/job_logger.rb:13:in `call'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:125:in `block (2 levels) in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/job_retry.rb:78:in `global'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:124:in `block in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/logger.rb:10:in `with'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/job_logger.rb:33:in `prepare'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:123:in `dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:162:in `process'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:78:in `process_one'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:68:in `run'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/util.rb:15:in `watchdog'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/util.rb:24:in `block in safe_thread'
worker_1  | 2021-01-06T16:10:42.474Z pid=1 tid=goz3l1ipx class=RunnerJob jid=b8459010d7bb2e49f7617ae5 INFO: start
worker_1  | I, [2021-

Thanks @Techbrunch. Just pushed a new docker image to fix that. Please re-pull 0.1.15.
docker pull gcr.io/opencspm/engine:0.1.15

Thanks @Techbrunch. Just pushed a new docker image to fix that. Please re-pull 0.1.15.
docker pull gcr.io/opencspm/engine:0.1.15

I think we are getting there :)
worker_1  | ..............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................E, [2021-01-06T17:15:20.928242 #1] ERROR -- : [ActiveJob] [RunnerJob] [f9cc5fca-775b-468e-8d28-645896b08009] Error performing RunnerJob (Job ID: f9cc5fca-775b-468e-8d28-645896b08009) from Sidekiq(default) in 9658.69ms: NameError (uninitialized constant AWSLoader::EMR):
worker_1  | /app/app/jobs/lib/loader/asset/loaders/aws_loader/_resource_loader.rb:21:in `const_get'
worker_1  | /app/app/jobs/lib/loader/asset/loaders/aws_loader/_resource_loader.rb:21:in `load'
worker_1  | /app/app/jobs/lib/loader/asset/loaders/aws_loader/_resource_loader.rb:8:in `initialize'
worker_1  | /app/app/jobs/lib/loader/asset/asset_router.rb:49:in `new'
worker_1  | /app/app/jobs/lib/loader/asset/asset_router.rb:49:in `aws_load'
worker_1  | /app/app/jobs/lib/loader/asset/asset_router.rb:29:in `route'
worker_1  | /app/app/jobs/lib/loader/asset/asset_router.rb:24:in `initialize'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:49:in `new'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:49:in `block (3 levels) in loop_over_asset_lines'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:509:in `call_with_index'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:477:in `process_incoming_jobs'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:459:in `block in worker'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:450:in `fork'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:450:in `worker'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:441:in `block in create_workers'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:440:in `each'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:440:in `each_with_index'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:440:in `create_workers'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:380:in `work_in_processes'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:283:in `map'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:232:in `each'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:46:in `block (2 levels) in loop_over_asset_lines'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:45:in `each_slice'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:45:in `block in loop_over_asset_lines'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:44:in `open'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:44:in `loop_over_asset_lines'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:35:in `block in loop_over_files'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:34:in `each'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:34:in `loop_over_files'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:27:in `load'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:23:in `initialize'
worker_1  | /app/app/jobs/lib/batch_importer.rb:26:in `new'
worker_1  | /app/app/jobs/lib/batch_importer.rb:26:in `import'
worker_1  | /app/app/jobs/loader_job.rb:25:in `perform'
worker_1  | /app/app/jobs/runner_job.rb:32:in `perform'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/execution.rb:40:in `block in perform_now'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:112:in `block in run_callbacks'
worker_1  | /gems/gems/i18n-1.8.7/lib/i18n.rb:313:in `with_locale'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/translation.rb:9:in `block (2 levels) in <module:Translation>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `instance_exec'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/core_ext/time/zones.rb:66:in `use_zone'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/timezones.rb:9:in `block (2 levels) in <module:Timezones>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `instance_exec'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:25:in `block (4 levels) in <module:Logging>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/notifications.rb:180:in `block in instrument'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/notifications/instrumenter.rb:24:in `instrument'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/notifications.rb:180:in `instrument'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:24:in `block (3 levels) in <module:Logging>'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:45:in `block in tag_logger'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/tagged_logging.rb:80:in `block in tagged'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/tagged_logging.rb:28:in `tagged'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/tagged_logging.rb:80:in `tagged'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:45:in `tag_logger'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:21:in `block (2 levels) in <module:Logging>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `instance_exec'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:139:in `run_callbacks'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/execution.rb:39:in `perform_now'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/execution.rb:25:in `block in execute'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:112:in `block in run_callbacks'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/railtie.rb:43:in `block (4 levels) in <class:Railtie>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/execution_wrapper.rb:88:in `wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/reloader.rb:72:in `block in wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/execution_wrapper.rb:84:in `wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/reloader.rb:71:in `wrap'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/railtie.rb:42:in `block (3 levels) in <class:Railtie>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `instance_exec'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:139:in `run_callbacks'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/execution.rb:23:in `execute'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/queue_adapters/sidekiq_adapter.rb:42:in `perform'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:196:in `execute_job'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:164:in `block (2 levels) in process'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/middleware/chain.rb:133:in `invoke'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:163:in `block in process'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:136:in `block (6 levels) in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/job_retry.rb:111:in `local'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:135:in `block (5 levels) in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/rails.rb:14:in `block in call'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/execution_wrapper.rb:88:in `wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/reloader.rb:72:in `block in wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/execution_wrapper.rb:88:in `wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/reloader.rb:71:in `wrap'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/rails.rb:13:in `call'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:131:in `block (4 levels) in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:257:in `stats'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:126:in `block (3 levels) in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/job_logger.rb:13:in `call'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:125:in `block (2 levels) in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/job_retry.rb:78:in `global'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:124:in `block in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/logger.rb:10:in `with'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/job_logger.rb:33:in `prepare'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:123:in `dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:162:in `process'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:78:in `process_one'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:68:in `run'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/util.rb:15:in `watchdog'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/util.rb:24:in `block in safe_thread'
worker_1  | 2021-01-06T17:15:20.939Z pid=1 tid=gnfn59299 class=RunnerJob jid=09a4194258480d49751e7fc2 elapsed=9.676 INFO: fail
worker_1  | 2021-01-06T17:15:20.940Z pid=1 tid=gnfn59299 WARN: {"context":"Job raised exception","job":{"retry":true,"queue":"default","class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","wrapped":"RunnerJob","args":[{"job_class":"RunnerJob","job_id":"f9cc5fca-775b-468e-8d28-645896b08009","provider_job_id":null,"queue_name":"default","priority":null,"arguments":[],"executions":0,"exception_executions":{},"locale":"en","timezone":"UTC","enqueued_at":"2021-01-06T17:15:11Z"}],"jid":"09a4194258480d49751e7fc2","created_at":1609953311.2627673,"enqueued_at":1609953311.2628284},"jobstr":"{\"retry\":true,\"queue\":\"default\",\"class\":\"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper\",\"wrapped\":\"RunnerJob\",\"args\":[{\"job_class\":\"RunnerJob\",\"job_id\":\"f9cc5fca-775b-468e-8d28-645896b08009\",\"provider_job_id\":null,\"queue_name\":\"default\",\"priority\":null,\"arguments\":[],\"executions\":0,\"exception_executions\":{},\"locale\":\"en\",\"timezone\":\"UTC\",\"enqueued_at\":\"2021-01-06T17:15:11Z\"}],\"jid\":\"09a4194258480d49751e7fc2\",\"created_at\":1609953311.2627673,\"enqueued_at\":1609953311.2628284}"}
worker_1  | 2021-01-06T17:15:20.940Z pid=1 tid=gnfn59299 WARN: NameError: uninitialized constant AWSLoader::EMR
worker_1  | 2021-01-06T17:15:20.941Z pid=1 tid=gnfn59299 WARN: /app/app/jobs/lib/loader/asset/loaders/aws_loader/_resource_loader.rb:21:in `const_get'
worker_1  | /app/app/jobs/lib/loader/asset/loaders/aws_loader/_resource_loader.rb:21:in `load'
worker_1  | /app/app/jobs/lib/loader/asset/loaders/aws_loader/_resource_loader.rb:8:in `initialize'
worker_1  | /app/app/jobs/lib/loader/asset/asset_router.rb:49:in `new'
worker_1  | /app/app/jobs/lib/loader/asset/asset_router.rb:49:in `aws_load'
worker_1  | /app/app/jobs/lib/loader/asset/asset_router.rb:29:in `route'
worker_1  | /app/app/jobs/lib/loader/asset/asset_router.rb:24:in `initialize'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:49:in `new'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:49:in `block (3 levels) in loop_over_asset_lines'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:509:in `call_with_index'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:477:in `process_incoming_jobs'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:459:in `block in worker'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:450:in `fork'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:450:in `worker'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:441:in `block in create_workers'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:440:in `each'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:440:in `each_with_index'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:440:in `create_workers'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:380:in `work_in_processes'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:283:in `map'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:232:in `each'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:46:in `block (2 levels) in loop_over_asset_lines'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:45:in `each_slice'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:45:in `block in loop_over_asset_lines'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:44:in `open'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:44:in `loop_over_asset_lines'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:35:in `block in loop_over_files'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:34:in `each'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:34:in `loop_over_files'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:27:in `load'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:23:in `initialize'
worker_1  | /app/app/jobs/lib/batch_importer.rb:26:in `new'
worker_1  | /app/app/jobs/lib/batch_importer.rb:26:in `import'
worker_1  | /app/app/jobs/loader_job.rb:25:in `perform'
worker_1  | /app/app/jobs/runner_job.rb:32:in `perform'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/execution.rb:40:in `block in perform_now'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:112:in `block in run_callbacks'
worker_1  | /gems/gems/i18n-1.8.7/lib/i18n.rb:313:in `with_locale'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/translation.rb:9:in `block (2 levels) in <module:Translation>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `instance_exec'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/core_ext/time/zones.rb:66:in `use_zone'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/timezones.rb:9:in `block (2 levels) in <module:Timezones>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `instance_exec'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:25:in `block (4 levels) in <module:Logging>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/notifications.rb:180:in `block in instrument'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/notifications/instrumenter.rb:24:in `instrument'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/notifications.rb:180:in `instrument'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:24:in `block (3 levels) in <module:Logging>'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:45:in `block in tag_logger'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/tagged_logging.rb:80:in `block in tagged'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/tagged_logging.rb:28:in `tagged'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/tagged_logging.rb:80:in `tagged'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:45:in `tag_logger'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:21:in `block (2 levels) in <module:Logging>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `instance_exec'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:139:in `run_callbacks'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/execution.rb:39:in `perform_now'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/execution.rb:25:in `block in execute'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:112:in `block in run_callbacks'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/railtie.rb:43:in `block (4 levels) in <class:Railtie>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/execution_wrapper.rb:88:in `wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/reloader.rb:72:in `block in wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/execution_wrapper.rb:84:in `wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/reloader.rb:71:in `wrap'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/railtie.rb:42:in `block (3 levels) in <class:Railtie>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `instance_exec'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:139:in `run_callbacks'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/execution.rb:23:in `execute'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/queue_adapters/sidekiq_adapter.rb:42:in `perform'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:196:in `execute_job'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:164:in `block (2 levels) in process'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/middleware/chain.rb:133:in `invoke'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:163:in `block in process'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:136:in `block (6 levels) in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/job_retry.rb:111:in `local'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:135:in `block (5 levels) in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/rails.rb:14:in `block in call'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/execution_wrapper.rb:88:in `wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/reloader.rb:72:in `block in wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/execution_wrapper.rb:88:in `wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/reloader.rb:71:in `wrap'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/rails.rb:13:in `call'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:131:in `block (4 levels) in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:257:in `stats'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:126:in `block (3 levels) in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/job_logger.rb:13:in `call'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:125:in `block (2 levels) in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/job_retry.rb:78:in `global'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:124:in `block in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/logger.rb:10:in `with'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/job_logger.rb:33:in `prepare'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:123:in `dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:162:in `process'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:78:in `process_one'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:68:in `run'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/util.rb:15:in `watchdog'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/util.rb:24:in `block in safe_thread'
worker_1  | 2021-01-06T17:15:46.677Z pid=1 tid=ouhq6p8q9 class=RunnerJob jid=09a4194258480d49751e7fc2 INFO: start

I think we are getting there :)
worker_1  | ..............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................E, [2021-01-06T17:15:20.928242 #1] ERROR -- : [ActiveJob] [RunnerJob] [f9cc5fca-775b-468e-8d28-645896b08009] Error performing RunnerJob (Job ID: f9cc5fca-775b-468e-8d28-645896b08009) from Sidekiq(default) in 9658.69ms: NameError (uninitialized constant AWSLoader::EMR):
worker_1  | /app/app/jobs/lib/loader/asset/loaders/aws_loader/_resource_loader.rb:21:in `const_get'
worker_1  | /app/app/jobs/lib/loader/asset/loaders/aws_loader/_resource_loader.rb:21:in `load'
worker_1  | /app/app/jobs/lib/loader/asset/loaders/aws_loader/_resource_loader.rb:8:in `initialize'
worker_1  | /app/app/jobs/lib/loader/asset/asset_router.rb:49:in `new'
worker_1  | /app/app/jobs/lib/loader/asset/asset_router.rb:49:in `aws_load'
worker_1  | /app/app/jobs/lib/loader/asset/asset_router.rb:29:in `route'
worker_1  | /app/app/jobs/lib/loader/asset/asset_router.rb:24:in `initialize'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:49:in `new'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:49:in `block (3 levels) in loop_over_asset_lines'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:509:in `call_with_index'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:477:in `process_incoming_jobs'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:459:in `block in worker'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:450:in `fork'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:450:in `worker'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:441:in `block in create_workers'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:440:in `each'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:440:in `each_with_index'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:440:in `create_workers'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:380:in `work_in_processes'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:283:in `map'
worker_1  | /gems/gems/parallel-1.20.1/lib/parallel.rb:232:in `each'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:46:in `block (2 levels) in loop_over_asset_lines'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:45:in `each_slice'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:45:in `block in loop_over_asset_lines'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:44:in `open'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:44:in `loop_over_asset_lines'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:35:in `block in loop_over_files'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:34:in `each'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:34:in `loop_over_files'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:27:in `load'
worker_1  | /app/app/jobs/lib/loader/file_loader.rb:23:in `initialize'
worker_1  | /app/app/jobs/lib/batch_importer.rb:26:in `new'
worker_1  | /app/app/jobs/lib/batch_importer.rb:26:in `import'
worker_1  | /app/app/jobs/loader_job.rb:25:in `perform'
worker_1  | /app/app/jobs/runner_job.rb:32:in `perform'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/execution.rb:40:in `block in perform_now'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:112:in `block in run_callbacks'
worker_1  | /gems/gems/i18n-1.8.7/lib/i18n.rb:313:in `with_locale'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/translation.rb:9:in `block (2 levels) in <module:Translation>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `instance_exec'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/core_ext/time/zones.rb:66:in `use_zone'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/timezones.rb:9:in `block (2 levels) in <module:Timezones>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `instance_exec'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:25:in `block (4 levels) in <module:Logging>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/notifications.rb:180:in `block in instrument'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/notifications/instrumenter.rb:24:in `instrument'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/notifications.rb:180:in `instrument'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:24:in `block (3 levels) in <module:Logging>'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:45:in `block in tag_logger'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/tagged_logging.rb:80:in `block in tagged'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/tagged_logging.rb:28:in `tagged'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/tagged_logging.rb:80:in `tagged'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:45:in `tag_logger'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/logging.rb:21:in `block (2 levels) in <module:Logging>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `instance_exec'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:139:in `run_callbacks'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/execution.rb:39:in `perform_now'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/execution.rb:25:in `block in execute'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:112:in `block in run_callbacks'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/railtie.rb:43:in `block (4 levels) in <class:Railtie>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/execution_wrapper.rb:88:in `wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/reloader.rb:72:in `block in wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/execution_wrapper.rb:84:in `wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/reloader.rb:71:in `wrap'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/railtie.rb:42:in `block (3 levels) in <class:Railtie>'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `instance_exec'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:139:in `run_callbacks'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/execution.rb:23:in `execute'
worker_1  | /gems/gems/activejob-<IP_ADDRESS>/lib/active_job/queue_adapters/sidekiq_adapter.rb:42:in `perform'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:196:in `execute_job'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:164:in `block (2 levels) in process'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/middleware/chain.rb:133:in `invoke'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:163:in `block in process'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:136:in `block (6 levels) in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/job_retry.rb:111:in `local'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:135:in `block (5 levels) in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/rails.rb:14:in `block in call'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/execution_wrapper.rb:88:in `wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/reloader.rb:72:in `block in wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/execution_wrapper.rb:88:in `wrap'
worker_1  | /gems/gems/activesupport-<IP_ADDRESS>/lib/active_support/reloader.rb:71:in `wrap'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/rails.rb:13:in `call'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:131:in `block (4 levels) in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:257:in `stats'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:126:in `block (3 levels) in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/job_logger.rb:13:in `call'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:125:in `block (2 levels) in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/job_retry.rb:78:in `global'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:124:in `block in dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/logger.rb:10:in `with'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/job_logger.rb:33:in `prepare'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:123:in `dispatch'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:162:in `process'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:78:in `process_one'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/processor.rb:68:in `run'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/util.rb:15:in `watchdog'
worker_1  | /gems/gems/sidekiq-6.1.2/lib/sidekiq/util.rb:24:in `block in safe_thread'
worker_1  | 2021-01-06T17:15:20.939Z pid=1 tid=gnfn59299 class=RunnerJob jid=09a4194258480d49751e7fc2 elapsed=9.676 INFO: fail