WIP Redact passwords

resolves RackHD/RackHD#241
@RackHD/corecommitters
new PR at the request of @brianparry 's to place logic into graph-object model.
due to a number of issues hit attempting to make my earlier code testable, I refactored to use encryption so passwords are not saved in our database in plain text to begin with and will be naturally concealed from the user on a query.
unit tests are complete for this feature.

so the most recent changes, we have gone with encrypting the password property, this way we can keep it encrypted on disk when a graph is shipped to a collection and not have an expensive recursive search done with each api call/query. new output looks like this:                 "repo": "{{api.server}}/esxi/{{options.version}}",                 "rootPassword": "ezgZbdAHMhU9+KViV5A4nA==.hSP54vlC8tvWWVhS37yb9m8elNOW2TLIeMyDpSGrZJU=",                 "users": [                     {                         "name": "onrack",                         "password": "FDWcWfw054YE9qWFmE87Ug==.6X+GyHREqap6VsSE9iuAi+BDXb6BWCc/VhImrgztwWQ=",                         "uid": 1010                     }                 ],

@brianparry so, I was wondering if I could get a second set of eye's on this (with ben being on vaca) so I'm worried, since this now encrypts passwords in the database... could other areas of our code expect plain text? and if that was the case... do these errors in jenkins seem to align with that thought process? thanks for any insight you can give!

test this please

*** BUILD #801 ***
Test Name: redfish_discovery_test
Error Details: Failure running Graph.Redfish.Discovery
-------------------- >> begin captured logging << --------------------
tests.api.v1_1.workflows_tests: INFO: Starting AMQP listener for Graph.Redfish.Discovery
amqp: DEBUG: Start from server, version: 0.9, properties: {u'information': u'Licensed under the MPL.  See http://www.rabbitmq.com/', u'product': u'RabbitMQ', u'copyright': u'Copyright (C) 2007-2013 GoPivotal, Inc.', u'capabilities': {u'exchange_exchange_bindings': True, u'connection.blocked': True, u'authentication_failure_close': True, u'basic.nack': True, u'consumer_priorities': True, u'consumer_cancel_notify': True, u'publisher_confirms': True}, u'platform': u'Erlang/OTP', u'version': u'3.2.4'}, mechanisms: [u'AMQPLAIN', u'PLAIN'], locales: [u'en_US']
amqp: DEBUG: Open OK!
kombu: INFO: Starting AMQP worker <unbound Queue graph.finished -> <unbound Exchange on.events(topic)> -> graph.finished.*>
amqp: DEBUG: Start from server, version: 0.9, properties: {u'information': u'Licensed under the MPL.  See http://www.rabbitmq.com/', u'product': u'RabbitMQ', u'copyright': u'Copyright (C) 2007-2013 GoPivotal, Inc.', u'capabilities': {u'exchange_exchange_bindings': True, u'connection.blocked': True, u'authentication_failure_close': True, u'basic.nack': True, u'consumer_priorities': True, u'consumer_cancel_notify': True, u'publisher_confirms': True}, u'platform': u'Erlang/OTP', u'version': u'3.2.4'}, mechanisms: [u'AMQPLAIN', u'PLAIN'], locales: [u'en_US']
amqp: DEBUG: Open OK!
kombu.mixins: INFO: Connected to amqp://guest:**@<IP_ADDRESS>:9091//
amqp: DEBUG: using channel_id: 1
amqp: DEBUG: Channel open
kombu: INFO: Stopping AMQP worker <unbound Queue graph.finished -> <unbound Exchange on.events(topic)> -> graph.finished.*>
tests.api.v1_1.workflows_tests: ERROR: {
    "active_task": {
        "0df65a46-8eed-45d6-a95d-96cb33e95027": {
            "friendlyName": "Create Default Pollers",
            "ignoreFailure": true,
            "implementsTask": "Task.Base.Pollers.Redfish.CreateDefault",
            "injectableName": "Task.Inline.Pollers.Redfish.CreateDefault",
            "instanceId": "0df65a46-8eed-45d6-a95d-96cb33e95027",
            "label": "create-emc-redfish-pollers",
            "name": "Task.Inline.Pollers.Redfish.CreateDefault",
            "options": {
                "_taskTimeout": 86400000,
                "pollers": [
                    {
                        "config": {
                            "command": "fabricservice"
                        },
                        "pollInterval": 10000,
                        "type": "redfish"
                    },
                    {
                        "config": {
                            "command": "elements.thermal"
                        },
                        "pollInterval": 10000,
                        "type": "redfish"
                    },
                    {
                        "config": {
                            "command": "elements.power"
                        },
                        "pollInterval": 10000,
                        "type": "redfish"
                    }
                ]
            },
            "properties": {
                "catalog": {}
            },
            "runJob": "Job.Pollers.CreateDefault",
            "state": "failed",
            "taskEndTime": "2016-08-11T19:46:44.014Z",
            "terminalOnStates": [
                "succeeded",
                "timeout",
                "cancelled",
                "failed"
            ],
            "waitingOn": {
                "6038e081-ed20-4553-81b5-1ffbab655fc3": "succeeded"
            }
        },
        "28f4c345-84fd-47d2-888a-7022a4ebf206": {
            "error": "Error: Unknown Error
    at /home/vagrant/src/on-taskgraph/node_modules/on-tasks/lib/utils/job-utils/redfish-tool.js:87:23
    at tryCatcher (/home/vagrant/src/on-taskgraph/node_modules/on-core/node_modules/bluebird/js/main/util.js:26:23)
    at Promise._settlePromiseFromHandler (/home/vagrant/src/on-taskgraph/node_modules/on-core/node_modules/bluebird/js/main/promise.js:507:31)
    at Promise._settlePromiseAt (/home/vagrant/src/on-taskgraph/node_modules/on-core/node_modules/bluebird/js/main/promise.js:581:18)
    at Async._drainQueue (/home/vagrant/src/on-taskgraph/node_modules/on-core/node_modules/bluebird/js/main/async.js:128:12)
    at Async._drainQueues (/home/vagrant/src/on-taskgraph/node_modules/on-core/node_modules/bluebird/js/main/async.js:133:10)
    at Immediate.Async.drainQueues [as _onImmediate] (/home/vagrant/src/on-taskgraph/node_modules/on-core/node_modules/bluebird/js/main/async.js:15:14)
    at processImmediate [as _immediateCallback] (timers.js:383:17)",
            "friendlyName": "Redfish Client Discovery",
            "ignoreFailure": false,
            "implementsTask": "Task.Base.Redfish.Discovery",
            "injectableName": "Task.Redfish.Discovery",
            "instanceId": "28f4c345-84fd-47d2-888a-7022a4ebf206",
            "label": "redfish-client-discovery",
            "name": "Task.Redfish.Discovery",
            "options": {
                "_taskTimeout": 86400000,
                "password": "oO2eqyKS5Q1ygJ0h0HnQ6g==.l4iAJ+iq+ju64npzLEXiRQ==",
                "uri": "https://<IP_ADDRESS>/redfish/v1",
                "username": "admin",
                "verifySSL": false
            },
            "properties": {
                "catalog": {}
            },
            "runJob": "Job.Redfish.Discovery",
            "state": "failed",
            "taskEndTime": "2016-08-11T19:46:44.014Z",
            "taskStartTime": "2016-08-11T19:46:43.844Z",
            "terminalOnStates": [
                "timeout",
                "cancelled",
                "failed"
            ],
            "waitingOn": {}
        },
        "4335af28-c838-488c-b8ff-d2758606933d": {
            "friendlyName": "Evaluate Condition",
            "ignoreFailure": false,
            "implementsTask": "Task.Base.Evaluate.Condition",
            "injectableName": "Task.Evaluate.Condition",
            "instanceId": "4335af28-c838-488c-b8ff-d2758606933d",
            "label": "when-catalog-emc",
            "name": "Task.Evaluate.Condition",
            "options": {
                "_taskTimeout": 86400000,
                "autoCatalogEmc": "true",
                "password": "buuErYdwruYKtC1YlAwGEw==.q6sIS+Ne0vF01gStXJxxOA==",
                "uri": "https://<IP_ADDRESS>/redfish/v1",
                "username": "admin",
                "when": "true"
            },
            "properties": {},
            "runJob": "Job.Evaluate.Condition",
            "schemaRef": "condition.json",
            "state": "failed",
            "taskEndTime": "2016-08-11T19:46:44.015Z",
            "terminalOnStates": [
                "timeout",
                "cancelled"
            ],
            "waitingOn": {
                "dce53aff-0d97-4760-9e4f-b1d393310b97": "succeeded"
            }
        },
        "6038e081-ed20-4553-81b5-1ffbab655fc3": {
            "friendlyName": "Evaluate Condition",
            "ignoreFailure": false,
            "implementsTask": "Task.Base.Evaluate.Condition",
            "injectableName": "Task.Evaluate.Condition",
            "instanceId": "6038e081-ed20-4553-81b5-1ffbab655fc3",
            "label": "when-pollers-emc",
            "name": "Task.Evaluate.Condition",
            "options": {
                "_taskTimeout": 86400000,
                "autoCreatePollerEmc": "true",
                "password": "UC+ekcCM1ln8v4C3uTUjig==.SOP++VjLwP03UlQ0k7pVNQ==",
                "uri": "https://<IP_ADDRESS>/redfish/v1",
                "username": "admin",
                "when": "true"
            },
            "properties": {},
            "runJob": "Job.Evaluate.Condition",
            "schemaRef": "condition.json",
            "state": "failed",
            "taskEndTime": "2016-08-11T19:46:44.015Z",
            "terminalOnStates": [
                "timeout",
                "cancelled"
            ],
            "waitingOn": {
                "dce53aff-0d97-4760-9e4f-b1d393310b97": "succeeded"
            }
        },
        "da3ce66b-4e2c-4ea2-a9cf-5b021340edfe": {
            "friendlyName": "noop",
            "ignoreFailure": false,
            "implementsTask": "Task.Base.noop",
            "injectableName": "Task.noop",
            "instanceId": "da3ce66b-4e2c-4ea2-a9cf-5b021340edfe",
            "label": "noop",
            "name": "Task.noop",
            "options": {
                "_taskTimeout": 86400000,
                "delay": 0,
                "option1": 1,
                "option2": 2,
                "option3": 3,
                "password": "GgR0ZfhxDDNz38EdWiAWwA==.IRS1uHBqy3ipYyOILrp+eQ==",
                "uri": "https://<IP_ADDRESS>/redfish/v1",
                "username": "admin"
            },
            "properties": {
                "noop": {
                    "foo": "bar",
                    "type": "null"
                }
            },
            "runJob": "Job.noop",
            "schemaRef": "noop-task.json",
            "state": "failed",
            "taskEndTime": "2016-08-11T19:46:44.015Z",
            "terminalOnStates": [
                "succeeded",
                "timeout",
                "cancelled",
                "failed"
            ],
            "waitingOn": {
                "4335af28-c838-488c-b8ff-d2758606933d": "failed",
                "6038e081-ed20-4553-81b5-1ffbab655fc3": "failed"
            }
        },
        "dce53aff-0d97-4760-9e4f-b1d393310b97": {
            "friendlyName": "Create Default Pollers",
            "ignoreFailure": false,
            "implementsTask": "Task.Base.Pollers.Redfish.CreateDefault",
            "injectableName": "Task.Inline.Pollers.Redfish.CreateDefault",
            "instanceId": "dce53aff-0d97-4760-9e4f-b1d393310b97",
            "label": "create-redfish-pollers",
            "name": "Task.Inline.Pollers.Redfish.CreateDefault",
            "options": {
                "_taskTimeout": 86400000,
                "pollers": [
                    {
                        "config": {
                            "command": "thermal"
                        },
                        "pollInterval": 10000,
                        "type": "redfish"
                    },
                    {
                        "config": {
                            "command": "power"
                        },
                        "pollInterval": 10000,
                        "type": "redfish"
                    },
                    {
                        "config": {
                            "command": "managers.logservices"
                        },
                        "pollInterval": 10000,
                        "type": "redfish"
                    },
                    {
                        "config": {
                            "command": "systems.logservices"
                        },
                        "pollInterval": 10000,
                        "type": "redfish"
                    }
                ]
            },
            "properties": {
                "catalog": {}
            },
            "runJob": "Job.Pollers.CreateDefault",
            "state": "failed",
            "taskEndTime": "2016-08-11T19:46:44.015Z",
            "terminalOnStates": [
                "timeout",
                "cancelled",
                "failed"
            ],
            "waitingOn": {
                "28f4c345-84fd-47d2-888a-7022a4ebf206": "succeeded"
            }
        },
        "e7f0061f-7de6-46de-ac1c-8eeb3299f4f0": {
            "friendlyName": "EMC Redfish Client Catalog",
            "ignoreFailure": true,
            "implementsTask": "Task.Base.Emc.Redfish.Catalog",
            "injectableName": "Task.Emc.Redfish.Catalog",
            "instanceId": "e7f0061f-7de6-46de-ac1c-8eeb3299f4f0",
            "label": "emc-redfish-catalog",
            "name": "Task.Emc.Redfish.Catalog",
            "options": {
                "_taskTimeout": 86400000
            },
            "properties": {
                "catalog": {}
            },
            "runJob": "Job.Emc.Redfish.Catalog",
            "state": "failed",
            "taskEndTime": "2016-08-11T19:46:44.015Z",
            "terminalOnStates": [
                "succeeded",
                "timeout",
                "cancelled",
                "failed"
            ],
            "waitingOn": {
                "4335af28-c838-488c-b8ff-d2758606933d": "succeeded"
            }
        }
    },
    "graph_name": "Graph.Redfish.Discovery",
    "route_id": "dbf34c8a-30f3-4b78-b1dc-e3db5d17836a",
    "status": "failed",
    "target": "Graph.Redfish.Discovery"
}
amqp: DEBUG: Closed channel #1
modules.worker: INFO: stopping subtask for Graph.Redfish.Discovery
--------------------- >> end captured logging << ---------------------
Stack Trace:   File "/usr/lib/python2.7/unittest/case.py", line 331, in run
    testMethod()
  File "/usr/lib/python2.7/unittest/case.py", line 1043, in runTest
    self._testFunc()
  File "/tmp/.venv/local/lib/python2.7/site-packages/proboscis/case.py", line 296, in testng_method_mistake_capture_func
    compatability.capture_type_error(s_func)
  File "/tmp/.venv/local/lib/python2.7/site-packages/proboscis/compatability/exceptions_2_6.py", line 27, in capture_type_error
    func()
  File "/tmp/.venv/local/lib/python2.7/site-packages/proboscis/case.py", line 350, in func
    func(test_case.state.get_state())
  File "/home/jenkins/workspace/on-core/RackHD/test/tests/api/v1_1/redfish_endpoint_tests.py", line 128, in redfish_discovery_test
    self.__post_unbound_workflow('Graph.Redfish.Discovery', body)
  File "/home/jenkins/workspace/on-core/RackHD/test/tests/api/v1_1/redfish_endpoint_tests.py", line 51, in __post_unbound_workflow
    workflow.post_unbound_workflow(graph_name, data=body)
  File "/home/jenkins/workspace/on-core/RackHD/test/tests/api/v1_1/workflows_tests.py", line 192, in post_unbound_workflow
    self.run_workflow_tasks(self.__tasks, timeout_sec)
  File "/home/jenkins/workspace/on-core/RackHD/test/tests/api/v1_1/workflows_tests.py", line 205, in run_workflow_tasks
    fail('Failure running {0}'.format(self.__graph_name))
  File "/tmp/.venv/local/lib/python2.7/site-packages/proboscis/asserts.py", line 220, in fail
    raise ASSERTION_ERROR(message)
'Failure running Graph.Redfish.Discovery\n-------------------- >> begin captured logging << --------------------\ntests.api.v1_1.workflows_tests: INFO: Starting AMQP listener for Graph.Redfish.Discovery\namqp: DEBUG: Start from server, version: 0.9, properties: {u\'information\': u\'Licensed under the MPL.  See http://www.rabbitmq.com/\', u\'product\': u\'RabbitMQ\', u\'copyright\': u\'Copyright (C) 2007-2013 GoPivotal, Inc.\', u\'capabilities\': {u\'exchange_exchange_bindings\': True, u\'connection.blocked\': True, u\'authentication_failure_close\': True, u\'basic.nack\': True, u\'consumer_priorities\': True, u\'consumer_cancel_notify\': True, u\'publisher_confirms\': True}, u\'platform\': u\'Erlang/OTP\', u\'version\': u\'3.2.4\'}, mechanisms: [u\'AMQPLAIN\', u\'PLAIN\'], locales: [u\'en_US\']\namqp: DEBUG: Open OK!\nkombu: INFO: Starting AMQP worker <unbound Queue graph.finished -> <unbound Exchange on.events(topic)> -> graph.finished.*>\namqp: DEBUG: Start from server, version: 0.9, properties: {u\'information\': u\'Licensed under the MPL.

Two questions:

Since you encrypt the options while persisting graph object, so how and when to decrypt the password before if the job wants to get the real value?
Output the ciphertext in API response still have security concern, since RackHD is open source, so it's clear that how to decrypt it. So this link redaction still leak the sensitive information.

@yyscamper For obms, I added some convenience methods in the model to decrypt, but ultimately this should be the responsibility of the job that uses the secret.
On your second point, the problem is not that we're revealing the cipher; it's that we're storing our keys insecurely.  Really, we should be using something like this: https://www.hashicorp.com/blog/vault.html

@yyscamper hey Felix, so I believe your first question is the root problem for why some of the integration tests are currently failing... we are not decrypting the password before it is being used.... so yes, this is an open question, I am assuming I'll have to hunt down the locations we need to decrypt and add those to the PR. For the short term I would agree with @brianparry to have the features that use passwords be expected to decrypt.

@yyscamper @brianparry I am open to any suggestions (and ripping this logic out if we add database tagging), and can look into the vault option as a perhaps long term solution. (at this time, I was focused on a short term fix for some blocking ODR stuff and this seemed like a more elegant method of masking our sensitive data then just transforming an object from a database that's storing plaintext ) I do however understand that it isn't entirely secure... just better then what we got lol

@brianparry just a heads up, I implemented most of what you asked for... still looking at shifting up the sanitizer function call into findAndModifyMongo~ (having a nubby moment). In addition, maybe seeking some help sorting out what jobs exactly are failing that could lead to where we expect the passwords to be decrypted...

test this please

*** BUILD #824 ***
Test Name: redfish_discovery_test
Error Details: Failure running Graph.Redfish.Discovery
-------------------- >> begin captured logging << --------------------
tests.api.v1_1.workflows_tests: INFO: Starting AMQP listener for Graph.Redfish.Discovery
amqp: DEBUG: Start from server, version: 0.9, properties: {u'information': u'Licensed under the MPL.  See http://www.rabbitmq.com/', u'product': u'RabbitMQ', u'copyright': u'Copyright (C) 2007-2013 GoPivotal, Inc.', u'capabilities': {u'exchange_exchange_bindings': True, u'connection.blocked': True, u'authentication_failure_close': True, u'basic.nack': True, u'consumer_priorities': True, u'consumer_cancel_notify': True, u'publisher_confirms': True}, u'platform': u'Erlang/OTP', u'version': u'3.2.4'}, mechanisms: [u'AMQPLAIN', u'PLAIN'], locales: [u'en_US']
amqp: DEBUG: Open OK!
kombu: INFO: Starting AMQP worker <unbound Queue graph.finished -> <unbound Exchange on.events(topic)> -> graph.finished.*>
amqp: DEBUG: Start from server, version: 0.9, properties: {u'information': u'Licensed under the MPL.  See http://www.rabbitmq.com/', u'product': u'RabbitMQ', u'copyright': u'Copyright (C) 2007-2013 GoPivotal, Inc.', u'capabilities': {u'exchange_exchange_bindings': True, u'connection.blocked': True, u'authentication_failure_close': True, u'basic.nack': True, u'consumer_priorities': True, u'consumer_cancel_notify': True, u'publisher_confirms': True}, u'platform': u'Erlang/OTP', u'version': u'3.2.4'}, mechanisms: [u'AMQPLAIN', u'PLAIN'], locales: [u'en_US']
amqp: DEBUG: Open OK!
kombu.mixins: INFO: Connected to amqp://guest:**@<IP_ADDRESS>:9091//
amqp: DEBUG: using channel_id: 1
amqp: DEBUG: Channel open
kombu: INFO: Stopping AMQP worker <unbound Queue graph.finished -> <unbound Exchange on.events(topic)> -> graph.finished.*>
tests.api.v1_1.workflows_tests: ERROR: {
    "active_task": {
        "530fe724-08f5-4894-8abe-ce198fec66ad": {
            "friendlyName": "Evaluate Condition",
            "ignoreFailure": false,
            "implementsTask": "Task.Base.Evaluate.Condition",
            "injectableName": "Task.Evaluate.Condition",
            "instanceId": "530fe724-08f5-4894-8abe-ce198fec66ad",
            "label": "when-pollers-emc",
            "name": "Task.Evaluate.Condition",
            "options": {
                "_taskTimeout": 86400000,
                "autoCreatePollerEmc": "true",
                "password": "UrIuwoUH4iXL1rtLUThg7g==.5Yt2CjCjIxrnSzqWMEK1IQ==",
                "uri": "https://<IP_ADDRESS>/redfish/v1",
                "username": "admin",
                "when": "true"
            },
            "properties": {},
            "runJob": "Job.Evaluate.Condition",
            "schemaRef": "condition.json",
            "state": "failed",
            "taskEndTime": "2016-08-18T20:52:52.506Z",
            "terminalOnStates": [
                "timeout",
                "cancelled"
            ],
            "waitingOn": {
                "cf5cd34d-ee87-4add-9300-47033444e1f0": "succeeded"
            }
        },
        "5cfff9c7-a354-4883-be26-199aeb1f81e9": {
            "friendlyName": "noop",
            "ignoreFailure": false,
            "implementsTask": "Task.Base.noop",
            "injectableName": "Task.noop",
            "instanceId": "5cfff9c7-a354-4883-be26-199aeb1f81e9",
            "label": "noop",
            "name": "Task.noop",
            "options": {
                "_taskTimeout": 86400000,
                "delay": 0,
                "option1": 1,
                "option2": 2,
                "option3": 3,
                "password": "vz9Gu/wVV7W/A6/iZS4S5A==.7Myi/ExhgTQMMJnqoaMW9g==",
                "uri": "https://<IP_ADDRESS>/redfish/v1",
                "username": "admin"
            },
            "properties": {
                "noop": {
                    "foo": "bar",
                    "type": "null"
                }
            },
            "runJob": "Job.noop",
            "schemaRef": "noop-task.json",
            "state": "failed",
            "taskEndTime": "2016-08-18T20:52:52.507Z",
            "terminalOnStates": [
                "succeeded",
                "timeout",
                "cancelled",
                "failed"
            ],
            "waitingOn": {
                "530fe724-08f5-4894-8abe-ce198fec66ad": "failed",
                "90565a25-d9ae-4754-a2c6-a1f60de0588c": "failed"
            }
        },
        "60623591-74b0-43ed-aa55-ecf7004b2e07": {
            "friendlyName": "Create Default Pollers",
            "ignoreFailure": true,
            "implementsTask": "Task.Base.Pollers.Redfish.CreateDefault",
            "injectableName": "Task.Inline.Pollers.Redfish.CreateDefault",
            "instanceId": "60623591-74b0-43ed-aa55-ecf7004b2e07",
            "label": "create-emc-redfish-pollers",
            "name": "Task.Inline.Pollers.Redfish.CreateDefault",
            "options": {
                "_taskTimeout": 86400000,
                "pollers": [
                    {
                        "config": {
                            "command": "fabricservice"
                        },
                        "pollInterval": 10000,
                        "type": "redfish"
                    },
                    {
                        "config": {
                            "command": "elements.thermal"
                        },
                        "pollInterval": 10000,
                        "type": "redfish"
                    },
                    {
                        "config": {
                            "command": "elements.power"
                        },
                        "pollInterval": 10000,
                        "type": "redfish"
                    }
                ]
            },
            "properties": {
                "catalog": {}
            },
            "runJob": "Job.Pollers.CreateDefault",
            "state": "failed",
            "taskEndTime": "2016-08-18T20:52:52.506Z",
            "terminalOnStates": [
                "succeeded",
                "timeout",
                "cancelled",
                "failed"
            ],
            "waitingOn": {
                "530fe724-08f5-4894-8abe-ce198fec66ad": "succeeded"
            }
        },
        "90565a25-d9ae-4754-a2c6-a1f60de0588c": {
            "friendlyName": "Evaluate Condition",
            "ignoreFailure": false,
            "implementsTask": "Task.Base.Evaluate.Condition",
            "injectableName": "Task.Evaluate.Condition",
            "instanceId": "90565a25-d9ae-4754-a2c6-a1f60de0588c",
            "label": "when-catalog-emc",
            "name": "Task.Evaluate.Condition",
            "options": {
                "_taskTimeout": 86400000,
                "autoCatalogEmc": "true",
                "password": "cnzoraNPzfRbcVgTomcL7g==.fwQelqaiDEZ9gXwMxzn/sA==",
                "uri": "https://<IP_ADDRESS>/redfish/v1",
                "username": "admin",
                "when": "true"
            },
            "properties": {},
            "runJob": "Job.Evaluate.Condition",
            "schemaRef": "condition.json",
            "state": "failed",
            "taskEndTime": "2016-08-18T20:52:52.507Z",
            "terminalOnStates": [
                "timeout",
                "cancelled"
            ],
            "waitingOn": {
                "cf5cd34d-ee87-4add-9300-47033444e1f0": "succeeded"
            }
        },
        "bb7394a2-6110-4ac1-9b04-8becf6c85bcd": {
            "error": "Error: Unknown Error
    at /home/vagrant/src/on-taskgraph/node_modules/on-tasks/lib/utils/job-utils/redfish-tool.js:87:23
    at tryCatcher (/home/vagrant/src/on-taskgraph/node_modules/on-core/node_modules/bluebird/js/main/util.js:26:23)
    at Promise._settlePromiseFromHandler (/home/vagrant/src/on-taskgraph/node_modules/on-core/node_modules/bluebird/js/main/promise.js:507:31)
    at Promise._settlePromiseAt (/home/vagrant/src/on-taskgraph/node_modules/on-core/node_modules/bluebird/js/main/promise.js:581:18)
    at Async._drainQueue (/home/vagrant/src/on-taskgraph/node_modules/on-core/node_modules/bluebird/js/main/async.js:128:12)
    at Async._drainQueues (/home/vagrant/src/on-taskgraph/node_modules/on-core/node_modules/bluebird/js/main/async.js:133:10)
    at Immediate.Async.drainQueues [as _onImmediate] (/home/vagrant/src/on-taskgraph/node_modules/on-core/node_modules/bluebird/js/main/async.js:15:14)
    at processImmediate [as _immediateCallback] (timers.js:383:17)",
            "friendlyName": "Redfish Client Discovery",
            "ignoreFailure": false,
            "implementsTask": "Task.Base.Redfish.Discovery",
            "injectableName": "Task.Redfish.Discovery",
            "instanceId": "bb7394a2-6110-4ac1-9b04-8becf6c85bcd",
            "label": "redfish-client-discovery",
            "name": "Task.Redfish.Discovery",
            "options": {
                "_taskTimeout": 86400000,
                "password": "RhRZ+opuzeVLwsAj3boKmg==.E2Q8oEdN81VbcCQC8zclIg==",
                "uri": "https://<IP_ADDRESS>/redfish/v1",
                "username": "admin",
                "verifySSL": false
            },
            "properties": {
                "catalog": {}
            },
            "runJob": "Job.Redfish.Discovery",
            "state": "failed",
            "taskEndTime": "2016-08-18T20:52:52.506Z",
            "taskStartTime": "2016-08-18T20:52:52.283Z",
            "terminalOnStates": [
                "timeout",
                "cancelled",
                "failed"
            ],
            "waitingOn": {}
        },
        "cf5cd34d-ee87-4add-9300-47033444e1f0": {
            "friendlyName": "Create Default Pollers",
            "ignoreFailure": false,
            "implementsTask": "Task.Base.Pollers.Redfish.CreateDefault",
            "injectableName": "Task.Inline.Pollers.Redfish.CreateDefault",
            "instanceId": "cf5cd34d-ee87-4add-9300-47033444e1f0",
            "label": "create-redfish-pollers",
            "name": "Task.Inline.Pollers.Redfish.CreateDefault",
            "options": {
                "_taskTimeout": 86400000,
                "pollers": [
                    {
                        "config": {
                            "command": "thermal"
                        },
                        "pollInterval": 10000,
                        "type": "redfish"
                    },
                    {
                        "config": {
                            "command": "power"
                        },
                        "pollInterval": 10000,
                        "type": "redfish"
                    },
                    {
                        "config": {
                            "command": "managers.logservices"
                        },
                        "pollInterval": 10000,
                        "type": "redfish"
                    },
                    {
                        "config": {
                            "command": "systems.logservices"
                        },
                        "pollInterval": 10000,
                        "type": "redfish"
                    }
                ]
            },
            "properties": {
                "catalog": {}
            },
            "runJob": "Job.Pollers.CreateDefault",
            "state": "failed",
            "taskEndTime": "2016-08-18T20:52:52.506Z",
            "terminalOnStates": [
                "timeout",
                "cancelled",
                "failed"
            ],
            "waitingOn": {
                "bb7394a2-6110-4ac1-9b04-8becf6c85bcd": "succeeded"
            }
        },
        "ded506ea-ae2b-49ea-9f4b-7e621b996cda": {
            "friendlyName": "EMC Redfish Client Catalog",
            "ignoreFailure": true,
            "implementsTask": "Task.Base.Emc.Redfish.Catalog",
            "injectableName": "Task.Emc.Redfish.Catalog",
            "instanceId": "ded506ea-ae2b-49ea-9f4b-7e621b996cda",
            "label": "emc-redfish-catalog",
            "name": "Task.Emc.Redfish.Catalog",
            "options": {
                "_taskTimeout": 86400000
            },
            "properties": {
                "catalog": {}
            },
            "runJob": "Job.Emc.Redfish.Catalog",
            "state": "failed",
            "taskEndTime": "2016-08-18T20:52:52.507Z",
            "terminalOnStates": [
                "succeeded",
                "timeout",
                "cancelled",
                "failed"
            ],
            "waitingOn": {
                "90565a25-d9ae-4754-a2c6-a1f60de0588c": "succeeded"
            }
        }
    },
    "graph_name": "Graph.Redfish.Discovery",
    "route_id": "0ef7163e-ed4b-4cb5-b646-2c1a14b895d7",
    "status": "failed",
    "target": "Graph.Redfish.Discovery"
}
amqp: DEBUG: Closed channel #1
modules.worker: INFO: stopping subtask for Graph.Redfish.Discovery
--------------------- >> end captured logging << ---------------------
Stack Trace:   File "/usr/lib/python2.7/unittest/case.py", line 331, in run
    testMethod()
  File "/usr/lib/python2.7/unittest/case.py", line 1043, in runTest
    self._testFunc()
  File "/tmp/.venv/local/lib/python2.7/site-packages/proboscis/case.py", line 296, in testng_method_mistake_capture_func
    compatability.capture_type_error(s_func)
  File "/tmp/.venv/local/lib/python2.7/site-packages/proboscis/compatability/exceptions_2_6.py", line 27, in capture_type_error
    func()
  File "/tmp/.venv/local/lib/python2.7/site-packages/proboscis/case.py", line 350, in func
    func(test_case.state.get_state())
  File "/home/jenkins/workspace/on-core/RackHD/test/tests/api/v1_1/redfish_endpoint_tests.py", line 128, in redfish_discovery_test
    self.__post_unbound_workflow('Graph.Redfish.Discovery', body)
  File "/home/jenkins/workspace/on-core/RackHD/test/tests/api/v1_1/redfish_endpoint_tests.py", line 51, in __post_unbound_workflow
    workflow.post_unbound_workflow(graph_name, data=body)
  File "/home/jenkins/workspace/on-core/RackHD/test/tests/api/v1_1/workflows_tests.py", line 192, in post_unbound_workflow
    self.run_workflow_tasks(self.__tasks, timeout_sec)
  File "/home/jenkins/workspace/on-core/RackHD/test/tests/api/v1_1/workflows_tests.py", line 205, in run_workflow_tasks
    fail('Failure running {0}'.format(self.__graph_name))
  File "/tmp/.venv/local/lib/python2.7/site-packages/proboscis/asserts.py", line 220, in fail
    raise ASSERTION_ERROR(message)
'Failure running Graph.Redfish.Discovery\n-------------------- >> begin captured logging << --------------------\ntests.api.v1_1.workflows_tests: INFO: Starting AMQP listener for Graph.Redfish.Discovery\namqp: DEBUG: Start from server, version: 0.9, properties: {u\'information\': u\'Licensed under the MPL.  See http://www.rabbitmq.com/\', u\'product\': u\'RabbitMQ\', u\'copyright\': u\'Copyright (C) 2007-2013 GoPivotal, Inc.\', u\'capabilities\': {u\'exchange_exchange_bindings\': True, u\'connection.blocked\': True, u\'authentication_failure_close\': True, u\'basic.nack\': True, u\'consumer_priorities\': True, u\'consumer_cancel_notify\': True, u\'publisher_confirms\': True}, u\'platform\': u\'Erlang/OTP\', u\'version\': u\'3.2.4\'}, mechanisms: [u\'AMQPLAIN\', u\'PLAIN\'], locales: [u\'en_US\']\namqp: DEBUG: Open OK!\nkombu: INFO: Starting AMQP worker <unbound Queue graph.finished -> <unbound Exchange on.events(topic)> -> graph.finished.*>\namqp: DEBUG: Start from server, version: 0.9, properties: {u\'information\': u\'Licensed under the MPL.    See http://www.rabbitmq.com/', u'product': u'RabbitMQ', u'copyright': u'Copyright (C) 2007-2013 GoPivotal, Inc.', u'capabilities': {u'exchange_exchange_bindings': True, u'connection.blocked': True, u'authentication_failure_close': True, u'basic.nack': True, u'consumer_priorities': True, u'consumer_cancel_notify': True, u'publisher_confirms': True}, u'platform': u'Erlang/OTP', u'version': u'3.2.4'}, mechanisms: [u'AMQPLAIN', u'PLAIN'], locales: [u'en_US']
amqp: DEBUG: Open OK!
kombu: INFO: Starting AMQP worker <unbound Queue graph.finished -> <unbound Exchange on.events(topic)> -> graph.finished.*>
amqp: DEBUG: Start from server, version: 0.9, properties: {u'information': u'Licensed under the MPL.  See http://www.rabbitmq.com/', u'product': u'RabbitMQ', u'copyright': u'Copyright (C) 2007-2013 GoPivotal, Inc.', u'capabilities': {u'exchange_exchange_bindings': True, u'connection.blocked': True, u'authentication_failure_close': True, u'basic.nack': True, u'consumer_priorities': True, u'consumer_cancel_notify': True, u'publisher_confirms': True}, u'platform': u'Erlang/OTP', u'version': u'3.2.4'}, mechanisms: [u'AMQPLAIN', u'PLAIN'], locales: [u'en_US']
amqp: DEBUG: Open OK!
kombu.mixins: INFO: Connected to amqp://guest:**@<IP_ADDRESS>:9091//
amqp: DEBUG: using channel_id: 1
amqp: DEBUG: Channel open
kombu: INFO: Stopping AMQP worker <unbound Queue graph.finished -> <unbound Exchange on.events(topic)> -> graph.finished.*>
tests.api.v1_1.workflows_tests: ERROR: {
    "active_task": {
        "30ed0cb1-b64c-4d8f-b64b-b2e46303b534": {
            "friendlyName": "noop",
            "ignoreFailure": false,
            "implementsTask": "Task.Base.noop",
            "injectableName": "Task.noop",
            "instanceId": "30ed0cb1-b64c-4d8f-b64b-b2e46303b534",
            "label": "noop",
            "name": "Task.noop",
            "options": {
                "_taskTimeout": 86400000,
                "delay": 0,
                "option1": 1,
                "option2": 2,
                "option3": 3,
                "password": "18GCIlLoep9sOP6FAJnUtw==.udYJNZTGYEUWBiaBiJi9Ig==",
                "uri": "https://<IP_ADDRESS>/redfish/v1",
                "username": "admin"
            },
            "properties": {
                "noop": {
                    "foo": "bar",
                    "type": "null"
                }
            },
            "runJob": "Job.noop",
            "schemaRef": "noop-task.json",
            "state": "failed",
            "taskEndTime": "2016-08-19T22:41:41.763Z",
            "terminalOnStates": [
                "succeeded",
                "timeout",
                "cancelled",
                "failed"
            ],
            "waitingOn": {
                "880ab249-a1ed-42d2-9ab6-6968fc503641": "failed",
                "93fb358e-91ea-4681-8966-b01d561df51c": "failed"
            }
        },
        "317ea631-3d05-489a-a41b-56efded07daf": {
            "friendlyName": "Create Default Pollers",
            "ignoreFailure": false,
            "implementsTask": "Task.Base.Pollers.Redfish.CreateDefault",
            "injectableName": "Task.Inline.Pollers.Redfish.CreateDefault",
            "instanceId": "317ea631-3d05-489a-a41b-56efded07daf",
            "label": "create-redfish-pollers",
            "name": "Task.Inline.Pollers.Redfish.CreateDefault",
            "options": {
                "_taskTimeout": 86400000,
                "pollers": [
                    {
                        "config": {
                            "command": "thermal"
                        },
                        "pollInterval": 10000,
                        "type": "redfish"
                    },
                    {
                        "config": {
                            "command": "power"
                        },
                        "pollInterval": 10000,
                        "type": "redfish"
                    },
                    {
                        "config": {
                            "command": "managers.logservices"
                        },
                        "pollInterval": 10000,
                        "type": "redfish"
                    },
                    {
                        "config": {
                            "command": "systems.logservices"
                        },
                        "pollInterval": 10000,
                        "type": "redfish"
                    }
                ]
            },
            "properties": {
                "catalog": {}
            },
            "runJob": "Job.Pollers.CreateDefault",
            "state": "failed",
            "taskEndTime": "2016-08-19T22:41:41.763Z",
            "terminalOnStates": [
                "timeout",
                "cancelled",
                "failed"
            ],
            "waitingOn": {
                "b5f79624-f03a-4277-8d84-16ab998436d6": "succeeded"
            }
        },
        "880ab249-a1ed-42d2-9ab6-6968fc503641": {
            "friendlyName": "Evaluate Condition",
            "ignoreFailure": false,
            "implementsTask": "Task.Base.Evaluate.Condition",
            "injectableName": "Task.Evaluate.Condition",
            "instanceId": "880ab249-a1ed-42d2-9ab6-6968fc503641",
            "label": "when-pollers-emc",
            "name": "Task.Evaluate.Condition",
            "options": {
                "_taskTimeout": 86400000,
                "autoCreatePollerEmc": "true",
                "password": "TPgZ9Y43SHSgwlE5q/Atug==.wFLQ6zwHvP5R2L5XQmvEFw==",
                "uri": "https://<IP_ADDRESS>/redfish/v1",
                "username": "admin",
                "when": "true"
            },
            "properties": {},
            "runJob": "Job.Evaluate.Condition",
            "schemaRef": "condition.json",
            "state": "failed",
            "taskEndTime": "2016-08-19T22:41:41.763Z",
            "terminalOnStates": [
                "timeout",
                "cancelled"
            ],
            "waitingOn": {
                "317ea631-3d05-489a-a41b-56efded07daf": "succeeded"
            }
        },
        "93fb358e-91ea-4681-8966-b01d561df51c": {
            "friendlyName": "Evaluate Condition",
            "ignoreFailure": false,
            "implementsTask": "Task.Base.Evaluate.Condition",
            "injectableName": "Task.Evaluate.Condition",
            "instanceId": "93fb358e-91ea-4681-8966-b01d561df51c",
            "label": "when-catalog-emc",
            "name": "Task.Evaluate.Condition",
            "options": {
                "_taskTimeout": 86400000,
                "autoCatalogEmc": "true",
                "password": "4jISAUllWruwrYK+q+N7Bw==.ikKdUltL/WMWv1kTS2lDwg==",
                "uri": "https://<IP_ADDRESS>/redfish/v1",
                "username": "admin",
                "when": "true"
            },
            "properties": {},
            "runJob": "Job.Evaluate.Condition",
            "schemaRef": "condition.json",
            "state": "failed",
            "taskEndTime": "2016-08-19T22:41:41.763Z",
            "terminalOnStates": [
                "timeout",
                "cancelled"
            ],
            "waitingOn": {
                "317ea631-3d05-489a-a41b-56efded07daf": "succeeded"
            }
        },
        "9c0da2fc-dee9-45aa-9833-f0dca5e5a0c2": {
            "friendlyName": "EMC Redfish Client Catalog",
            "ignoreFailure": true,
            "implementsTask": "Task.Base.Emc.Redfish.Catalog",
            "injectableName": "Task.Emc.Redfish.Catalog",
            "instanceId": "9c0da2fc-dee9-45aa-9833-f0dca5e5a0c2",
            "label": "emc-redfish-catalog",
            "name": "Task.Emc.Redfish.Catalog",
            "options": {
                "_taskTimeout": 86400000
            },
            "properties": {
                "catalog": {}
            },
            "runJob": "Job.Emc.Redfish.Catalog",
            "state": "failed",
            "taskEndTime": "2016-08-19T22:41:41.763Z",
            "terminalOnStates": [
                "succeeded",
                "timeout",
                "cancelled",
                "failed"
            ],
            "waitingOn": {
                "93fb358e-91ea-4681-8966-b01d561df51c": "succeeded"
            }
        },
        "b2087373-53c9-4631-8514-c47516cbe412": {
            "friendlyName": "Create Default Pollers",
            "ignoreFailure": true,
            "implementsTask": "Task.Base.Pollers.Redfish.CreateDefault",
            "injectableName": "Task.Inline.Pollers.Redfish.CreateDefault",
            "instanceId": "b2087373-53c9-4631-8514-c47516cbe412",
            "label": "create-emc-redfish-pollers",
            "name": "Task.Inline.Pollers.Redfish.CreateDefault",
            "options": {
                "_taskTimeout": 86400000,
                "pollers": [
                    {
                        "config": {
                            "command": "fabricservice"
                        },
                        "pollInterval": 10000,
                        "type": "redfish"
                    },
                    {
                        "config": {
                            "command": "elements.thermal"
                        },
                        "pollInterval": 10000,
                        "type": "redfish"
                    },
                    {
                        "config": {
                            "command": "elements.power"
                        },
                        "pollInterval": 10000,
                        "type": "redfish"
                    }
                ]
            },
            "properties": {
                "catalog": {}
            },
            "runJob": "Job.Pollers.CreateDefault",
            "state": "failed",
            "taskEndTime": "2016-08-19T22:41:41.763Z",
            "terminalOnStates": [
                "succeeded",
                "timeout",
                "cancelled",
                "failed"
            ],
            "waitingOn": {
                "880ab249-a1ed-42d2-9ab6-6968fc503641": "succeeded"
            }
        },
        "b5f79624-f03a-4277-8d84-16ab998436d6": {
            "error": "Error: Unknown Error
    at /home/vagrant/src/on-taskgraph/node_modules/on-tasks/lib/utils/job-utils/redfish-tool.js:87:23
    at tryCatcher (/home/vagrant/src/on-taskgraph/node_modules/on-core/node_modules/bluebird/js/main/util.js:26:23)
    at Promise._settlePromiseFromHandler (/home/vagrant/src/on-taskgraph/node_modules/on-core/node_modules/bluebird/js/main/promise.js:507:31)
    at Promise._settlePromiseAt (/home/vagrant/src/on-taskgraph/node_modules/on-core/node_modules/bluebird/js/main/promise.js:581:18)
    at Async._drainQueue (/home/vagrant/src/on-taskgraph/node_modules/on-core/node_modules/bluebird/js/main/async.js:128:12)
    at Async._drainQueues (/home/vagrant/src/on-taskgraph/node_modules/on-core/node_modules/bluebird/js/main/async.js:133:10)
    at Immediate.Async.drainQueues [as _onImmediate] (/home/vagrant/src/on-taskgraph/node_modules/on-core/node_modules/bluebird/js/main/async.js:15:14)
    at processImmediate [as _immediateCallback] (timers.js:383:17)",
            "friendlyName": "Redfish Client Discovery",
            "ignoreFailure": false,
            "implementsTask": "Task.Base.Redfish.Discovery",
            "injectableName": "Task.Redfish.Discovery",
            "instanceId": "b5f79624-f03a-4277-8d84-16ab998436d6",
            "label": "redfish-client-discovery",
            "name": "Task.Redfish.Discovery",
            "options": {
                "_taskTimeout": 86400000,
                "password": "KfSjnIoB6IAcY4kUTjs0jw==.tDmwwwPIBkNJkmgLfWSC3Q==",
                "uri": "https://<IP_ADDRESS>/redfish/v1",
                "username": "admin",
                "verifySSL": false
            },
            "properties": {
                "catalog": {}
            },
            "runJob": "Job.Redfish.Discovery",
            "state": "failed",
            "taskEndTime": "2016-08-19T22:41:41.764Z",
            "taskStartTime": "2016-08-19T22:41:41.601Z",
            "terminalOnStates": [
                "timeout",
                "cancelled",
                "failed"
            ],
            "waitingOn": {}
        }
    },
    "graph_name": "Graph.Redfish.Discovery",
    "route_id": "6f1c7efb-646d-4e30-9f0f-1005fa95b273",
    "status": "failed",
    "target": "Graph.Redfish.Discovery"
}
amqp: DEBUG: Closed channel #1
modules.worker: INFO: stopping subtask for Graph.Redfish.Discovery
--------------------- >> end captured logging << ---------------------
Stack Trace:   File "/usr/lib/python2.7/unittest/case.py", line 331, in run
    testMethod()
  File "/usr/lib/python2.7/unittest/case.py", line 1043, in runTest
    self._testFunc()
  File "/tmp/.venv/local/lib/python2.7/site-packages/proboscis/case.py", line 296, in testng_method_mistake_capture_func
    compatability.capture_type_error(s_func)
  File "/tmp/.venv/local/lib/python2.7/site-packages/proboscis/compatability/exceptions_2_6.py", line 27, in capture_type_error
    func()
  File "/tmp/.venv/local/lib/python2.7/site-packages/proboscis/case.py", line 350, in func
    func(test_case.state.get_state())
  File "/home/jenkins/workspace/on-core/RackHD/test/tests/api/v1_1/redfish_endpoint_tests.py", line 128, in redfish_discovery_test
    self.__post_unbound_workflow('Graph.Redfish.Discovery', body)
  File "/home/jenkins/workspace/on-core/RackHD/test/tests/api/v1_1/redfish_endpoint_tests.py", line 51, in __post_unbound_workflow
    workflow.post_unbound_workflow(graph_name, data=body)
  File "/home/jenkins/workspace/on-core/RackHD/test/tests/api/v1_1/workflows_tests.py", line 192, in post_unbound_workflow
    self.run_workflow_tasks(self.__tasks, timeout_sec)
  File "/home/jenkins/workspace/on-core/RackHD/test/tests/api/v1_1/workflows_tests.py", line 205, in run_workflow_tasks
    fail('Failure running {0}'.format(self.__graph_name))
  File "/tmp/.venv/local/lib/python2.7/site-packages/proboscis/asserts.py", line 220, in fail
    raise ASSERTION_ERROR(message)
'Failure running Graph.Redfish.Discovery\n-------------------- >> begin captured logging << --------------------\ntests.api.v1_1.workflows_tests: INFO: Starting AMQP listener for Graph.Redfish.Discovery\namqp: DEBUG: Start from server, version: 0.9, properties: {u\'information\': u\'Licensed under the MPL.  See http://www.rabbitmq.com/\', u\'product\': u\'RabbitMQ\', u\'copyright\': u\'Copyright (C) 2007-2013 GoPivotal, Inc.\', u\'capabilities\': {u\'exchange_exchange_bindings\': True, u\'connection.blocked\': True, u\'authentication_failure_close\': True, u\'basic.nack\': True, u\'consumer_priorities\': True, u\'consumer_cancel_notify\': True, u\'publisher_confirms\': True}, u\'platform\': u\'Erlang/OTP\', u\'version\': u\'3.2.4\'}, mechanisms: [u\'AMQPLAIN\', u\'PLAIN\'], locales: [u\'en_US\']\namqp: DEBUG: Open OK!\nkombu: INFO: Starting AMQP worker <unbound Queue graph.finished -> <unbound Exchange on.events(topic)> -> graph.finished.*>\namqp: DEBUG: Start from server, version: 0.9, properties: {u\'information\': u\'Licensed under the MPL.  See http://www.rabbitmq.com/\', u\'product\': u\'RabbitMQ\', u\'copyright\': u\'Copyright (C) 2007-2013 GoPivotal, Inc.\', u\'capabilities\': {u\'exchange_exchange_bindings\': True, u\'connection.blocked\': True, u\'authentication_failure_close\': True, u\'basic.nack\': True, u\'consumer_priorities\': True, u\'consumer_cancel_notify\': True, u\'publisher_confirms\': True}, u\'platform\': u\'Erlang/OTP\', u\'version\': u\'3.2.4\'}, mechanisms: [u\'AMQPLAIN\', u\'PLAIN\'], locales: [u\'en_US\']\namqp: DEBUG: Open OK!\nkombu.mixins: INFO: Connected to amqp://guest:**@<IP_ADDRESS>:9091//\namqp: DEBUG: using channel_id: 1\namqp: DEBUG: Channel open\nkombu: INFO: Stopping AMQP worker <unbound Queue graph.finished -> <unbound Exchange on.events(topic)> -> graph.finished.*>\ntests.api.v1_1.workflows_tests: ERROR: {\n    "active_task": {\n        "30ed0cb1-b64c-4d8f-b64b-b2e46303b534": {\n            "friendlyName": "noop",\n            "ignoreFailure": false,\n            "implementsTask": "Task.Base.noop",\n            "injectableName": "Task.noop",\n            "instanceId": "30ed0cb1-b64c-4d8f-b64b-b2e46303b534",\n            "label": "noop",\n            "name": "Task.noop",\n            "options": {\n                "_taskTimeout": 86400000,\n                "delay": 0,\n                "option1": 1,\n                "option2": 2,\n                "option3": 3,\n                "password": "18GCIlLoep9sOP6FAJnUtw==.udYJNZTGYEUWBiaBiJi9Ig==",\n                "uri": "https://<IP_ADDRESS>/redfish/v1",\n                "username": "admin"\n            },\n            "properties": {\n                "noop": {\n                    "foo": "bar",\n                    "type": "null"\n                }\n            },\n            "runJob": "Job.noop",\n            "schemaRef": "noop-task.json",\n            "state": "failed",\n            "taskEndTime": "2016-08-19T22:41:41.763Z",\n            "terminalOnStates": [\n                "succeeded",\n                "timeout",\n                "cancelled",\n                "failed"\n            ],\n            "waitingOn": {\n                "880ab249-a1ed-42d2-9ab6-6968fc503641": "failed",\n                "93fb358e-91ea-4681-8966-b01d561df51c": "failed"\n            }\n        },\n        "317ea631-3d05-489a-a41b-56efded07daf": {\n            "friendlyName": "Create Default Pollers",\n            "ignoreFailure": false,\n            "implementsTask": "Task.Base.Pollers.Redfish.CreateDefault",\n            "injectableName": "Task.Inline.Pollers.Redfish.CreateDefault",\n            "instanceId": "317ea631-3d05-489a-a41b-56efded07daf",\n            "label": "create-redfish-pollers",\n            "name": "Task.Inline.Pollers.Redfish.CreateDefault",\n            "options": {\n                "_taskTimeout": 86400000,\n                "pollers": [\n                    {\n                        "config": {\n                            "command": "thermal"\n                        },\n                        "pollInterval": 10000,\n                        "type": "redfish"\n                    },\n                    {\n                        "config": {\n                            "command": "power"\n                        },\n                        "pollInterval": 10000,\n                        "type": "redfish"\n                    },\n                    {\n                        "config": {\n                            "command": "managers.logservices"\n                        },\n                        "pollInterval": 10000,\n                        "type": "redfish"\n                    },\n                    {\n                        "config": {\n                            "command": "systems.logservices"\n                        },\n                        "pollInterval": 10000,\n                        "type": "redfish"\n                    }\n                ]\n            },\n            "properties": {\n                "catalog": {}\n            },\n            "runJob": "Job.Pollers.CreateDefault",\n            "state": "failed",\n            "taskEndTime": "2016-08-19T22:41:41.763Z",\n            "terminalOnStates": [\n                "timeout",\n                "cancelled",\n                "failed"\n            ],\n            "waitingOn": {\n                "b5f79624-f03a-4277-8d84-16ab998436d6": "succeeded"\n            }\n        },\n        "880ab249-a1ed-42d2-9ab6-6968fc503641": {\n            "friendlyName": "Evaluate Condition",\n            "ignoreFailure": false,\n            "implementsTask": "Task.Base.Evaluate.Condition",\n            "injectableName": "Task.Evaluate.Condition",\n            "instanceId": "880ab249-a1ed-42d2-9ab6-6968fc503641",\n            "label": "when-pollers-emc",\n            "name": "Task.Evaluate.Condition",\n            "options": {\n                "_taskTimeout": 86400000,\n                "autoCreatePollerEmc": "true",\n                "password": "TPgZ9Y43SHSgwlE5q/Atug==.wFLQ6zwHvP5R2L5XQmvEFw==",\n                "uri": "https://<IP_ADDRESS>/redfish/v1",\n                "username": "admin",\n                "when": "true"\n            },\n            "properties": {},\n            "runJob": "Job.Evaluate.Condition",\n            "schemaRef": "condition.json",\n            "state": "failed",\n            "taskEndTime": "2016-08-19T22:41:41.763Z",\n            "terminalOnStates": [\n                "timeout",\n                "cancelled"\n            ],\n            "waitingOn": {\n                "317ea631-3d05-489a-a41b-56efded07daf": "succeeded"\n            }\n        },\n        "93fb358e-91ea-4681-8966-b01d561df51c": {\n            "friendlyName": "Evaluate Condition",\n            "ignoreFailure": false,\n            "implementsTask": "Task.Base.Evaluate.Condition",\n            "injectableName": "Task.Evaluate.Condition",\n            "instanceId": "93fb358e-91ea-4681-8966-b01d561df51c",\n            "label": "when-catalog-emc",\n            "name": "Task.Evaluate.Condition",\n            "options": {\n                "_taskTimeout": 86400000,\n                "autoCatalogEmc": "true",\n                "password": "4jISAUllWruwrYK+q+N7Bw==.ikKdUltL/WMWv1kTS2lDwg==",\n                "uri": "https://<IP_ADDRESS>/redfish/v1",\n                "username": "admin",\n                "when": "true"\n            },\n            "properties": {},\n            "runJob": "Job.Evaluate.Condition",\n            "schemaRef": "condition.json",\n            "state": "failed",\n            "taskEndTime": "2016-08-19T22:41:41.763Z",\n            "terminalOnStates": [\n                "timeout",\n                "cancelled"\n            ],\n            "waitingOn": {\n                "317ea631-3d05-489a-a41b-56efded07daf": "succeeded"\n            }\n        },\n        "9c0da2fc-dee9-45aa-9833-f0dca5e5a0c2": {\n            "friendlyName": "EMC Redfish Client Catalog",\n            "ignoreFailure": true,\n            "implementsTask": "Task.Base.Emc.Redfish.Catalog",\n            "injectableName": "Task.Emc.Redfish.Catalog",\n            "instanceId": "9c0da2fc-dee9-45aa-9833-f0dca5e5a0c2",\n            "label": "emc-redfish-catalog",\n            "name": "Task.Emc.Redfish.Catalog",\n            "options": {\n                "_taskTimeout": 86400000\n            },\n            "properties": {\n                "catalog": {}\n            },\n            "runJob": "Job.Emc.Redfish.Catalog",\n            "state": "failed",\n            "taskEndTime": "2016-08-19T22:41:41.763Z",\n            "terminalOnStates": [\n                "succeeded",\n                "timeout",\n                "cancelled",\n                "failed"\n            ],\n            "waitingOn": {\n                "93fb358e-91ea-4681-8966-b01d561df51c": "succeeded"\n            }\n        },\n        "b2087373-53c9-4631-8514-c47516cbe412": {\n            "friendlyName": "Create Default Pollers",\n            "ignoreFailure": true,\n            "implementsTask": "Task.Base.Pollers.Redfish.CreateDefault",\n            "injectableName": "Task.Inline.Pollers.Redfish.CreateDefault",\n            "instanceId": "b2087373-53c9-4631-8514-c47516cbe412",\n            "label": "create-emc-redfish-pollers",\n            "name": "Task.Inline.Pollers.Redfish.CreateDefault",\n            "options": {\n                "_taskTimeout": 86400000,\n                "pollers": [\n                    {\n                        "config": {\n                            "command": "fabricservice"\n                        },\n                        "pollInterval": 10000,\n                        "type": "redfish"\n                    },\n                    {\n                        "config": {\n                            "command": "elements.thermal"\n                        },\n                        "pollInterval": 10000,\n                        "type": "redfish"\n                    },\n                    {\n                        "config": {\n                            "command": "elements.power"\n                        },\n                        "pollInterval": 10000,\n                        "type": "redfish"\n                    }\n                ]\n            },\n            "properties": {\n                "catalog": {}\n            },\n            "runJob": "Job.Pollers.CreateDefault",\n            "state": "failed",\n            "taskEndTime": "2016-08-19T22:41:41.763Z",\n            "terminalOnStates": [\n                "succeeded",\n                "timeout",\n                "cancelled",\n                "failed"\n            ],\n            "waitingOn": {\n                "880ab249-a1ed-42d2-9ab6-6968fc503641": "succeeded"\n            }\n        },\n        "b5f79624-f03a-4277-8d84-16ab998436d6": {\n            "error": "Error: Unknown Error\n    at /home/vagrant/src/on-taskgraph/node_modules/on-tasks/lib/utils/job-utils/redfish-tool.js:87:23\n    at tryCatcher (/home/vagrant/src/on-taskgraph/node_modules/on-core/node_modules/bluebird/js/main/util.js:26:23)\n    at Promise._settlePromiseFromHandler (/home/vagrant/src/on-taskgraph/node_modules/on-core/node_modules/bluebird/js/main/promise.js:507:31)\n    at Promise._settlePromiseAt (/home/vagrant/src/on-taskgraph/node_modules/on-core/node_modules/bluebird/js/main/promise.js:581:18)\n    at Async._drainQueue (/home/vagrant/src/on-taskgraph/node_modules/on-core/node_modules/bluebird/js/main/async.js:128:12)\n    at Async._drainQueues (/home/vagrant/src/on-taskgraph/node_modules/on-core/node_modules/bluebird/js/main/async.js:133:10)\n    at Immediate.Async.drainQueues [as _onImmediate] (/home/vagrant/src/on-taskgraph/node_modules/on-core/node_modules/bluebird/js/main/async.js:15:14)\n    at processImmediate [as _immediateCallback] (timers.js:383:17)",\n            "friendlyName": "Redfish Client Discovery",\n            "ignoreFailure": false,\n            "implementsTask": "Task.Base.Redfish.Discovery",\n            "injectableName": "Task.Redfish.Discovery",\n            "instanceId": "b5f79624-f03a-4277-8d84-16ab998436d6",\n            "label": "redfish-client-discovery",\n            "name": "Task.Redfish.Discovery",\n            "options": {\n                "_taskTimeout": 86400000,\n                "password": "KfSjnIoB6IAcY4kUTjs0jw==.tDmwwwPIBkNJkmgLfWSC3Q==",\n                "uri": "https://<IP_ADDRESS>/redfish/v1",\n                "username": "admin",\n                "verifySSL": false\n            },\n            "properties": {\n                "catalog": {}\n            },\n            "runJob": "Job.Redfish.Discovery",\n            "state": "failed",\n            "taskEndTime": "2016-08-19T22:41:41.764Z",\n            "taskStartTime": "2016-08-19T22:41:41.601Z",\n            "terminalOnStates": [\n                "timeout",\n                "cancelled",\n                "failed"\n            ],\n            "waitingOn": {}\n        }\n    },\n    "graph_name": "Graph.Redfish.Discovery",\n    "route_id": "6f1c7efb-646d-4e30-9f0f-1005fa95b273",\n    "status": "failed",\n    "target": "Graph.Redfish.Discovery"\n}\namqp: DEBUG: Closed channel #1\nmodules.worker: INFO: stopping subtask for Graph.Redfish.Discovery\n--------------------- >> end captured logging << ---------------------'

@DavidjohnBlodgett :
The biggest problem of the PR is that it lacks the peer function of santize, the data is encrypted but it is not decrypted in job, so that if your PR is merged, I can think of there is a lot jobs will not work well, such as install-os.

@yyscamper I cannot agree more, I've been trying to wrap my brain around all possible effected areas. If you can think of any other area's we may need to unencrypt aside from OS installs and maybe ssh command tasks... That would be very helpful... I'm assuming I'll have to post a dependent PR in on-tasks to implement to decrypting stage for each specific job... unless there are any possible layers of abstraction I am unaware of that decryption could reside. There's a part of me that feels we may have bitten off more then we can chew, and may need to revert back to simple redaction at the api layer if this is a blocking issue for any internal group...

please don't merge yet... manual testing has found this new logic seems to miss a rootPassword property in my example, thats found outside of a workflows options... investigating now...

@RackHD/corecommitters ok guys... I think this is it... we made some changes to insure that decryption happens when we actually use it by tasks... and I manually tested that the recursion and regex work. This should be merge ready. Let me know if you have any questions, or concerns!

👍 although you could squash the commit history down a little.

@rolandpoulter understood.

@RackHD/corecommitters Need merge for this one to resolve the P1 ODR for upcoming release.

Looks good to me :+1:


Fix usage in help text

Currently "usage: Maast.py  [options]" but should be "usage: maast   [options]"

Good catch, fixed now.

Not fixed for ant of the individual modules: ./maast end_to_end

This is seemingly due to the default error message of argparse. Not sure if there is a easy way to change it. I changed the name of Maast.py to maast.py to make it less obvious. Let me know please if you have a better way.


Scan inverse datamatrix not working

I'm trying to scan datamatrix code on black background and it not working. I think it need to be reverse. How I can do it?

The image itself needs to be rotated? I think your best option is to ensure that the input image is in the correct orientation when it is going to be scanned. I don't think we have options to tell MLKit that the source image is actually rotated.

The image itself needs to be rotated? I think your best option is to ensure that the input image is in the correct orientation when it is going to be scanned. I don't think we have options to tell MLKit that the source image is actually rotated.

No rotate needed. Inverse colors need only. Black -> white, white -> black. I worked with flutter_zxing it has arguments: tryInverse, need argument like it.

I don't think MLKit exposes such a configuration option. You could try opening a feature proposal at MLKit for this.

I don't think MLKit exposes such a configuration option. You could try opening a feature proposal at MLKit for this.

is there an opportunity to do this from flutter? for example, I will try to invert the colors before transferring the image to MLKit


Issue running ocsmesh.Hfun function

I get an error (pasted below) running any of the code blocks that includes ocsmesh.Hfun function, for example, the following in the toturial:
hfun_obj_4 = ocsmesh.Hfun(rasters[3], hmin=1000, hmax=5000)
hfun_msh_t_4 = hfun_obj_4.msh_t()

And here is the error:
---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
Cell In[28], line 2
      1 hfun_obj_4 = ocsmesh.Hfun(rasters[3], hmin=1000, hmax=5000)
----> 2 hfun_msh_t_4 = hfun_obj_4.msh_t()

File /scratch2/STI/coastal/save/Bahram.Khazaei/env/miniconda3/envs/myenv/lib/python3.10/site-packages/ocsmesh/hfun/raster.py:465, in HfunRaster.msh_t(self, window, marche, verbosity)
    462 if marche is True:
    463     libsaw.marche(opts, hfun)
--> 465 libsaw.jigsaw(opts, geom, window_mesh, hfun=hfun)
    467 del geom
    468 # do post processing

File /scratch2/STI/coastal/save/Bahram.Khazaei/env/miniconda3/envs/myenv/lib/python3.10/site-packages/jigsawpy/libsaw.py:746, in jigsaw(opts, geom, mesh, init, hfun)
    740 retv = JLIB.jigsaw(ct.byref(ojig),
    741                    ct.byref(gmsh),
    742                    iptr, hptr,
    743                    ct.byref(mmsh))
    745 if (retv != +0):
--> 746     raise Exception(
    747         "JIGSAW returned code: " + str(retv))
    749 #--------------------------------- copy buffers to MSH_t
    751 get_msh_t(mesh, mmsh)

Exception: JIGSAW returned code: 4

Thanks for testing @BahramKhazaei-NOAA. Please see my suggestions below:

ocsmesh.Hfun(rasters[3], hmin=1000, hmax=5000)

Can you please try hmin=1000.0 and hmax=5000.0 (with the added .0) and see if it works? If not it might just be a Jigsaw installation issue I guess.

Also, I believe first line in cell 55 of the tutorial should be:

No, this is correct, it is supposed to create a combination geometry from all the 4 raster files. Do you run into an issue if you don't put [1]? The warnings are OK to see. I'll need to address it on OCSMesh side at some point.

Got the same error after adding .0

Thanks for testing @BahramKhazaei-NOAA. Please see my suggestions below:

ocsmesh.Hfun(rasters[3], hmin=1000, hmax=5000)

Can you please try hmin=1000.0 and hmax=5000.0 (with the added .0) and see if it works? If not it might just be a Jigsaw installation issue I guess.

Without the [1] the code runs for hours, and after completion the next code block plots an empty figure.

Also, I believe first line in cell 55 of the tutorial should be:

No, this is correct, it is supposed to create a combination geometry from all the 4 raster files. Do you run into an issue if you don't put [1]? The warnings are OK to see. I'll need to address it on OCSMesh side at some point.

Got the same error after adding .0

Were you able to run any meshing commands using your environment? I think it this fails too, there must be an issue with the jigsaw library installation

Without the [1] the code runs for hours, and after completion the next code block plots an empty figure.

It is also possible sometimes that pygeos package causes an error. In your environment, if it is installed, can you try
conda remove pygeos

and also
conda install -cconda-forge "shapely<2"

Because sometimes the wrong version of these two can result in errors or delays.

OK, thanks! The whole making sure libraries work together thing is why I usually want to avoid too many different dependencies. In this case if Jigsaw was released in PyPI, we wouldn't have had this issue. Please let me know if you notice anything else

In this line rivers_ex4 = geom_poly_5.intersection(gdf_nwm_rivers.unary_union) of Example 4, I got the following error:
organizePolygons() received a polygon with more than 100 parts.  The processing may be really slow.  You can skip the processing by setting METHOD=SKIP.
TopologyException: Input geom 0 is invalid: Holes are nested at -73.946092835981489 40.998176055050735
---------------------------------------------------------------------------
TopologicalError                          Traceback (most recent call last)
Cell In[48], line 2
      1 gdf_nwm_rivers = gpd.read_file(data/'NWM_v2.1_channel_hydrofabric_10262020/nwm_v2_1_hydrofabric.gdb', layer=0) #read conus reaches
----> 2 rivers_ex4 = geom_poly_5.intersection(gdf_nwm_rivers.unary_union)

File /scratch2/STI/coastal/save/Bahram.Khazaei/env/miniconda3/envs/ocsmeshenv/lib/python3.9/site-packages/shapely/geometry/base.py:695, in BaseGeometry.intersection(self, other)
    693 def intersection(self, other):
    694     """Returns the intersection of the geometries"""
--> 695     return geom_factory(self.impl['intersection'](self, other))

File /scratch2/STI/coastal/save/Bahram.Khazaei/env/miniconda3/envs/ocsmeshenv/lib/python3.9/site-packages/shapely/topology.py:73, in BinaryTopologicalOp.__call__(self, this, other, *args)
     70 if product is None:
     71     err = TopologicalError(
     72             "This operation could not be performed. Reason: unknown")
---> 73     self._check_topology(err, this, other)
     74 return product

File /scratch2/STI/coastal/save/Bahram.Khazaei/env/miniconda3/envs/ocsmeshenv/lib/python3.9/site-packages/shapely/topology.py:38, in Delegating._check_topology(self, err, *geoms)
     36 for geom in geoms:
     37     if not geom.is_valid:
---> 38         raise TopologicalError(
     39             "The operation '%s' could not be performed. "
     40             "Likely cause is invalidity of the geometry %s" % (
     41                 self.fn.__name__, repr(geom)))
     42 raise err

TopologicalError: The operation 'GEOSIntersection_r' could not be performed. Likely cause is invalidity of the geometry <shapely.geometry.multipolygon.MultiPolygon object at 0x7f56bc5e0a60>

It also took more than 2 hours for the code to crash.

the following code cell would be data/'NWM_v2.1_channel_hydrofabric_10262020/nwm_v2_1_hydrofabric.gdb'

Thank you. I don't know how to download version 2.0, but on my system when I ran it, I had 2.0, that's why there's this discrepancy. I can add some notes to the notebook about this. Thanks!

In this line rivers_ex4 = geom_poly_5.intersection(gdf_nwm_rivers.unary_union) of Example 4 ...

On you new environment, do you have shapely pre version 2?

It also took more than 2 hours for the code to crash.

This is happening on a compute node on Hera?

Here are some updates for future reference. conda remove pygeos does not work so I removed it with pip uninstall pygeos. After removing pygeos, I got the same error and long runtime for the river intersection. Also, the following code block in tutorial runs very slow (it took more than 6 hours and stopped the code):
geom_rasters = list()
for f in dem_paths:
    geom_rasters.append(ocsmesh.Raster(f))
    geom = ocsmesh.Geom(
        geom_rasters,
        base_shape=base_gs.unary_union,
        base_shape_crs=base_gs.crs,
        zmax=0)

I tried everything with a fresh installation of the package and jigsaw and no pygeos.

Thank you for testing it. I put this on my todo list to try and rerun the notebook.

@BahramKhazaei-NOAA I think I see what is happening with example 4. Please see https://github.com/SorooshMani-NOAA/OCSMeshTutorial/issues/4#issuecomment-1474413935
I still need to look into the second issue you mentioned above

@BahramKhazaei-NOAA the other issue is related to how conda initializes PROJ environment. For some reason it sets the environment to use network, while network is not available. What you need to do to fix this is to modify the file <your_conda_installation_path>/envs/<your_env_name>/etc/conda/activate.d/proj4-activate.sh and set PROJ_NETWORK to OFF. To do that you can just comment the lines:
if [ -f "${CONDA_PREFIX}/share/proj/copyright_and_licenses.csv" ]; then
  # proj-data is installed because its license was copied over
  export PROJ_NETWORK="OFF"
else
  export PROJ_NETWORK="ON"
fi

and add
export PROJ_NETWORK="OFF"

This way every time conda is activated, it will automatically set it to OFF.
You can also manually set PROJ_NETWORK to OFF every time you activate your conda environment. Please let me know if this works for you or not.

@BahramKhazaei-NOAA with the above two comments, I think you should be able to go through all the OCSMesh sections of the tutorial. Please let me know if you see any other issues doing so. Thanks!

@SorooshMani-NOAA do you have a working environment on Hera. I was wondering if I can use your conda env to run the tutorial?
Tagging @yunfangsun for reference here.

Here is the environment I use and get errors:
/scratch2/STI/coastal/save/Bahram.Khazaei/env/miniconda3/envs/ocsmesh

Since this seems to have been an environment related issue, I'll close it. Note that now you can install ocsmesh from conda to avoid issues like this.


docs(readme0): remove readme0

Codecov Report

Merging #71 into master will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff          @@
##           master    #71   +/-   ##
=====================================
  Coverage     100%   100%
=====================================
  Files          16     16
  Lines          50     50
=====================================
  Hits           50     50

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c6999f3...dea0ece. Read the comment docs.


[BUG] Get device list failed / couldn't get HA device

ha-addon version: 1.2.6
Home Assistant Core 2022.5.5
Home Assistant Supervisor 2022.05.3
Home Assistant OS 7.6
Successfully login ewelink account and all the ewelink devices are listed, but not able to get HA device list.

Same here...any news or solutions?

have you been able to sync HA Devices? I'm still having the message "Get device list failed"...


Add regex exclude support and python-inotify ExcludeFilter support.

This prevents subfolders from being watched when they should be excluded. (/var/log, /var/spool, etc.)
I think this solves the running out of inotify watches problem.

Heh, we should talk more, I was working on this as well. Luckily I didn't get very far.
I'll have to look into the exclude_filter feature in pyinotify to make sure it's doing what we expect it to. My route was going to be more along the lines of removing watches based on excludes, but this is much cleaner.

Don't know if this is relevant, but ExcludeFilter has a limitation with files directly below the named file: https://github.com/seb-m/pyinotify/issues/31

Ah, and that explains why we can't rid ourselves of the old exclude behavior, it's required for the files directly under watched directories.

Added a comment inline, otherwise looks perfect.


Deprecate DAQ Compliance App

The original sponsor of the app left DAQ a few years ago. Since then DAQ has developed a Survey123 app and some other AGOL capabilities to replace their need for the compliance application.

Application Name: DAQ Compliance
Application Url: complianceapp.utah.gov (complianceapp.dev.utah.gov)

A short summary of the situation.
(see summary above)
Action items

Assign a person who should complete the task by replacing name with their github @name.
Check [x] the box when the task is completed and add the date of completion.
~Strike~ out all items that do not apply.

~[ ] Update relevant gis.utah.gov application pages (name, completed: 2022/00/00)~
~[ ] Remove forklift pallet (@steveoh , completed: 2022/00/00)~
~[ ] Remove application-specific Databases (-, completed: 2022/00/00)~
[x] AGRC Projects team drive folder (@nathankota, completed: 2022/05/05) - only a notes doc was found and deleted

Is there a website?

[ ] Remove DNS entry (@nathankota , completed: 2022/00/00)
[x] Archive source code repository (@steveoh , completed: 2022/05/05)
~[ ] Remove app-specific quad word from Discover system (-, completed: 2022/00/00)~

Choose one.

[x] Remove from the web server (@steveoh, completed: 2022/05/05)
~[ ] Replace app with a static page with information (name, completed: 2022/00/00)~
~[ ] Redirect somewhere else (name, completed: 2022/00/00)~

Is there a map service?

~[ ] Stop (-, completed: 2022/00/00)~
~[ ] Delete (-, completed: 2022/00/00)~
~[ ] Unregister Data Store in ArcGIS Server (-, completed: 2022/00/00)~

Is there a forklift pallet?

~[ ] Remove repo (-, completed: 2022/00/00)~
~[ ] Remove stale data from forklift hashing and receiving (-, completed: 2022/00/00)~
~[ ] Remove row from data/hashed/changedetection.gdb/TableHashes (-, completed: 2022/00/00)~

Are there service dependencies?

~[ ] Validate that the web api does not query it (-, completed: 2022/00/00)~
~[ ] Validate that AGRC or applications widgets do not reference it (-, completed: 2022/00/00)~

Notification

~[ ] Twitter (@steveoh)~

Group Task Assignments

Check [x] the box when you have assigned all the tasks relevant to your group.

~[ ] Data Team (@gregbunce)~
[x] Dev Team (@steveoh)
~[ ] Cadastre Team (@rkelson)~

I'm done.

INC1648775 for the DNS records.

messaged Lexie to remove agrc_deqedocs_ro user from ITDB222SP

conductor results for tasks - 203

check
status

@nathankota has completed 1 out of 2 tasks
:no_entry:

steveoh has completed 3 out of 3 tasks
:+1:


feat(nrql alert condition)add signal.aggregation_window

Add support for NRQL condition aggregationWindow (how frequently to bucket a NRQL condition query for evaluation)
This is a prerequisite for #932 in the New Relic Terraform Provider

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.Jay McGrath seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.

Thanks @jaymcgrath !


Refining to an array no longer allows me to edit the array using splice

Missing/Incorrect APIs
Before (last version I checked is 0.92.1), it was possible to refine mixed to an array using Array.isArray and edit the array using splice:
const arr : mixed = foo()

if (Array.isArray(x)) {
  const s = arr.splice(0, 3)
}

Before, arr was an Array<mixed> and this would work. But since<PHONE_NUMBER>954e7f66a968af5e68f76a762a9c14, this will no longer work since arr will be a $ReadOnlyArray.
I still think my code should be valid though, since there's no writing of values into the array so I'm not changing the type (which is what<PHONE_NUMBER>954e7f66a968af5e68f76a762a9c14 was designed to tackle).
Maybe there needs to be an intermediate array type that does not allow writing values but does allow for slicing up the array
If not, what would be a good way to work around this? I'm writing code that works with arrays without caring about the values in the array, but that needs to be able to split up the arrays.

Isn't splice mutating method?

@goodmind Yes it is, so I understand it can not be part of $ReadOnlyArray, but the issue here is that refining mixed to an array of mixed is creating a $ReadOnlyArray<mixed>, which prevents some valid programs (like mine) to be type checked.
The reason $ReadOnlyArray<mixed> was chosen in 5639532  is because Flow needs to make sure the type of the underlying array cannot be changed  or invalid programs can be written (see the example in 5639532).
My point is that $ReadOnlyArray<mixed> is too strict to reach that goal, because it misses the slice method which, as I show in my example above, also leaves the type of underlying array untouched.

I think you're probably right in this case that there would need to be an intermediate array type to model this problem, one where you can maybe move elements around and delete them but not add anything.
There's a second problem though, which is with splice. In theory you can use splice to add elements to an array which would be writing its contents and potentially changing its type. I don't think you can currently target a specific class based on function arity with flow, but I think that's what you would want to do, basically break up the splice definition based on arity. If you did introduce a new intermediate array type, splice would still require a full Array I think unless you could target its implementation based on arity.

So it would be something like this?
declare class $MovableArray<+T> extends $ReadOnlyArray<T> {
  copyWithin(target: number, start: number, end?: number): Array<T>;
  every(
    callbackfn: (value: T, index: number, array: $MovableArray<T>) => any,
    thisArg?: any
  ): boolean;
  filter(callbackfn: typeof Boolean): Array<$NonMaybeType<T>>;
  filter(
    callbackfn: (value: T, index: number, array: $MovableArray<T>) => any,
    thisArg?: any
  ): Array<T>;
  find(
    callbackfn: (value: T, index: number, array: $MovableArray<T>) => any,
    thisArg?: any
  ): T | void;
  findIndex(
    callbackfn: (value: T, index: number, array: $MovableArray<T>) => any,
    thisArg?: any
  ): number;
  forEach(
    callbackfn: (value: T, index: number, array: $MovableArray<T>) => any,
    thisArg?: any
  ): void;
  map<U>(
    callbackfn: (value: T, index: number, array: $MovableArray<T>) => U,
    thisArg?: any
  ): Array<U>;
  pop(): T;
  reduce(
    callbackfn: (
      previousValue: T,
      currentValue: T,
      currentIndex: number,
      array: $MovableArray<T>
    ) => T,
    initialValue: void
  ): T;
  reduce<U>(
    callbackfn: (
      previousValue: U,
      currentValue: T,
      currentIndex: number,
      array: $MovableArray<T>
    ) => U,
    initialValue: U
  ): U;
  reduceRight(
    callbackfn: (
      previousValue: T,
      currentValue: T,
      currentIndex: number,
      array: $MovableArray<T>
    ) => T,
    initialValue: void
  ): T;
  reduceRight<U>(
    callbackfn: (
      previousValue: U,
      currentValue: T,
      currentIndex: number,
      array: $MovableArray<T>
    ) => U,
    initialValue: U
  ): U;
  reverse(): Array<T>;
  shift(): T;
  some(
    callbackfn: (value: T, index: number, array: $MovableArray<T>) => any,
    thisArg?: any
  ): boolean;
  sort(compareFn?: (a: T, b: T) => number): Array<T>;
  splice(start: number, deleteCount?: number): Array<T>;
}

Yeah I think so. But @goodmind shouldn't sort and splice return $PermutableArray<T> also since they don't return a new array? Returning Array<T> here would allow the element type of the original array to change, I think.
@romeovs if I were you I would just pull the above type into your project and then cast your $ReadOnlyArray to this $PermutableArray and see how that works out for you, like:
const arr : mixed = foo()

if (Array.isArray(arr)) {
  ((arr: any): $PermutableArray<mixed>);
  const s = arr.splice(0, 3)
}

or maybe wrap it in a helper:
const withPermutation = <+T>(arr: $ReadOnlyArray<T>): $PermutableArray<T> =>
  ((arr: any): $PermutableArray<T>);

(Try Flow)
Actually getting something like this into flow would I think mean changing array refinement to return a $PermutableArray rather than a $ReadOnlyArray which might be a breaking change that would introduce all kinds of subtle errors to existing code. I'm not sure how big a deal it would be, because a $PermutableArray can, of course, cast directly to a $ReadOnlyArray. I don't know enough about how classes work in flow, I guess. Either way, I'm sure adding another basic array type would have pretty wide-ranging implications.

/cc @dsainati1

changing array refinement to return a $PermutableArray

Then any $ReadOnlyArray can be cast to $PermutableArray, would be that safe?
declare var ra: $ReadOnlyArray<string>
const ram: mixed = ra
if (Array.isArray(ram) {
  const pa: $PermutableArray<mixed> = ram
  pa.sort()
}


Inconsitant UI Update

Description:
Clicking the "Mark as Complete" button does not instantly update the sidebar checkbox. The UI inconsistency persists until a manual reload is performed.
Steps to Reproduce:

Click on the "Mark as Complete" button.
Observe that the sidebar checkbox does not update immediately.
Perform a manual reload to see the updated status.

Expected Behavior:
The UI should reflect the updated status of the task in real time when using the "Mark as Complete" button.
Screenshots
After Clicking on Button

After Hard Refresh

Probably can be solved with useOptimistic react hook

I have confidence that I can solve this one. Here's what I learnt from the codebase:-
The inconsistency lies in the face that the sidebar is only rendered once when the whole page loads data from the backend, what's happening right now is that whenever the loadingMarkAs variable(which is responsible for the Mark as Incomplete text to Mark as complete ) is changing, it's not re-rendering the sidebar to also re-render the check icon, meaning, the check mark is probably working independent of the state of the button, and only checks the value when the data is fetched from the database as to whether the course has been marked done or not
what I am going to do is:-
In the parent component of the page that renders both the sidebar and the course video components to that specific page, I will crate a context, which will store both the function and value of that loadingMarkAs variable through useState, and then, make the sidebar re-render the check mark if there is a change in the variable.
I doubt that always fetching to get whether the course is done is not important since it won't create any kind of severe bug. So, having variables take care of this is much better, and only saving the value in the database in case the user arrives back to see which courses have they done.
@hkirat Please let me know if this approach is good and if I can work on this. Or maybe I can create the pull request for you to check if I implemented this correctly?


remove unnecessary use of boolean variables to enable workers

Issue #, if available: #102
Description of changes:
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

e2e tests passed!