Incorrect S3 endpoint

Title: One of AWS Account has an error about incorrect S3 endpoint and crashed
Description:
I have started sync for multiple AWS Accounts according to the documentation of cartography. About ten accounts were finished the sync successful. But the one account has an error on the S3 bucket's endpoint and sync was crashed.
To Reproduce:
cartography --neo4j-uri bolt://localhost:7687 --aws-sync-all-profiles
python3 -v
Python 3.6.9 (default, Jan 26 2021, 15:33:00)
[GCC 8.4.0] on Linux
Modules:
Successfully installed PyJWT-1.7.1 adal-1.2.6 applicationinsights-0.11.9 argcomplete-1.12.2 azure-cli-core-2.20.0 azure-cli-telemetry-1.0.6 azure-common-1.1.26 azure-core-1.12.0 azure-mgmt-compute-19.0.0 azure-mgmt-core-1.2.2 azure-mgmt-resource-16.0.0 bcrypt-3.2.0 boto3-1.17.32 botocore-1.20.32 cartography-0.33.2 certifi-2020.12.5 cffi-1.14.5 chardet-4.0.0 colorama-0.4.4 cryptography-3.3.2 humanfriendly-9.1 idna-2.10 importlib-metadata-3.7.3 isodate-0.6.0 jmespath-0.10.0 jsonpickle-2.0.0 knack-0.8.0rc2 msal-1.10.0 msrest-0.6.21 msrestazure-0.6.4 neo4j-1.7.6 neotime-1.7.4 oauthlib-3.1.0 paramiko-2.7.2 pkginfo-1.7.0 portalocker-1.7.1 psutil-5.8.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.20 pygments-2.8.1 pynacl-1.4.0 pyopenssl-20.0.1 python-dateutil-2.8.1 python-digitalocean-1.16.0 requests-oauthlib-1.3.0 rsa-4.7.2 s3transfer-0.3.6 tabulate-0.8.9 typing-extensions-<IP_ADDRESS>
#cat /etc/os-release
NAME="Ubuntu"
VERSION="18.04.5 LTS (Bionic Beaver)"
Logs:
cartography --neo4j-uri bolt://localhost:7687 --aws-sync-all-profiles
INFO:cartography.sync:Starting sync with update tag '1616134146'
INFO:cartography.sync:Starting sync stage 'create-indexes'
INFO:cartography.intel.create_indexes:Creating indexes for cartography node types.
INFO:cartography.sync:Finishing sync stage 'create-indexes'
INFO:cartography.sync:Starting sync stage 'aws'
WARNING:neo4j:FIXME: should always disconnect before connect
INFO:cartography.intel.aws:Syncing AWS account with ID 'XXXXXXXXXXXX' using configured profile 'MY-product-team'.
INFO:cartography.intel.aws:Trying to autodiscover accounts.
WARNING:cartography.intel.aws:The current account (XXXXXXXXXXXX) doesn't have enough permissions to perform autodiscovery.
INFO:cartography.intel.aws.iam:Syncing IAM for account 'XXXXXXXXXXXX'.
WARNING:neo4j:FIXME: should always disconnect before connect
WARNING:neo4j:FIXME: should always disconnect before connect
INFO:cartography.intel.aws.s3:Syncing S3 for account 'XXXXXXXXXXXX'.
WARNING:neo4j:FIXME: should always disconnect before connect
WARNING:cartography.intel.aws.s3:get_bucket_policy(MY-s3backup-server-ip-172-XX-YY-ZZ.eu-central-1.compute.int) failed because the bucket no longer exists
ERROR:cartography.sync:Unhandled exception during sync stage 'aws'
Traceback (most recent call last):
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/urllib3/connection.py", line 170, in _new_conn
(self._dns_host, self.port), self.timeout, **extra_kw
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/urllib3/util/connection.py", line 73, in create_connection
for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
File "/usr/lib/python3.6/socket.py", line 745, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/botocore/httpsession.py", line 323, in send
chunked=self._chunked(request.headers),
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 756, in urlopen
method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/urllib3/util/retry.py", line 507, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/urllib3/packages/six.py", line 735, in reraise
raise value
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 706, in urlopen
chunked=chunked,
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 382, in _make_request
self._validate_conn(conn)
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 1010, in _validate_conn
conn.connect()
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/urllib3/connection.py", line 353, in connect
conn = self._new_conn()
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/urllib3/connection.py", line 182, in _new_conn
self, "Failed to establish a new connection: %s" % e
urllib3.exceptions.NewConnectionError: <botocore.awsrequest.AWSHTTPSConnection object at 0x7f9f84d5eb70>: Failed to establish a new connection: [Errno -2] Name or service not known
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/cartography/sync.py", line 73, in run
stage_func(neo4j_session, config)
File "/usr/local/lib/python3.6/dist-packages/cartography/util.py", line 66, in timed
return method(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/cartography/intel/aws/init.py", line 201, in start_aws_ingestion
requested_syncs,
File "/usr/local/lib/python3.6/dist-packages/cartography/intel/aws/init.py", line 139, in _sync_multiple_accounts
aws_requested_syncs=aws_requested_syncs,  # Could be replaced later with per-account requested syncs
File "/usr/local/lib/python3.6/dist-packages/cartography/intel/aws/init.py", line 57, in _sync_one_account
RESOURCE_FUNCTIONSfunc_name
File "/usr/local/lib/python3.6/dist-packages/cartography/util.py", line 66, in timed
return method(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/cartography/intel/aws/s3.py", line 397, in sync
load_s3_details(neo4j_session, acl_and_policy_data_iter, current_aws_account_id, update_tag)
File "/usr/local/lib/python3.6/dist-packages/cartography/util.py", line 66, in timed
return method(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/cartography/intel/aws/s3.py", line 194, in load_s3_details
for bucket, acl, policy in s3_details_iter:
File "/usr/local/lib/python3.6/dist-packages/cartography/intel/aws/s3.py", line 64, in get_s3_bucket_details
acl = get_acl(bucket, client)
File "/usr/local/lib/python3.6/dist-packages/cartography/util.py", line 66, in timed
return method(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/cartography/intel/aws/s3.py", line 103, in get_acl
acl = client.get_bucket_acl(Bucket=bucket['Name'])
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/botocore/client.py", line 357, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/botocore/client.py", line 663, in _make_api_call
operation_model, request_dict, request_context)
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/botocore/client.py", line 682, in _make_request
return self._endpoint.make_request(operation_model, request_dict)
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/botocore/endpoint.py", line 102, in make_request
return self._send_request(request_dict, operation_model)
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/botocore/endpoint.py", line 137, in _send_request
success_response, exception):
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/botocore/endpoint.py", line 256, in _needs_retry
caught_exception=caught_exception, request_dict=request_dict)
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/botocore/hooks.py", line 356, in emit
return self._emitter.emit(aliased_event_name, **kwargs)
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/botocore/hooks.py", line 228, in emit
return self._emit(event_name, kwargs)
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/botocore/hooks.py", line 211, in _emit
response = handler(**kwargs)
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/botocore/retryhandler.py", line 183, in call
if self._checker(attempts, response, caught_exception):
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/botocore/retryhandler.py", line 251, in call
caught_exception)
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/botocore/retryhandler.py", line 277, in _should_retry
return self._checker(attempt_number, response, caught_exception)
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/botocore/retryhandler.py", line 317, in call
caught_exception)
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/botocore/retryhandler.py", line 223, in call
attempt_number, caught_exception)
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/botocore/retryhandler.py", line 359, in _check_caught_exception
raise caught_exception
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/botocore/endpoint.py", line 200, in _do_get_response
http_response = self._send(request)
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/botocore/endpoint.py", line 269, in _send
return self.http_session.send(request)
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/botocore/httpsession.py", line 343, in send
raise EndpointConnectionError(endpoint_url=request.url, error=e)
botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: "https://YYYYYsite.s3.EU.amazonaws.com/?acl"
Traceback (most recent call last):
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/urllib3/connection.py", line 170, in _new_conn
(self._dns_host, self.port), self.timeout, **extra_kw
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/urllib3/util/connection.py", line 73, in create_connection
for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
File "/usr/lib/python3.6/socket.py", line 745, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/botocore/httpsession.py", line 323, in send
chunked=self._chunked(request.headers),
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 756, in urlopen
method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/urllib3/util/retry.py", line 507, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/urllib3/packages/six.py", line 735, in reraise
raise value
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 706, in urlopen
chunked=chunked,
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 382, in _make_request
self._validate_conn(conn)
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 1010, in _validate_conn
conn.connect()
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/urllib3/connection.py", line 353, in connect
conn = self._new_conn()
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/urllib3/connection.py", line 182, in _new_conn
self, "Failed to establish a new connection: %s" % e
urllib3.exceptions.NewConnectionError: <botocore.awsrequest.AWSHTTPSConnection object at 0x7f9f84d5eb70>: Failed to establish a new connection: [Errno -2] Name or service not known
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/bin/cartography", line 11, in
sys.exit(main())
File "/usr/local/lib/python3.6/dist-packages/cartography/cli.py", line 441, in main
return CLI(default_sync, prog='cartography').main(argv)
File "/usr/local/lib/python3.6/dist-packages/cartography/cli.py", line 421, in main
return cartography.sync.run_with_config(self.sync, config)
File "/usr/local/lib/python3.6/dist-packages/cartography/sync.py", line 148, in run_with_config
return sync.run(neo4j_driver, config)
File "/usr/local/lib/python3.6/dist-packages/cartography/sync.py", line 73, in run
stage_func(neo4j_session, config)
File "/usr/local/lib/python3.6/dist-packages/cartography/util.py", line 66, in timed
return method(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/cartography/intel/aws/init.py", line 201, in start_aws_ingestion
requested_syncs,
File "/usr/local/lib/python3.6/dist-packages/cartography/intel/aws/init.py", line 139, in _sync_multiple_accounts
aws_requested_syncs=aws_requested_syncs,  # Could be replaced later with per-account requested syncs
File "/usr/local/lib/python3.6/dist-packages/cartography/intel/aws/init.py", line 57, in _sync_one_account
RESOURCE_FUNCTIONSfunc_name
File "/usr/local/lib/python3.6/dist-packages/cartography/util.py", line 66, in timed
return method(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/cartography/intel/aws/s3.py", line 397, in sync
load_s3_details(neo4j_session, acl_and_policy_data_iter, current_aws_account_id, update_tag)
File "/usr/local/lib/python3.6/dist-packages/cartography/util.py", line 66, in timed
return method(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/cartography/intel/aws/s3.py", line 194, in load_s3_details
for bucket, acl, policy in s3_details_iter:
File "/usr/local/lib/python3.6/dist-packages/cartography/intel/aws/s3.py", line 64, in get_s3_bucket_details
acl = get_acl(bucket, client)
File "/usr/local/lib/python3.6/dist-packages/cartography/util.py", line 66, in timed
return method(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/cartography/intel/aws/s3.py", line 103, in get_acl
acl = client.get_bucket_acl(Bucket=bucket['Name'])
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/botocore/client.py", line 357, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/botocore/client.py", line 663, in _make_api_call
operation_model, request_dict, request_context)
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/botocore/client.py", line 682, in _make_request
return self._endpoint.make_request(operation_model, request_dict)
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/botocore/endpoint.py", line 102, in make_request
return self._send_request(request_dict, operation_model)
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/botocore/endpoint.py", line 137, in _send_request
success_response, exception):
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/botocore/endpoint.py", line 256, in _needs_retry
caught_exception=caught_exception, request_dict=request_dict)
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/botocore/hooks.py", line 356, in emit
return self._emitter.emit(aliased_event_name, **kwargs)
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/botocore/hooks.py", line 228, in emit
return self._emit(event_name, kwargs)
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/botocore/hooks.py", line 211, in _emit
response = handler(**kwargs)
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/botocore/retryhandler.py", line 183, in call
if self._checker(attempts, response, caught_exception):
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/botocore/retryhandler.py", line 251, in call
caught_exception)
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/botocore/retryhandler.py", line 277, in _should_retry
return self._checker(attempt_number, response, caught_exception)
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/botocore/retryhandler.py", line 317, in call
caught_exception)
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/botocore/retryhandler.py", line 223, in call
attempt_number, caught_exception)
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/botocore/retryhandler.py", line 359, in _check_caught_exception
raise caught_exception
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/botocore/endpoint.py", line 200, in _do_get_response
http_response = self._send(request)
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/botocore/endpoint.py", line 269, in _send
return self.http_session.send(request)
File "/var/lib/neo4j/.local/lib/python3.6/site-packages/botocore/httpsession.py", line 343, in send
raise EndpointConnectionError(endpoint_url=request.url, error=e)
botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: "https://YYYYYsite.s3.EU.amazonaws.com/?acl"
Additional context:

I have the S3 bucket YYYYYsite in the eu-west-1 region.

Getting exactly same issue. And i have got the same issue on multiple AWS accounts. Why it cant handle the exception and move on with other resources?
cartography was looking interesting but if it keeps breaking like that, it loses the reliability.

Hey @copoka5, haven't seen an EndpointConnectionError before. Few follow-up questions:

Does this happen every time when hitting that specific bucket?
Is cartography able to sync any buckets from that AWS account at all?
Can you confirm that the role you are running cartography with has at minimum the SecurityAudit policy in AWS on that target account?

Same issue and yes it is happening on same exact bucket.
`RROR:cartography.sync:Unhandled exception during sync stage 'aws'
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 170, in _new_conn
(self._dns_host, self.port), self.timeout, **extra_kw
File "/usr/local/lib/python3.7/site-packages/urllib3/util/connection.py", line 73, in create_connection
for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
File "/usr/lib64/python3.7/socket.py", line 752, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/botocore/httpsession.py", line 320, in send
chunked=self._chunked(request.headers),
File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 756, in urlopen
method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
File "/usr/local/lib/python3.7/site-packages/urllib3/util/retry.py", line 506, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/usr/local/lib/python3.7/site-packages/urllib3/packages/six.py", line 735, in reraise
raise value
File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 706, in urlopen
chunked=chunked,
File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 382, in _make_request
self._validate_conn(conn)
File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 1010, in _validate_conn
conn.connect()
File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 353, in connect
conn = self._new_conn()
File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 182, in _new_conn
self, "Failed to establish a new connection: %s" % e
urllib3.exceptions.NewConnectionError: <botocore.awsrequest.AWSHTTPSConnection object at 0x7ff123ec1a10>: Failed to establish a new connection: [Errno -2] Name or service not known
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/cartography/sync.py", line 71, in run
stage_func(neo4j_session, config)
File "/usr/local/lib/python3.7/site-packages/cartography/util.py", line 63, in timed
return method(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/cartography/intel/aws/init.py", line 150, in start_aws_ingestion
_sync_multiple_accounts(neo4j_session, aws_accounts, config.update_tag, common_job_parameters)
File "/usr/local/lib/python3.7/site-packages/cartography/intel/aws/init.py", line 98, in _sync_multiple_accounts
_sync_one_account(neo4j_session, boto3_session, account_id, sync_tag, common_job_parameters)
File "/usr/local/lib/python3.7/site-packages/cartography/intel/aws/init.py", line 29, in _sync_one_account
s3.sync(neo4j_session, boto3_session, account_id, sync_tag, common_job_parameters)
File "/usr/local/lib/python3.7/site-packages/cartography/util.py", line 63, in timed
return method(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/cartography/intel/aws/s3.py", line 383, in sync
load_s3_details(neo4j_session, acl_and_policy_data_iter, current_aws_account_id, aws_update_tag)
File "/usr/local/lib/python3.7/site-packages/cartography/util.py", line 63, in timed
return method(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/cartography/intel/aws/s3.py", line 183, in load_s3_details
for bucket, acl, policy in s3_details_iter:
File "/usr/local/lib/python3.7/site-packages/cartography/intel/aws/s3.py", line 56, in get_s3_bucket_details
acl = get_acl(bucket, client)
File "/usr/local/lib/python3.7/site-packages/cartography/util.py", line 63, in timed
return method(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/cartography/intel/aws/s3.py", line 95, in get_acl
acl = client.get_bucket_acl(Bucket=bucket['Name'])
File "/usr/local/lib/python3.7/site-packages/botocore/client.py", line 357, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/usr/local/lib/python3.7/site-packages/botocore/client.py", line 663, in _make_api_call
operation_model, request_dict, request_context)
File "/usr/local/lib/python3.7/site-packages/botocore/client.py", line 682, in _make_request
return self._endpoint.make_request(operation_model, request_dict)
File "/usr/local/lib/python3.7/site-packages/botocore/endpoint.py", line 102, in make_request
return self._send_request(request_dict, operation_model)
File "/usr/local/lib/python3.7/site-packages/botocore/endpoint.py", line 137, in _send_request
success_response, exception):
File "/usr/local/lib/python3.7/site-packages/botocore/endpoint.py", line 256, in _needs_retry
caught_exception=caught_exception, request_dict=request_dict)
File "/usr/local/lib/python3.7/site-packages/botocore/hooks.py", line 356, in emit
return self._emitter.emit(aliased_event_name, **kwargs)
File "/usr/local/lib/python3.7/site-packages/botocore/hooks.py", line 228, in emit
return self._emit(event_name, kwargs)
File "/usr/local/lib/python3.7/site-packages/botocore/hooks.py", line 211, in _emit
response = handler(**kwargs)
File "/usr/local/lib/python3.7/site-packages/botocore/retryhandler.py", line 183, in call
if self._checker(attempts, response, caught_exception):
File "/usr/local/lib/python3.7/site-packages/botocore/retryhandler.py", line 251, in call
caught_exception)
File "/usr/local/lib/python3.7/site-packages/botocore/retryhandler.py", line 277, in _should_retry
return self._checker(attempt_number, response, caught_exception)
File "/usr/local/lib/python3.7/site-packages/botocore/retryhandler.py", line 317, in call
caught_exception)
File "/usr/local/lib/python3.7/site-packages/botocore/retryhandler.py", line 223, in call
attempt_number, caught_exception)
File "/usr/local/lib/python3.7/site-packages/botocore/retryhandler.py", line 359, in _check_caught_exception
raise caught_exception
File "/usr/local/lib/python3.7/site-packages/botocore/endpoint.py", line 200, in _do_get_response
http_response = self._send(request)
File "/usr/local/lib/python3.7/site-packages/botocore/endpoint.py", line 269, in _send
return self.http_session.send(request)
File "/usr/local/lib/python3.7/site-packages/botocore/httpsession.py", line 340, in send
raise EndpointConnectionError(endpoint_url=request.url, error=e)
botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: "https://SAME-BUCKET-eu-west-1.s3.EU.amazonaws.com/?acl"
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 170, in _new_conn
(self._dns_host, self.port), self.timeout, **extra_kw
File "/usr/local/lib/python3.7/site-packages/urllib3/util/connection.py", line 73, in create_connection
for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
File "/usr/lib64/python3.7/socket.py", line 752, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/botocore/httpsession.py", line 320, in send
chunked=self._chunked(request.headers),
File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 756, in urlopen
method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
File "/usr/local/lib/python3.7/site-packages/urllib3/util/retry.py", line 506, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/usr/local/lib/python3.7/site-packages/urllib3/packages/six.py", line 735, in reraise
raise value
File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 706, in urlopen
chunked=chunked,
File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 382, in _make_request
self._validate_conn(conn)
File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 1010, in _validate_conn
conn.connect()
File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 353, in connect
conn = self._new_conn()
File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 182, in _new_conn
self, "Failed to establish a new connection: %s" % e
urllib3.exceptions.NewConnectionError: <botocore.awsrequest.AWSHTTPSConnection object at 0x7ff123ec1a10>: Failed to establish a new connection: [Errno -2] Name or service not known
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/bin/cartography", line 33, in
sys.exit(load_entry_point('cartography==0.31.0', 'console_scripts', 'cartography')())
File "/usr/local/lib/python3.7/site-packages/cartography/cli.py", line 348, in main
return CLI(default_sync, prog='cartography').main(argv)
File "/usr/local/lib/python3.7/site-packages/cartography/cli.py", line 328, in main
return cartography.sync.run_with_config(self.sync, config)
File "/usr/local/lib/python3.7/site-packages/cartography/sync.py", line 145, in run_with_config
return sync.run(neo4j_driver, config)
File "/usr/local/lib/python3.7/site-packages/cartography/sync.py", line 71, in run
stage_func(neo4j_session, config)
File "/usr/local/lib/python3.7/site-packages/cartography/util.py", line 63, in timed
return method(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/cartography/intel/aws/init.py", line 150, in start_aws_ingestion
_sync_multiple_accounts(neo4j_session, aws_accounts, config.update_tag, common_job_parameters)
File "/usr/local/lib/python3.7/site-packages/cartography/intel/aws/init.py", line 98, in _sync_multiple_accounts
_sync_one_account(neo4j_session, boto3_session, account_id, sync_tag, common_job_parameters)
File "/usr/local/lib/python3.7/site-packages/cartography/intel/aws/init.py", line 29, in _sync_one_account
s3.sync(neo4j_session, boto3_session, account_id, sync_tag, common_job_parameters)
File "/usr/local/lib/python3.7/site-packages/cartography/util.py", line 63, in timed
return method(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/cartography/intel/aws/s3.py", line 383, in sync
load_s3_details(neo4j_session, acl_and_policy_data_iter, current_aws_account_id, aws_update_tag)
File "/usr/local/lib/python3.7/site-packages/cartography/util.py", line 63, in timed
return method(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/cartography/intel/aws/s3.py", line 183, in load_s3_details
for bucket, acl, policy in s3_details_iter:
File "/usr/local/lib/python3.7/site-packages/cartography/intel/aws/s3.py", line 56, in get_s3_bucket_details
acl = get_acl(bucket, client)
File "/usr/local/lib/python3.7/site-packages/cartography/util.py", line 63, in timed
return method(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/cartography/intel/aws/s3.py", line 95, in get_acl
acl = client.get_bucket_acl(Bucket=bucket['Name'])
File "/usr/local/lib/python3.7/site-packages/botocore/client.py", line 357, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/usr/local/lib/python3.7/site-packages/botocore/client.py", line 663, in _make_api_call
operation_model, request_dict, request_context)
File "/usr/local/lib/python3.7/site-packages/botocore/client.py", line 682, in _make_request
return self._endpoint.make_request(operation_model, request_dict)
File "/usr/local/lib/python3.7/site-packages/botocore/endpoint.py", line 102, in make_request
return self._send_request(request_dict, operation_model)
File "/usr/local/lib/python3.7/site-packages/botocore/endpoint.py", line 137, in _send_request
success_response, exception):
File "/usr/local/lib/python3.7/site-packages/botocore/endpoint.py", line 256, in _needs_retry
caught_exception=caught_exception, request_dict=request_dict)
File "/usr/local/lib/python3.7/site-packages/botocore/hooks.py", line 356, in emit
return self._emitter.emit(aliased_event_name, **kwargs)
File "/usr/local/lib/python3.7/site-packages/botocore/hooks.py", line 228, in emit
return self._emit(event_name, kwargs)
File "/usr/local/lib/python3.7/site-packages/botocore/hooks.py", line 211, in _emit
response = handler(**kwargs)
File "/usr/local/lib/python3.7/site-packages/botocore/retryhandler.py", line 183, in call
if self._checker(attempts, response, caught_exception):
File "/usr/local/lib/python3.7/site-packages/botocore/retryhandler.py", line 251, in call
caught_exception)
File "/usr/local/lib/python3.7/site-packages/botocore/retryhandler.py", line 277, in _should_retry
return self._checker(attempt_number, response, caught_exception)
File "/usr/local/lib/python3.7/site-packages/botocore/retryhandler.py", line 317, in call
caught_exception)
File "/usr/local/lib/python3.7/site-packages/botocore/retryhandler.py", line 223, in call
attempt_number, caught_exception)
File "/usr/local/lib/python3.7/site-packages/botocore/retryhandler.py", line 359, in _check_caught_exception
raise caught_exception
File "/usr/local/lib/python3.7/site-packages/botocore/endpoint.py", line 200, in _do_get_response
http_response = self._send(request)
File "/usr/local/lib/python3.7/site-packages/botocore/endpoint.py", line 269, in _send
return self.http_session.send(request)
File "/usr/local/lib/python3.7/site-packages/botocore/httpsession.py", line 340, in send
raise EndpointConnectionError(endpoint_url=request.url, error=e)
botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: "https://SAME-BUCKET-eu-west-1.s3.EU.amazonaws.com/?acl"`

The bucket exist, what can you suggest i can check on the bucket?

Thanks alex, you have added hot fix to avoid the issue but its now gonna ignore the bucket and move on. I quickly fired below command on the same instance and it gave me BucketACL for the bucket in error.
aws s3api get-bucket-acl --bucket

I have seen a pattern here and the code is failing on buckets created for cloudfomation templates. Same type of buckets in two different accounts. Naming like: cf-templates-ggt56ss1r-eu-west-1
I even tried executing same piece of code (just extracted the code and hard coded the Bucket name) as above which @achantavy provided fix on the same instance where Cartography SYNC failed and it worked with no issues and printed the ACL.
This is now confusing as why during SYNC failed.

As said above, copied the same function as modified in the PR.
Hard coded the bucket name
executed on same instance where SYNC failed.
env is same so IAM role, OS, python version, cartography version all are same

import boto3
from botocore.exceptions import ClientError

def test():
    boto3_session = boto3.session.Session(profile_name="default")
    client = boto3_session.client('s3')
    """
       Gets the S3 bucket ACL. Returns ACL string
       """
    try:
        bucket_name  = "cf-templates-idd4333gghhyy1r-eu-west-1"
        acl = client.get_bucket_acl(Bucket=bucket_name)
    except ClientError as e:
        if "AccessDenied" in e.args[0]:
            print("Failed to retrieve S3 bucket {} ACL - Access Denied".format(bucket_name))
            return None
        elif "NoSuchBucket" in e.args[0]:
            print("Failed to retrieve S3 bucket {} ACL - No Such Bucket".format(bucket_name))
            return None
        elif "AllAccessDisabled" in e.args[0]:
            print("Failed to retrieve S3 bucket {} ACL - Bucket is disabled".format(
                bucket_name))
            return None
        elif "EndpointConnectionError" in e.args[0]:
            print("Failed to retrieve S3 bucket {} ACL - EndpointConnectionError".format(
                bucket_name))
            return None
        else:
            raise
    print("ACL: ", acl)

if __name__ == "__main__":
    test()

Response:
{
   "ResponseMetadata":{
      "RequestId":"HBDC0B169HC0SMYN",
      "HostId":"2vr/FTbdMppIUlQu0Dam3sl9PCkIAkceRx5DEjBGSopWDck=",
      "HTTPStatusCode":200,
      "HTTPHeaders":{
         "x-amz-request-id":"HBDC0B169HC0SMYN",
         "date":"Fri, 26 Mar 2021 16:56:44 GMT",
         "content-type":"application/xml",
         "transfer-encoding":"chunked",
         "server":"AmazonS3"
      },
      "RetryAttempts":0
   },
   "Owner":{
      "DisplayName":"aws-account",
      "ID":"33cae4189491afe8a1ad52aaa140a24505"
   },
   "Grants":[
      {
         "Grantee":{
            "DisplayName":"aws-account",
            "ID":"33d6dd9c491afe8a1ad52aaa140a505",
            "Type":"CanonicalUser"
         },
         "Permission":"FULL_CONTROL"
      }
   ]
}

@sing22 - This is really weird. I have several cf-* buckets in my inventory but have never seen this problem and am unable to repro :\

@achantavy - The release 0.34.1 didn't fix this one. Got the same error again. I can see the exception block on s3.py but somehow it didnt go in that and still failed. It's surprising.
  File "/usr/local/lib/python3.7/site-packages/cartography/util.py", line 68, in timed
    return method(*args, **kwargs)
  **File "/usr/local/lib/python3.7/site-packages/cartography/intel/aws/s3.py", line 103, in get_acl
    acl = client.get_bucket_acl(Bucket=bucket['Name'])**
  File "/usr/local/lib/python3.7/site-packages/botocore/client.py", line 357, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/lib/python3.7/site-packages/botocore/client.py", line 663, in _make_api_call
    operation_model, request_dict, request_context)
  File "/usr/local/lib/python3.7/site-packages/botocore/client.py", line 682, in _make_request
    return self._endpoint.make_request(operation_model, request_dict)
  File "/usr/local/lib/python3.7/site-packages/botocore/endpoint.py", line 102, in make_request
    return self._send_request(request_dict, operation_model)
  File "/usr/local/lib/python3.7/site-packages/botocore/endpoint.py", line 137, in _send_request
    success_response, exception):
  File "/usr/local/lib/python3.7/site-packages/botocore/endpoint.py", line 256, in _needs_retry
    caught_exception=caught_exception, request_dict=request_dict)
  File "/usr/local/lib/python3.7/site-packages/botocore/hooks.py", line 356, in emit
    return self._emitter.emit(aliased_event_name, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/botocore/hooks.py", line 228, in emit
    return self._emit(event_name, kwargs)
  File "/usr/local/lib/python3.7/site-packages/botocore/hooks.py", line 211, in _emit
    response = handler(**kwargs)
  File "/usr/local/lib/python3.7/site-packages/botocore/retryhandler.py", line 183, in __call__
    if self._checker(attempts, response, caught_exception):
  File "/usr/local/lib/python3.7/site-packages/botocore/retryhandler.py", line 251, in __call__
    caught_exception)
  File "/usr/local/lib/python3.7/site-packages/botocore/retryhandler.py", line 277, in _should_retry
    return self._checker(attempt_number, response, caught_exception)
  File "/usr/local/lib/python3.7/site-packages/botocore/retryhandler.py", line 317, in __call__
    caught_exception)
  File "/usr/local/lib/python3.7/site-packages/botocore/retryhandler.py", line 223, in __call__
    attempt_number, caught_exception)
  File "/usr/local/lib/python3.7/site-packages/botocore/retryhandler.py", line 359, in _check_caught_exception
    raise caught_exception
  File "/usr/local/lib/python3.7/site-packages/botocore/endpoint.py", line 200, in _do_get_response
    http_response = self._send(request)
  File "/usr/local/lib/python3.7/site-packages/botocore/endpoint.py", line 269, in _send
    return self.http_session.send(request)
  File "/usr/local/lib/python3.7/site-packages/botocore/httpsession.py", line 343, in send
    raise EndpointConnectionError(endpoint_url=request.url, error=e)
botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: "https://cf-templates-p3333dddd0edd3-eu-west-1.s3.EU.amazonaws.com/?acl"

Hi @copoka5, haven't heard from you in a while so I'm going to mark as closed. Please feel free to reopen though - hopefully we can help with your issue.


Fixed #1276

Summary
Fixed [BUG] Invalid regular expression: /^[ -íŸ¿î€€-ï¿½ð€€-ô¿¿]$/: Range out of order in character class.
This was due to terser wrongly converting characters inside string literals.
Ref: https://github.com/terser/terser/issues/410
Test plan
Tried new min js. Works fine. No such issue exists.

"I've created a web page using exceljs, and it works fine in Korea, China, and Europe, but the same error occurs only in the US: exceljs.min.js:24 Uncaught SyntaxError: Invalid regular expression:." how can i solve?


WIP: Add initial version of Marshmallow implementation

packit-service/packit#635

handled in #320


Switch to C

Could you use the original C version of N-API : https://nodejs.org/api/n-api.html
Example: https://github.com/drom/node-iio/blob/master/src/iio.c

I can switch to C but the verilator testbench framework is in c++.


Replace deprecated CLIError with new error types

All references to of the the deprecated error type CLIError have been replaced with the new error types described in Azure CLI Error Handling Guidelines

This checklist is used to make sure that common guidelines for a pull request are followed.


Set maxLength attribute for username and password for input validation

No max length was set on Username and password, limit is recommended simply to avoid exhausting resources on server and would be an easy target for Denial Of Service attack. So for security purpose I have set the maxLength for username is 32 and password of 16.

Is there a limit on username and password in the st2 login command? Or rbac?
If there is then we should get these to match or impose limits on st2 as well - if possible.

I often use passwords over 40 characters long. I know others that use full sentences as passwords. I find websites that limit passwords length extremely irritating. Why limit this to 32? If there must be a limit then choose something much longer, like 100 or 120 for both password and username.

TBH, I don't really think frontend level validation gives us some level of protection against the attacks, especially DoS.
I'd leave it as is with no change, I agree with @cognifloyd it looks like a user annoying practice.

We can agree on a length and make a change. I think the intent of this exercise (one of many to come) is to put a max length on all form inputs.  So if there's need to be 100 chars for password, then let's do that.

On previous comments, then the suggestion was to also increase username length as well.
If we do put in a limit, can we raise an issue against ST2 core? As it will be annoying if a user can login via the CLI but not via the web, so if imposing limits on the Web UI for username/password, we should do the same on the CLI.

@amanda11 Yes, please raise an issue on st2 core so this is at least known. There's going to be some challenges there because the st2auth backends are not managed by st2 (i.e. htpasswd, LDAP, etc.).  If those auth providers allow longer usernames and passwords, there's not much we can do other than throw an error that says username/passwords is beyond supported length when user login via the CLI.

@pradnya-orchestral As commented by other reviewers, can you also increase username length to 120, the same as password?


Improved chain page Open graph metadata + image

Problem solved
Short description of the bug fixed or feature added

PR-Codex overview
This PR focuses on enhancing the Open Graph metadata and improving the layout of the chainPage component in the dashboard application.
Detailed summary

In layout.tsx, added an openGraph object containing title and description.
In opengraph-image.tsx, added the objectFit property with the value "contain" for image styling.

✨ Ask PR-Codex anything about this PR by commenting with /codex {your question}

#5081  👈
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @MananTank and the rest of your teammates on  Graphite


Fix to how |bash= expressions are parsed when path contains whitespace

I wanted to run the jtokoph's Spotify plugin, but unfortunately it didn't work when the plugin was in a directory with whitespace. I decided it would be too much work to rename the directory, so I spent a few orders of magnitude more time editing the code. As far as I know this doesn't break anything (are there ever legitimate words after the pipe that don't have an equal sign?), so hopefully this fix will save other folks more time than it saved me.

OK, I'm getting a t-shirt with this on it: "I decided it would be too much work to rename the directory, so I spent a few orders of magnitude more time editing the code."
Would you be interested in contributing to the project? I'm considering a Swift rewrite.

Sure, I haven't written any Swift before so it'd be fun to have a reason to learn some


Switch to 64 bit field elements for data availability checks

New CPUs have special instructions that can do 64 bit binary field operations efficiently (the main requirement for this is to have a carryless multiplication). This gets rid of the logarithm lookup tables to compute field multiplications. This benchmark written by @kilic and @saitima shows that this is 4x faster than the 16 bit lookup table approach (it can do a 64 bit operation in the same time it takes to do 16 bit operations using lookup tables)
https://github.com/kilic/gf
This indicates we should switch to 64 bit field elements to improve performance.

I feel like this decision depends critically on whether we end up using fraud proofs or Kate commitments or STARKs. If we intend to rule out Kate commitments, then it's certainly reasonable.

Hmm, wasn't our current roadmap on this that even if we're going to use Kate for state, it would still be more efficient to do data availability using the old scheme (since Kate doesn't save that much, needing a sqrt commitment anyway, and computing all the proofs is time consuming)?


Ao/tech shares to one

I think we are still missing the new calibrate_current_pop and calibrate_urban_rural_split (calibrate_urban_rural_split is the new name of calibrate_urban_current_and_future_GHS). Other than that I think it looks good!


Experimental support for sparse matrices

Edit: The second commit implements experimental sparse matrix support. Closes #178.
The code should be reviewed, tested and refactored. In particular, mixing dense and sparse matrices should be handled. It might be cleaner to create a new sparse_mask_job job type and forbid mixing? @sk1p how would you handle this?
As a side note, sparse matrices can be constructed correctly from dense matrices as well, it is just not efficient if the matrix is actually dense.

Was:
This commit adds very basic support for sparse matrices
The modification of _make_mask_slicer() allows handling sparse matrices
since scipy.sparse matrices allow reshaping into matrices with (-1, 1),
not into vectors with (-1, ), and they support hstack() and vstack(),
but not stack(). The result of the function is the same.
The computation itself doesn't work yet for reasons that have to be
investigated. The test for sparse matrices allows to handle the situation
gracefully with a NotImplementedError instead of crashing.
The example currently triggers this error.

Added a comment on why there's so much transpose going on

Changing the point analysis to sparse matrix gives a substantial speedup on my laptop. To be tested on Linux and on systems with more RAM and CPU.


Adding this commando project.

I’ve been unable to add this to my bot using the commando framework, is this possible or am I doing something wrong?

Hello, I was not aware of the existence of this project, so i have not tested it yet.
But I will check and get back to you soon!

I'm back to you with informations, @GDWingWangWong. Even if the support is not native, you can easily add your custom Commando command to start a TicTacToe game. Here is a sample of what I have done to make it works:
In your main file:
const path = require('path');
const Commando = require('discord.js-commando');

const client = new Commando.Client({
    owner: '...'
});

client.registry
    .registerDefaults()
    .registerCommandsIn(path.join(__dirname, 'commands'));

client.login('MY_BOT_TOKEN');

In the file commands/game/tictactoe.js:
const Command = require("discord.js-commando").Command;
const TicTacToe = require('discord-tictactoe');

module.exports = class TicTacToeCommand extends Command {
    constructor(client) {
        super(client, {
            name: 'tictactoe',
            group: 'commands',
            memberName: 'tictactoe',
            aliases: ['ttt'],
            description: 'Start a tictactoe game'
        })

        this.game = new TicTacToe();
    }

    async run(message) {
        this.game.handleMessage(message);
    }
}

You just have to create a TicTacToe instance (you can pass an object with options, see more info here) and use the method handleMessage(message) in the run(message) method. Is it ok for you?

It works well! Thanks.


Proxy error

Hello i have proxy errors but i have the fomrat ip:port do i have to use another format or is it just not working anymore?
eGen>> Something is wrong :(
eGen>> <IP_ADDRESS>:9026
eGen>> Something is wrong :(
eGen>> <IP_ADDRESS>:8773
eGen>> Something is wrong :(
eGen>> <IP_ADDRESS>:7114
eGen>> Something is wrong :(
eGen>> <IP_ADDRESS>:8176
eGen>> Something is wrong :(
eGen>> <IP_ADDRESS>:5111
eGen>> Something is wrong :(
eGen>> <IP_ADDRESS>:7259
eGen>> Something is wrong :(
eGen>> <IP_ADDRESS>:7681
eGen>> Something is wrong :(
eGen>> <IP_ADDRESS>:6461
eGen>> Something is wrong :(
eGen>> <IP_ADDRESS>:7259
eGen>> Something is wrong :(
eGen>> <IP_ADDRESS>:8809
eGen>> Something is wrong :(
eGen>> <IP_ADDRESS>:8593
eGen>> Something is wrong :(
eGen>> <IP_ADDRESS>:7159
eGen>> Something is wrong :(
eGen>> <IP_ADDRESS>:9026

proxy problem
try with http https proxy

It is http

My Proxys work for sure because i have used some other tools too worked fine

no, its okey i test it

Whats the problem then

No errors


[Chore] Fix compatibility for dbt 1.6.2

PR checklist

[x] Ensure you have added or ran the appropriate tests for your PR.
[x] DCO signed

What type of PR is this?
chore
What this PR does / why we need it:

dbt-core 1.6.2 is released and it caused a compatibility issue.

Which issue(s) this PR fixes:
sc-32254
Special notes for your reviewer:
Does this PR introduce a user-facing change?:

NONE

Duplicated #884 884


Support wildcard paths

Support paths in the format of /account/*path
The generated Play route would look something like
   GET      /accounts/account/*path              controllers.Accounts.getAccountByPath(path: String)

I don't think this is RESTful though, so perhaps it's inappropriate.

curious - you need to do this in the rest api or the web app server itself? (personally find I do this sort of thing in web app, but then turn into specific rest calls to api server)


Support for Keras LSTM/CuDNNLSTM Layer?

I saw in this issue that Keras is not fully supported yet. Do any of you know when this feature may be available? If not, may you point me to a resource that shows a LSTM implementation using the default tensorflow api? The following link is the training python script for my existing model:  https://github.com/jingle1000/Natural-Language-Processing/blob/master/MLPython/train.py I'm not sure if there is an easy way to transfer that model to this framework or not.

This example uses Keras interface, unfortunately we have't implemented Keras yet.


[BUG] - typescript type inference is not working in menu component

NextUI Version
2.0.12
Describe the bug
import { Menu, Item as MenuItem } from 'react-aria-components'
import { Menu as Menu2, MenuItem as MenuItem2 } from '@nextui-org/react'

const Foo = () =>  {
  const TestA = <Menu aria-label='menu' items={menuItems}>
        {item => {
            return <MenuItem key={item.name}>{item.name}</MenuItem>
        }}
    </Menu>
    const TestB = <Menu2 aria-label='menu' items={menuItems}>
        {item => <MenuItem2 key={item.name}>{item.name} next</MenuItem2>}
    </Menu2>
   return <div>{TestA}{TestB}</div>
}

Your Example Website or App
No response
Steps to Reproduce the Bug or Issue
by comparison in vscode editor, you can see clearly the type inference is not working in nextui/menu component, however, it's working fine in react-aria-components/menu.
Expected behavior
type inference works fine
Screenshots or Videos

can also find same problem in test code, use any for type inference

Operating System Version
Windows
Browser
Chrome

I tried to fix this issue, but it seems too complicated.


WIP: Reimbursement fix bugs #2891

https://github.com/edenlabllc/ehealth.api/issues/2891

Coverage increased (+0.005%) to 91.872% when pulling 41faf8cc0208c8e0bde678f89c92a1df272b9f63 on reimbursement_fix_bugs#2891 into 945b4376b052eaa53e16cc5ffbcc39eaf8cc4934 on develop.


Refactoring & Changes

building on windows machines could be seriously impacted by long file names. Shortened file names, condensed verbose names, de-nested several directories.
Removed some extraneous wrapping elements.
Updated the senzing theme to fit with app.
Added a number of new theme variables to help style search box/search list components.
Bumped package minor version number up.

Issue number: #40


Gesture handling always fails on iOS

Can you please explain it a bit

When swiping vertically in Screen B and then returning by swiping left, you will find that the gesture is disabled

It stopped working in 1.7.0 due to https://youtrack.jetbrains.com/issue/CMP-6622/Gestures-dont-work-on-native-ios-view-embedded-with-UIKitView-for-compose-1.7.0-beta01
Also the interactive pop gesture recognizer works better when you ditch the whole swipe gesture recognition
So you can remove this:
val directions =
            listOf(UISwipeGestureRecognizerDirectionRight, UISwipeGestureRecognizerDirectionLeft)
        for (direction in directions) {
            val gesture = UISwipeGestureRecognizer(
                target = this,
                action = NSSelectorFromString("handleSwipe:")
            )
            gesture.direction = direction
            gesture.delegate = this
            this.view.addGestureRecognizer(gesture)
        }

and this:
    @OptIn(BetaInteropApi::class)
    @ObjCAction
    fun handleSwipe(sender: UISwipeGestureRecognizer) {
        NSLog("Swipe detected: ${sender.direction}")
    }

In UIViewControllerWrapper.kt if you want only the navigation gestures to be visible.

can you please move this issue here: https://github.com/Kashif-E/Voyant @master-lzh

OK, I've moved over.

is there a fix to the back gesture on iOS? it seems that the hadleSwipe does write to log that there was a swipe but there is no pop action happening

@TomerGlick  please use https://github.com/kashif-e/voyant


Fix fulltrust process sometimes exiting on start

Resolved / Related Issues
Items resolved / related issues by this PR.

Closes #9572 (for real)

Details of Changes
Add details of changes here.

Previous PR #9573 only fixed a symptom but not the issue.
The "TerminateUwp" command was sometimes left saved in local settings so on startup the FTP was exiting. Call "TerminateUwp" only when starting Files though command line (it appears the only case in which it's needed to terminate "zombie" Files processes)
Added a couple of comments to remember what we're doing there
Removed some old code

Validation
How did you test these changes?

[x] Built and ran the app

for real

I think I've seen you say this before 😅


Document branches in contributing guidelines

Also resolves #1232

@yurishkuro @vprithvi @black-adder  @tiffon please do not push feature branches to upstream and rather submit PR from your fork.
https://github.com/jaegertracing/jaeger/branches/all

I have already changed my git clone.


WIP: Possible fix for discord rich presence memory leak

I tested this on my machine and it seems to fix the memory leak, however removing this code also doesn't cause the memory leak on my machine, so I'm contributing it as a possible fix although I can't test it anymore.

Will leave the actual issue open until I have to validate this fix but it looks good to me

Sorry for the massive CI backup, was trying to get my commit to show up as signed.


dask_key_name="foo"

dask 2.7.0.
Ok, this is weird.
If one invokes a delayed function with dask_key_name exactly equal to "foo", and then it computes it without explicitly setting the scheduler, it returns the wrapped function instead of the function output O_O
>>> from dask import delayed
>>> f = delayed(lambda: 1)
>>> d1 = f(dask_key_name="bar")
>>> d2 = f(dask_key_name="foo")
>>> d1.compute()
1
>>> d2.compute()
<function __main__.f()>
>>> d2.compute(scheduler=d2.__dask_scheduler__)
1

Interesting!  Any interest in tracking this down?
On Tue, Nov 12, 2019 at 10:11 AM crusaderky<EMAIL_ADDRESS>wrote:

dask 2.7.0.
Ok, this is weird.
If one invokes a dask.distributed with dask_key_name exactly equal to
"foo", and then it computes it without explicitly setting the scheduler, it
breaks O_O

from dask import delayed>>> f = delayed(lambda: 1)>>> d1 = f(dask_key_name="bar")>>> d2 = f(dask_key_name="foo")>>> d1.compute()1>>> d2.compute()<function main.f()>>>> d2.compute(scheduler=d2.dask_scheduler)1

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/dask/dask/issues/5582?email_source=notifications&email_token=AACKZTBBTOSMZFFPKO7NNG3QTLWVVA5CNFSM4JMHELAKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HYYXRTA,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AACKZTEV3M55WXWLF6D5WQLQTLWVVANCNFSM4JMHELAA
.

Interest, plenty.
The issue was that I was connecting to a distributed cluster where, several hours before, I persisted (and never cleaned up) an object with key="foo".


I am a Graph API Support Engineer. These changes are related to what …

I am a Graph API Support Engineer. These changes are related to what the engineer said in this ICM: https://portal.microsofticm.com/imp/v5/incidents/details/555516257/summary.

@microsoft-github-policy-service agree company="Microsoft"

@kevindavidsofronie , thanks for your contribution. We need to update the permissions model that generates the include files; when we've made that update we can merge this change.


[#126126973] Tooling to use STS tokens once we enforce MFA

What
In another PR we enforce users to have MFA enabled using an IAM policy that blocks all access to AWS if a token has not been registered. Because regular AWS keys are not MFA'd, any request made using them will be blocked by the policy.
The solution to this is to move to security tokens which are MFA'd, and are issued using the regular AWS keys. These tokens are time-limited with a maximum lifetime of 36 hours, however we set the expiry to 12 hours so they last a working day.
This PR introduces tooling that both checks you have valid keys before running Make tasks, and tooling to issue & cache tokens.
We acknowledge that the interaction with this tooling may not be perfect yet, however we would like to see how people use it before integrating it more deeply or adding more magic ✨.
It is suggested that developers create bash aliases to both create and source the correct credentials for an environment. Such as:
alias fetch_token="${PAAS_CF_DIR}/scripts/create_sts_token.sh" && \
    source "${HOME}/.aws_sts_tokens/${AWS_ACCOUNT}_${DEPLOY_ENV}.sh"

How to review

Run make showenv and observe that you are warned about not having credentials.
Run ./scripts/create_sts_token.sh to create an STS token in ~/.aws_sts_tokens/.
Run source ~/.aws_sts_tokens/$AWS_ACCOUNT_$DEPLOY_ENV.sh (Assuming you have AWS_ACCOUNT set to dev or similar).
Run make showenv and observe it working using the temporary credentials.
Optionally wait 12 hours and try make showenv again, then observe a notice telling you your credentials are now invalid.

Who can review
Preferably not @benhyland or @jonty.

Fixed the typos @alext found

an STS token that is cached on the local filesystem

Out of interest, is it necessary to cache it on disk? Is there an overhead or limit in requesting new tokens?

@dcarley The overhead is that you need to re-enter a 2FA code every time you request one, the caching is to make using STS less irritating.


fix: only request reviewers when PR no requested reviewers

Part of https://github.com/ti-community-infra/tichi/issues/315
Because we support multiple sigs and in practice we find that people assign revewers manually, we need to stop auto-assignment when there are manually assigned viewers. This feature is similar to not auto-assigning when we have /cc.

/lgtm

/merge


Can't get feature specs to hit the mock server.

I'm using Capybara with a Selenium driver to submit a Stripe form in my feature specs. I can see that the requests aren't hitting the mock server at all.  I've tried to do both StripeMock.start & the other way, StripeMock.start_client
Is there something I'm missing to get feature specs working?

Same situation, the gem handles every request from the backend but in the frontend 'https://api.stripe.com/v1/tokens' isn't intercepted.

There's two problems here:

Your local machine needs to intercept the requests going to api.stripe.com.
Your local machine needs to bypass SSL somehow. The whole point of SSL is that you cannot pretend to be someone you are not. If the browser is trying to hit https://api.stripe.com, and api.stripe.com responds with a certificate that is not signed by a trusted authority, the request will die. I tried to do a MITM attack on my test server by self-signing a certificate and telling the browser to trust it, but I could not get it to work for whatever reason.

Neither of these problems are easy to solve cleanly. The best solution might be some thing that scrubs your Rails responses, replacing the string https://api.stripe.com with some http://localhost or something similar.

I think this is somewhat solvable, although I ended up just trying to work around it.
Instead of scrubbing out Rails responses, one would need to replace calls to https://api.stripe.com with the address of our local Stripe server.
Not sure how to do that, and it's not worth my time for now.
In the meanwhile I've resorted to getting real tokens from the browser, then mocking the responses to my backend requests using WebMock.

@yjukaku: Thanks for sharing your experience. I'm having the same issue, and I'm fine with just letting my browser get real tokens, but I'm getting this JS error in my Capybara test:
XMLHttpRequest cannot load https://api.stripe.com/v1/tokens. Origin https://js.stripe.com is not allowed by Access-Control-Allow-Origin.", :source=>"https://js.stripe.com/v2/channel.html?stripe_xdm_e=http%3A%2F%2F<IP_ADDRESS>%3A55210&stripe_xdm_c=default817163&stripe_xdm_p=1#__stripe_transport__

Did you get that error, too, and if so, how did you work around it?

Stripe.endpoint = 'localhost:4999' seems to control the endpoint, but I am still having trouble getting the server to respond to requests properly.

@stevehanson Sorry, I don't remember anymore!
@mattvanhorn Nice find with the Stripe.endpoint.  So are you just running javascript from the spec to set that variable? Also, what trouble are you having?

Here's the solution I went with, which is admittedly a little messy.
First, I set the token as an ENV variable in my test:
ENV["STRIPE_TOKEN"] = stripe_helper.generate_card_token(last_4: '4242', exp_year: '2025', exp_month: '12')

And inject the fake token into the page:
<% if Rails.env.test? && ENV["STRIPE_TOKEN"] %>

<%= ENV["STRIPE_TOKEN"] %>

<% end %>

and then in my JS form submit handler, I bypass the Stripe call if the token is present on the page:
var self = this;

if($('#js_test_token').length) { // mock stripe call in feature tests
  setTimeout(function() { self.stripeResponseHandler('success', { id: $('#js_test_token').text() }); }, 0);
} else {
  Stripe.card.createToken({
    number: $form.find('#cc_number').val(),
    cvc: $form.find('#cc_cvc').val(),
    exp: $form.find('#cc_exp').val(),
    address_zip: $form.find('#cc_postal_code').val()
  }, self.stripeResponseHandler.bind(self));
}

This way, in my feature tests, I bypass the Stripe call and just invoke the handler with the token generated by stripe-ruby-mock.

I was trying to set it by executing javascript w/ capybara. We gave up on mocking everything, because we needed to move faster, but when I go back to it, I will post updates here.

@stevehanson - that was going to be my fallback approach, I don't love having "if test" conditionals in my code, but if it works, that's at least an option.

@mattvanhorn executing JS with capybara seems like a much better idea. Didn't think of that. I bet something like this would work:
page.execute_script "window.testStripeToken = '#{my_token}';"

and then can just check if(window.testStripeToken) in the Stripe JS code. That's a little cleaner. No need for anything in the DOM. I agree that it still sucks to have to have conditionals or that the code has to know about testing at all. Still a little better than not having tests over a critical part of our app, though :)

For anyone stumbling on this, I want to point out that setting Stripe.endpoint to a different endpoint (such as a fake local server) is not officially supported by Stripe.js.   This did work until recently, but Stripe.js recently changed, and setting Stripe.endpoint doesn't necessary have any effect on the server that Stripe.js hits anymore.  Therefore, if you really want to hit a fake server, you'll need to use some mechanism other than just setting Stripe.endpoint.


The property 'Name' cannot be found on this object.

Hello,
Trying to run PublishADFTask@1, I'm getting right down to the end before receiving what seems to be a PowerShell error.
STEP: Replacing all properties environment-related...
Config file:   D:\a\1\s\deployment\config-***.csv
Validation of config file completed.
##[error]The property 'Name' cannot be found on this object. Verify that the property exists.
Finishing: Push code to Data Factory

I'm just running a single task, updating some values via a CSV.
  - task: PublishADFTask@1
    displayName: 'Push code to Data Factory'
    inputs:
      ConnectedServiceName: ${{ parameters.serviceConnectionName }}
      ResourceGroupName: $(CustomerResourceGroupName)
      DataFactoryName: $(CustomerDataFactoryName)
      Location: $(CustomerLocation)
      DataFactoryCodePath: '$(Agent.BuildDirectory)\s'
      StopStartTriggers: false
      CreateNewInstance: false
      IgnoreLackOfReferencedObject: false
      StageType: Stage
      StageCode: $(Environment)
      FilteringType: 'Inline'
      ${{ if eq(parameters.useSelfHostedIntegrationRuntime, true) }}:
        FilterText: |
          -trigger.*
          -factory.*
      ${{ else }}:
        FilterText: |
          -pipeline.*-IR-*
          -*.*viaSelfHostedIR*
          -linkedService.*_IR
          -integrationRuntime.*
          -trigger.*
          -factory.*
      PublishMethod: 'AzResource'

Do you have any ideas?

Just to add some further information, it works when my CSV file looks like this:
type,name,path,value
linkedService,AzureKeyVault_LinkedService,typeProperties.baseUrl,"https://***.vault.azure.net/"

type,name,path,value
linkedService,AzureKeyVault_LinkedService,typeProperties.baseUrl,"https://***.vault.azure.net/"
integrationRuntime,SelfHostedIntegrationRuntime,description,"Linked Customer Self-Hosted Integration Runtime"
integrationRuntime,SelfHostedIntegrationRuntime,+typeProperties.linkedInfo,"{'resourceId':'/subscriptions/***/resourcegroups/***/providers/Microsoft.DataFactory/factories/***/integrationruntimes/***','authorizationType':'Rbac'}"

Looks fine to me. The name of my IR is SelfHostedIntegrationRuntime (as per the CSV)

Can you prepare a test environment and script on your local machine?
You can use this example as guidance:
https://github.com/SQLPlayer/azure.datafactory.tools/tree/master/adhoc/issue-240-v2


Handle Generic Arguments for CSTs

CSTs can be generic structs and some generic arguments can be redundant after some fields are dropped. See example below
#[derive(AST)]
pub struct SumCST<'a> {
    lhs: Spanned<i64>,
    #[space]
    lw: Spanned<&'a str>,
    #[space]
    rw: Spanned<&'a str>,
    rhs: Spanned<i64>,
}

Lifetime argument 'a becomes redundant in generated AST.

Generic argument for each field and derived AST should be dropped. Since macros are expanded at compile time, we don't know signatures of new types so for CSTs that use other CSTs it is impossible to know whether their generic arguments are present or not.


Update Readme.md to make it clear that the .HEX file can be imported into Microsoft Makecode

https://github.com/DeskPi-Team/microcar/blob/main/README.md
Could this file be updated to state that you can import the .hex files from the individual module folder into the extensions section of Microsoft Makecode? This was not immediately apparent and would help future customers.

Not sure if it can be imported directly. It may require support and review of a lot of files? I'm not sure, currently each project is independent, we would be happy to add it to Microsoft's repository if possible


Sega CD games Cue/bin issue

I am using the guide from redump.org I have ripped one of my games but the Cue/Bin  when loaded in an emulator only sees it as a music CD.  did I rip it incorrectly? the command I used was ( DiscImageCreator.exe cd d Eye_of_the_Beholder_dump1.bin 8 /c2 /nl ) do I need different commands for unprotected games?

Does your dumped image match the database of redump.org?

there is not one, that is what I wanted to do.  but I found out that it might be my drive.  I have an LG not a plexor.  when I used Imgburn the game was playable on the emulator... but it doesnt give me any data.  I did see that the Hex was missing some code from the games I ripped with Disc Image Creator, compared to the working ISOs


Add '%' to quote_plus's safe parameter in response.redirect

https://github.com/huge-success/sanic/blob/30e6a310f132752669a74927530e8bc52a51e98e/sanic/response.py#L367
from urllib.parse import urlencode
args = urlencode({"redirect_uri": "http://example.com"})
response.redirect(...args...)
>>> 'redirect_uri=http%253A%252F%252Fexample.com'

In this case it will be quote_plus twice, and generate extra '%25'。

@sjsadowski Thanks for your attention.
Because the URL to redirect is generated by concatenate multiple parameters, just like Tornado's url_concat function. These operations use functions such as urlparse, parse_qsl, urlencode for convenience, so, the URL is encoded before it is passed to response.redirect.
I have solved this problem by  passing '%' to my own urlencode function's safe parameter first, and then calling response.redirect is correct.
But I think '%' should still be a part of quote_plus which in response.redirect. It should be treated the same as ":/#?&=@[]!$&'()*+,;".

Yeah, it has to quote path and query params that including sth. like white space, while constructing redirect url.


Force floating point division!

There's currently no way to cast int vars as real vars, and assigning the quotient of two ints to a var results in truncated division.

Works for me. Don't know what you're smoking!


Deprecate cass_cluster_set_use_hostname_resolution()

Add documentation that MITM is possible when using
CASS_SSL_VERIFY_PEER_IDENTITY_DNS with
cass_cluster_set_use_hostname_resolution()

Something else just occurred to me a little bit ago: we probably still need a changelog mention for this even though it isn't actually changing (behaviour in) code, right?

Merging this in for now, will address the changelog update separately once we get closer to a release


(feat): nnnnat guest form

Resolves 225 in starterkit
Impact: minor
Type: feature
Component
Created a GuestForm component using reacto-form and the RC form element components. This form only captures a users email address and will be used inside the checkout's login view.
Screenshots
Desktop

Mobile

Testing

Verify it looks and works as expected

🚀 Preview deployed
Built with commit 76dd14701d2227d7a6cf612300d890cd307cd743
https://deploy-preview-246--stoic-hodgkin-c0179e.netlify.com

:tada: This PR is included in version 0.26.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


ci: switch to pnpm

Using Lerna with pnpm wasn't a problem, but since pnpm works a bit differently (doesn't have node_modules like yarn install does), during pnpm build there are inferred types problems, an example:
The inferred type of 'FormContainer' cannot be named without a reference to '.pnpm/@mui+system@5.14.3_@emotion+react@11.11.1_@emotion+styled@11.11.0_@types+react@18.2.16_react@18.2.0/node_modules/@mui/system'. This is likely not portable. A type annotation is necessary.

(Here's a TypeScript issue talking about a similar problem)
Those inferred types errors can be manually solved, but not sure if you want that

@dnnnvx can we solve this by adding @mui/system as dependency?

can we solve this by adding @mui/system as dependency?

I'll try

@dnnnvx as I understand, this is not going to fly and we can close it until better times? 🙂

@dnnnvx as I understand, this is not going to fly and we can close it until better times? 🙂

Yes, also other FE repos have some problems migrating to it. I think we should pause it for now and wait and see if it makes sense, since on the BE we still have some open discussion about it, for example: Dependabot doesn't support pnpm, and its developers are not that active

@dnnnvx Thank you for trying this out! 🙂 I'll close it then.


Make libfreetype submodule

I notice that there is a hard-coded git clone command in the Makefile, which seems a bit out of place to me.
Would it make any sense to make this a submodule instead?
That way, you could clone it automatically (and just once) with "git submodule init && git submodule update".
PS: I can make this a PR.

Yeah, definitely that would be a good idea. Any help is appreciated.
Note that with the Rust implementation, it looks like it will be much more expedient to get to a stage where we can plug into an already established UI framework. rusttype for instance got everything about text rendering bootstrapped really nicely.
I will add some photos and videos tomorrow with the new rust-poc. It is basically 99.5% the way there in terms of matching the xochitl I/O latency and this is end to end.

Cool! That's good to know. I've never written a line of Rust before, but that may convince me to learn it. What's it like setting up a cross-compiler environment with Rust?
For the time being, the low-level C stuff is good enough for me now. I definitely want to be able to poke at things and learn about some of the guts.

@PaulBatchelor It is actually really straightforward to set it up, and I definitely would recommend it.
Assuming you have the arm-linux-gnueabihf toolchain already installed and you have arm-linux-gnueabihf-gcc in your $PATH, just add the following to your ~/.cargo/config:
[target.armv7-unknown-linux-gnueabihf]
linker = "arm-linux-gnueabihf-gcc"
ar = "arm-linux-gnueabihf-ar"

Run rustup target add armv7-unknown-linux-gnueabihf to download the toolchain and the std crate for the architecture.
After that you can simply cd into rust-poc and run cargo build --release --target=armv7-unknown-linux-gnueabihf.
You'll have the executable under ./target/armv7-unknown-linux-gnueabihf/release/rust-poc.
The reason why I am recommending this is because the c-implementation already has missing features at this point. Translation from multitouch and Wacom Digitizer coordinates to eInk display coordinates would be one of the first to come to mind.
Aside from that, rusttype is providing a lot of text rendering features out of the box while libfreetype is leaving a lot of the work to us, and the implementation we have in place actually quite hacky, using the letter T to adjust the horizontal and vertical positioning of glyphs etc.
On the other hand, the Rust implementation is moving forward. Here is what an example application would look like (from https://github.com/canselcik/RemarkableFramebuffer/blob/master/rust-implementation/rust-poc/src/main.rs):
// Takes callback functions as arguments
// They are called with the event and the &mut framebuffer
let mut app = uix::ApplicationContext::new(
  on_button_press,
  on_wacom_input,
  on_touch,
);

// Clear the screen
app.clear(true);

// A rudimentary way to declare a scene and layout
app.draw_elements(&vec![
        UIElement::Text {
            text: "Remarkable Tablet".to_owned(),
            y: 200, x: 100,
            scale: 100,
            refresh: UIConstraintRefresh::NoRefresh
        },
        UIElement::Image {
            img: image::load_from_memory(include_bytes!("../rustlang.bmp")).unwrap(),
            y: 10, x: 900,
            refresh: UIConstraintRefresh::Refresh },
        UIElement::Text {
            text: "Current Waveform: ".to_owned(),
            y: 350, x: 120,
            scale: 65, refresh: UIConstraintRefresh::NoRefresh
        },
        UIElement::Text {
            text: "Current Dither Mode: ".to_owned(),
            y: 410, x: 120,
            scale: 65,
            refresh: UIConstraintRefresh::NoRefresh
        },
        UIElement::Text {
            text: "Current Quant: ".to_owned(),
            y: 470, x: 120,
            scale: 65,
            refresh: UIConstraintRefresh::RefreshAndWait
        },
]);

// Get a &mut to the framebuffer object, exposing many convenience functions
let fb = app.get_framebuffer();
let clock_thread = std::thread::spawn(move || {
  loop_print_time(fb, 100, 100, 65);
});

// Blocking call to process events from digitizer + touchscreen + physical buttons
app.dispatch_events(4096, 1024);
clock_thread.join().unwrap();


FIX# circular dependency

Strictly speaking, this is a broken API. Now,

wlang has few users
the "require dialect directly" behavior is not used AFAIK
wlang/html is always required by wlang.rb anyway

Could you please require wlang/dummy alongside wlang/html in wlang.rb? I'll merge this PR then.


New app for Client sekoia.io by Patryk Dros(FDSE)

Name of the app: sekoiaio
Integration: https://www.sekoia.io/
About: New SOAR App whose actions will be able to manipulate STIX bundles and represent an external system that can be queried to enrich observables.

Repo created at https://github.com/splunk-soar-connectors/sekoiaio


swupdate: Add support for arp target

Signed-off-by: Tariq Ansari<EMAIL_ADDRESS>(cherry picked from commit 0c50590f1f6e1785c90a9aadd304c106384eadd0)

cherry picked from master in to sumo

Forgot to tell you: I thought it would be easier to once in a while rebase the branch on master than cp every commit.


Submission client should provide the driver with a pod template to use for executors

Currently we have a construct of a SparkPodInitContainerBootstrap, which is a module that can bootstrap an arbitrary pod with the init container that fetches dependencies. This code is used in both the submission client and the scheduler backend to bootstrap the init container into the driver and executor pods, respectively. However, in order to make sure the bootstrap is identical between the driver and the executors, the submission client passes along internal configuration to the driver to inform the driver of the name of a specific config map to load onto the executors. This also requires the scheduler backend to know to interpret the configurations sent from the submission client and to construct a SparkPodInitContainerBootstrap accordingly. This is not ideal for two reasons:

The submission client and the driver code need to agree on these internal configurations and the interpretation of them. If the submission client is upgraded, it necessarily requires an upgrade of the driver image also.
One could argue that the scheduler backend is doing unnecessary work, in the sense that it's merely repeating the work of setting up the executor's pod spec YAML despite the fact that the submission client had already done this for the driver. Here, the scheduler backend is merely translating Spark configuration from SparkConf key-value pairs into the pod spec that will be used in the executors.

The issue is amplified by #393 because we're intending to programmatically create another configmap to load small files from the user's local disk. We can use the submission steps architecture to fix the driver accordingly, but the files still need to be mounted on the executors. If we follow the existing precedent, that means another block of specific bootstrap code that needs to be abstracted out and shared between the driver and the executors. What's more, even though the interpretation of spark.files seems to be a responsibility specific to the submission client and not the driver, the user will need to upgrade the driver image alongside the submission client again.
I'm suggesting here that the submission client should be responsible for providing some of the executor pod's specification directly. The submission client can build a loose template for the scheduler backend to apply to all executors that it builds. Note that this is similar to, but decidedly distinct from, https://github.com/apache-spark-on-k8s/spark/issues/38, as in said linked issue, we want the user to provide the executor pod's YAML specification, but in this case, the YAML specification to be applied to the executors is being determined by the submission client, based on parameters passed to spark-submit.
This would also require a change in the submission steps API because now steps need to be able to specify customizations for the executor pod template.
The significant concern to this kind of design is that it becomes unclear what the scheduler backend is responsible for building versus what the submission client is responsible for building. For example, should the submission client build the executor's template that specifies the amount of resources the executor needs? The division of responsibilities of what should be applied by the driver versus what should be applied by the submission client may end up being uncomfortably arbitrary.

@aash @erikerlandson @ifilonenko @foxish for thoughts. I'm undecided on if this is a philosophically sound approach.

Is the idea that a template rendered as yaml is an approach to reduce the duplicated effort, in a way that can be passed into the cluster from the submission client?

Closing because this wasn't necessary yet to accomplish what we needed to do


Add Likes button to the navbar

when we want to access our liked tweets the current flow is:
click profile picture > profile > find likes button
with this new button you can click whenever you are at

The Likes button is added quite slowly. Many hundred milliseconds after the page has loaded.
And if I press the Likes button, it's added again. You need to check and prevent duplicate buttons. Example: https://github.com/sindresorhus/refined-github/blob/0d25d590e828fb54db2ad8342f8115aa4aa23ec8/source/features/add-gists-link-to-profile.js#L7-L11

@sindresorhus thanks for the review! I just fixed them.
let me know if there's still something to be done 😁

It still loads pretty slowly. It's not a good experience when the button is visibly loaded after page load. @bfred-it Any tips for making it appear before the page is painted?

Don’t wait for domready, use safeElementReady (this means you have to restore run_at: document_start)

@bfred-it Cool. I reverted your commit then: https://github.com/sindresorhus/refined-twitter/commit/fe35d0a98162a5c5aa7a0c6bedf5aa9d81192d01

@vcapretz Can you use safeElementReady? Example here: https://github.com/sindresorhus/refined-twitter/blob/fe35d0a98162a5c5aa7a0c6bedf5aa9d81192d01/source/content.js#L22

@vcapretz Can you also add this feature to the feature listing in the readme?

sure @sindresorhus, I'll update it

hey @sindresorhus I just applied the suggested changes!
now the function runs on init, and I saw it has really improved the loading performance.
I also added the safeElementReady function to the main list were the new item is about to be added!

Great. Thanks :)


Changed reference to .btn-dark.active to .btn-inverse.active.

.btn-dark has been changed to .btn-inverse but the change was missed in one spot.  This fixes that.

I have an open pull request which fixes the same problem + adds inverse examples to dropdown buttons and split dropdown buttons.
https://github.com/twitter/bootstrap/pull/1940

Sorry I checked, but didn't see that one.


RUMM-1948 Make 'service' and 'status' properties optional in web events

…ewEvent
What does this PR do?
A brief description of the change being made with this pull request.
Motivation
What inspired you to submit this pull request?
Additional Notes
Anything else we should know when reviewing?
Review checklist (to be filled by reviewers)

[ ] Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
[ ] Make sure you discussed the feature or bugfix with the maintaining team in an Issue
[ ] Make sure each commit and the PR mention the Issue number (cf the CONTRIBUTING doc)

Codecov Report

Merging #831 (579c627) into release/1.12.0 (da1d4d0) will decrease coverage by 0.01%.
The diff coverage is n/a.

@@                Coverage Diff                 @@
##           release/1.12.0     #831      +/-   ##
==================================================
- Coverage           80.59%   80.58%   -0.01%
==================================================
  Files                 254      254
  Lines                8552     8552
  Branches             1453     1453
==================================================
- Hits                 6892     6891       -1
  Misses               1319     1319
- Partials              341      342       +1

Impacted Files
Coverage Δ

...g/android/rum/internal/domain/scope/RumEventExt.kt
90.52% <0.00%> (-0.86%)
:arrow_down:


Better handle non-24/32-bit images

The image processing functions work on 24 and 32-bit images, but have issues with 1/2/4/8-bit images (due to color palette, byte padding, bit/byte packing, etc).
Better handle these image types, or at least check for and ignore these images.

8-bit images are correctly handled, removing padding before processing, or converting to 24- or 32-bit as required. Issue #4 raised to track 1- and 4-bit image types.


mavlink autopilot_version: add vendor version

Goes together with https://github.com/mavlink/qgroundcontrol/pull/7326.
Allows downstream adopters to use git tags in the form <PX4-version>-<vendor-version>, e.g. v1.8.2-0.2.0.
@hamishwillee we should document this (not saying you need to, but I'm not sure where to place this best?).

Hi @bkueng

Allows downstream adopters to use git tags in the form -, e.g. v1.8.2-0.2.0.

I need to know how this works to advise
How does someone using a git tag of the form <PX4-version>-<vendor-version> in their own fork result in a custom build that has a useful value for AUTOPILOT_VERSION.flight_custom_version?
Looking at AUTOPILOT_VERSION.flight_custom_version it says  Custom version field, commonly the first 8 bytes of the git hash. So guessing, would I be right in assuming that:

using a tag of the above format will result in a SHA where the first 8 bytes can be turned back into the above string by QGC? (and presumably the remaining SHA bytes for specific commit are just lost?)
Building firmware somehow checks for the current SHA and stores the info?

Further questions:

Is this info available for other boards than pixhawk series?
What happens if they use a different tag format than our desired above (in terms of displayed custom version)?
Is there anything else that a developer should do when they are creating a downstream build? What I am trying to work out is if this should be tacked on to "Building the code" or part of a bigger topic on "All the things you need to do if you're releasing a custom build".

Do we also need to also tell end users how to check the firmware version?

using a tag of the above format will result in a SHA where the first 8 bytes can be turned back into the above string by QGC? (and presumably the remaining SHA bytes for specific commit are just lost?)

The first 3 bytes of flight_custom_version will be the vendor version, encoded in the same way as flight_sw_version:
        majorVersion = autopilotVersion.flight_custom_version[2];
        minorVersion = autopilotVersion.flight_custom_version[1];
        patchVersion = autopilotVersion.flight_custom_version[0];

The rest of flight_custom_version will be the git hash (5 bytes).
Example
QGC will show the vendor version as:

The tag was v120.83.3-186.39.56 in that case.

Building firmware somehow checks for the current SHA and stores the info?

Yes, plus git tag.

Is this info available for other boards than pixhawk series?

Yes all boards.

What happens if they use a different tag format than our desired above (in terms of displayed custom version)?

Things will break unexpectedly.

Is there anything else that a developer should do when they are creating a downstream build? What I am trying to work out is if this should be tacked on to "Building the code" or part of a bigger topic on "All the things you need to do if you're releasing a custom build".

I don't see anything else at the moment.

Do we also need to also tell end users how to check the firmware version?

Might be interesting for them. For reports we get it from the log.


tests/drivers/pwm/pwm_api failed on reel_board.

To Reproduce
Steps to reproduce the behavior:

west build -b=reel_board tests/drivers/pwm/pwm_api
west flash
See error

Screenshots or console output
****** Booting Zephyr OS build zephyr-v2.0.0-1547-gda5a595f248f *****
Running test suite pwm_basic_test

starting test - test_pwm_usec
[PWM]: 0, [period]: 2000, [pulse]: 1000
E: PWM pin 0 not enabled through DTS configuration.
Fail to set the period and pulse width

    Assertion failed at ZEPHYR_BASE/tests/drivers/pwm/pwm_api/src/test_pwm.c:112: test_pwm_usec: test_task(DEFAULT_PWM_PORT, DEFAULT_PERIOD_USEC, DEFAULT_PULSE_USEC, UNIT_USECS) == TC_PASS is false

FAIL - test_pwm_usec

starting test - test_pwm_nsec
[PWM]: 0, [period]: 2000000, [pulse]: 1000000
E: PWM pin 0 not enabled through DTS configuration.
Fail to set the period and pulse width

    Assertion failed at ZEPHYR_BASE/tests/drivers/pwm/pwm_api/src/test_pwm.c:130: test_pwm_nsec: test_task(DEFAULT_PWM_PORT, DEFAULT_PERIOD_NSEC, DEFAULT_PULSE_NSEC, UNIT_NSECS) == TC_PASS is false

FAIL - test_pwm_nsec

starting test - test_pwm_cycle
[PWM]: 0, [period]: 64000, [pulse]: 32000
E: PWM pin 0 not enabled through DTS configuration.
Fail to set the period and pulse width

    Assertion failed at ZEPHYR_BASE/tests/drivers/pwm/pwm_api/src/test_pwm.c:148: test_pwm_cycle: test_task(DEFAULT_PWM_PORT, DEFAULT_PERIOD_CYCLE, DEFAULT_PULSE_CYCLE, UNIT_CYCLES) == TC_PASS is false

FAIL - test_pwm_cycle

Test suite pwm_basic_test failed.

PROJECT EXECUTION FAILED

Environment (please complete the following information):

OS: Fedora28
Toolchain: zephyr-sdk-0.10.3
Commit ID: da5a595f

Hi @jfischer-phytec-iot any updates on this?

This test was not customized for reelboard, it does need board specific settings AFAICT. I doubt it worked before...

I do not think it worked for any nRF based board.
boards/arm/nrf52_pca10040/nrf52_pca10040.dts:156:       ch0-pin = <17>;
boards/arm/decawave_dwm1001_dev/decawave_dwm1001_dev.dts:117:   ch0-pin = <22>;
boards/arm/nrf52833_pca10100/nrf52833_pca10100.dts:166: ch0-pin = <13>;
boards/arm/nrf5340_dk_nrf5340/nrf5340_dk_nrf5340_cpuapp_common.dts:105: ch0-pin = <28>;
boards/arm/nrf52832_mdk/nrf52832_mdk.dts:98:    ch0-pin = <22>;
boards/arm/nrf52840_pca10056/nrf52840_pca10056.dts:166: ch0-pin = <13>;
boards/arm/holyiot_yj16019/holyiot_yj16019.dts:54:      ch0-pin = <29>;
boards/arm/nrf52840_mdk/nrf52840_mdk.dts:100:   ch0-pin = <22>;
boards/arm/nrf52811_pca10056/nrf52811_pca10056.dts:119: ch0-pin = <13>;
boards/arm/bl654_dvk/bl654_dvk.dts:114: ch0-pin = <13>;
boards/arm/actinius_icarus/actinius_icarus_common.dts:119:      ch0-pin = <10>;
boards/arm/bl652_dvk/bl652_dvk.dts:90:  ch0-pin = <17>;
boards/arm/nrf9160_pca10090/nrf9160_pca10090_common.dts:123:    ch0-pin = <2>;
boards/arm/nrf52840_papyr/nrf52840_papyr.dts:99:        ch0-pin = <13>;
boards/arm/reel_board/dts/reel_board.dtsi:137:  ch0-pin = <13>;


Unable to write files when executable is removed from project directory.

I have a project that writes an html file and a .txt file.
I am bundling using the command `pkg index.js'
I am receiving no errors during bundling
when the executable it moved to a different location, everything works, except for the files being created.
Here is my file writing code:
fs.writeFile("./output.html", result, function(err){
    if(err){
      return console.log(err);
    }
    opn('./output.html');
  });

  fs.writeFile("./output.txt", result, function(err){
    if(err){
      return console.log(err);
    }
    opn('./output.txt');
  });

Issue was caused by trying to read a local file before writing.