Global Async Queries doesn't work

Hi.
I tried to enable GAQ feature with apache/superset:latest docker image. After I enabled it I experienced some dashboard stopped working and all I see is white screen. When I checked network activity I saw too much async request and they weren't stoping and returing emtpy results.
Expected results
Dashboard should load faster and async
Actual results
Charts and dashboards aren't loading. Also see this exception in worker
worker_1    | [2021-01-26 10:16:36,308: ERROR/ForkPoolWorker-11] Task load_explore_json_into_cache[069996ac-d5d9-4c29-a6e4-c99e3faf6899] raised unexpected: NoSuchColumnError("Could not locate column in row for column 'table_columns.id'")
worker_1    | Traceback (most recent call last):
worker_1    |   File "/usr/local/lib/python3.7/site-packages/celery/app/trace.py", line 412, in trace_task
worker_1    |     R = retval = fun(*args, **kwargs)
worker_1    |   File "/app/superset/app.py", line 116, in __call__
worker_1    |     return task_base.__call__(self, *args, **kwargs)
worker_1    |   File "/usr/local/lib/python3.7/site-packages/celery/app/trace.py", line 704, in __protected_call__
worker_1    |     return self.run(*args, **kwargs)
worker_1    |   File "/app/superset/tasks/async_queries.py", line 108, in load_explore_json_into_cache
worker_1    |     raise exc
worker_1    |   File "/app/superset/tasks/async_queries.py", line 84, in load_explore_json_into_cache
worker_1    |     payload = viz_obj.get_payload()
worker_1    |   File "/app/superset/viz.py", line 476, in get_payload
worker_1    |     columns = set(self.datasource.column_names)
worker_1    |   File "/app/superset/connectors/base/models.py", line 145, in column_names
worker_1    |     return sorted([c.column_name for c in self.columns], key=lambda x: x or "")
worker_1    |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/attributes.py", line 287, in __get__
worker_1    |     return self.impl.get(instance_state(instance), dict_)
worker_1    |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/attributes.py", line 723, in get
worker_1    |     value = self.callable_(state, passive)
worker_1    |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/strategies.py", line 760, in _load_for_state
worker_1    |     session, state, primary_key_identity, passive
worker_1    |   File "<string>", line 1, in <lambda>
worker_1    |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/strategies.py", line 902, in _emit_lazyload
worker_1    |     .with_post_criteria(set_default_params)
worker_1    |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/ext/baked.py", line 544, in all
worker_1    |     return list(self)
worker_1    |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 100, in instances
worker_1    |     cursor.close()
worker_1    |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__
worker_1    |     with_traceback=exc_tb,
worker_1    |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
worker_1    |     raise exception
worker_1    |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 80, in instances
worker_1    |     rows = [proc(row) for row in fetch]
worker_1    |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 80, in <listcomp>
worker_1    |     rows = [proc(row) for row in fetch]
worker_1    |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 524, in _instance
worker_1    |     tuple([row[column] for column in pk_cols]),
worker_1    |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 524, in <listcomp>
worker_1    |     tuple([row[column] for column in pk_cols]),
worker_1    |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/result.py", line 686, in _key_fallback
worker_1    |     replace_context=err,
worker_1    |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
worker_1    |     raise exception
worker_1    | sqlalchemy.exc.NoSuchColumnError: "Could not locate column in row for column 'table_columns.id'"

Screenshots

How to reproduce the bug

Create superset and celery environment with apache/superset:latest image
Enable GAQ feature and configure secret.

Environment

superset version: latest
python version: 3.7
mysql 5.7.32
redis 6.0.9

Checklist
Make sure to follow these steps before submitting your issue - thank you!

[X] I have checked the superset logs for python stacktraces and included it here as text if there are any.
[X] I have reproduced the issue with at least the latest released version of superset.
[x] I have checked the issue tracker for the same issue and I haven't found one similar.

@robdiciuccio 🙏

@robdiciuccio 🙏

Hi @iercan, it is expected that the event polling mechanism makes continuous requests to the /api/v1/async_event/ endpoint (presumably what's in the above screenshot). The sqlalchemy.exc.NoSuchColumnError trace you posted above appears to be a SQL query error against your analytics DB. Does the same query function correctly in synchronous mode?
Also, not sure if you've seen this, but check out the documentation added here: https://github.com/apache/superset/pull/12573

Hi @iercan, it is expected that the event polling mechanism makes continuous requests to the /api/v1/async_event/ endpoint (presumably what's in the above screenshot). The sqlalchemy.exc.NoSuchColumnError trace you posted above appears to be a SQL query error against your analytics DB. Does the same query function correctly in synchronous mode?
Also, not sure if you've seen this, but check out the documentation added here: https://github.com/apache/superset/pull/12573

@robdiciuccio There is no problem with synchronous mode. I already checked documentation and configured accordingly.

@robdiciuccio There is no problem with synchronous mode. I already checked documentation and configured accordingly.

Today I made some more test. I disabled thumbnails which was enabled on other test. This time dashoard charts stuck on loading states and async_event requests started  returning 401. On worker logs I see queries are executing. Really couldn't figure out what's wrong here.

Today I made some more test. I disabled thumbnails which was enabled on other test. This time dashoard charts stuck on loading states and async_event requests started  returning 401. On worker logs I see queries are executing. Really couldn't figure out what's wrong here.

Permission error gone after I run superset init. But problem I mentioned on my first post still persist.

Permission error gone after I run superset init. But problem I mentioned on my first post still persist.

Here is another exception I catch. I don't know what is different with async and sync queries but looks like the way celery worker send queries cause these exceptions
worker_1    | [2021-01-27 13:21:35,231: ERROR/ForkPoolWorker-7] Task load_explore_json_into_cache[2f6fdc73-84e8-4eac-9c3e-dbad93852f49] raised unexpected: SupersetVizException('[{\'message\': "Unknown exception (org.apache.calcite.tools.ValidationException): org.apache.calcite.runtime.CalciteContextException: From line 7, column 10 to line 7, column 20: Column \'algorithm\' not found in any table", \'error_type\': <SupersetErrorType.GENERIC_DB_ENGINE_ERROR: \'GENERIC_DB_ENGINE_ERROR\'>, \'level\': <ErrorLevel.ERROR: \'error\'>, \'extra\': {\'engine_name\': \'Apache Druid\', \'issue_codes\': [{\'code\': 1002, \'message\': \'Issue 1002 - The database returned an unexpected error.\'}]}}]')
worker_1    | Traceback (most recent call last):
worker_1    |   File "/usr/local/lib/python3.7/site-packages/celery/app/trace.py", line 412, in trace_task
worker_1    |     R = retval = fun(*args, **kwargs)
worker_1    |   File "/app/superset/app.py", line 116, in __call__
worker_1    |     return task_base.__call__(self, *args, **kwargs)
worker_1    |   File "/usr/local/lib/python3.7/site-packages/celery/app/trace.py", line 704, in __protected_call__
worker_1    |     return self.run(*args, **kwargs)
worker_1    |   File "/app/superset/tasks/async_queries.py", line 108, in load_explore_json_into_cache
worker_1    |     raise exc
worker_1    |   File "/app/superset/tasks/async_queries.py", line 86, in load_explore_json_into_cache
worker_1    |     raise SupersetVizException(errors=payload["errors"])
worker_1    | superset.exceptions.SupersetVizException: [{'message': "Unknown exception (org.apache.calcite.tools.ValidationException): org.apache.calcite.runtime.CalciteContextException: From line 7, column 10 to line 7, column 20: Column 'algorithm' not found in any table", 'error_type': <SupersetErrorType.GENERIC_DB_ENGINE_ERROR: 'GENERIC_DB_ENGINE_ERROR'>, 'level': <ErrorLevel.ERROR: 'error'>, 'extra': {'engine_name': 'Apache Druid', 'issue_codes': [{'code': 1002, 'message': 'Issue 1002 - The database returned an unexpected error.'}]}}]

Here is another exception I catch. I don't know what is different with async and sync queries but looks like the way celery worker send queries cause these exceptions
worker_1    | [2021-01-27 13:21:35,231: ERROR/ForkPoolWorker-7] Task load_explore_json_into_cache[2f6fdc73-84e8-4eac-9c3e-dbad93852f49] raised unexpected: SupersetVizException('[{\'message\': "Unknown exception (org.apache.calcite.tools.ValidationException): org.apache.calcite.runtime.CalciteContextException: From line 7, column 10 to line 7, column 20: Column \'algorithm\' not found in any table", \'error_type\': <SupersetErrorType.GENERIC_DB_ENGINE_ERROR: \'GENERIC_DB_ENGINE_ERROR\'>, \'level\': <ErrorLevel.ERROR: \'error\'>, \'extra\': {\'engine_name\': \'Apache Druid\', \'issue_codes\': [{\'code\': 1002, \'message\': \'Issue 1002 - The database returned an unexpected error.\'}]}}]')
worker_1    | Traceback (most recent call last):
worker_1    |   File "/usr/local/lib/python3.7/site-packages/celery/app/trace.py", line 412, in trace_task
worker_1    |     R = retval = fun(*args, **kwargs)
worker_1    |   File "/app/superset/app.py", line 116, in __call__
worker_1    |     return task_base.__call__(self, *args, **kwargs)
worker_1    |   File "/usr/local/lib/python3.7/site-packages/celery/app/trace.py", line 704, in __protected_call__
worker_1    |     return self.run(*args, **kwargs)
worker_1    |   File "/app/superset/tasks/async_queries.py", line 108, in load_explore_json_into_cache
worker_1    |     raise exc
worker_1    |   File "/app/superset/tasks/async_queries.py", line 86, in load_explore_json_into_cache
worker_1    |     raise SupersetVizException(errors=payload["errors"])
worker_1    | superset.exceptions.SupersetVizException: [{'message': "Unknown exception (org.apache.calcite.tools.ValidationException): org.apache.calcite.runtime.CalciteContextException: From line 7, column 10 to line 7, column 20: Column 'algorithm' not found in any table", 'error_type': <SupersetErrorType.GENERIC_DB_ENGINE_ERROR: 'GENERIC_DB_ENGINE_ERROR'>, 'level': <ErrorLevel.ERROR: 'error'>, 'extra': {'engine_name': 'Apache Druid', 'issue_codes': [{'code': 1002, 'message': 'Issue 1002 - The database returned an unexpected error.'}]}}]

Yet another
worker_1    | [2021-01-27 13:32:29,087: ERROR/ForkPoolWorker-1] Task load_explore_json_into_cache[60028736-ba7e-49e2-8513-f4a66d63f6b1] raised unexpected: TypeError("'NoneType' object does not support item assignment")
worker_1    | Traceback (most recent call last):
worker_1    |   File "/usr/local/lib/python3.7/site-packages/celery/app/trace.py", line 412, in trace_task
worker_1    |     R = retval = fun(*args, **kwargs)
worker_1    |   File "/app/superset/app.py", line 116, in __call__
worker_1    |     return task_base.__call__(self, *args, **kwargs)
worker_1    |   File "/usr/local/lib/python3.7/site-packages/celery/app/trace.py", line 704, in __protected_call__
worker_1    |     return self.run(*args, **kwargs)
worker_1    |   File "/app/superset/tasks/async_queries.py", line 108, in load_explore_json_into_cache
worker_1    |     raise exc
worker_1    |   File "/app/superset/tasks/async_queries.py", line 84, in load_explore_json_into_cache
worker_1    |     payload = viz_obj.get_payload()
worker_1    |   File "/app/superset/viz.py", line 470, in get_payload
worker_1    |     payload["data"] = self.get_data(df)
worker_1    |   File "/app/superset/viz.py", line 2048, in get_data
worker_1    |     df[col] = []
worker_1    | TypeError: 'NoneType' object does not support item assignment

Yet another
worker_1    | [2021-01-27 13:32:29,087: ERROR/ForkPoolWorker-1] Task load_explore_json_into_cache[60028736-ba7e-49e2-8513-f4a66d63f6b1] raised unexpected: TypeError("'NoneType' object does not support item assignment")
worker_1    | Traceback (most recent call last):
worker_1    |   File "/usr/local/lib/python3.7/site-packages/celery/app/trace.py", line 412, in trace_task
worker_1    |     R = retval = fun(*args, **kwargs)
worker_1    |   File "/app/superset/app.py", line 116, in __call__
worker_1    |     return task_base.__call__(self, *args, **kwargs)
worker_1    |   File "/usr/local/lib/python3.7/site-packages/celery/app/trace.py", line 704, in __protected_call__
worker_1    |     return self.run(*args, **kwargs)
worker_1    |   File "/app/superset/tasks/async_queries.py", line 108, in load_explore_json_into_cache
worker_1    |     raise exc
worker_1    |   File "/app/superset/tasks/async_queries.py", line 84, in load_explore_json_into_cache
worker_1    |     payload = viz_obj.get_payload()
worker_1    |   File "/app/superset/viz.py", line 470, in get_payload
worker_1    |     payload["data"] = self.get_data(df)
worker_1    |   File "/app/superset/viz.py", line 2048, in get_data
worker_1    |     df[col] = []
worker_1    | TypeError: 'NoneType' object does not support item assignment

@iercan can you provide some additional information about your situation?

Which visualizations are triggering the errors?
It appears from the stacktraces that you're querying an Apache Druid DB, is that correct? Are you seeing errors with other DB types?
Are you able to run asynchronous queries in SQL Lab?

The same code is used to query the analytics DB in sync and async mode, so I'm also very curious why there's a discrepancy between the two. I should note that celery does not support hot reloads, so workers need to be restarted on any config changes.

@iercan can you provide some additional information about your situation?

Which visualizations are triggering the errors?
It appears from the stacktraces that you're querying an Apache Druid DB, is that correct? Are you seeing errors with other DB types?
Are you able to run asynchronous queries in SQL Lab?

The same code is used to query the analytics DB in sync and async mode, so I'm also very curious why there's a discrepancy between the two. I should note that celery does not support hot reloads, so workers need to be restarted on any config changes.

Which visualizations are triggering the errors?

Dashboard I tested we have filter box, line chart, time series chart, big number and table. Also we have defined metrics on that datasource.

It appears from the stacktraces that you're querying an Apache Druid DB, is that correct? Are you seeing errors with other DB types?

Yes It come from druid but I got this error from mysql too.
worker_1    | [2021-01-28 06:17:13,108: ERROR/ForkPoolWorker-8] Task load_explore_json_into_cache[fd7b0575-725e-4b19-925b-4f6530cfc478] raised unexpected: SupersetVizException('[{\'message\': \'"Could not locate column in row for column \\\'sql_metrics.id\\\'"\', \'error_type\': <SupersetErrorType.VIZ_GET_DF_ERROR: \'VIZ_GET_DF_ERROR\'>, \'level\': <ErrorLevel.ERROR: \'error\'>, \'extra\': None}]')
worker_1    | Traceback (most recent call last):
worker_1    |   File "/usr/local/lib/python3.7/site-packages/celery/app/trace.py", line 412, in trace_task
worker_1    |     R = retval = fun(*args, **kwargs)
worker_1    |   File "/app/superset/app.py", line 116, in __call__
worker_1    |     return task_base.__call__(self, *args, **kwargs)
worker_1    |   File "/usr/local/lib/python3.7/site-packages/celery/app/trace.py", line 704, in __protected_call__
worker_1    |     return self.run(*args, **kwargs)
worker_1    |   File "/app/superset/tasks/async_queries.py", line 108, in load_explore_json_into_cache
worker_1    |     raise exc
worker_1    |   File "/app/superset/tasks/async_queries.py", line 86, in load_explore_json_into_cache
worker_1    |     raise SupersetVizException(errors=payload["errors"])
worker_1    | superset.exceptions.SupersetVizException: [{'message': '"Could not locate column in row for column \'sql_metrics.id\'"', 'error_type': <SupersetErrorType.VIZ_GET_DF_ERROR: 'VIZ_GET_DF_ERROR'>, 'level': <ErrorLevel.ERROR: 'error'>, 'extra': None}]

Are you able to run asynchronous queries in SQL Lab?

Sql lab works fine.

Which visualizations are triggering the errors?

Dashboard I tested we have filter box, line chart, time series chart, big number and table. Also we have defined metrics on that datasource.

It appears from the stacktraces that you're querying an Apache Druid DB, is that correct? Are you seeing errors with other DB types?

Yes It come from druid but I got this error from mysql too.
worker_1    | [2021-01-28 06:17:13,108: ERROR/ForkPoolWorker-8] Task load_explore_json_into_cache[fd7b0575-725e-4b19-925b-4f6530cfc478] raised unexpected: SupersetVizException('[{\'message\': \'"Could not locate column in row for column \\\'sql_metrics.id\\\'"\', \'error_type\': <SupersetErrorType.VIZ_GET_DF_ERROR: \'VIZ_GET_DF_ERROR\'>, \'level\': <ErrorLevel.ERROR: \'error\'>, \'extra\': None}]')
worker_1    | Traceback (most recent call last):
worker_1    |   File "/usr/local/lib/python3.7/site-packages/celery/app/trace.py", line 412, in trace_task
worker_1    |     R = retval = fun(*args, **kwargs)
worker_1    |   File "/app/superset/app.py", line 116, in __call__
worker_1    |     return task_base.__call__(self, *args, **kwargs)
worker_1    |   File "/usr/local/lib/python3.7/site-packages/celery/app/trace.py", line 704, in __protected_call__
worker_1    |     return self.run(*args, **kwargs)
worker_1    |   File "/app/superset/tasks/async_queries.py", line 108, in load_explore_json_into_cache
worker_1    |     raise exc
worker_1    |   File "/app/superset/tasks/async_queries.py", line 86, in load_explore_json_into_cache
worker_1    |     raise SupersetVizException(errors=payload["errors"])
worker_1    | superset.exceptions.SupersetVizException: [{'message': '"Could not locate column in row for column \'sql_metrics.id\'"', 'error_type': <SupersetErrorType.VIZ_GET_DF_ERROR: 'VIZ_GET_DF_ERROR'>, 'level': <ErrorLevel.ERROR: 'error'>, 'extra': None}]

Are you able to run asynchronous queries in SQL Lab?

Sql lab works fine.

I tested locally running async queries against a MySQL database, and am still unable to replicate the issue. All of the stack traces you've posted appear to have something to do with a missing column in the dataset you're querying. I'm not sure how async queries could be the culprit here, as the query execution code is fundamentally the same, but please do let us know if you find any more information.

I tried too many config combinations but unfortunately couldn't resolve issue. For now, I decided to disable that feature and go on.

@robdiciuccio I did some further tests. I tried to change celery execution pool from prefork to gevent and interestingly errors gone. But this time I started to experience more slow load times or even some charts can not load even if I don't see any error in logs.

@robdiciuccio Here is my final report. Chart load problem gone after I set GLOBAL_ASYNC_QUERIES_POLLING_DELAY from 500 to 1500. I observed dashboard load times with GAQ enabled and disabled. It seems while GAQ disabled dashboards loads faster. I believe it is because I had to increase pooling delay and our dashboards usually have 5-10 charts. As a result we decided to go on with GAQ disabled for now. I'd glad test it again after some improvements. Thanks

Thanks for the updates @iercan. Interesting that the errors were resolved when increasing the polling delay. I had this running on my Macbook at 200ms delay with no errors. What kind of hardware or virtual machine are you running this on?
Also, websocket support should be shipping soon, which will negate the need for polling.

We deployed superset via docker on a dedicated server which has amd cpu. As a client I'm using ubuntu 20.04 with chrome also have amd cpu.

I've been able to reproduce the issue using Celery prefork concurrency. It seems to be the same issue documented here: https://github.com/apache/superset/issues/10530
Continuing to investigate.

@iercan can you test the fix in #13350 to see if that resolves the issue?

@robdiciuccio I'm still getting errors.
worker_1    | [2021-02-26 07:05:27,835: INFO/ForkPoolWorker-7] Cache key: fd68af649f2858ffb1156862272ba704
worker_1    | Serving from cache
worker_1    | [2021-02-26 07:05:27,836: INFO/ForkPoolWorker-7] Serving from cache
worker_1    | Cache key: 926fcd84c4ea3e42956ae51436269022
worker_1    | Cache key: a98643384e8bc016e02fe68b97211a32
worker_1    | [2021-02-26 07:05:27,840: INFO/ForkPoolWorker-1] Cache key: a98643384e8bc016e02fe68b97211a32
worker_1    | [2021-02-26 07:05:27,840: INFO/ForkPoolWorker-8] Cache key: 926fcd84c4ea3e42956ae51436269022
worker_1    | Serving from cache
worker_1    | [2021-02-26 07:05:27,842: INFO/ForkPoolWorker-1] Serving from cache
worker_1    | Serving from cache
worker_1    | [2021-02-26 07:05:27,842: INFO/ForkPoolWorker-8] Serving from cache
worker_1    | Cache key: e497d4025ce15e3ac7f4d1304bbfd1e8
worker_1    | [2021-02-26 07:05:27,846: INFO/ForkPoolWorker-10] Cache key: e497d4025ce15e3ac7f4d1304bbfd1e8
worker_1    | Serving from cache
worker_1    | [2021-02-26 07:05:27,848: INFO/ForkPoolWorker-10] Serving from cache
worker_1    | [2021-02-26 07:05:27,854: INFO/ForkPoolWorker-1] Task load_explore_json_into_cache[530e7fcb-c6e6-448b-be9b-4ff0307dd745] succeeded in 0.019942453131079674s: None
worker_1    | Cache key: e3f5fbde933b2a77404ac5ba57910469
worker_1    | [2021-02-26 07:05:27,861: INFO/ForkPoolWorker-10] Cache key: e3f5fbde933b2a77404ac5ba57910469
worker_1    | Serving from cache
worker_1    | [2021-02-26 07:05:27,862: INFO/ForkPoolWorker-10] Serving from cache
worker_1    | [2021-02-26 07:05:27,864: INFO/ForkPoolWorker-7] Task load_explore_json_into_cache[7e47b489-286a-43e4-9bb4-ff5706aa5050] succeeded in 0.0350632518529892s: None
worker_1    | [2021-02-26 07:05:27,871: ERROR/ForkPoolWorker-8] Task load_explore_json_into_cache[53952638-59ac-481b-b30f-381469c1b4c8] raised unexpected: NoSuchColumnError("Could not locate column in row for column 'table_columns.id'")
worker_1    | Traceback (most recent call last):
worker_1    |   File "/usr/local/lib/python3.7/site-packages/celery/app/trace.py", line 412, in trace_task
worker_1    |     R = retval = fun(*args, **kwargs)
worker_1    |   File "/app/superset/app.py", line 116, in __call__
worker_1    |     return task_base.__call__(self, *args, **kwargs)
worker_1    |   File "/usr/local/lib/python3.7/site-packages/celery/app/trace.py", line 704, in __protected_call__
worker_1    |     return self.run(*args, **kwargs)
worker_1    |   File "/app/superset/tasks/async_queries.py", line 108, in load_explore_json_into_cache
worker_1    |     raise exc
worker_1    |   File "/app/superset/tasks/async_queries.py", line 84, in load_explore_json_into_cache
worker_1    |     payload = viz_obj.get_payload()
worker_1    |   File "/app/superset/viz.py", line 454, in get_payload
worker_1    |     columns = set(self.datasource.column_names)
worker_1    |   File "/app/superset/connectors/base/models.py", line 145, in column_names
worker_1    |     return sorted([c.column_name for c in self.columns], key=lambda x: x or "")
worker_1    |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/attributes.py", line 287, in __get__
worker_1    |     return self.impl.get(instance_state(instance), dict_)
worker_1    |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/attributes.py", line 723, in get
worker_1    |     value = self.callable_(state, passive)
worker_1    |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/strategies.py", line 760, in _load_for_state
worker_1    |     session, state, primary_key_identity, passive
worker_1    |   File "<string>", line 1, in <lambda>
worker_1    |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/strategies.py", line 902, in _emit_lazyload
worker_1    |     .with_post_criteria(set_default_params)
worker_1    |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/ext/baked.py", line 544, in all
worker_1    |     return list(self)
worker_1    |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 100, in instances
worker_1    |     cursor.close()
worker_1    |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__
worker_1    |     with_traceback=exc_tb,
worker_1    |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
worker_1    |     raise exception
worker_1    |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 80, in instances
worker_1    |     rows = [proc(row) for row in fetch]
worker_1    |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 80, in <listcomp>
worker_1    |     rows = [proc(row) for row in fetch]
worker_1    |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 524, in _instance
worker_1    |     tuple([row[column] for column in pk_cols]),
worker_1    |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 524, in <listcomp>
worker_1    |     tuple([row[column] for column in pk_cols]),
worker_1    |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/result.py", line 686, in _key_fallback
worker_1    |     replace_context=err,
worker_1    |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
worker_1    |     raise exception
worker_1    | sqlalchemy.exc.NoSuchColumnError: "Could not locate column in row for column 'table_columns.id'"

Sorry I realized I made test from wrong branch thats why I removed previous answer. I retested it with both prefork and gevent pools and it looks fine. Thanks for the fix.

@robdiciuccio I also make some performance tests. I loaded a dashboard that have more than 20 charts. Looks like with GAQ dashboards loads far slower. Also chart loads not finish until second explode request, I don't know if this is expected behavior
with GAQ dashboard loads in 13 sec

without GAQ dashboard loads in 5 sec

@iercan thanks for the report on performance. See my tests here. It looks like you're using prefork with Celery; what's your concurrency setting (how many processes)?

@robdiciuccio I did my test with gevent and concurrency was 500. I used Refresh dashboard options which reload dashboard without cache.

@iercan in my tests, celery with gevent performed the worst, with prefork returning results comparable to synchronous operation.

@robdiciuccio I see. I prefer to use gevent because prefork leaves defunct webdriver processes.

@robdiciuccio I also made some performance tests. I loaded a dashboard that have more than 20 charts. Looks like with GAQ dashboards loads far slower. Also chart loads does not finish until second explode request, I don't know if this is expected behavior
with GAQ dashboard loads in 13 sec  without GAQ dashboard loads in 5 sec

Hi @robdiciuccio , I also have performance problem with dashboard have 25 charts. My superset is v2.1.0 and launched on docker. I have tried sharding domains, increase CPU, RAM, config increase gunicorns workers and threads but didn't work. I also test a dashboard with all chart query to a csv file on postgresql to see whether the connection to ouside database is the bottleneck but even with the postgre db, the response time of the server from 25 query requests still not improve.
Have you overcome this performance problem?
Thank you


Easy to mismatch residual norm criteria and actual norm used by solver

Some of our iterative solvers use the exact residual norm for their stopping criteria, while others use the implicit residual norm. For a user, it is not possible to know which one is actually used. This results in an extra reduction, or an exception. If an exception is thrown, it might be easy to fix, but the extra reduction will just happen silently.
We should make it easier for users to select the correct criteria. Some solution ideas:

add residual type (explicit, implicit, none) to solver traits
introduce new criteria which allows the solver to decide if ResidualNorm or ImplicitResidualNorm is used. (this should still allow users to explicitly specify either of them)

Here is a list of solvers and their used residual norm. Both Ir and Multigrid use neither.
Check exact residual norm:

[Cb_]Gmres
Idr

Check implicit residual norm:

Bicg
Bicgstab
Cg
Cgs
Fcg

Any solver can use the ResidualNorm as the stop criterion.
If the solver does not have it during solving, ResidualNorm will introduce an additional spmv and norm as requests.
Thus, solver does not always have this additional operation when they do not need residual norm during operation.
My concern about the automatic ResidualNorm/ImplicitResidualNorm is that it will introduce another confusion out of expectation.
When user use it on Gmres/CbGmres, they may think they get the actual residual reduction. However, when they use it on Cg, they do not get the actual residual norm from their expectation.
In the original settings, if they want to use ResidualNormReduction as stop criterion, they can use it for all solver.
When they use ImplicitResidualNormReduction, they should know it checks the implicit residual norm different from the residual norm.


pull request for e2e test at 12/09/2016 16:15:05

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

optest/docs/updated_at.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


✨ Added flow list handling for flow_of_flows

Summary

Importance

Checklist

This PR:

[x] adds new tests (if appropriate)
[ ] updates docstrings for any new functions or function arguments (if appropriate)
[ ] updates CHANGELOG.md with a summary of the changes

@lzuchowska looks good, only the changelog entry is missing

Pls re-add the changelog entry here (it was "- Added flow_of_flows list handling")

Looks good, just make 4 test flows for the test in the "test" project and use those (you're only checking task & flow execution anyway, so the content of the tasks/flows doesn't matter).


gemini annotate error with no variants updated

just need to check if to_update == []

this is in master


[Bug]: Possibly unnecessary prefetch during GroupIntoBatches

What happened?
As I was inspecting why the Dataflow "Total streaming data processed" metric was so much higher (2-4× depending on the pipeline) than the actual data that was being processed as it only has one shuffle/gbk/gib/etc transform I stumbled upon something that I couldn't justify being there:
https://github.com/apache/beam/blob/f549fd33abdc672143ccbe3f0f66104995d30fe6/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/GroupIntoBatches.java#L566-L569
I might be missing some context here, but I'm not even sure what benefit this provides for any generic GroupIntoBatches transform, but it seems to be especially useless if called on a GroupIntoBatches.WithShardedKey (when every data the batch contains comes from the same worker already).
I would appreciate any insight before I submit a PR to change this.
Issue Priority
Priority: 3 (minor)
Issue Components

[ ] Component: Python SDK
[X] Component: Java SDK
[ ] Component: Go SDK
[ ] Component: Typescript SDK
[ ] Component: IO connector
[ ] Component: Beam examples
[ ] Component: Beam playground
[ ] Component: Beam katas
[ ] Component: Website
[ ] Component: Spark Runner
[ ] Component: Flink Runner
[ ] Component: Samza Runner
[ ] Component: Twister2 Runner
[ ] Component: Hazelcast Jet Runner
[X] Component: Google Cloud Dataflow Runner

.

Thanks for reporting this. This if block exists at the first place when GroupIntoBatches was implemented (#2610), assuming there was some reason for that. WithShardedKey was added some time later (still 3y ago). Haven't looked into detail, would like to dig into it.

"Total streaming data processed" metric was so much higher (2-4× depending on the pipeline) than the actual data

Have you tested that this is caused by the pointed code path? Would be nice if there is some benchmark data to share with

"assuming there was some reason for that"
I assume too, but there is literally no comment or discussion about it anywhere I could find. Meanwhile when I just think it though I can't find any justification for it.
"Have you tested that this is caused by the pointed code path? Would be nice if there is some benchmark data to share with"
Kinda. I didn't launch any pipeline with modified code to pinpoint this as 100% certainty.
Also I don't think my pipeline result can be considered good enough for benchmarking regarding consistency and straightforwardness.
... but basically what I did was that I ran the pipelines on the same kafka streams with different configs concurrently. So theoretically the "shuffled" data amount should be equal. It wasn't. Meanwhile everything else (read data from kafka, written data into bq, etc) were essentially identical. My original goal wasn't to test this, but to actually optimize the pipelines by using non-default config values as the default seemed rather unoptimized based on the GCS storage usage pattern, and average batch size, and sometimes the pipeline couldn't even scale down due to the estimated backlog size (when cpu was clearly available)
In order to do that I was playing with these configs:
options.setMaxBufferingDurationMilliSec(...);
options.setGcsUploadBufferSizeBytes(...);

When I increased them - as expected - the behaviour, cpu utilization, storage usage pattern, etc changed in a way that corresponds with having bigger batches, but I noticed increased costs due to the "processed data amount". So it was obvious for some reason it handles data differently, so I started checking the code what it might be, and this seems like the only thing that could cause this.
For example one run like I mentioned:
Pipeline1:

gcsUploadBufferSizeBytes: 2MB
maxBufferingDurationMilliSec: 30s
every other config identical
Total vCPU time 2 139,443 vCPU hr
Kafka read:

Elements added (Approximate) 8 617 148 267
Estimated size 5,89 TB

BQ Write:

8 617 148 273 + 8 617 148 224
5,97 TB + 5,99 TB

Total streaming data processed 26,66 TB

depending on if both reading and writing counts or only one of them this should be either around ~12TB or ~24TB, so 26.66 seems okay

Pipeline2:

gcsUploadBufferSizeBytes: 16MB
maxBufferingDurationMilliSec: 30s
every other config identical
Total vCPU time 2 232,822 vCPU hr
Kafka read:

Elements added (Approximate) 8 637 509 081
Estimated size 6,16 TB

BQ Write:

8 637 509 008 + 8 637 509 087
6,06 TB + 6,09 TB

Total streaming data processed 46,06 TB

12/24 vs 46

@Abacn
I have created two PRs for the two most likely implementations, feel free to pick what you prefer :)

Using an experimental flag
https://github.com/apache/beam/pull/26618
Completely removing it
https://github.com/apache/beam/pull/26619

Thanks for sharing benchmark result! Let me help find people with experience to review

Also thanks for being persistent. I am also looking into it in the mean time, and the benchmark result looks impressive. To my understanding, at least there is no harm to add a parameter to the GroupIntoBatches PTransform specifically, something like GroupIntoBatches.disablePreFetch

Before I created the PRs I was thinking what option to choose, and I decided against having a specific method on the GroupIntoBatches (GIB from now) itself for various reasons:

GIB is becoming a widely used transform, not quite as popular - and never will be - as GBK, but it's getting there

it's already being used at multiple places and I don't see why this change would be recommended only for a few but not for the others, so we would have to add this at every place
I would assume every streaming sink that could benefit from batch insertion will eventually utilize GIB, and I see no reason to force developers to be aware of this performance optimization (not to mention "non-beam" devs who utilize GIB in business pipelines)

if we decide to let the developers configure this externally at a transform instance level it would mean a lot of boilerplate code, as GIB functionality is usually deep inside other transforms
this functionality isn't strictly related to GIB itself, grouping by size, count, duration etc are, but having the need/or not to prefetch data sounds like an implementation detail of the stateful functionality

So to finally sum it up, either stateful processing needs it, and we need it everywhere, or it doesn't and we don't need it at all either. I saw no reason not to so I would have just removed it completely, but I also created the experiment version, just so we can release/use it without potentially breaking a release.

yeah, thanks, get it. Experimental flag sounds good in turns of not changing the behavior of the default config.
I am trying to run some test also in this weekend.

CC: @kennknowles (from file history) may have insight

Update: I tested #26618 with a test pipeline and found no noticable change with the switch on/off. Basically I am generating a steady stream of 100k element per second and each of 1 kB, so it is 1 MB/s throughput.
pipeline option provided: -Dexec.args="--project=*** --tempLocation=gs://***/temp --region=us-central1 --runner=DataflowRunner --numWorkers=5 --autoscalingAlgorithm=NONE --dataflowWorkerJar=/Users/***beam/runners/google-cloud-dataflow-java/worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.48.0-SNAPSHOT.jar --experiments=enable_streaming_engine"
Both jobs are drained after 15 min.

Metric
prefetch disabled
prefetch enabled (current master)

Elapsed time
19 min 23 sec
19 min 10 sec

elements_read
28,003,335
54,323,517

Total streaming data
84.76 GB
167.62 GB

ratio (kB / element)
3.03
3.09

Both settings having similar ratio (3kB per element, which is 3 times the actual data size), however, disabling the prefetch show significant regression on the through put, and causing surging backlog:
Backlog:
(pregetch disabled)

(prefetch enabled)

@nbali Have you been able to test your changed code with a pipeline? To test modified java-core code with your pipeline, one can build both sdks-java-core and dataflow worker jar (for runner v1) sth like
./gradlew -Ppublishing  sdks:java:core:jar
./gradlew :runners:google-cloud-dataflow-java:worker:shadowJar

and use the compiled jar as the dependency of java core, and pass the shadow worker jar to dataflow as pipeline option above.

test pipeline:

public class GroupIntoBatchesTest {

  public static void main(String[] argv) {
    PipelineOptions options = PipelineOptionsFactory.fromArgs(argv).withValidation().as(PipelineOptions.class);
    Pipeline p = Pipeline.create(options);

    p.apply(GenerateSequence.from(0).withRate(100000, Duration.standardSeconds(1)))
        .apply(Window.into(FixedWindows.of(Duration.standardSeconds(10))))
        .apply(MapElements.via(new MapToKVFn(1000)))
        .apply(GroupIntoBatches.ofSize(50000))
        .apply(ParDo.of(new DoFn<KV<Integer, Iterable<byte[]>>, Void>(){
          @ProcessElement
          public void procees(ProcessContext ctx) {
            System.out.println("grouped key:" + Objects.requireNonNull(ctx.element()).getKey());
          }
        }));
    p.run();

  }

  static class MapToKVFn extends SimpleFunction<Long, KV<Integer, byte[]>> {

    private transient Random rd;

    private final int valueSize;

    public MapToKVFn(int valueSize) {
      this.valueSize = valueSize;
    }

    @Override
    public KV<Integer, byte[]> apply(Long input) {
      if (rd == null) {
        rd = new Random();
      }
      byte[] data = new byte[valueSize];
      rd.nextBytes(data);
      return KV.of(Long.hashCode(input)%10, data);
    }
  }
}

I haven't had the time yet - I will try to make time in the upcoming days -, but your example has a theoretical maximum at 100k element/sec, executed for 900s, meanwhile it's only 28M and 54M. Which means it was 30-60k/sec. The hashing split it into 10 keys. So a single key received about 3-6k msg/sec. The windowing essentially closed every batch after 10s. So a window for a key received 30-60k element, which indicates totally different amount of prefetching (even if both are using the master version), yet the data amount is the same. Seems odd to me. Do you have the input/output stats of the GIB transform? Also over a minute delays with both cases with 10s windowing? It's seems CPU limited. I mean isn't this essentially single threaded?

@Abacn
Well, I only had time now at work. At least some. I haven't tried running it with the modified runner yet, only just by modifying parameters.
First of all I'm not saying your tests are invalid in any way, but I also did mine and saw huge differences. I didn't mention originally as I thought it's irrelevant and should happen with every GIB- my bad - but where I noticed this is the GroupIntoBatches inside BigQueryIO. More specifically this one:
https://github.com/apache/beam/blob/bb044f4863cd151214f61174ac3c097bea724098/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BatchLoads.java#L686-L689
This is what I use on the BQ.Write:
.withMethod(BigQueryIO.Write.Method.FILE_LOADS)
.withTriggeringFrequency(...)
.withAutoSharding()

For the benchmark I removed some irrelevant parts from my pipeline, but kept the core logic of Kafka.Read->BigQuery.Write. First deployed a flex template, then launched the template with different configurations to start consuming.
About the configurations:

the size is hardcoded, so that is 500'000 in every test.
the byteSize is configurable, I tested with 2 MB and 100 MB.
the bufferingDuration is configurable as well, I tested with 1 minute and 1 hour.

That gives me 4 scenario.
The source is a kafka stream with 12 partitions, about 1k elements/sec and 1 MB/sec for every partition.
Reaching the prefetchSize (500'000 / 5) takes typically almost 2 minutes, so if we trigger the batches more often we never call the "prefetch" code. The 2 MB limit most likely triggers it immediately in a few seconds, so we should never "prefetch". The 100 MB limit most likely triggers 1 prefetch (100MB / 1KB ~100'000).
After 22 min of runtime:

read from kafka: 13-15 GB, 17 million
streamed data:
2 MB + 1 min -> 30,77 GB (~2)
2 MB + 1 hour -> 30,99 GB (~2)
100 MB + 1 min -> 63,38 GB (~4)
100 MB + 1 hour -> 65,09 GB (~4)
GroupIntoBatches.WithShardedKey metrics from 18 million element input:
2 MB + 1 min ->~7000 batches
2 MB + 1 hour -> ~7000 batches
100 MB + 1 min -> ~700 batches
100 MB + 1 hour -> ~120 batches

As you can see there are huge differences in the results, but also compared to your test pipeline in the code as well, for example you don't use .withShardedKey() and trigger it at a much higher frequency.
For the next run I have increased the 100 MB option to 256 MB to trigger prefetching more (and also switched to n1-highmem-8 from n1-standard-8 just to be sure avoid an OOME). Everything else is the same.
Drained after 40 min:

read from kafka: 16-19 GB, 20-21 million
streamed data:
2 MB + 1 min -> 39 GB (~2)
2 MB + 1 hour -> 39 GB (~2)
256 MB + 1 min -> 75 GB (~4)
256 MB + 1 hour -> 96 GB (~5)

I could only guess the reason you couldn't see it is that you fire too quickly and it might be still cached? I would modify the rate from 100'000/sec to 10'000/sec for example (so with the key creation, it will be 1'000/sec for every key on average).
When I have time again, I will try with a modified runner.

Quick update, runtime 11h, same as the previous example with the exception that the message size has been reduced to ~1/10, but I process every message 10 times, so still 1MB/sec for every partition, but 10k elements/sec.

read from kafka: 270-280 GB, 380 million (it's 470 GB and 3.8 billion at the GIB)
streamed data:
2 MB + 1 min -> 1 TB (~2)
2 MB + 1 hour -> 1 TB (~2)
256 MB + 1 min -> 1,48 TB (~3)
256 MB + 1 hour -> 1,58 TB (~3)

@Abacn Tested with the modified GroupIntoBatches transform too, and it didn't influence the processed data amount for me either. Something is still causing non-linear increase in the processed data amount, but this isn't it.

We identified a cause of large ratio of (shuffled data) / (processed data): #27283 which is recently fixed in #29517. It may/may not related to this use case

@Abacn I see how that would cause a similar pattern in unexpected shuffled data amount, and I also get it how that change fixes that issue, but what I can't see is where do we use an Iterable in the tested scenarios that would cause this? When checking GroupIntoBatchesDoFn the obvious pick would be BagState, but that is what this test was about, and we clearly didn't read it. Looking at the other states (CombiningState, ValueState) nothing really sticks out for me... OR is that the registerByteObserver() was called regularly on the BagState's Iterable content by the framework and although we clearly didn't explicitly read it, it was read as this unintended side-effect therefore causing the costs?


Investigate Auth0 / Keycloak implementation

Context:
Currently I mostly rely on service built-in authentication mechanisms. It would be better / easier to set up a separate authentication mechanism so that I can centralise my user accounts, and possibly even implement SSO.
Related issues:

https://github.com/billimek/k8s-gitops/issues/36

I just went through this and found vouch-proxy and keycloak slightly easier to setup

I now have oauth2-proxy and Keycloak up and running after a brief dalliance with Auth0 so this one can finally be closed :)


Absent replacement setters in AbstractArchiveTask

Expected Behavior

Setters to replace the deprecating setArchiveName() and setDestinationDir()
Current Behavior

setArchiveName() and setDestinationDir() are marked @Deprecated but both point to get methods with no replacement set methods.
Context

I'm working on 6.5.1, and using --warning-mode all to check

I'm doing this to prepare for 7.0 compatibility. In our use case, we make use of Zip (which extends AbstractArchiveTask) inside our own custom tasks, and have some simple code that uses the inherited setters.

Steps to Reproduce

Perhaps easiest way to see it is in the generated javadocs or the source code?

https://docs.gradle.org/current/javadoc/org/gradle/api/tasks/bundling/AbstractArchiveTask.html#setArchiveName-java.lang.String-
https://docs.gradle.org/current/javadoc/org/gradle/api/tasks/bundling/AbstractArchiveTask.html#setDestinationDir-java.io.File-

Note both point to "get" methods, and a similarly named "set" method doesn't exist.
Your Environment

N/A

There aren't supposed to be setters.  The getter returns a Property, which has a separate methods for configuring the value to allow plugins to lazily wire things together.
So this:
archiveTask.setDestinationDir(someFile)
becomes:
archiveTask.getDestinationDirectory().fileValue(someFile)
There are other setter methods (convention, set, etc), which one you need depends on what you're doing: https://docs.gradle.org/current/javadoc/org/gradle/api/file/DirectoryProperty.html

Thanks, I see!
It's a feature I wasn't familiar with.


automate security headers and add them in google sheets

all sites have security headers and we have to keep track of it so i made a script to automate that work

Hey! I'm just starting to get involved in open source and, based on the tags, thought this might be a good place to start. I'm still a bit confused on what needs to be done though. Do you mind elaborating on the task ?

Hey @avyayjain can you please explain it a Lil bit more

So there is a site called securityheaders.com where you can check how secure the site is and where does the website lacks so I made a script where you just have to enter the website url and you can check the tags

can i make a pr for it

Cool, go ahead

pls check my pr as it is showing flake8 error and greeting but i have pass it in my code but it is showing error for other codes


how to get product ids?

hi!
I wanted to try out your library but I am a bit overwhelmed how to use it correctly. I would like to get some charting ID from a stock, for example for the AEX. I cant seem to find in the examples how to get the product id that belongs to a stock from a company defined by an isin. Could you provide an example for that? Thanks!

Hi,
what you are calling charting ID is actually called like this in Degiro`s API :

vwdId
issueid
vwdIdSecondary

This list is not exhaustive.
Not to confuse with the id parameter which is used in Degiro`s API for some "trading related operations".
I think once you will fixed #26 and will be able to see the data it will become easier to understand for you.
If you need charts these issues might interest you :

#9
#4
#2


Changes send and recv functions in the C++ interface to virtual

This is required in order to override the mip::DeviceInterface in the case where the user wants to do something like record data agnostic of the connection type

I think it would be better to implement a RecordingConnection class which acts as a proxy for the real connection, but which also logs the data. See my example below. This way data recording can be set up independently of the code which declares the DeviceInterface.
class RecordingConnection : public Connection
{
public:
  RecordingConnection(Connection& connection, std::ostream* recvFile, std::ostream* sendFile=nullptr) :
    mConnection(connection), mSendFile(sendFile), mRecvFile(recvFile)
  {
  }

  virtual bool sendToDevice(const uint8_t* data, size_t length) {
    // May want to make this conditional on the data successfully sending
    writeToFile(mSendFile, data, length);
    return mConnection.sendToDevice(data, length);
  }

  virtual bool recvFromDevice(uint8_t* buffer, size_t* count_out, size_t max_count, Timestamp* timestamp_out) {
    bool ok = mConnection.recvFromDevice(buffer, count_out, max_count, timestamp_out);
    if( ok )
      writeToFile(mRecvFile, buffer, *count_out);
    return ok;
  }

private:
  static void writeToFile(std::ostream* file, const uint8_t* data, size_t count);

private:
  // Files may be NULL to not record one direction or the other
  std::ostream* mSendFile;
  std::ostream* mRecvFile;
  Connection& mConnection;
};

Closing as this was superseded by https://github.com/LORD-MicroStrain/mip_sdk/pull/24


Item click with a button inside the view

Hi,
my item xml in the horizontal list looks like this:
<RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android"
 android:layout_width="200dp"
    android:layout_height="200dp"
    android:descendantFocusability="blocksDescendants" >
    <ImageView
        android:layout_width="match_parent"
        android:layout_height="match_parent" />
    <ImageButton
        android:id="@+id/imageButton"
        android:layout_width="50dp"
        android:layout_height="50dp"  />
</RelativeLayout>

Then I have registered the horizontal  ItemClick and the ImageButton Click events.
Actual behaviour: If I click on the ImageButton, both events are triggered.
Expected behaviour (and actual behaviour on standard components like a gridview): If I click on the ImageButton only its event is triggered
Is this a bug or am I missing something? Can you reproduce this on your side?
Thanks in advance!

Read the nice notice in the readme:

NOTE: No longer actively maintained. I do take pull requests. Consider thinking about whether ViewPager solves your problem instead of using this library.

Basically... don't use this unless you really, really have to!


ARM disassembly support

Proposes a change that introduces Thumb-v2 and Arm instruction set disassembly for ARM ELF files.

Thank you for your contribution and a bit of cursed knowledge :)

0.2.44 is out


don't export unnamed tests / runs

Quick revert: for unnamed tests, use a symbol as a temporary name instead of an identifier.
When it's an identifier, it gets exported by the top-level (provide (except-out ....)). That leads to a conflict when one file imports two others.
Long term question: do we want unnamed tests to get provided?
cc @conradz

Ah, so named tests are currently being exported? What is exactly is exported, and how is it referenced? FWIW, I don't particularly care about unnamed tests, but it would be nice to have them exported with arbitrary identifiers.

Oops, no. Only runs get exported, not tests.


Fix PowerShell to save Defender rule in GPO

Save the new Firewall Rule in GPO

@takeokams : Thanks for your contribution! The author(s) have been notified to review your proposed change.

#sign-off

#sign-off


Transitive references do not show all the time in the Dependency node

File -> New ConsoleApp (ConsoleApp)
File -> New Library (A)
File -> New Library (B)
Add reference from ConsoleApp -> A
Add reference from A -> B

Expected:

Actual:

To show the nodes, I had to restart VS. Even if I closed/reopened solution it didn't work.

Wonder if this related to: https://github.com/dotnet/roslyn-project-system/pull/1392

it might be, true. since if you look at the pic B also does not have anything under A project, not only under ConsoleApp A's dependencies are missing. However it might be something else also. I suppose repro steps are the only thing your did? What was the restore status after adding project references? It's is hard to repro in general such things, if you see this (or similar) again , i would really need you to attach a debugger and set break points in http://index/?rightProject=Microsoft.VisualStudio.ProjectSystem.Managed.VS&file=ProjectSystem\VS\Tree\Dependencies\DependenciesSubTreeProviderBase.cs&line=242 (to see if we get any events from CPS) and in http://index/?rightProject=Microsoft.VisualStudio.ProjectSystem.Managed.VS&file=ProjectSystem\VS\Tree\Dependencies\DependenciesGraphProvider.cs&line=209 to see if it is even called for new dependency nodes or updated ones... on my machine i never could see such things , will try again.

Can you verify if this still repros?

I investigated that further and found a potential issue here , testing a fix right now.

I repro this consistently with the MVC repo:

git clone https://github.com/aspnet/Mvc
cd Mvc
git checkout nimullen/migration.badvs
Open MVC.sln

tag @NTaylorMullen as FYI


convert_error assumes fixed-width characters

Hi,
a nom and Rust newbie here. Thanks for developing such a great library :pray:
Recently, I have been tinkering with parsing a file format containing tab characters. It just so happens that tabs can make convert_error output really confusing.
A minimal example (using nom 6.0.0-alpha3),
use nom::{
    bytes::complete::tag,
    character::complete::space1,
    error::{convert_error, ParseError, VerboseError},
    sequence::delimited,
    Err, IResult,
};

fn foobar<'a, E: ParseError<&'a str>>(input: &'a str) -> IResult<&'a str, &'a str, E> {
    delimited(tag("foo"), space1, tag("bar"))(input)
}

fn main() {
    let input = "foo\tbaz";

    match foobar::<VerboseError<&str>>(input) {
        Err(Err::Error(e)) => {
            println!("{}", convert_error(input, e));
        }
        _ => {}
    }
}

produces (YMMV, depending on terminal configuration)
0: at line 1, in Tag:
foo     baz
    ^

That is, convert_error assumes every output character (including tabs) to be one column wide.
This particular example isn't too bad yet. Given a more complex sequence of tokens after a tab character, you can probably see how misleading it can get, though.
Sure, I'd like my use case to "just work". However, looking at how the problem is solved in, say rustc1, it seems somewhat nontrivial. As a newcomer I really can't tell if a typical user needs nom to provide high-quality, human-readable error traces at all. Maybe you can tell me if it's something you want to see nom handling in the future?
IMHO, it would be nice to at least mention this assumption in the documentation and/or examples.
As said, I'm not too confident of a rustacean yet but I would like to get into contributing if this is something I can help with :)

[1]
See NonNarrowChar, Loc, source_map::SourceMap::lookup_char_pos in rustc_span.
↩

it would be good to have convert_error support this use case, but I'm not sure I have the bandwidth to work on this? Could you send a PR?

Hey, thanks for the reply. I will try hacking something together. However, being busy with other things™ too, it might take a while.
Since last time, I did a little bit of research discovering codespan-reporting and annotate-snippets both of which have, unsurprisingly, encountered effectively the exact same issue already. They also seem to solve the problem of error reporting pretty well. Thus, I infer that it really makes no sense for nom to delve too deep into their problem space, as some one using nom as a parser in a more complex application will likely set up user-facing error reporting using such a library anyway, avoiding convert_error altogether.
For simpler use cases (such as mine) I still see the value of convert_error error trace. With my current knowledge, I would suggest nom to take the rustc route, replacing tabs in the input with a fixed amount of spaces. Alternatively, one could try something more sophisticated, e.g. via unicode-width, but that would add a dependency and yet wouldn't be a 100% solution.
Any thoughts or comments are warmly welcome.


Added missing USB device types

There were some smartctl's device types missing. Added the "snt*" types, which I use. Information taken from https://www.smartmontools.org/wiki/USB

Thanks!


Setup update

This PR is intended to solve user issues where the environment isn't up to date with the latest releases from efabless.

change make setup to use a python script, making it easier to download and prettier
make setup now uses central CI repo to fetch the versions from upstream
make setup outputs tool_versions.json to indicate which tools the user is using in their environment
make setup now has a log file called setup.log
all targets run check_environment which checks their tools versions and compares it to the upstream json file, if any version is outdated it will throw an error and exit

@marwaneltoukhy the docs reference make install, make install_mcw and other make targets that are removed by this PR. The docs should be updated accordingly.

They are included

run make setup (with no PDK/OPENLANE ROOT set)
got a warning: Makefile:48: warning: undefined variable 'MPW_TAG'
also it says it's installing cocotb, which might not be what's wanted as it could overwrite a version I already have. I thought cocotb was usually used inside the dockers?
afterwards get the expected files in dependencies.
I get tool_versions.json created (attached)
tool_versions.json
setup.log looks fine, (feature request - add timestamps)
next I run make user_proj_example (I need to remember to use a different demo project, this takes tooooo long)
wasn't sure how to test the check_environment functionality.
make open-gui-user_proj_example
seems to hang:

open-odb-gui-user_proj_example
fails with a stacktrace

I then removed dependencies, set PDK and OPENLANE ROOT and installed again. I was able to build the user project again.

@mattvenn I can't replicate the issue you're seeing with opening the gui, this is an OS specific issue. Can you try adding --privileged in the docker_extra_args in openlane/Makefile, and let me know if that works for you?


Fundraising for Open Ulmus

This issue tackles finding funds for Open Ulmus through open collective and other crowdsourcing platforms. We to find a way to support our daily operational costs through a collective.
Issues that relate:

[ ] Establish Open Ulmus Collective
[ ] Draft Initial Open Ulmus Budget
[ ] Plan the flow of money back to collectives using this platform.

Since Open Ulmus could be looked at as a subproject or flavor of ELMS (and any new features/functionality/docs/etc will be contributed back to the mothership), possibly we should direct our fundraising efforts (once we put together a plan) to https://opencollective.com/elmsln
Then a % could flow to the Open Ulmus project team via it’s collective. That way the hosting cost could be recouped from funds coming into the ELMS collective. Also can dole out incoming funds to support the overall ELMS project.
We’ll also be able to collect and send a % of funds from Open Ulmus once the service is up an running… but initially I think ELMS already has a lot of name recognition and value to multiple communities.


Omitting 'redis' key in config.yml gives Undefined index: redis error

Omitting 'redis' key in config.yml gives
(1/1) ContextErrorException
Notice: Undefined index: redis

so you can't configure the bundle without using a redis caching service?

Redis is a strict requirement at the moment as locking mechanism and all the counters are stored in there.
There's an open issue in library repository, but I don't know if someone would really use any of other integrations.
In bundle configuration that key should be made required so that developer would have nicer error message.


clear console

Issue Type: Feature Request
There is no shortcut key to clear the console
VS Code version: Code 1.40.2 (f359dd69833dd8800b54d458f6d37ab7c78df520, 2019-11-25T14:54:45.096Z)
OS version: Windows_NT x64 10.0.17763

Go ahead and assign your own:
https://code.visualstudio.com/docs/getstarted/keybindings


ebiten: NewImage freezes when called after RunGame finishes

Ebiten should raise more explicit errors.
CC @wasedaigo

I also experienced in gopherjs and wasm.

I have not expected what happens after RunGame. Every operation should be forbidden...

There is no problem in window about new image open after run ().
Chrome, edge, i.e. using gopherjs All browsers freeze.
The wasm version of the chrome browser is also stopped.

Do this after #1122


Set RC4 defines on libcrypto/liblegacy

Also add missing prototype for rc4_md5_enc.
Fixes #21150

This pull request is ready to merge

Merged to master, 3.1, and 3.0 branches. Thank you for the reviews.


MSID: 88431 Version: 2 DOI: 10.1101/2<IP_ADDRESS>3802

MSID: 88431
Version: 2
Preprint DOI: https://doi.org/10.1101/2<IP_ADDRESS>3802
Step 1. Awaiting reviews
Editorial to post reviews via hypothesis
Useful links:

DocMap: https://data-hub-api.elifesciences.org/enhanced-preprints/docmaps/v2/by-publisher/elife/get-by-manuscript-id?manuscript_id=88431
New model tracking: https://docs.google.com/spreadsheets/d/1_fHaoOy7hjyocptKtVJRijeNpUY4hBS7Ck_aVmx6ZJk/
Reviews on sciety: https://sciety.org/articles/activity/10.1101/2<IP_ADDRESS>3802

For trouble shooting (e.g. no Docmaps available):

DocMap issue addressing: https://miro.com/app/board/uXjVNCwK6EI=/
Explore DataHub DocMaps API: https://lookerstudio.google.com/reporting/4c2f0368-babb-4beb-b5b3-497e7e7b0f08/page/ejphD
Unmatched submissions and preprints: https://lookerstudio.google.com/u/0/reporting/9f86204f-3bf7-477c-9b18-5c5ef141bf69/page/p_gxi57ha93c
Unmatched manuscripts spreadsheet: https://docs.google.com/spreadsheets/d/15QcK8w-ssB7109RQEDtFpJPZ0J5HTGxoHa_2TtpMBbg/edit#gid=1336081641

Step 2. Preview reviewed preprint
Production QC content ahead of publication
Instructions:

QC preview: https://prod--epp.elifesciences.org/previews/88431v2
Update ticket with any problems (add blocked label)
When QC OK, add QC OK label to ticket and add publication date and time to https://docs.google.com/spreadsheets/d/1amAlKvdLcaDp5W8Z8g77NmkwbMF5n_u89ArSqPMO8jg
Move card to next column
(At end of the day post link in #enhanced-preprint and ask for PDF to be generated)

Useful links:

Preprint DOI :  https://doi.org/10.1101/2<IP_ADDRESS>3802
Confirm reviews returned by EPP: https://prod--epp.elifesciences.org/api/reviewed-preprints/88431/v2/reviews
To update the MECA path in the docmap: https://docs.google.com/spreadsheets/d/1mctCQuNFBjSn97Lihy7_vBO6z7-N-oqyLv4clyi6zHg

Step 3: Awaiting search reindex
This step adds the reviewed preprint to the homepage: https://elifesciences.org
The search reindex is triggered once an hour. We need the reviewed preprint to be indexed as the search application serves the journal homepage.
Useful links:

Jenkins pipeline to reindex search can be triggered sooner or monitored here: https://alfred.elifesciences.org/job/process/job/process-reindex-reviewed-preprints/

Step 4: Published! PDF requested
Waiting for PDF to be generated
Useful links:

PDF tracking: https://docs.google.com/spreadsheets/d/106_XeDjmuBae7gexOTNzg60lapeqjl2aRn9DzupGyS8/

Step 5: Introduce PDF to data folder and git repo
Upload PDF to relevent folder in git repo https://github.com/elifesciences/enhanced-preprints-data/
Step 6: Done!

[ ] Kettle is on!

@FionaBryant 'solid' is bolded twice, should the 1st instance of 'solid' be altered?, otherwise fine to QC.

Thanks, I have corrected this to remove the 1st use of the word 'solid'.
On Fri, 5 Apr 2024 at 15:51, Fred Atherden @.***> wrote:

Assigned #3321
https://github.com/elifesciences/enhanced-preprints-import/issues/3321
to @FionaBryant https://github.com/FionaBryant.
—
Reply to this email directly, view it on GitHub
https://github.com/elifesciences/enhanced-preprints-import/issues/3321#event-12375458401,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ARMNBZFOQYODFXBOLEW2EXTY3227PAVCNFSM6AAAAABFF63XMCVHI2DSMVQWIX3LMV45UABCJFZXG5LFIV3GK3TUJZXXI2LGNFRWC5DJN5XDWMJSGM3TKNBVHA2DAMI
.
You are receiving this because you were assigned.Message ID:
<elifesciences/enhanced-preprints-import/issue/3321/issue_event/12375458401
@github.com>

--
Fiona Bryant
Senior Editorial Assistant, eLife
--
elifesciences.org https://elifesciences.org
eLife Sciences
Publications, Ltd is a limited liability non-profit non-stock corporation
incorporated in the State of Delaware, USA, with company number 5030732,
and is registered in the UK with company number FC030576 and branch number
BR015634 at the address Westbrook Centre, Milton Road, Cambridge, CB4 1YG.

Thanks @FionaBryant, will reset the article so it includes this update.


Sticky book list sorting across sessions

Is your feature request related to a problem? Please describe.
Currently, the sorting in the main table resets to default -title- after each launch of the app -or after the browser times the tab out if in active for a while-
Describe the solution you'd like
To save the sorting as a user preference, either as an option in settings or to save the last one the user sorted by.
Additional context
Title sorting doesn't work for non-alphabetical languages like Chinese and Japanese. It also doesn't work if you have a ton of books, as you're not necessary reading in alphabetical order, and having to set it each time is a bit inconvenient.

For a first pass, not having a user preference is easier.  Too many prefs = too many headaches :-)
Thanks for the issue.  :wave:

Hm, I'm not able to reproduce this one -- tried opening new tabs, starting stopping in dev and prod mode, closing and reopening etc.  The sorting and filtering for the book stays the same.  @shetozz what am I missing?

Not sure, I'm still on 3.1.3a (mandarin fork), so there might be a difference, but I can replicate it with just closing the tab and opening a new one.

Right, I perhaps try updating ... but I can't see a commit in there that would change it.

Updating fixed the issue, I'll close it with this comment.


Update node-glfw to go from nan 0.8 to 1.6 to support node12/iojs

AFAICT, still works with node 10.26  as well. Did not test on node 11.

is this request going to be merged anytime soon?  i am interested in trying it out...

:+1: can we have some progress on NAN integration? :)

@mikeseven Do we need this for node-opencl? (We'll be trying to merge into mainline of that next week, not sure if this part is still relevant.)

Node-opencl doesn't use any graphical library such Opengl and thus no
windowing system, so no dependency on glfw and no dependency on node-webgl.
Node-opencl is tested on node 0.12 and iojs.
I plan on deprecate node-webcl for node-opencl this month. If you use
OpenCL, use node-opencl. It supports up to OpenCL 2.0 for now, while webcl
is only 1.1.
--mike
On Sep 3, 2015 11:19 AM, "lmeyerov"<EMAIL_ADDRESS>wrote:

@mikeseven https://github.com/mikeseven Do we need this for
node-opencl? (We'll be trying to merge into mainline of that next week, not
sure if this part is still relevant.)
—
Reply to this email directly or view it on GitHub
https://github.com/mikeseven/node-glfw/pull/17#issuecomment-137533459.


Update transformer e2e tests

What this PR does / why we need it:
Update transformer e2e test to use image built from CI pipeline.
Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #1667
Special notes for your reviewer:

Fix the tfserving input validation, inputs is a Dict in tfserving v1 protocol while it is a List in v2 kserve inference protocol.
Fix the issue for e2e transformer test that it was not testing using the image built from CI and instead it used the image with latest tag which can potentially cause issue when several PRs simultaneously running CI.

Release note:

@Suresh-Nakkeran Please take a look, this PR fixes the transformer e2e test that always uses the image built from CI.

\lgtm

/lgtm


App getting crash in iOS 7.

App getting crash in iOS 7.
setClockwise, setHidesWhenStopped those setter methods crashing
Example app also getting crashing.
Crash report.
TYMActivityIndicatorView[10581:607] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '*** Illegal property type, c for appearance setter, _installAppearanceSwizzlesForSetter:'

UIAppearance proxy doesn't support BOOL type in iOS 7. It's fixed now.

Can you update the Cocoapod so it points to the latest version. The current version has caused my live app to crash for all iOS 7 users.

Sorry, it's my fault. I forgot to push the spec file to CocoaPods. Now, it works.


Added the ability to mark files as spoilers

Checklist

[x] I have run nox and all the pipelines have passed.
[x] I have made unittests according to the code I have added/modified/deleted.

What about bytes, webresources/urls?
Can embeds have spoilt images? If so, we'd want to adjust the logic in the embed serializer to upload any spoilt resources again so that we can inject a file name, right?

What about bytes, webresources/urls?
Will add it to Bytes, websources and urls im not really sure can be spoilt (since they are text). Maybe adding them between || if the user selects to spoil them?

Can embeds have spoilt images? If so, we'd want to adjust the logic in the embed serializer to upload any spoilt resources again so that we can inject a file name, right?
No, looks like they cant, they just render them as if they were not spoilers


CNF-9796: Add backup and restore plugin

Initial PR to create backup and restore APIs to achieve

Apply backup CRs extracted from oadp configmaps
Store restore CRs extracted from oadp configmaps to a given location
Store oadp configuration to a given location
Apply oadp configuration stored in a given location
Apply restore CRs stored in a given location

/cc @browsell @jc-rh @donpenney

/cc @pixelsoccupied

/lgtm

/approve

/cherrypick release-4.15

nvm... had the wrong link...


Isn't working.

input:
PS C:WINDOWS\system32> startupmanager list
output:
Unhandled exception: System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.  ---> System.ArgumentException: The path is empty. (Parameter 'path')    at System.IO.Path.GetFullPath(String path)    at System.IO.Enumeration.FileSystemEnumerator1..ctor(String directory, Boolean isNormalized, EnumerationOptions options)
at System.IO.Enumeration.FileSystemEnumerable1..ctor(String directory, FindTransform transform, EnumerationOptions options, Boolean isNormalized)    at System.IO.Enumeration.FileSystemEnumerableFactory.UserFiles(String directory, String expression, EnumerationOptions options)    at System.IO.Directory.InternalEnumeratePaths(String path, String searchPattern, SearchTarget searchTarget, EnumerationOptions options)    at System.IO.Directory.EnumerateFiles(String path, String searchPattern, SearchOption searchOption)    at StartupManager.Services.Directories.DirectoryService.<>c__DisplayClass12_0.<GetFiles>b__0(String searchPattern)    at System.Linq.Enumerable.SelectManySingleSelectorIterator2.MoveNext()
at System.Linq.Enumerable.SelectEnumerableIterator2.ToList()    at System.Linq.Enumerable.ToList[TSource](IEnumerable1 source)
at StartupManager.Services.Directories.DirectoryService.GetShellStartup(Boolean allUsers, String path, IEnumerable1 startupStates)    at StartupManager.Services.Directories.DirectoryService.GetStartupPrograms(IEnumerable1 startupStates)
at StartupManager.Commands.StartupList.ListCommandHandler.Run()
at StartupManager.Commands.StartupList.ListCommand.Run(Boolean detailed)
--- End of inner exception stack trace ---
at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor, Boolean wrapExceptions)
at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at System.Delegate.DynamicInvokeImpl(Object[] args)
at System.Delegate.DynamicInvoke(Object[] args)
at System.CommandLine.Invocation.ModelBindingCommandHandler.InvokeAsync(InvocationContext context)
at System.CommandLine.Invocation.InvocationPipeline.<>c__DisplayClass4_0.<b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c.<b__20_0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass15_0.<b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass24_0.<b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass22_0.<b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c.<b__21_0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c.<b__19_0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c.<b__11_0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c.<b__10_0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass13_0.<b__0>d.MoveNext()`
I installed with scoop

Could you try 1.6.1 and see if this fixes the issue?

Solved, thanks a lot.


🐛 [Bug] Failed to compile a FP16 QAT resnet

Bug Description
After trained a Q/DQ-inserted fake-quantized resnet, we did model.half() and then torch.jit.trace() to generate this torchscript model. However, it failed to compile.
To Reproduce
Steps to reproduce the behavior:

Download the FP16 QAT checkpoint resnet50_quant.ts
Run the following script

import torch
imgs = torch.zeros(32, 3, 224, 224).cuda().half()
resnet50_model = torch.jit.load("resnet50_quant.ts").cuda()
out = resnet50_model(imgs)
for k, v in resnet50_model.state_dict().items():
    print(k, v.dtype, v.shape)

import torch_tensorrt

trt_model_int8 = torch_tensorrt.ts.compile(resnet50_model,
    inputs = [
        torch_tensorrt.Input(
            min_shape = tuple([1, 3, 224, 224]),
            opt_shape = tuple([16, 3, 224, 224]),
            max_shape = tuple([32, 3, 224, 224]),
            dtype=torch.float16
        )
    ],
    enabled_precisions = torch.int8,
    workspace_size = 1 << 32,
)
trt_model_int8.save("trt_64.pt")
trt_model_int8 = torch.jit.load("trt_64.pt")
trt_output = trt_model_int8(imgs)
resnet50_outputs = resnet50_model(imgs)
print(trt_output)
print(resnet50_outputs)

print(trt_output[1].dtype)
diff = abs(trt_output - resnet50_outputs)
print('diff:', diff.mean())

Expected behavior
Compile with no error and result matches.
Environment

Build information about Torch-TensorRT can be found by turning on debug messages

Torch-TensorRT Version (e.g. 1.0.0):  1.3.0a0+975f6387
PyTorch Version (e.g. 1.0): 1.13.0.dev20220921+cu116
CPU Architecture:
OS (e.g., Linux):
How you installed PyTorch (conda, pip, libtorch, source):
Build command you used (if compiling from source):
Are you using local sources or building from archives:
Python version: 3.9
CUDA version: 11.6
GPU models and configuration:
Any other relevant information:

Additional context

Fixed when turning to trt 8.5.1


bug: exec failed, "curl" not found in $PATH

Issue description
I am using apisix ingress controller according to this document
https://apisix.apache.org/docs/ingress-controller/tutorials/proxy-the-httpbin-service/
When I got to this point in the execution I found the error
kubectl exec -it -n ${namespace of Apache APISIX} ${Pod name of Apache APISIX} -- curl http://<IP_ADDRESS>:9180/apisix/admin/routes -H 'X-API-Key: edd1c9f034335f136f87ad84b625c8f1'

The errors are as follows:
Defaulted container "apisix" out of: apisix, wait-etcd (init)
OCI runtime exec failed: exec failed: unable to start container process: exec: "curl": executable file not found in $PATH: unknown
command terminated with exit code 126

After troubleshooting, we found that apisix changed the image to debian, which caused the curl command to be unavailable, so we may need to update the documentation.
Environment

your apisix-ingress-controller version (output of apisix-ingress-controller version --long):
your Kubernetes cluster version (output of kubectl version):
if you run apisix-ingress-controller in Bare-metal environment, also show your OS version (uname -a):

Minimal test code / Steps to reproduce
https://apisix.apache.org/docs/ingress-controller/tutorials/proxy-the-httpbin-service/
Actual result
Following the documentation can be successfully executed.
Error log
Defaulted container "apisix" out of: apisix, wait-etcd (init)
OCI runtime exec failed: exec failed: unable to start container process: exec: "curl": executable file not found in $PATH: unknown
command terminated with exit code 126

Expected result
No response

We can modify all the examples to enter the Pod of apisix-ingress-controller to operate

Is the solution to this fix is to install curl command processing in the container? @tao12345666333

see https://github.com/apache/apisix-ingress-controller/pull/1648 @tao12345666333

Hi @fengshunli, we need to add curl in the APISIX docker image, you can refer to https://github.com/apache/apisix-docker, apisix-ingress-controller no need to change.

Do you need to modify the dockerfile in debian and centos? @Baoyuantop

Just modify the debian, cc @tao12345666333

fixed by https://github.com/apache/apisix-docker/pull/419


Setup of chat.gateway has been done along with removal of all errors

This PR has the following
1).Controllers and services specifically for the room and its create data transfer object
2).chat.gateway.ts initialized and made error-free
continuation of PR #79
followup:-The final testing is left and further integration might be made

Please change it to drafts.


Selecting provider: iam for API not updating unauth cognito identity role

Before opening, please confirm:

[X] I have installed the latest version of the Amplify CLI (see above), and confirmed that the issue still persists.
[X] I have searched for duplicate or closed issues.
[X] I have read the guide for submitting bug reports.
[X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.
[X] I have removed any sensitive information from my code snippets and submission.

How did you install the Amplify CLI?
npm
If applicable, what version of Node.js are you using?
16.13.0
Amplify CLI Version
7.2.26
What operating system are you using?
mac
Did you make any manual changes to the cloud resources managed by Amplify? Please describe the changes made.
no.
Amplify Categories
auth, api
Amplify Commands
add
Describe the bug
Per the allow public data access section of the docs, I should be able to allow public access to my API by using the @auth directive with a rule of { allow: public, provider: iam, operations: [read] } (auth is already configured).
However when running the app, I get the following error:

I have both cognito and IAM set as auth modes, with cognito set as the default. As such, I specify the auth mode in my code as such:
API.graphql({
			query: listTodos,
			authMode: 'AWS_IAM',
		}).then((data) => console.log(data.data.listTodos.items))
	}, [])

Further, when running the command amplify status api -acm Todo
I get the following access table:

That is based off of the following schema:
type Todo
	@model
	@auth(
		rules: [
			{ allow: owner }
			{ allow: public, provider: iam, operations: [read] }
		]
	) {
	id: ID!
	name: String!
	description: String
}

Expected behavior
Given the above API.graphql query, I would expect to receive back an empty array of Todo items given that there are no Todo's in my database.
Reproduction steps

amplify init -y
amplify add auth (accept defaults)
amplify add api (add the provided schema)
amplify push -y
add the above query (after setting up Amplify.configure in main file)
Observe error in console.

GraphQL schema(s)

# Put schemas below this line
type Todo
	@model
	@auth(
		rules: [
			{ allow: owner }
			{ allow: public, provider: iam, operations: [read] }
		]
	) {
	id: ID!
	name: String!
	description: String
}

Log output

# Put your logs below this line

Additional information

This is a nextjs app, though I'm not using anything nextjs specific at this point.
I tried to remove auth, and re-add, but that triggered an error:

Hey @mtliendo :wave: thanks for raising this! While the UnrecognizedClientException seems interesting, can you confirm the auth resource is configured to "Allow unauthenticated logins"? I noticed in the reproduction steps it's noted to accept defaults, where we will need to run amplify update auth > Walkthrough all configurations > choose "Yes" to allow unauthenticated logins

I tried to remove auth, and re-add, but that triggered an error:

To clarify, did this occur after running amplify remove auth and amplify add auth?

Additionally, I think you may need to call await Auth.currentCredentials() after Amplify.configure() to pull down the credentials of the unauthenticated IAM role.

Thanks @josefaidt that did it 🎉 (going through amplify update auth flow).
A couple of notes on that:

It's the 3rd option of ~15, and there wasn't a way to break out of it after selecting my option.
The options default to "yes", which meant I can't just hit enter to bypass them. Not the best DX
Is it possible for the CLI to jump me to that unauth role option and exit out... better yet, do it automatically for me?

@josefaidt wrt not being able to remove auth, that still isn't working.
steps:

ampify add auth
amplify push -y
amplify remove auth
amplify push -y
observe error

@mtliendo that's a separate issue that will be fixed by https://github.com/aws-amplify/amplify-cli/pull/10072
It occurs because the API resource is still trying to depend on the deleted auth resource. The way to fix it is to run amplify update api and remove Cognito as an auth provider for the API

Agree with all your points about DX. I think this issue would best be turned into a feature request to improve the auth workflow for deeply nested config

Makes total sense :) Thanks for the update and the workaround!


fix : Add/Update Background Color for Archived Notes

Description
This PR addressed the issue regarding working of adding or updating background color on multiple archived notes.

Fixes #335

Type of change
-bug fix
Please delete options that are not relevant.

[x] Bug fix (non-breaking change which fixes an issue)

Checklist

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] My changes generate no new warnings
[x] New and existing unit tests pass locally with my changes
[x] I have maintained a clean commit history by using the necessary Git commands
[x] I have checked that my code does not cause any merge conflicts

Screenshots (if applicable)
Screencast from 2024-10-29 14-36-53.webm

resolve conflicts

resolve conflicts

@yashmandi , Done


Add Policy requirement for application autoscaling to the README

Description of changes:
Add steps to attach PassRole Policy to the IAM role required by ApplicationAutoscaling ScalableTarget Update Operation.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

closing in favor of #99


python312Packages.textual: 0.82.0 -> 0.85.2

Automatic update generated by nixpkgs-update tools. This update was made based on information from passthru.updateScript.
meta.description for python312Packages.textual is: TUI framework for Python inspired by modern web development
meta.homepage for python312Packages.textual is: https://github.com/Textualize/textual
meta.changelog for python312Packages.textual is: https://github.com/Textualize/textual/releases/tag/v0.85.2
Updates performed

Ran passthru.UpdateScript

To inspect upstream changes
Impact
Checks done

built on NixOS
The tests defined in passthru.tests, if any, passed
found 0.85.2 with grep in /nix/store/5rvqmhmvi64dwck6w4s6lrd6vay5ks45-python3.12-textual-0.85.2
found 0.85.2 in filename of file in /nix/store/5rvqmhmvi64dwck6w4s6lrd6vay5ks45-python3.12-textual-0.85.2

Rebuild report (if merged into master) (click to expand)

32 total rebuild path(s)

32 package rebuild(s)

First fifty rebuilds by attrpath

browsr
dooit
frogmouth
harlequin
isponsorblocktv
memray
mutmut
oterm
python311Packages.manifestoo
python311Packages.pytest-textual-snapshot
python311Packages.textual
python311Packages.textual-dev
python311Packages.textual-fastdatatable
python311Packages.textual-slider
python311Packages.textual-textarea
python311Packages.textual-universal-directorytree
python312Packages.manifestoo
python312Packages.pytest-textual-snapshot
python312Packages.textual
python312Packages.textual-dev
python312Packages.textual-fastdatatable
python312Packages.textual-slider
python312Packages.textual-textarea
python312Packages.textual-universal-directorytree
rich-cli
smassh
streamcontroller
tftui
toolong
upiano
wsrepl

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/5rvqmhmvi64dwck6w4s6lrd6vay5ks45-python3.12-textual-0.85.2 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A python312Packages.textual https://github.com/r-ryantm/nixpkgs/archive/c5e1e81aa01fb6d69a2df81928849e5b6bd0789e.tar.gz

Or:
nix build github:r-ryantm/nixpkgs/c5e1e81aa01fb6d69a2df81928849e5b6bd0789e#python312Packages.textual

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/5rvqmhmvi64dwck6w4s6lrd6vay5ks45-python3.12-textual-0.85.2
ls -la /nix/store/5rvqmhmvi64dwck6w4s6lrd6vay5ks45-python3.12-textual-0.85.2/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
nixpkgs-review result
Generated using nixpkgs-review.
Command: nixpkgs-review

x86_64-linux

:x: 2 packages failed to build:

isponsorblocktv
isponsorblocktv.dist

:white_check_mark: 59 packages built:

browsr
browsr.dist
dooit
dooit.dist
frogmouth
frogmouth.dist
harlequin
harlequin.dist
memray
memray.dist
mutmut
mutmut.dist
oterm
oterm.dist
python311Packages.manifestoo
python311Packages.manifestoo.dist
python311Packages.pytest-textual-snapshot
python311Packages.pytest-textual-snapshot.dist
python311Packages.textual
python311Packages.textual-dev
python311Packages.textual-dev.dist
python311Packages.textual-fastdatatable
python311Packages.textual-fastdatatable.dist
python311Packages.textual-slider
python311Packages.textual-slider.dist
python311Packages.textual-textarea
python311Packages.textual-textarea.dist
python311Packages.textual-universal-directorytree
python311Packages.textual-universal-directorytree.dist
python311Packages.textual.dist
python312Packages.manifestoo
python312Packages.manifestoo.dist
python312Packages.pytest-textual-snapshot
python312Packages.pytest-textual-snapshot.dist
python312Packages.textual
python312Packages.textual-dev
python312Packages.textual-dev.dist
python312Packages.textual-fastdatatable
python312Packages.textual-fastdatatable.dist
python312Packages.textual-slider
python312Packages.textual-slider.dist
python312Packages.textual-textarea
python312Packages.textual-textarea.dist
python312Packages.textual-universal-directorytree
python312Packages.textual-universal-directorytree.dist
python312Packages.textual.dist
rich-cli
rich-cli.dist
smassh
smassh.dist
streamcontroller
tftui
tftui.dist
toolong
toolong.dist
upiano
upiano.dist
wsrepl
wsrepl.dist

Maintainer pings

[!TIP]
As a maintainer, if your package is located under pkgs/by-name/*, you can comment @NixOS/nixpkgs-merge-bot merge to automatically merge this update using the nixpkgs-merge-bot.

Add a :+1: reaction to pull requests you find important.

updated in #356997


Add reason for white flag conflicts in API

The explorer only shows that a transaction was conflicting.
It would be much better to add the reason why the conflict happened.

Initial draft implementation in #808

Initial draft implementation in #808


Add accessibility_label to button block element

Summary
This pull request adds "accessibility_label" to the "button" type block element.

https://api.slack.com/reference/block-kit/block-elements#button
https://api.slack.com/changelog

Category (place an x in each of the [ ])

[ ] slack_sdk.web.WebClient (sync/async) (Web API client)
[ ] slack_sdk.webhook.WebhookClient (sync/async) (Incoming Webhook, response_url sender)
[ ] slack_sdk.socket_mode (Socket Mode client)
[ ] slack_sdk.signature (Request Signature Verifier)
[ ] slack_sdk.oauth (OAuth Flow Utilities)
[x] slack_sdk.models (UI component builders)
[ ] slack_sdk.scim (SCIM API client)
[ ] slack_sdk.audit_logs (Audit Logs API client)
[ ] slack_sdk.rtm_v2 (RTM client)
[ ] /docs-src (Documents, have you run ./scripts/docs.sh?)
[ ] /docs-src-v2 (Documents, have you run ./scripts/docs-v2.sh?)
[ ] /tutorial (PythOnBoardingBot tutorial)
[x] tests/integration_tests (Automated tests for this library)

Requirements (place an x in each [ ])

[x] I've read and understood the Contributing Guidelines and have done my best effort to follow them.
[x] I've read and agree to the Code of Conduct.
[x] I've run python3 -m venv .venv && source .venv/bin/activate && ./scripts/run_validation.sh after making the changes.

Thanks for your reviews!


fix: session was not closed issue in Android

Hi Guys,
I fixed error in Android!
In version 0.6.0, the app turned off after 30 times dispose.
In version 0.6.1, the app turned off after 70 times dispose.
And now, The app does not turn off after dispose!
The animation continued to be soft and the amount of heat generated became very low.
Overall, it was optimized.

nice! I'll have a look at the new code this afternoon or at the latest this weekend and merge it into main :)

@Koobonik I left a few questions in your code, after they are clarified I'll merge it :)

@Koobonik I left a few questions in your code, after they are clarified I'll merge it :)

I always feel grateful for your efforts.

hey @Koobonik , can you have a look at the questions I asked on this PR? if we clarify this I can go on and merge it :)

@CariusLars Yes, You can.
How can I see your questions?

they are further up in this discussion thread, I've attached a screenshot on how it looks for me:

In the fisrt case, So many flutter libs set null after dispose. because, It helps to prevent memory leak.
second case, you right, The difference from arcore was compared. I think I made a mistake there. you can delete "faceSceneUpdateListener" function

are you sure the method call handlers are automatically disposed?
also, could you remove the faceSceneUpdateListener code pieces so I can compare and test the code better?
Thanks @Koobonik !

hi @CariusLars
sorry about too late
I will send a new code in Sunday. :)

@Koobonik Any update #71 ? Am i the only one facing this issue ?

@naveenbharadwaj19
Does it mean an issue that Turn OFF when you go home while running the app and then return right?

@Koobonik Well yeah. It's terminating whole app . I've added a demo video in #71 how the crash is happening

@Koobonik I've merged your changes into main in the current v0.6.2

@CariusLars Thanks! If I find something. I will PR :)


[Paper]Using the IPython Notebook as Lab Notebook

I'm really happy to see that, it is a nice showcase on how you can extend the notebook, and I know that @juhasch have spend a lot of time writing extensions and made them available to user.
The ‘ 20 m / 15 s‘ seem to be badly rendered.
In figure 2, if you wish you can describe what can be seen/not seen on the picture which is due to extension (button in the toolbar, code cell hidden ?).
Figure 3, you could add in the description, something like "StatelessInputTransformer allow non python syntax like 10 um" which emphasize what you have done.
These are just suggestion. Good job otherwise !

In reply to the invitation to review this paper: I accept, but I can start only in about a week.

Here are my first impressions.
What this paper requires most of all is a clearer focus. What's the intended message? It reads a bit like "some things you can do with IPython notebooks". Given that the IPython notebook is already quite popular, it is not obvious what the intention of the author is. A big part of the article is about extensions, so perhaps that should become the focus.
On a related note, the title and introduction suggest a comparison to traditional lab notebooks. However, that is not the role that the IPython notebook has in most scientists' workflows, and it is not even particularly well suited to this task. An IPython notebook is more a scratchpad for developing ideas than a lab notebook.
As a reminder, a lab notebook is used for keeping time-stamped notes of what was done. It's a bit like a bank ledger: you append at the end, but never change what was written earlier. The closest equivalent we have in the digital world is certain version control systems such as Mercurial or subversion. Even git doesn't fully qualify because it encourages rewriting history. The IPython notebook doesn't integrate well with version control, as the article mentions, so it seems strange to see the reference to lab notebooks at all.
Section 2 contains mostly unqualified judgements. I'd expect a more rigorous approach in a scientific environment. What exactly is shocking about the use of MS Word? In what way is Mathematica less easy to use than Python? Easy for who? BTW, none of the tools in section 2 fulfills the role of a lab notebook.

I going to have decline this paper, as the reviewers' comment were not addressed.


Tigo CTM support fee sender

In the case of a support provider like Tigo CTM, a fee transaction should be sent out every time a normal transaction is made. Whether it is a BTC buy or sell, x% of the value should be sent to the BTC address configured. This should be made available on a separate branch, and generally made into an opt-in setting.

Git Guild Bounty Program

Total
Dev
Sales
Reporter
Tester(s)
Maintainer
Documenter

N/A
assignee(s)
@isysd
@isysd
@isysd
@isysd
@d3brouille

$1,833.33
$1,100.00
$91.67
$91.67
$366.67
$91.67
$91.67


[MQ reliability] The Service Bus tests occasionally fail in canary

Now that we've reached a stable spot with our SDK we need to be particularly attentive to failures in the canary region, given that it's a first signal for potential larger customer issues.

Please add a test that runs indefinitely or at least for several days to catch the production issues that customers (we) actually faced. I would say 30-60 days would be long enough, or to just keep it running until the next release is cut.

Just adding in some more context - there is conversation in this issue (#12609) that is discussing what we have and what we're putting in place for long term tests.

This original issue was handle some specific cases that were breaking in canary. We have broken those down into specific issues that we'll handle individually (for instance, #14441).
I am closing this particular issue in favor of tracking the more specific failures that have already been logged.
@jmealo - we've actually started doing more long-term and stress related testing across all of our messaging packages. We have really appreciated your feedback across this and other issues.


Add a license

Hi, Thanks for maintaining this awesome library. I was going to use polyfillPath (unmodified) and polyfillContent ( a bit modified for my use case) functions from this package but as I only needed these functions also needed to modify, so I wasn't installing full deps. But I saw that you don't have any license files in this repo. Can u please add a license here so I can mention that license while using the codes? Thanks again ❤

done!