Charm requests keystone data before keystone is in ready state

On a deploy with a clustered keystone, one keystone unit is ready, the other 2 are waiting.  The charm requested data from keystone resulting in a http 500 error.
keystone:
charm: cs:keystone-292
series: bionic
os: ubuntu
charm-origin: jujucharms
charm-name: keystone
charm-rev: 292
exposed: false
application-status:
current: waiting
message: 'Incomplete relations: database'
since: 28 Feb 2019 04:58:47Z
relations:
cluster:
- keystone
ha:
- hacluster-keystone
identity-credentials:
- ceilometer
- openstack-service-checks
- prometheus-openstack-exporter
identity-service:
- aodh
- ceph-radosgw
- cinder
- designate
- glance
- gnocchi
- heat
- neutron-api
- nova-cloud-controller
- openstack-dashboard
juju-info:
- filebeat
- telegraf
nrpe-external-master:
- nrpe-container
shared-db:
- mysql
2019-02-28 05:35:23 ERROR juju-log Hook error:
Traceback (most recent call last):
File "/var/lib/juju/agents/unit-openstack-service-checks-0/.venv/lib/python3.6/site-packages/charms/reactive/init.py", line 73, in main
bus.dispatch(restricted=restricted_mode)
File "/var/lib/juju/agents/unit-openstack-service-checks-0/.venv/lib/python3.6/site-packages/charms/reactive/bus.py", line 390, in dispatch
_invoke(other_handlers)
File "/var/lib/juju/agents/unit-openstack-service-checks-0/.venv/lib/python3.6/site-packages/charms/reactive/bus.py", line 359, in _invoke
handler.invoke()
File "/var/lib/juju/agents/unit-openstack-service-checks-0/.venv/lib/python3.6/site-packages/charms/reactive/bus.py", line 181, in invoke
self._action(*args)
File "/var/lib/juju/agents/unit-openstack-service-checks-0/charm/reactive/service_checks.py", line 184, in render_config
render_checks()
File "/var/lib/juju/agents/unit-openstack-service-checks-0/charm/reactive/service_checks.py", line 162, in render_checks
endpoint_checks = create_endpoint_checks()
File "/var/lib/juju/agents/unit-openstack-service-checks-0/charm/reactive/service_checks.py", line 236, in create_endpoint_checks
endpoints = keystone_client.endpoints.list()
File "/usr/lib/python3/dist-packages/keystoneclient/v3/endpoints.py", line 125, in list
**kwargs)
File "/usr/lib/python3/dist-packages/keystoneclient/base.py", line 75, in func
return f(*args, **new_kwargs)
File "/usr/lib/python3/dist-packages/keystoneclient/base.py", line 397, in list
self.collection_key)
File "/usr/lib/python3/dist-packages/keystoneclient/base.py", line 125, in _list
resp, body = self.client.get(url, **kwargs)
File "/usr/lib/python3/dist-packages/keystoneauth1/adapter.py", line 304, in get
return self.request(url, 'GET', **kwargs)
File "/usr/lib/python3/dist-packages/keystoneauth1/adapter.py", line 463, in request
resp = super(LegacyJsonAdapter, self).request(*args, **kwargs)
File "/usr/lib/python3/dist-packages/keystoneauth1/adapter.py", line 189, in request
return self.session.request(url, method, **kwargs)
File "/usr/lib/python3/dist-packages/keystoneclient/session.py", line 343, in request
auth_headers = self.get_auth_headers(auth)
File "/usr/lib/python3/dist-packages/keystoneclient/session.py", line 660, in get_auth_headers
return auth.get_headers(self, **kwargs)
File "/usr/lib/python3/dist-packages/keystoneclient/auth/base.py", line 159, in get_headers
token = self.get_token(session)
File "/usr/lib/python3/dist-packages/keystoneclient/auth/identity/base.py", line 206, in get_token
return self.get_access(session).auth_token
File "/usr/lib/python3/dist-packages/keystoneclient/auth/identity/base.py", line 252, in get_access
self.auth_ref = self.get_auth_ref(session)
File "/usr/lib/python3/dist-packages/keystoneclient/auth/identity/v3/base.py", line 189, in get_auth_ref
authenticated=False, log=False, **rkwargs)
File "/usr/lib/python3/dist-packages/keystoneclient/session.py", line 543, in post
return self.request(url, 'POST', **kwargs)
File "/usr/lib/python3/dist-packages/keystoneclient/session.py", line 443, in request
raise exceptions.from_response(resp, method, url)
keystoneauth1.exceptions.http.InternalServerError: An unexpected error prevented the server from fulfilling your request. (HTTP 500) (Request-ID: req-58a1d5ba-d15c-41ea-8f94-cb6b01a16356)
2019-02-28 05:35:23 ERROR juju.worker.uniter.operation runhook.go:132 hook "update-status" failed: exit status 1
Full crashdump is attached

(by asbalderson)

(by jason-hobbs)
I suspect this is something to do with the recent updates to the charm.
It's not clear if keystone is ready or not. In Alex's run, it shows as not in juju status, but all of the units have said they are ready at some point:
http://paste.ubuntu.com/p/Pz64Yt8rBv/

(by jason-hobbs)
Sub'd to field high as this is causing a sudden spike in failures

(by ajkavanagh)
Hi Alex, please, could you provide the bundles you are using so that we can chase this down a bit.  Have those bundles changed recently (i.e. are they using stable or next, etc.)
Thanks.

(by jason-hobbs)
This is happening in the new create_endpoint_check() that was added here:
https://git.launchpad.net/charm-openstack-service-checks/commit/?id=7895a97e4ee052c44971576a691e4ce23d6969f2

(by jason-hobbs)
ajkavanagh here is the bundle http://paste.ubuntu.com/p/b5YXFdyPfp/
We haven't changed the bundle recently; the change that triggered this was to the openstack service checks charm. It's not clear to me where root cause is.

(by xavpaice)
Looks like the openstack-service-checks is calling the relation for keystone before keystone is ready - should keystone not be blocking the relation until it's ready?
The relation to Keystone has been there for a while, but the recent change actually uses that information to query Keystone immediately - so the relation shouldn't be made available until Keystone is actually ready to service requests.

(by afreiberger)
just after identity-credentials-relation-changed hook ran where we were passed authentication credentials from keystone, keystone is immediately not available for the next update-status for the charm to use keystone.
from openstack-service-checks-0.log:
2019-02-28 05:32:06 INFO juju-log identity-credentials:220: Reactive main running for hook identity-credentials-relation-changed
2019-02-28 05:32:12 INFO juju-log identity-credentials:220: render_config: Got credentials for username=nagios
2019-02-28 05:32:28 INFO juju.worker.uniter.operation runhook.go:138 ran "identity-credentials-relation-changed" hook
2019-02-28 05:33:35 INFO juju-log Reactive main running for hook update-status
2019-02-28 05:33:41 ERROR juju-log Hook error:
keystoneauth1.exceptions.http.InternalServerError: An unexpected error prevented the server from fulfilling your request. (HTTP 500) (Request-ID: req-058d844c-4b70-42f0-8bda-a7c3b6789bde)
So, somehow keystone charm is able to create the user account from the relation hooks, but the endpoint is not available to authenticate against.  This seems either inconsistent or a race condition of clustering of keystone.
I also see a possible race condition where render_config may be called before keystone is ready if nrpe-external-master relation happens first.
Keystone's shared-db isn't even set up until 05:35 or later, so it appears that keystone is able to provide credentials for what "will" be created users when the db is online?
from keystone/0:
2019-02-28 05:35:38 INFO juju-log identity-service:41: database relation's interface, shared-db, is related awaiting the following data from the relationship: database_host, database_password.
Long term, this should probably handled with broker type handling as is done with ceph-mon:client relations.  Short term, we'll capture the exception and work around it.

(by xavpaice)
Update released in cs:~canonical-bootstack/openstack-service-checks-11
Perhaps we should be altering the Keystone charm to use goal state? The fact remains, that until the Keystone cluster is ready (even just one unit) it shouldn't be completing the identity credentials relation, and non-ready units shouldn't be serving up 500's to clients.

(by afreiberger)
We found there was a race-condition between layer:apt installing packages and the reactive handlers kicking off in service_checks.py which lead to service_checks.py imports failing due to apt packages providing those dist-packages for python modules.  We have updated the code to use layer:basic's package installs on bootstrap and are working on review/release.

(by asbalderson)
Something very similar to this came up in another run, but at a slightly different place:
2019-04-23 12:39:50 ERROR juju-log identity-credentials:240: Hook error:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 601, in urlopen
chunked=chunked)
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 387, in _make_request
six.raise_from(e, None)
File "", line 3, in raise_from
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 383, in _make_request
httplib_response = conn.getresponse()
File "/usr/lib/python3.6/http/client.py", line 1331, in getresponse
response.begin()
File "/usr/lib/python3.6/http/client.py", line 297, in begin
version, status, reason = self._read_status()
File "/usr/lib/python3.6/http/client.py", line 266, in _read_status
raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/requests/adapters.py", line 440, in send
timeout=timeout
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 639, in urlopen
_stacktrace=sys.exc_info()[2])
File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 357, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/var/lib/juju/agents/unit-openstack-service-checks-0/.venv/lib/python3.6/site-packages/six.py", line 692, in reraise
raise value.with_traceback(tb)
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 601, in urlopen
chunked=chunked)
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 387, in _make_request
six.raise_from(e, None)
File "", line 3, in raise_from
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 383, in _make_request
httplib_response = conn.getresponse()
File "/usr/lib/python3.6/http/client.py", line 1331, in getresponse
response.begin()
File "/usr/lib/python3.6/http/client.py", line 297, in begin
version, status, reason = self._read_status()
File "/usr/lib/python3.6/http/client.py", line 266, in _read_status
raise RemoteDisconnected("Remote end closed connection without"
urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/keystoneclient/session.py", line 462, in _send_request
resp = self.session.request(method, url, **kwargs)
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 520, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 630, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python3/dist-packages/requests/adapters.py", line 490, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/var/lib/juju/agents/unit-openstack-service-checks-0/.venv/lib/python3.6/site-packages/charms/reactive/init.py", line 73, in main
bus.dispatch(restricted=restricted_mode)
File "/var/lib/juju/agents/unit-openstack-service-checks-0/.venv/lib/python3.6/site-packages/charms/reactive/bus.py", line 390, in dispatch
_invoke(other_handlers)
File "/var/lib/juju/agents/unit-openstack-service-checks-0/.venv/lib/python3.6/site-packages/charms/reactive/bus.py", line 359, in _invoke
handler.invoke()
File "/var/lib/juju/agents/unit-openstack-service-checks-0/.venv/lib/python3.6/site-packages/charms/reactive/bus.py", line 181, in invoke
self._action(*args)
File "/var/lib/juju/agents/unit-openstack-service-checks-0/charm/reactive/openstack_service_checks.py", line 165, in render_config
helper.render_checks(creds)
File "lib/lib_openstack_service_checks.py", line 140, in render_checks
self.create_endpoint_checks(creds)
File "lib/lib_openstack_service_checks.py", line 178, in create_endpoint_checks
endpoints = keystone_client.endpoints.list()
File "/usr/lib/python3/dist-packages/keystoneclient/v3/endpoints.py", line 125, in list
**kwargs)
File "/usr/lib/python3/dist-packages/keystoneclient/base.py", line 75, in func
return f(*args, **new_kwargs)
File "/usr/lib/python3/dist-packages/keystoneclient/base.py", line 397, in list
self.collection_key)
File "/usr/lib/python3/dist-packages/keystoneclient/base.py", line 125, in _list
resp, body = self.client.get(url, **kwargs)
File "/usr/lib/python3/dist-packages/keystoneauth1/adapter.py", line 304, in get
return self.request(url, 'GET', **kwargs)
File "/usr/lib/python3/dist-packages/keystoneauth1/adapter.py", line 463, in request
resp = super(LegacyJsonAdapter, self).request(*args, **kwargs)
File "/usr/lib/python3/dist-packages/keystoneauth1/adapter.py", line 189, in request
return self.session.request(url, method, **kwargs)
File "/usr/lib/python3/dist-packages/keystoneclient/session.py", line 343, in request
auth_headers = self.get_auth_headers(auth)
File "/usr/lib/python3/dist-packages/keystoneclient/session.py", line 660, in get_auth_headers
return auth.get_headers(self, **kwargs)
File "/usr/lib/python3/dist-packages/keystoneclient/auth/base.py", line 159, in get_headers
token = self.get_token(session)
File "/usr/lib/python3/dist-packages/keystoneclient/auth/identity/base.py", line 206, in get_token
return self.get_access(session).auth_token
File "/usr/lib/python3/dist-packages/keystoneclient/auth/identity/base.py", line 252, in get_access
self.auth_ref = self.get_auth_ref(session)
File "/usr/lib/python3/dist-packages/keystoneclient/auth/identity/v3/base.py", line 189, in get_auth_ref
authenticated=False, log=False, **rkwargs)
File "/usr/lib/python3/dist-packages/keystoneclient/session.py", line 543, in post
return self.request(url, 'POST', **kwargs)
File "/usr/lib/python3/dist-packages/keystoneclient/session.py", line 428, in request
resp = send(**kwargs)
File "/usr/lib/python3/dist-packages/keystoneclient/session.py", line 472, in _send_request
raise exceptions.ConnectionRefused(msg)
keystoneauth1.exceptions.connection.ConnectFailure: Unable to establish connection to http://<IP_ADDRESS>:35357/v3/auth/tokens
2019-04-23 12:39:50 ERROR juju.worker.uniter.operation runhook.go:132 hook "identity-credentials-relation-changed" failed: exit status 1
2019-04-23 12:39:50 INFO juju.worker.uniter resolver.go:124 awaiting error resolution for "relation-changed" hook
2019-04-23 12:39:55 INFO juju.worker.uniter resolver.go:124 awaiting error resolution for "relation-changed" hook
2019-04-23 12:39:55 INFO juju-log identity-credentials:240: Reactive main running for hook identity-credentials-relation-changed

(by asbalderson)
Attaching logs

(by asbalderson)
Saw a very similar issue with octavia today:
It requested data from keystone but keystone was not yet available.
1 keystone unit is active; the other 2 are waiting for database relation and running the "leader setting changed" hook.
Attaching the crashdump

(by asbalderson)
attaching bundle

(by asbalderson)
versioned overlay

(by 1chb1n)
The example bundles regarding comments 13 through 15 seem to be lacking a race-condition avoidance requirement of specifying the cluster_count on hacluster.  Can you confirm that this problem still exists when that is set?

(by mskalka)
Ryan,
The hacluster charm defaults that value to 3: https://jaas.ai/hacluster#charm-config-cluster_count. As we do not override that value in our bundle and deploy 3 units of each service I don't see how that would come into play.

(by 1chb1n)
Actually, the hacluster charm config default of 3 units happens to align with the deployed number of units, and I would expect that to squash my concern in my comment #16.

(by alai)
We hit it again.  Crashdump is attached.

(by nikolay.vinogradov)
Happens from time to time in our deployment as well.

(by nikolay.vinogradov)
However it affects octavia charm, not the openstack-service-checks charm. Symptoms: https://paste.ubuntu.com/p/vjCD4vrV9z/

(by nikolay.vinogradov)
'juju resolved' fixes the problem

(by jog)
Solutions QA continues to hit this bug. All occurrences are listed here:
https://solutions.qa.canonical.com/#/qa/bug/1818113

(by jog)
SQA hit this bug 3 more times over the weekend.

(by gnuoy)
The crashdump from comment #13 shows octavia trying to communicate with other
services once it has received credentials from keystone. Looking at the logs
all octavia units fail running setup_hm_port at around 11:48. But looking at
the keystone log at the same time keystone/0 and keystone/2 have yet to
setup their database connection and are still talking to their local sqllite
db, this results in them returning a 500 error.
Looking at the keystone charm it delays advertising client credentials until
it is ready. Unfortunately it only checks its own readiness and not that of its
peers. This can lead to the situation from comment #13 where only one keystone
unit is able to service requests so there is a 2 in 3 chance of getting a 500
back.

(by mskalka)
Hit this again: https://solutions.qa.canonical.com/#/qa/testRun/a69e03d1-fcb5-43b1-8c66-cdc2721e8282

(by asbalderson)
Twice over the weekend: https://solutions.qa.canonical.com/#/qa/testRun/493aa388-7575-4a43-a084-08f5f6994189

(by hudson-openstack)
Fix proposed to branch: master
Review: https://review.opendev.org/721558

(by hudson-openstack)
Reviewed:  https://review.opendev.org/721558
Committed: https://git.openstack.org/cgit/openstack/charm-keystone/commit/?id=53bcfd0a064c11d90009d42f9fbe61d79f963e9c
Submitter: Zuul
Branch:    master
commit 53bcfd0a064c11d90009d42f9fbe61d79f963e9c
Author: Liam Young
Date:   Tue Apr 21 09:26:13 2020 +0000
Check peers before emitting identity data

The checks that keystone was performing before emitting identity
data were applicable to any Openstack api charm so the check
definitions have been moved to charmhelpers so other charms can
use them. The checks as they were are encapsulated in
`check_api_unit_ready` *1

Bug 1818113 was caused by keystone emitting identity data
as soon as the leader was ready but ignoring the state of the
peer units. This is now covered by a new check
`check_api_application_ready` which performs all the local
unit checks and then checks that all peers have reported
as ready too.

In addition `check_api_unit_ready` is now used when
setting the units workload status and `check_api_application_ready`
is used when setting the application workload status.

*1 https://github.com/juju/charm-helpers/blob/master/charmhelpers/contrib/openstack/utils.py#L2289
*2 https://github.com/juju/charm-helpers/blob/master/charmhelpers/contrib/openstack/utils.py#L2330

Change-Id: I99830ab2c2482e8beef174424820907ce96fd5d7
Closes-Bug: #1818113

(by alai)
We are still hitting this bug using the keystone -next charm.  The artifacts can be found on the bottom of the test run.
https://solutions.qa.canonical.com/#/qa/testRun/753c0272-f052-4450-8a5b-e205760774e6

(by alai)
https://solutions.qa.canonical.com/#/qa/testRun/794aaeef-0d8d-438c-b21b-55faec2b8380
In the run above, one of the keystone unit is updating endpoint for octavia.  Not sure if it helps in debugging.
keystone/2*                       maintenance  executing  12/lxd/7   <IP_ADDRESS>    5000/tcp                                 Updating endpoint for octavia

(by alai)
octavia/2 failed to connect to keystone in the run above.

(by 1chb1n)
Basing this on the premise that this stble run is actually using the next charms as reported:
https://solutions.qa.canonical.com/#/qa/testRun/753c0272-f052-4450-8a5b-e205760774e6

(by fnordahl)
Looking at the artifacts referenced in #33
units:
  keystone/0:
    workload-status:
      current: active
      message: Unit is ready
      since: 07 May 2020 20:14:44Z
    juju-status:
      current: executing
      message: running identity-service-relation-joined hook
      since: 07 May 2020 20:14:55Z
      version: 2.7.6
    leader: true
  keystone/1:
    workload-status:
      current: blocked
      message: Charm and its dependencies not yet at expected scale
      since: 07 May 2020 20:04:49Z
    juju-status:
      current: executing
      message: running identity-credentials-relation-changed hook
      since: 07 May 2020 20:15:13Z
  keystone/2:
    workload-status:
      current: blocked
      message: Charm and its dependencies not yet at expected scale
      since: 07 May 2020 20:03:55Z
    juju-status:
      current: executing
      message: running identity-service-relation-joined hook
      since: 07 May 2020 20:15:17Z

octavia_0/var/log/juju/unit-octavia-0.log:2020-05-07 20:09:25 DEBUG juju-log identity-service:135: tracer: set flag identity-service.available
octavia_0/var/log/juju/unit-octavia-0.log:2020-05-07 20:09:28 DEBUG juju-log identity-service:135: tracer: set flag identity-service.available.auth
octavia_1/var/log/juju/unit-octavia-1.log:2020-05-07 20:09:24 DEBUG juju-log identity-service:135: tracer: set flag identity-service.available
octavia_1/var/log/juju/unit-octavia-1.log:2020-05-07 20:09:27 DEBUG juju-log identity-service:135: tracer: set flag identity-service.available.auth
octavia_2/var/log/juju/unit-octavia-2.log:2020-05-07 20:09:23 DEBUG juju-log identity-service:135: tracer: set flag identity-service.available
octavia_2/var/log/juju/unit-octavia-2.log:2020-05-07 20:09:26 DEBUG juju-log identity-service:135: tracer: set flag identity-service.available.auth
So this could either be that the fix provided for Keystone does not cover all cases, or it could be something wrong with the detection of availability in the keystone reactive interface.
While we look for local a reproducer would it be possible to get a dump of relation data the next time this hits:

from the peer relation from the perspective of all the keystone units
from the identity-service relation from the perspective of all the octavia units

(by aurelien-lourot)
debug_log.txt contains lines like:
unit-keystone-2: 19:59:30 DEBUG unit.keystone/2.juju-log cluster:30: Telling peers this unit is: NOTREADY
unit-keystone-2: 20:12:50 INFO unit.keystone/2.juju-log Keystone charm unit not ready - deferring identity-relation updates
but never contains lines like:
DEBUG unit.keystone/?.juju-log Telling peers this unit is: READY
DEBUG unit.keystone/?.juju-log Application Ready
INFO unit.keystone/?.juju-log Firing identity_credentials_changed hook for all related services.
Based on the source code this proves that the keystone charm has correctly assessed that at least one unit wasn't ready and thus didn't mark the application as ready and also didn't update the identity-* relations. Debugging the charm locally shows that these things correctly happen later eventually.
However there is a special code-path [0] which sets 'service_password' on the identity-service relation with no regard for peer-readiness whatsoever. This path finally prints this line, which has been seen early in the logs:
unit-keystone-1: 19:57:06 INFO unit.keystone/1.juju-log identity-service:115: Deferring identity_changed() to service leader.
I don't have a precise solution yet but I'm now pretty sure this is the root cause.
0: https://github.com/openstack/charm-keystone/blob/master/hooks/keystone_hooks.py#L486

(by hudson-openstack)
Fix proposed to branch: master
Review: https://review.opendev.org/729781

(by mskalka)
David,
Has there been any progress on this issue? We are hitting this extremely often in our stable and master testing: https://solutions.qa.canonical.com/#/qa/bug/1818113

(by thedac)
Michael,
I believe Aurelien [0] is working on this. I will double check with him and have him update the bug.
[0] https://review.opendev.org/#/c/729781/

(by aurelien-lourot)
Hi, this is currently blocked on this fix [0] to be packaged and published.
0: https://bugs.launchpad.net/ubuntu/+source/python-oslo.policy/+bug/1880959

(by aurelien-lourot)
Still blocked. Temporarily published to https://jaas.ai/u/aurelien-lourot/keystone for testing.

(by aurelien-lourot)
Still blocked as long as https://bugs.launchpad.net/ubuntu/+source/python-oslo.policy/+bug/1880959 hasn't been backported to Queens

(by jason-hobbs)
We've been testing with aurelien-lourot's fix and appear to still hit this issue, here is an example test run:
https://solutions.qa.canonical.com/qa/testRun/b6233bf9-3d55-421d-a463-7e5465e29021
This uses cs:~aurelien-lourot/keystone-1

(by thedac)
After some more analysis, we can also attack this bug from the Octavia side. The root cause is with Keystone but Octavia is also not handling the error condition gracefully.
A trivial change may get this bug unblocked:
https://review.opendev.org/741304

(by aurelien-lourot)
I confirm that the last failing run is running my charm and that it contains my fix, since I can see these new traces [0] in debug_log.txt.
Note that in the log I can see that there is one unit (keystone/1) that has never been considered ready and also that keystone has never advertised the application as ready. This is also reflected in juju_status.yaml:
keystone:
charm: cs:~aurelien-lourot/keystone-1
series: bionic
os: ubuntu
charm-origin: jujucharms
charm-name: keystone
charm-rev: 1
charm-version: 07de96c
exposed: false
application-status:
current: waiting
message: Some units are not ready
since: 13 Jul 2020 16:42:12Z
So indeed I would now start looking at keystone's consumers as potential culprits.
0: https://github.com/juju/charm-helpers/pull/476/files#diff-935a00f40074c3c59d617c88f71e42efR2246

(by mskalka)
aurelion-lourot
Not sure if this is related to your fix, but in https://solutions.qa.canonical.com/qa/testRun/5589fe2d-bac5-4f35-80a1-57f4a3e454c4 it took the keystone charm 45 minutes to recognize that percona was clustered and ready before starting any endpoint updates (those updates then ran for 1.5hr+ before the run timed out). This is not behavior we have previously seen from keystone before using your fork. This seems like an inordinate amount of time for a deployment which already takes 5+ hours.
/var/log/juju/unit-mysql-1.log
...
2020-07-15 20:29:50 DEBUG juju-log cluster:32: All 3 percona units reporting clustered
...
/var/log/juju/unit-keystone-1.log
...
2020-07-15 21:19:02 DEBUG juju-log identity-service:144: Not at expected scale, not enough units on shared-db relation
...
...
2020-07-15 21:23:12 INFO juju-log shared-db:80: Updating endpoint for gnocchi
...

(by aurelien-lourot)
We found the new culprit:
According to the logs Octavia fails in setup_hm_port() which on the condition @reactive.when('identity-service.available'). Also we know from the previous keystone fix [0] that the issue was keystone setting service_password too early on the identity-service interface, thus signaling application readiness too early. The implementation of the interface [1] however shows that service_password is taken into account in identity-service.available.auth and not in identity-service.available. So either:
a. Octavia is waiting on the wrong condition, or
b. the definition of identity-service.available should be fixed to wait for service_password.
We prefer b as waiting for {relation_name}.available for application readiness is a widely used pattern and we would rather not have a special case for keystone.
0: https://review.opendev.org/729781
1: https://github.com/openstack/charm-interface-keystone/blob/master/requires.py#L91

(by aurelien-lourot)
Thanks mskalka for noticing! In the previous run with my charm [0] it took only 5 minutes (not 45):
unit-mysql-2: 16:51:01 DEBUG unit.mysql/2.juju-log All 3 percona units reporting clustered
...
unit-keystone-0: 16:55:55 INFO unit.keystone/0.juju-log identity-service:88: Updating endpoint for gnocchi
I went through my patch [1] again and I don't think this can cause this but I might not be very objective. Checking now the latest mentioned run [2].
0: https://solutions.qa.canonical.com/qa/testRun/b6233bf9-3d55-421d-a463-7e5465e29021
1: https://review.opendev.org/729781
2: https://solutions.qa.canonical.com/qa/testRun/5589fe2d-bac5-4f35-80a1-57f4a3e454c4

(by hudson-openstack)
Fix proposed to branch: master
Review: https://review.opendev.org/741547

(by hudson-openstack)
Reviewed:  https://review.opendev.org/729781
Committed: https://git.openstack.org/cgit/openstack/charm-keystone/commit/?id=8b46dfd637908adeb5a817f85fc97fad249be8a0
Submitter: Zuul
Branch:    master
commit 8b46dfd637908adeb5a817f85fc97fad249be8a0
Author: Aurelien Lourot
Date:   Wed May 20 10:28:59 2020 +0200
Fix peer readiness detection

Sharing the admin password with peers over the 'cluster' relation
was needed in case the leader would die and the next leader would
then need that information. This was implemented years ago when
the leader DB didn't exist.

This led to a race condition described in the mentioned bug and
can now be safely removed.

Validated by deploying several keystone and glance units, then
removing the keystone leader, then adding a glance unit and
checking that this new unit gets its service credentials.

Also added useful traces, made linter happy and increased fernet
token expiration to avoid spurious test failures.

Closes-Bug: #1818113
Change-Id: I004903e50f51e190467d71691982de26518d7149

(by aurelien-lourot)
mskalka I'm not sure why the latest run [0] timed out but from the debug_log.txt I see only 3 minutes (not 45):
unit-mysql-2: 22:49:36 DEBUG unit.mysql/2.juju-log All 3 percona units reporting clustered
...
unit-keystone-1: 22:52:55 INFO unit.keystone/1.juju-log identity-service:100: Updating endpoint for gnocchi
I suggest we continue this topic in a separate bug if needed because I doubt this is a regression introduced here. (I hope I'm not wrong.)
BTW you can now stop using ~aurelien-lourot/keystone as the fix [1] has now been published to ~openstack-charmers. And you were right, the issue is still not fully solved and we're working on the next fix [2].
0: https://solutions.qa.canonical.com/qa/testRun/5589fe2d-bac5-4f35-80a1-57f4a3e454c4
1: https://review.opendev.org/729781
2: https://review.opendev.org/741547

(by hudson-openstack)
Reviewed:  https://review.opendev.org/741547
Committed: https://git.openstack.org/cgit/openstack/charm-interface-keystone/commit/?id=a48a1e826ac0a74ab05a16c441cda4e4b34b1241
Submitter: Zuul
Branch:    master
commit a48a1e826ac0a74ab05a16c441cda4e4b34b1241
Author: David Ames
Date:   Thu Jul 16 14:14:44 2020 -0700
Collapse available and available.auth

Most reactive charms react on identity-service.available but the
current interface distinguishes between available and available.auth. It
is somewhat assumed by most charms that identity-service.available is
equivalent to identity-service.available.auth, as what charms are
concerned with is the ability to authenticate against the cloud.

Collapse the difference between identity-service.available and
identity-service.available.auth.

Continue to set identity-service.available.auth for any charms that may
rely on it.

Change-Id: I100cb1cfbff0b764932acccb0fdbb76ce6d1715c
Closes-Bug: #1818113


Estimation for web-based polls

What if we build a system to distribute polls via web/facebook and let people submit these web forms. Let's work this out and give it an estimation.

authentication (ask email, explain why + mention tombola if relevant)
7 components
sections (header/title/description)
navigation (between sections)
maybe register intent action

± 10d


Moving completed tasks to Obsidian's daily notes

Is your feature request related to a problem? Please describe.
Not related to a problem, but something that might be on the "good-to-have" list. It would be great if we could use Obsidian's daily note as an archive for completed tasks (someone mentioned they do this in the Discord server with another todo app).
Describe the solution you'd like
The process could be a little manual (at least for starters):

Open yesterday's (or whatever date I've selected) daily note
Invoque the command palette to add completed tasks from todoist
2a. I think an alternative could be the current syntax with the codeblocks if we're able to show completed tasks there
Add them as text to a (configurable) section, e.g. ## Completed tasks[...]

## Completed tasks
### Project 1
- Do thing A
- Read paper X

### Project 2
- Call person Y

### Home
- Water plants

3a. It would work just as the existing syntax, although completed tasks would still be stored in Todoist

Describe alternatives you've considered
As far as I know this isn't possible with filters. Probably the only way to get the completed tasks out of Todoist is through the API, but I could be wrong.
I assume the Todoist archive doesn't necessarily need to go in the daily notes, but could go into any other note too, but it's probably a lot more complex to define what to sync out of the completed tasks in that case.
Additional context
N/A

I can implement some variant of this, but sadly you cannot get completed tasks through the API.
What I can implement is something like: "when you complete a task from Obsidian it gets copied to a different section of the document". Is that something that's interesting or useful to you?

@jamiebrynes7 since I saw you plan one day to migrate to the sync api, could this be implemented then ?

Potentially yes, I'd need to investigate what kind of extra data the sync API exposes, but it seems more likely!

What I can implement is something like: "when you complete a task from Obsidian it gets copied to a different section of the document". Is that something that's interesting or useful to you?

@jamiebrynes7 this would be an absolute godsend for my workflow. The only thing keeping me in using Obsidian ToDos alone (instead of a preferable combo of ToDoist and Obsidian via this plugin) - is the fact that once a task has been completed, I lose all visibility over it from Obsidian.
I would assume that it would be like another "Project" added to your ToDoist view, that says "Completed" - where the items would appear as a completed item formatted in markdown.

I've checked the new Sync API, and it seems like you can get the completed items from it and you can even uncheck them.
I'd prefer if the completed items stayed where they were actually instead of them moving to daily notes.

I'd love to be able to keep completed tasks somewhere in my Daily Note. Preferably under a configurable heading. That way I can grab them in my Weekly Review template.

This would be great, and getting completed task info is pretty straightforward with the sync API (e.g. https://github.com/wbthomason/import-todoist.nvim/blob/master/rplugin/python3/todoist_import.py#L17-L45, as an example with the Python sync API library).

Looking at the sync API, it doesn't support filtering, so things like listing done tasks for a specific day will probably not work.
Let's hope Todoist adds this feature to their API. I pinged them on Twitter. If enough people asks for this, they might consider implementing it.

@jamiebrynes7, can you build this feature on plugin side? i.e. after checking as "completed", plugin can create markdown version of completed task under todoist query.

Daily note or otherwise.. being able to keep completed tasks somewhere in Obsidian would be huge. Then linking tasks to notes would still be valuable.

Daily note or otherwise.. being able to keep completed tasks somewhere in Obsidian would be huge. Then linking tasks to notes would still be valuable.
While not exactly the same feature requests in the original post, intent would be to have tasks imported via Todoist to carry the same functionality as a task created in Obsidian through something like: https://github.com/schemar/obsidian-tasks

it doesn't support filtering, so things like listing done tasks for a specific day will probably not work.
It does support filtering, at least for Pro users. I have a custom script for something similar in my daily notes that grabs any tasks that were completed that day. I'd be happy to open a pull request if you're open to contributions?

it doesn't support filtering, so things like listing done tasks for a specific day will probably not work.

It does support filtering, at least for Pro users. I have a custom script for something similar in my daily notes that grabs any tasks that were completed that day. I'd be happy to open a pull request if you're open to contributions?

Could you share the script regardless of pull request? This would save so much manual work or using alternatives like VCRecur, which isn't quite the same.

I have figured out a way to make this happen, although there are a lot of working parts. I'm using IFTTT, Dropbox, and Hazel (Noodlesoft utility to manage files based on rules on Mac), and a little bash script to make it work. The gist:

IFTTT recipe creates a text file in Dropbox when a Todoist task is completed
Hazel monitors that folder, changes the file name to match the date style I use for the daily note, but with a .txt extension
Hazel runs the bash script which appends the content of the .txt to the matching .md file in my "Journal" folder
Hazel deletes the .txt file on Dropbox

I can share the specific IFTTT recipe and bash script if it's of use to anyone - just thought I'd post the gist of it if anyone wants to try recreating it with their own tools and utilities. It works pretty well!

I'm very interested in this functionality as well.
Currently I achieve it by using this Todoist task in conjunction with another: https://github.com/wesmoncrief/obsidian-todoist-text/issues
I use this plugin to display todos in the notes that are not today's daily note. When I open the daily note of the current day in the morning, I used the obsidian-todoist-text plugin to 'print' my todo list to text in obsidian, delete the embed from this app, keep track of my todos in obsidian, and then catch up my actual todoist list at the end of the day. The goal being to a.) preserve a list of completed todos b.) enable myself to link 'i have a meeting' tasks to notes about said meetings at the beginning of the day.

I think there's now a plugin than can do this - https://github.com/Ledaryy/obsidian-todoist-completed-tasks


MTL-1968: stop copying out csm_ntp.py

Description
This was necessary through CSM-1.2. It has been redundant / straight code duplication since then. This rsync was done to overwrite a buggy csm_ntp.py that shipped in CSM-1.x.

Checklist

[ ] If I added any command snippets, the steps they belong to follow the prompt conventions (see example).
[ ] If I added a new directory, I also updated .github/CODEOWNERS with the corresponding team in Cray-HPE.
[ ] My commits or Pull-Request Title contain my JIRA information, or I do not have a JIRA.

/backport release/1.6


Automated Package Upgrade

:tada: This PR is included in version 12.4.136 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Documentation

Extend the readme with more detailed information. (e. g. Environment, function, example)

See branch fh-58-documentation


Updated the Using nextTick section

The currrent description for Using nextTick was very confusing for a beginner so updated that for easier understanding.
The example provided for nextTick would have not worked unless the user would import Vue (which had not been stated anywhere so far, nor was necessary) so made changes for the example to work with wrapper.vm.$nextTick().

What kind of change does this PR introduce? (check at least one)

[x] Bugfix
[ ] Feature
[ ] Code style update
[ ] Refactor
[ ] Build-related changes
[ ] Other, please describe:

Does this PR introduce a breaking change? (check one)

[ ] Yes
[x] No

If yes, please describe the impact and migration path for existing applications:
The PR fulfills these requirements:

[x] It's submitted to the dev branch.
[ ] When resolving a specific issue, it's referenced in the PR's title (e.g. fix #xxx[,#xxx], where "xxx" is the issue number)
[ ] All tests are passing: https://github.com/vuejs/vue-test-utils/blob/dev/.github/CONTRIBUTING.md#development-setup
[ ] New/updated tests are included

If adding a new feature, the PR's description includes:

[ ] A convincing reason for adding this feature (to avoid wasting your time, it's best to open a suggestion issue first and wait for approval before working on it)

Other information:

Nice. Thank you for this.
Context: @bhaskar-nair2 and I spoke in Vue Land's testing channel after he read the docs and needed additional context on what/why nextTick was needed.
This helps users who are visiting the docs/VTU for the first time understand WHY and WHEN they will need nextTick.


[I Don't Have Spotify] Select default service

Extension
https://www.raycast.com/sjdonado/idonthavespotify
Description
Would be cool if the extension had the option to select which streaming service is preferred as the default option!
Who will benefit from this feature?
Everyone, I think!
Anything else?
No response

@third774 good idea, I'll work on it!

Commenting to remove stale marker — I think this is a worthwhile feature!

Sorry that I haven't replied before. I have plans to implement this feature soon :)


Is this package abandonned?

Doesn't work with the latest omnipay :c

Omnipay 3.x was released one month ago. So please use Omnipay 2.x if you want this package or you can update the code and send a pull request :)

Ok, but do you plan to update it? I know nothing about package development.

Yes but i can not tell how much time it will take.

Ok, thank you :)

You are welcome. Please keep this issue open :)

Okay np 💯


Do not supply discovery client when Consul discovery client disabled

Fixes #785

@spencergibb let me know what you think about this change....the other option is to continue to return null but then we will also need fix ConfigServerInstanceProvider to handle the null function.  Ultimately the end result is the same...an IllegalStateException will be thrown withe the same error message indicating not being able to find the config server.

@ryanjbaxter I'm wondering if we should extend these changes to other discovery client implementations that could suffer the same problem (Zookeeper, Kubernetes, ...) ?


Handle loading LST DL1 file from lstchain v0.9

Fine for me. Here you are now assuming LST-1 ID = 1, so you loose a bit the IDs generalization, but we can accept this and fix it in the future by importing the configuration file (BTW, real data now only have ID =1)

Well, it can be fixed now I would say. We can just simply pass the dictionary of the mc tel ids (not all the config, unless we need something else from there to be passed to the function), so that even if there are more LSTs added in the future in the real data, the function is already usable.

But I wanted to spare him some extra work both on the coincidence script and on the tests. He has already offered to fix the SubArray!

@aleberti This fix is meant to handle loading dl1 data from existing lstchain v0.9 files. So it will never be applied to dl1 data from LST2-4.
In case a dl1 file contains the data from multiple telescopes (not sure if this is the plan) and such a fix is still needed in the future, the tel_ids could probably be read from the dl1 file.

Well, in any case the ID is changed later in the coincidence script. Also, we cannot even get the telescope ID from the broken subarray. So, not much to do anyway.

There are no new comment since 2 weeks. @jsitarek green light from your side to merge? You have been assigned as 2nd reviewer.

thx @Elisa-Visentin
as soon as the checks finishes correctly I think we can merge


Updated certifi version

This PR updates certifi to the latest version to fix a security vulnerability.
There's an absolutely huge number of changes in poetry.lock, I think this is because I've recently updated poetry (to 1.5). Hopefully they've worked out backwards compatibility this time

Codecov Report

Merging #82 (3b733f6) into release/3.0.0 (b5b021a) will not change coverage.
The diff coverage is n/a.

@@              Coverage Diff               @@
##           release/3.0.0      #82   +/-   ##
==============================================
  Coverage          69.00%   69.00%
==============================================
  Files                 12       12
  Lines               3639     3639
==============================================
  Hits                2511     2511
  Misses              1128     1128

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


Cannot play soundboard audio after audio has been played

Describe the bug
After you play any type of audio (using either /music or /soundboard), you cannot play any /soundboard sounds again until the bot disconnects.
Steps to reproduce

Make sure esmBot is not already in the VC.
Use /music or /soundboard to play an audio.
The bot should join and play what you request.
Either skip the audio or wait for it to finish.
Try to use a /soundboard command again.
The bot will state "I can't play a sound effect while other audio is playing!"

Expected behavior
The bot should play the sound after the audio finishes/is skipped.
Screenshots
No response
Self-hosted instance?
Yes
Additional context
Reproducible with both slash commands and message commands.
Additional music (using /music play) will play after music/soundboard sound has been played.
Bot is configured to not leave voice channel after audio finishes playing.
Running bot in Docker container.
Music and soundboard commands cause an "Unknown return type" console warning.

Commands ran to produce above image output:

/soundboard oof (successfully played)
/music play <query> (successfully played)
/music skip (successfully skipped)
/soundboard bruh (did not play)
/music stop (bot disconnected)

I can't seem to reproduce this issue on the main bot, latest commit, or lavalink-v4 branch; is it still happening for you?

It's still happening for me... does it have anything to do with me using the Docker image?

As of 60b0d8e79272acbeab0ff9b96998e0a0e257676a, soundboard commands have been removed, meaning that this issue is no longer possible.


Sourcemaps for css/sass are not working

Environment

Operating System: Linux
Node Version:     v16.15.1
Nuxt Version:     3.0.0
Nitro Version:    1.0.0
Package Manager<EMAIL_ADDRESS>Builder:          vite
User Config:      telemetry, ssr, devServer, typescript, components, css, webpack, sourcemap, dev, postcss, modules, plugins
Runtime Modules<EMAIL_ADDRESS>Build Modules:    -

Reproduction
Steps to reproduce:

Try to enable css/scss sourcemaps in nuxt.config.ts

Describe the bug
I'm using sass in the project and sourcemaps are not toggling on. All the sass and css are all being converted to inline styles.

Here's my  nuxt.config.ts
export default defineNuxtConfig({
  ssr: false,
  devServer: {},
  typescript: {
    strict: false,
    typeCheck: true
  },
  components: {
    dirs: [
      {
        path: "~/components/global",
        global: true,
        pathPrefix: false
      }
    ]
  },
  css<EMAIL_ADDRESS><EMAIL_ADDRESS>
  webpack: {
    cssSourceMap: true
    loaders: {
      scss: {
        sourceMap: true
      }
    }
  },

  sourcemap: {
    server: true,
    client: true
  },

  dev: true

  postcss: {
     plugins: {
      tailwindcss: {},
      autoprefixer: {}
    }
  },

  modules: [
    // "nuxt-typed-router"
    "@pinia/nuxt"
  ],

  plugins: []
});

Additional context
No response
Logs
No response

Hey! Could you kindly provide a reproduction? 🙏

@manniL https://stackblitz.com/edit/github-x4a2um  👍

Sourcemaps in CSS are not currently supported in vite. The issue is being tracked in https://github.com/vitejs/vite/issues/2830.

@danielroe Thanks for linking that issue, found a temporary fix there that is working. Added this to the config,
  vite: {
    css: {
      devSourcemap: true
    }
  },


DBZ-7732 Add ConnectorSpecific annotation to manage multiple SPI connector specific implementation

closes: https://issues.redhat.com/browse/DBZ-7732

@mfvitale I wonder if we can add a test to Debezium Server rep verifying this fix. Ideally it should have multiple connectors on the classpath and test two of them to make sure they are working. If necessary a new module would me introduced for it. WDYT?

@mfvitale I wonder if we can add a test to Debezium Server rep verifying this fix. Ideally it should have multiple connectors on the classpath and test two of them to make sure they are working.

Yeah, I was thinking to add it too but I need to understand how tests works on Debezium Server. Any point where to start? Seems that as how now it just start PostgreSQL, isn't it?

If necessary a new module would me introduced for it.

Can you please explain better?

Well, the idea is that there will be a new module added to debezium-server project, like debezium-server-testing. This one will have multiple connectors on the classpath to make sure there are clashing classes. There will be one basic test for for example PostgreSQL and on for MySQL - generally get one record from snapshot, one from streaming, check there are no errors. Such tests should exhibit the existing bug and start passing with the fix.

The other option is to try to add it to the core library - a new module under debezium-testing and test it at Debezium Engine level as again I suppose there should be the issue if there are multiple connectors on the module classpath, should not it?

The other option is to try to add it to the core library - a new module under debezium-testing and test it at Debezium Engine level as again I suppose there should be the issue if there are multiple connectors on the module classpath, should not it?

Yes and maybe this is a better solution. But I was looking at debezium-testing-system module that seems to do exact this thing but seems to be designed to run manually. Is there any place where we execute it?

debezium-testing-system is module that contains product tests and these are executed on product build candidates - @obabec can tell you more

@jpechane At the end I added it to Debezium Server since it can be the base for further tests on it.
https://github.com/debezium/debezium-server/pull/85
I have verified that without the fix the tests fails.

@mfvitale Applied, thanks!


Persist generated subscription keywords

We currently have code that generates a keyword for a subscription from
the subject and job_title in its search criteria, if the subscription
doesn't have a keyword to begin with. This used to live in a separate
AlertSearchBuilder, which we removed in a recent PR, and is now a
method in Subscription.
Newly created subscriptions no longer need this because keywords are now
generated in CriteriaDeviser, so we can just persist the generated
keywords against the existing "old" subscriptions, and then after get
rid of the Subscription#keyword method.
Also removes the now unneeded subscription location category task.

Review app deployed to https://teaching-vacancies-review-pr-2884.london.cloudapps.digital


chore: release main

:robot: I have created a release beep boop
core: 0.7.4
0.7.4 (2022-11-22)
Features

parser: Code syntax highlight, no css. (ca9d5ef)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/timgreen/Anki.md/releases/tag/core/v0.7.4 :sunflower:


Automaking project with java 1.6_045 fails

Hello,
i've got problems with automatic building of project with jdk 1.6_045.
Here is my test example repo : https://github.com/Hejwo/LombokPluginApp .
When i'm building project with Maven (JAVA_HOME set to jdk 1.6_045) it work fine. Also similar project witch we develop in our team in Eclipse compiles without problems. But when I use IntelliJ and make with same jdk it fails. Here is my StackOverflow topic :
http://stackoverflow.com/questions/29165157/intellij-14-lombok-plugin-builder-error
I'm out of ideas... Can you help me ?

Hello Hejwo,
I don’t use java6 anymore, so a can’t help you. With Java7 or Java8 I can compile and build  your project with Intellij 14.1 without any problems.
What is you Intellij version? You can also update your build jdk.

Hi,
thanks for quick response.
I use newest IntelliJ 14.1.
Well... main problem is that our project is really old and we can't upgrade to newer jdk's. It works only under 1.6 due to deprecated code.
Linked above project on GitHub is just an example to issolate the error.
A lot of our developers use Eclipse and they have no problems. Only few people use IDEA and they all have same problems.

You can use jdk7/8 and use source/target parameter 1.6 to compile your project to be compatable with Java6 on Runtime, is it not an option for you?

Unfortunately not :(.
I've tried it allready. Lombok annotations are working but project does not compile because of some old stuff...


Could not get the value of "alias" correctly when an email forwarded and bounced

The alias and recipient are the same value

The recipient is the final destination address

It occurs in forwarding to Gmail or other ESP
To read Received: headers in the original message for resolving this issue
Sisimai::RFC5322->received should get the envelope-to address after for

Read and parse the value of Sisimai::Message->rfc822->received

This issue fixed at v5.0.2


[BUG]  Navbar of Augmented reality is not visible

Describe the bug:
the navbar in the AR section is not correctly visible due to the same color background gif the parameters not accessible

Expected behavior:
A clear and concise description of what you expected to happen.
Actual behavior:
A clear and concise description of what actually happened.
Reproduction Steps
Steps to reproduce the behavior:

Go to '...'
Click on '....'
Scroll down to '....'
See error

Expected result:
A clear and concise description of what you expected to happen at each step.
Actual result:
A clear and concise description of what actually happened at each step.
Screenshots
If applicable, add screenshots to help explain your problem.
Additional Information
Device (please complete the following information):

OS: [e.g., Windows, macOS, Linux]
Browser [e.g., Chrome, Safari, Firefox]
Version [e.g., 22]

Additional context:
Add any other context about the problem here.

@akshitagupta15june  do assign me


platform/api/equinixmetal: use httpS for PXE URL

PXE boot over httpS is now supported since this PR: https://github.com/flatcar-linux/Flatcar/issues/527

We switched to ISRG Root X1 preferred chain, the handshake data is now short enough that iPXE can digest it.

Signed-off-by: Mathieu Tortuyaux<EMAIL_ADDRESS>Testing done

Run an amd64 test in Equinix Metal

The URL used for testing is passed from the script environment and uses GCS (where the image to test is stored)


chore: disable some stories from loading in the website

We don't want to show all of the custom style and dark theme demos. The light/dark theme will be
a toggle on the website and the custom style demos should remain in advanced content.
PR Checklist
Please check if your PR fulfills the following requirements:

[ ] Tests for the changes have been added (for bug fixes / features)
[x] Docs have been added / updated (for bug fixes / features)
[ ] If applicable, have a visual design approval

PR Type
What kind of change does this PR introduce?

[ ] Bugfix
[ ] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Build related changes
[ ] CI related changes
[x] Documentation content changes
[ ] clarity.design website / infrastructure changes
[ ] Other... Please describe:

What is the current behavior?

Issue Number: N/A
What is the new behavior?
Does this PR introduce a breaking change?

[ ] Yes
[x] No

Other information

A possible cleaner way for the commit history chore(website): disable some stories from loading


Refactor client.today and client.yesterday

Since most parts of client.today and client.yesterday are the same code, i extracted it as client._daily.

Sorry, I don't know how the refactoring should be described in keep a changelog.

Thanks @noriapi, this is great. For the CHANGELOG, I would just add something like this:
### Changed

- (internal) Refactored daily note creation

Thanks @noriapi, this is great. For the CHANGELOG, I would just add something like this:
### Changed

- (internal) Refactored daily note creation

Thank you, I added it to CHANGELOG.


text encoder network details?

I noticed that in your code, for the text encoder part, you used the clip and T5 models. Based on this, I have two questions: 1. Will the clip and T5 models update parameters during training? 2. What is the difference between these two clip models?
Looking forward to your answer, thank you

We followed SD3 in the text encoder design, where these models are frozen during training. The two CLIP models are used separately for the text branch of MM-DiT and AdaLN.

In training process, only the parameters of MM-DiT and VAE will be updated?

In training process, only the parameters of MM-DiT and VAE will be updated?

The VAE is also frozen during training. We used pre-extracted VAE latents to train the MM-DiT.

In training process, only the parameters of MM-DiT and VAE will be updated?

The VAE is also frozen during training. We used pre-extracted VAE latents to train the MM-DiT.
Thank you very much for your answer, can I understand it as: The paper states that vae comes from MAGVIT, which means that vae encoder and vae decoder both come from the MAGVIT model and are not need training. only the MM-DiT needs trained?

In training process, only the parameters of MM-DiT and VAE will be updated?

The VAE is also frozen during training. We used pre-extracted VAE latents to train the MM-DiT.
Thank you very much for your answer, can I understand it as: The paper states that vae comes from MAGVIT, which means that vae encoder and vae decoder both come from the MAGVIT model and are not need training. only the MM-DiT needs trained?

Thank you for your great work. I really want to figure out the composition of each part of pyramid-flow: I now know that CLIP_1, CLIP_2, T5, MM-DiT come from SD3, of which only MM-DiT needs to be trained in pyramid-flow, VAE-encoder comes from MAGVIT and does not need to be trained. So where does VAE-decoder come from and does it need to be trained?

The VAE encoder and decoder were trained from scratch, with some of the key design choices following MAGVIT-v2.

The VAE encoder and decoder were trained from scratch, with some of the key design choices following MAGVIT-v2.

Got it, thank you very much for your answer


ghcide does not seem to report non-exhaustive pattern matches as diagnostics

I'm surprised there hasn't been an issue raised about this already, so I suspect my setup is to blame. ghcide does not seem to be reporting warnings such as:
src/Nng.hs:336:19: warning: [-Wincomplete-patterns]
    Pattern match(es) are non-exhaustive
    In a case alternative: Patterns not matched: _
    |
336 | addressFromText = \case
    |                   ^^^^^...

-Wincomplete-patterns is not enabled by default. Are you sure that your cradle enables it? E.g., if you use the stack or cabal cradles you need to enable it in your cabal file.

This is a dupe of https://github.com/digital-asset/ghcide/issues/107 which explains why it hapoens

Ah good point, thanks. I got confused because I was sure that I had seen this warning for DAML but of course we generate Core there. Closing as a duplicate.


Rebranding the readme for DataTrails

Textual rebranding. I didn't change the scripts, to avoid impacting any automation
 Mention [stepsize] in a comment if you'd like to report some technical debt. See examples here.

I merged this so that I can bring a similar PR that I am working on up to date.


Database should be toggleable

The Database provided should really be off by default, and toggleable. That or other database options should be provided, by using an interface with many subclasses for different Database types. Storing player data in flat file will grow over time, and if the server is used a lot, it has the chance that the server may be slowed down by storing the large database document in RAM.
I can see that it is used for settings, and snowflakes, which personally I believe would be best stored in YAML, or in an actual Database, such as MongoDB or MySQL. Although ideally, for scrimmages, it should be off completely, unless of course you provide an open database for all Cardinal servers to use.

I am aware of the weaknesses of the "database." I am looking at possibly having settings/snowflake/other stats stored remotely and acquired when a player logs in.

@twizmwazin like remote DB?

@RubeGamerV perhaps

Is this being worked on?

@intenseoxygen more like the other way arround, database is now needed for ranks to work (in cardinal 1.9), moved all the data for who has what ranks to there, instead of the config-based saving that we had before

@Pablete1234 nice keep it up


Excessive throttling when running ACA-py agents containerized cloud environments

TL;DR
I've noticed that a number of our agents, mediators or otherwise, have a very high amount throttling when run in a containerized cloud environment. This would be OpenShift in our case, but more broadly impact any Kubernets platform. In this case throttling is when the process didn't get all the CPU time it needed to complete its work.
Scheduling on Linux and Kubernets
You cannot be on a fractional part of a CPU core. You're either on a core or you're not. Modern Linux systems, and by extension Kubernets work around this by converting your CPU into time slice. When a containers starts it is given an infinite amount of 100ms blocks of time. Your container resource limits, like limit.cpu is converted into its fractional mount of that 100ms block of time. For example, 100ms is 1/10 of a vCore which is 10ms of run time. Out of the 100ms block of time, the container has 10ms to be running on a physical CPU core. The other 90ms is waiting time where it is unscheduled and waiting for the next 100ms block of time.
Note: This applies to any container run-time. Its all managed by cfgroups in Linux as this is not a Kubernets specific feature. Containers are just three linux features working together (cfgroups, namespaces, and UFS).
Throttling
I noted above that throtteling means you needed more time to do work but were un-scheduled by the Linux scheduler. This situation is tracked by linux in the following file:
1004450000@aries-mediator-agent-35-77gk5:~$ pwd
/home/indy
1004450000@aries-mediator-agent-35-77gk5:~$ cat /sys/fs/cgroup/cpu/cpu.stat
nr_periods 19040
nr_throttled 4905
throttled_time<PHONE_NUMBER>18
1004450000@aries-mediator-agent-35-77gk5:~$

Where nr_periods is the total number of 100ms blocks of time the container was given since it started up and nr_throttled is the number of those blocks the container / process did not get enough time to do its work. To figure out the % we just 4905 / 19040 * 100 = ~25%.
I think some throttling is acceptable but I regularly see Agents run at 50% throttling unless the limit.cpu is set above 800m or 8/10 of a vCPU and I can get it below 1% if I use 1000m or 1 vCPU. Setting to 1 vCPU means the container can run 100% of the time.
Threads
The problem of throttling can be made worse by threads. In the example above 10ms of run time might be fine for 1 thread. But if you have 12 threads each one is in a death match for some CPU time  and may only get 1-2ms to do their work. Maybe not enough time to do heavy cryptographic workload. I believe ACA-py uses a thread pool whenever run_in_executor is called.
Impact
Our specific case was running a mediator on ACA-py 0.8.x with enough CPU request / limit to make things work fine. This update started causing quite a few timeouts in our AFJ client. When I did a deep dive to triage the issue I noticed the throttling metric. I increased the CPU until I got the percentage below 25 and everything is working as expected (so far). What we saw was that with a few hundred millicores the throttling was about 50% causing mediation connection (probably web-socket) timeouts.
Remedy
I think you need to notably over-provision a container's CPU in a containerized cloud environment. It would be worth further evaluation to find out the impacts of running at high levels of CPU throttling. At a minimum, I think its worth noting in the docs.

Thanks, @jleach — great write up.  So, in your specific case, what parameters did you change — from what to what?
When you say “until I got the percentage below 25” — is that the equivalent of CPU usage — e.g. that you got the percent CPU to below 25? Or for a simple person like me to understand — like calling up the Mac Activity Monitor or Windows Task Manager and looking at the %CPU column? You altered the “limit.cpu” until you got the % CPU to where you wanted it.

@swcurran I had greatly over provision the pod, and the percentage (25) was throttling meaning the ratio of the number of run time it was hungry for more CPU time. When I gave the pod < 800m (8/10 of a CPU core) I would see that about 45% of its run time it was throttled. We would see erratic behaviour like WebSocket timeouts when AFJ clients were connecting. When I gave it 800m the percentage of throttling dropped to 18% and the problem went away: overnight regression tests passed and team testing all worked. If I give it 1 full CPU core it drops below 1% throttling.
A good first step to resolve this would be, if possible, to make ACA-py log thread pool count (number of threads or tasks or whatever is the Python equivalent) when in debug mode. Then we can see if when throttling is high how the thread count varies.

I believe this is more of the impact of Python Global Interpreter Lock or GIL. Since the Python process is limited to running in basically a single thread, if that thread is limited CPU time, it will be limited to a single core. If that core has a CPU limiting on it, than it simply cannot go faster and will spend a good amount of time in the throttled state.
NodeJS has similiar limitations as it also has a single cpu thread design.
The majority or at least half of the aca-py time is spent inside of the C APIs, which by default locks the GIL. See https://pythonspeed.com/articles/python-gil/
Any service that is blocked may not be able to respond to websocket requests etc, resulting in timeouts.

Sounds like we need to do some profiling of ACA-Py and AFJ on Kubernetes to understand what is needed when provisioning and scaling environments.  Based on the mediator example, we know we can reduce throttling significantly by increasing the provisioning, and in turn we reduce the timeouts we were experiencing.

Can we monitor timeouts?
Can we correlate that with throttling?
Can we get some metrics to set provisioning, or is that use case specific?

Is it necessary/sufficient to differentiate by mediators, issuers and verifiers?

Can/Should we use throttling as a scaling indicator?
Do ACA-Py and AFJ behave differently?

Top priority is figuring out how to investigate these issues. What tests can we run to understand what is happening and what we can do to influence the behaviours?
And of course, being able to scale mediators is a crucial element of this. How do we scale with web sockets?

@reflectivedevelopment I say threading because its consistent with the behaviour I see: An ACA-py agent will have very low CPU utilization, typically < 1% but a very high throttling ratio, typically > 45%. Usually this pattern indicates problem with threads. I also see this often in Postgres but in that case I think its because it has several processes inside the "container"; each process has the same effect as a thread - they fight for CPU time and in a unimportant process consumes too much then the important ones are starved.

Can we monitor timeouts?

These only happen with Websockets when using ACA-py as a mediator. I do have a test script that can easily reproduce and monitor.

Can we correlate that with throttling?

I've done this and can reproduce it easily:

Set limit.cpu and cpu.request to 2/10th of a core. Run the test script. Watch CPU usage at < 1%, throttling at ~45%.
Set limit.cpu and cpu.request to 8/10th of a core. Run the test script. Watch CPU usage at < 1%, throttling at ~16%.
Set limit.cpu and cpu.request to 1 core. Run the test script. Watch CPU usage at < 1%, throttling at < 1%.

Can we get some metrics to set provisioning, or is that use case specific?

More details needed.

Is it necessary/sufficient to differentiate by mediators, issuers and verifiers?

I can't empirically tell if its a problem for issuers and verifiers. They seem to work fine in this situation and have for a while now. Maybe they preform slower than they should. Its only the mediators that have serious problems.

Can/Should we use throttling as a scaling indicator?
Do ACA-Py and AFJ behave differently?

I had a look at the AFJ agent used for our showcase. It does present a similar problem but its fall less prominent. More research required. One example is just a data point - not a trend.

Top priority is figuring out how to investigate these issues. What tests can we run to understand what is happening and what we can do to influence the behaviours?

I think we need to understand what ACA-py is doing to have low CPU utilization but high throttling? Too many processes in the container? Threading? Is Indy doing something?

And of course, being able to scale mediators is a crucial element of this. How do we scale with web sockets?

I feel like in its current state a container can't deal with even a few WS connections.

This doesn't sound quite right to me. Is it perhaps related to a bug in the CFS scheduler? See
https://medium.com/omio-engineering/cpu-limits-and-aggressive-throttling-in-kubernetes-c5b20bd8a718
https://github.com/kubernetes/kubernetes/issues/51135#issuecomment-373454012
https://github.com/kubernetes/kubernetes/issues/67577
https://gist.github.com/bobrik/2030ff040fad360327a5fab7a09c4ff1

@reflectivedevelopment I know of this issue and checked with RedHat to see if the fix was back-ported to our kernel version - it was. It's a good point though, it would be worth deploying the mediator to different k8s infrastructure to see how it preforms there.

I added run-2157.sh to test for the Websocket timeout issue. Just run it and in the background tweak the amount of CPU a pod gets. You'll see the failure rate change accordingly.

I think that it would be tricky to test like that. We would need to have something reproduce the problem on our k8s implementation, and then run the identical test on another k8s. Do we have such a portable test?
It was a bit hard to follow all of the links to determine what versions of the Linux kernel has the bug and the fix, but from what I can see — it is fixed in 4.18 and later, that was released circa 2020.  So I would think the easiest check is to see if our OpenShift deployment is running with a fixed kernel. I assume that the fix is on the infra, not in containers (so many kernels 😄 ) so that should be sort of easy to check. Asking Red Hat is probably the right way to go.

@reflectivedevelopment I say threading because its consistent with the behaviour I see: An ACA-py agent will have very low CPU utilization, typically < 1% but a very high throttling ratio, typically > 45%. Usually this pattern indicates problem with threads. I also see this often in Postgres but in that case I think its because it has several processes inside the "container"; each process has the same effect as a thread - they fight for CPU time and in a unimportant process consumes too much then the important ones are starved.

If we are waiting on other threads, we should see the CPU time be 45%, just with unimportant threads. The threads are either blocked, waiting for resources from postgres, disk, etc, or are being starved by the CPU scheduler. If we are seeing 100% CPU usage with no quota, but 1% CPU usage with a 90% quota, I'm inclined to think the scheduler is not scheduling the process.

@reflectivedevelopment 🤔 I think waiting on db would show a process bing in the WAIT state - it wouldn't count as a throttled block. The only way I can account for the behaviour I see is with threads (or many processes in the container).
The way I understand the scheduler to work is like this: If a container has a request.cpu of 50m and a limit.cpu of 100m it gets 10ms to run. If you have 10 threads they can all run at the same time if cores are free, but, that means you burn through your 10ms faster. Each thread gets 1ms to run. Because CPU usage is based on the request.cpu it shows as low CPU usage because 1ms is < 5ms (50m).
This meats all the criteria. Its throttled because it used up its quota in the first 1ms (much less than its 10ms as per the limit) and has a very low CPU utilization because 1ms is much lower than 5ms.

@jleach This would be interesting if Aca-Py used more than one CPU core on a regular basis. It appears from my testing that Aca-Py almost always uses one core and has very little if any parallel processing, and is mostly serialized in its CPU usage.
However, if the statment is true that the Aca-Py requires more than one CPU to run, it would be necessary to include additional CPU time to keep it operating correctly.  I imagine some programs behave in a similar manner.
Still, this doesn't explain the 1% cpu utilization. If the schedular is properly scheduling the threads, then each thread should register CPU time, and we should see that in the CPU utilization. If I have a multi-threaded application using two CPUs, it will register as using 200% of the CPU. So even if Aca-Py has threads running on multiple cores, it should show up as part of the CPU usage.

@jleach for the websocket disconnects, we may want to adjust the ping and timeout settings.
My recommendation is based off of research of the QUIC protocol, used for HTTP3. QUIC was designed by a google engineer and makes up half of Google Chromes traffic. The reason for the selection is to base decisions upon prior art that has been successfully deployed.
RFC 9308: Applicability of the QUIC Transport Protocol .
In RFC 9308, it indicates the following:
"According to a 2010 study ([Hatonen10]), UDP applications can assume that any NAT binding or other state entry can expire after just thirty seconds of inactivity. Section 3.5of [RFC8085] further discusses keep-alive intervals for UDP: it requires that there is a minimum value of 15 seconds, but recommends larger values, or that keep-alive is omitted entirely."
And
"Sending PING frames more frequently than every 30 seconds over long idle periods may result in excessive unproductive traffic in some situations and unacceptable power usage for power-constrained (mobile) devices. Additionally, timeouts shorter than 30 seconds can make it harder to handle transient network interruptions, such as Virtual Machine (VM) migration or coverage loss during mobility. See [RFC8085], especially Section 3.5."
I would therefor recommend setting the keep alive to 15 seconds, the minimum defined in RFC8085, and setting the timeout to 60 seconds to allow connections to survive coverage loss during mobility.
The current Aca-Py Default timeout is too short, and may result in unnecessary mobile disconnects.
The current Aca-Py Default keepalive is too short and may result in excessive network traffic.
I believe these can be tuned with --ws-heartbeat-interval ( 3 seconds default) and --ws-timeout-interval ( 15 seconds default)

@jleach, We can easily try out the settings by adding ACAPY_WS_HEARTBEAT_INTERVAL=15 and ACAPY_WS_TIMEOUT_INTERVAL=60 to the environment variables of the agent's DC.

I've seen this error pop back again during an onboarding session last Friday (Match 17, 2:30 - 3PM PT). Fresh install, first wallet initialization. The workaround was to delete the app and try again. The 2nd time it worked with no problem.

@cvarjao We only rolled out the above changes to test. Also, in prod we have 800m/800m which drops throttling to about 18%. Need to go 100m/1,000m probably (assuming request can be low) to get it below 1%.

@jleach, I made some those updates to prod being tracked here; https://github.com/bcgov/bc-wallet-mobile/issues/990#issuecomment-1478532913

We think this issue is largely understood and resolved. Closing -- please open other issues with new details.


Relationship fields returning null

First, thanks for creating this extension!
All the data is coming in great, the only issue I am having is relationship fields showing as null when I try to query them. I have a screenshot below of what I am seeing. Is there a specific naming convention this is looking for or is there something else I should be doing to get this? While doing this I also noticed the Internationalization field was returning null as well. Not sure if that is related or not.

Thanks!

Have you added the Legal Page as a collection type included in config and enabled access to find on the public role? FYI will add api access support asap also

Yes, I believe I have all those things set. I can query the Legal Page collection separately it's just when trying to get it from a relationship field.
Also, API docs would be amazing. Thank you for all your work!

Maybe you can log in the resolver for the relationship type:
https://github.com/relate-app/gatsby-source-strapi/blob/master/build-types.js#L41
You can fork the repo and replace the source using resolve: require('../gatsby-source-strapi') in your gatsby-config.js

Oh wait, have you enabled find on the Site collection type?

Is it same for all types with relationships, just a though cause gatsby does also have a type name Site, maybe it is colliding with your type somehow. Otherwise are you up for a short meet, maybe we can figure it out together?

Any luck with this?

@dustyndoyle I figured it out, the relationships return null if they are one-to-many, this is fixed in v4.2.0
So I close this for now

Awesome work @timbrandin 🚀

Awesome job! I had to back off this project for more pressing things but this looks amazing.


what is flag_product?

when i run the code ogbn-product/gat/gat_ns.py, i didn't find where the 'flag_product' function be defined. So i would like to know what the function mean and where i can find it.

Hello, the flag_products function is defined at https://github.com/devnkong/FLAG/blob/c19ae31ff80a41fe4fa0c011cc1e38b83511df45/ogb/attacks.py#L66-L105.
The function is to carry out the forward and backward computation for GNNs using FLAG.

Hello, the flag_products function is defined at
https://github.com/devnkong/FLAG/blob/c19ae31ff80a41fe4fa0c011cc1e38b83511df45/ogb/attacks.py#L66-L105
.
The function is to carry out the forward and backward computation for GNNs using FLAG.

thank you very much!  I have found it.


Benchmarking and various changes

This just pulls in some changes from my local copy to get the remote up to speed.


Question about training samples visualization.

Hi, I‘m training an unconditional GAN on my own dataset and confused about the training samples visualization. The visualization result iter_{}.png has 8 rows and 4 cols and I find that the generated images in first 4 rows are different with last 4 rows. Can you tell me the difference?
Thanks a lot.

Can you provide your visualization configs?

Hi， @LeoXing1996 , here is my configs.

training config:

"""Note that this config is just for testing."""

_base_ = [
    '../_base_/datasets/polyu_256.py',
    '../_base_/models/stylegan/stylegan2_base.py',
    '../_base_/default_runtime.py'
]

model = dict(generator=dict(out_size=256), discriminator=dict(in_size=256))

data = dict(
    samples_per_gpu=4, train=dict(dataset=dict(imgs_root='./data/PolyU/ROI/inROI/NIR')))

ema_half_life = 10.  # G_smoothing_kimg

custom_hooks = [
    dict(
        type='VisualizeUnconditionalSamples',
        output_dir='training_samples',
        interval=1000),
    dict(
        type='ExponentialMovingAverageHook',
        module_keys=('generator_ema', ),
        interval=1,
        interp_cfg=dict(momentum=0.5**(32. / (ema_half_life * 1000.))),
        priority='VERY_HIGH')
]

checkpoint_config = dict(interval=10000, by_epoch=False, max_keep_ckpts=30)
lr_config = None

log_config = dict(
    interval=100,
    hooks=[
        dict(type='TextLoggerHook'),
        # dict(type='TensorboardLoggerHook'),
    ])

total_iters = 100002  # need to modify

base dataset config:

# Style-based GANs do not perform any augmentation for the LSUN datasets
dataset_type = 'UnconditionalImageDataset'

train_pipeline = [
    dict(
        type='LoadImageFromFile',
        key='real_img',
        io_backend='disk',
        flag='grayscale',
    ),
    dict(type='Resize', keys=['real_img'], scale=(256, 256)),
    dict(
        type='Normalize',
        keys=['real_img'],
        mean=[127.5],
        std=[127.5],
        to_rgb=False),
    dict(type='ImageToTensor', keys=['real_img']),
    dict(type='Collect', keys=['real_img'], meta_keys=['real_img_path'])
]

# `samples_per_gpu` and `imgs_root` need to be set.
data = dict(
    samples_per_gpu=None,
    workers_per_gpu=4,
    train=dict(
        type='RepeatDataset',
        times=100,
        dataset=dict(
            type=dataset_type, imgs_root=None, pipeline=train_pipeline))
)

In VisualizeUnconditionalSamples.after_train_iter, images are sampled from BaseGAN.sample_from_noise, which sample images from the original model and the EMA model at the same time with the same input noise.
Therefore the first 4 rows are generated by the EMA model and the last 4 rows are generated by the original model.

I get it. Thanks a lot.


[SPARK-21960][Streaming] Spark Streaming Dynamic Allocation should respect spark.executor.instances

What changes were proposed in this pull request?
Removes check that spark.executor.instances is set to 0 when using Streaming DRA.
How was this patch tested?
Manual tests
My only concern with this PR is that spark.executor.instances (or the actual initial number of executors that the cluster manager gives Spark) can be outside of spark.streaming.dynamicAllocation.minExecutors to spark.streaming.dynamicAllocation.maxExecutors. I don't see a good way around that, because this code only runs after the SparkContext has been created.

Can one of the admins verify this patch?

Can one of the admins verify this patch?

This is actually a good design solution. Right now, it is not very clear (not even docs, examples, google searches) how to set an initial number of executors for a Streaming Application that haves StreamingDynamicAllocation enabled. I considerer this important, cause for some streams, like Kinesis... a minimun number of executors are needed, to match shards.
Any quick workaround for this?

Just commenting to be subscribed!

Also one point, the lack of tests leaves me a little concerned about this change, maybe look at ./streaming/src/test/scala/org/apache/spark/streaming/scheduler/ExecutorAllocationManagerSuite.scala and see if it would make sense to add something there?
This is a bit out of my usual range so I'm going to CC @koeninger to take a more detailed look.

Test build #90232 has finished for PR 19183 at commit 4c9769e.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90232/
Test PASSed.

I don't have personal experience with streaming dynamic allocation, but this patch makes sense to me and I don't see anything obviously wrong.
I agree with Holden regarding tests.

I can add a test on ExecutorAllocationManagerSuite.scala to assert spark.executor.instances correctly allocates that number of executors initially when uaing Streaming DA

@sansagara sounds reasonable to me

@sansagara go for it -- it'll be a few days until I'll have time to look at this again. I'll close my PR if/when you make a new one :)

@karth295 I think you could validate the config here, in the validateSettings method for SparkConfig. I guess this is where you have all the info required.

Can one of the admins verify this patch?

I thought this check also existed in the non-streaming code; the theory was that if you have set a fixed number of executors but enabled dynamic allocation, then that's probably a configuration error. But given that many people run on clusters with dynamic allocation defaulting to 'on' globally, that could be confusing or a little inconvenient to work around.
I don't think that check exists in the non-streaming code anymore though, and I see a test to that effect too. Therefore I think this is reasonable for consistency.
CC @tdas

@srowen ah, thanks for the background -- that does make sense.
@skonto I agree that the message should be logged when the SparkContext gets initialized, but I don't like the idea of putting it in validateSettings. Spark Core's ExecuterAllocationManager does its own validation on construction, and it is constructed while the SparkContext is getting initialized. I like that each module validates its own configs, rather than having validation separate from code.
Spark Streaming's ExecutorAllocationManager also validates on construction, but it gets created when a job starts, not when the StreamingContext is initialized. This is because it depends on two things that only are available when the job starts: ReceiverTracker and the micro-batch interval.
I see two options:

Initialize ExecutorAllocationManager excluding those two things, and pass them in start()
Make validateSettings a static method and call that when creating the streaming context (or SparkConf), then construct it later.

I'm inclined to implement option 1 to avoid parsing the properties in two places -- constructor and validateSettings.


Uplift third_party/tt-metal to 8ab8736293d4213be6dd9e7585a80c2acdcac855 2024-12-20

This PR uplifts the third_party/tt-metal to the 8ab8736293d4213be6dd9e7585a80c2acdcac855

this moves back to tt-metal main instead of branch.
passing tt-forge-fe except 2 xpass tests, link: https://github.com/tenstorrent/tt-forge-fe/actions/runs/12436381823/job/34724130155
going to merge.


Add documentation for -- delimiter in command line

Fixes https://github.com/gradle/gradle/issues/21158
Staged docs:

New bullet point under "limitations": https://builds.gradle.org/repository/download/Gradle_Release_Check_BuildDistributions/55882785:id/distributions/gradle-7.6-docs.zip!/gradle-7.6-20220909043056%2B0000/docs/userguide/custom_tasks.html#limitations
New "Specify options for a task section (and subsection)": https://builds.gradle.org/repository/download/Gradle_Release_Check_BuildDistributions/55882785:id/distributions/gradle-7.6-docs.zip!/gradle-7.6-20220909043056%2B0000/docs/userguide/command_line_interface.html#sec:disambiguate_task_options_from_built_in_options

Contributor Checklist

[X] Review Contribution Guidelines
[X] Make sure that all commits are signed off to indicate that you agree to the terms of Developer Certificate of Origin.
[X] Make sure all contributed code can be distributed under the terms of the Apache License 2.0, e.g. the code was written by yourself or the original code is licensed under a license compatible to Apache License 2.0.
[X] Check "Allow edit from maintainers" option in pull request so that additional changes can be pushed by Gradle team
[X] Provide integration tests (under <subproject>/src/integTest) to verify changes from a user perspective
[X] Provide unit tests (under <subproject>/src/test) to verify logic
[X] Update User Guide, DSL Reference, and Javadoc for public-facing changes
[X] Ensure that tests pass sanity check: ./gradlew sanityCheck
[X] Ensure that tests pass locally: ./gradlew <changed-subproject>:quickTest

Gradle Core Team Checklist

[ ] Verify design and implementation
[ ] Verify test coverage and CI build status
[ ] Verify documentation
[ ] Recognize contributor in release notes

@bot-gradle test BD

OK, I've already triggered the following builds for you:

BuildDistributions build

@bot-gradle test BD

OK, I've already triggered the following builds for you:

BuildDistributions build

@bot-gradle test BD

OK, I've already triggered the following builds for you:

BuildDistributions build

@bot-gradle test QFL

@bot-gradle test QFL

OK, I've already triggered the following builds for you:

QuickFeedbackLinux build

@bot-gradle test QFL

OK, I've already triggered the following builds for you:

QuickFeedbackLinux build

@bot-gradle test QFL

OK, I've already triggered the following builds for you:

QuickFeedbackLinux build

@bot-gradle test QFL

OK, I've already triggered the following builds for you:

QuickFeedbackLinux build

@bot-gradle test and merge

@bot-gradle test BD

Pre-tested commit build failed.

@bot-gradle test BD

OK, I've already triggered the following builds for you:

BuildDistributions build

@bot-gradle test and merge

Your PR is queued. See the queue page for details.

OK, I've already triggered a build for you.

Pre-tested commit build failed.

@bot-gradle test BD

OK, I've already triggered the following builds for you:

BuildDistributions build

@bot-gradle test BD

OK, I've already triggered the following builds for you:

BuildDistributions build

@bot-gradle test and merge

OK, I've already triggered a build for you.

Pre-tested commit build failed.

@bot-gradle test this

@bot-gradle test this

OK, I've already triggered the following builds for you:

PullRequestFeedback build

Your PR is queued. See the queue page for details.

OK, I've already triggered a build for you.


tls: Send close notify alert

The alert was buffered but not sent to the peer by the socket.
Closes #87
The PR fulfills these requirements

[ ] This change is a single feature in a single commit.
[ ] The commit message follows our commit guidelines.
[ ] The code formatting follows our coding guidelines.
[ ] Tests have been added for the changes.
[ ] Docs have been added / updated (docstrings and README).
[ ] A one-liner has been added to the ChangeLog (for new features).

More details in CONTRIBUTING.
I am submitting a …

[ ] Bug fix
[ ] New feature
[ ] Tests update (new test vector, etc.)
[ ] Docs update
[ ] Other (please complete)

Description

Other information

coverage: 88.805% (-0.2%) from 89.009% when pulling 9aed5eb3d867be1f2f119e6e68d4dc42b2258346 on issue-87 into 5cd5f728b12182e545dac5728c90b6c619e77fbe on master.


Add missing "get" and "set" property options

As describe in mongoose doc, we can define "get" and "set" options to be used as property getter and setter : https://mongoosejs.com/docs/api.html#schematype_SchemaType-set

Coverage remained the same at 91.228% when pulling c2aa6478d70292bd535459390d0df9b296b4807b on neilime:patch-1 into 287a209ef7d80ccef3ba2866d707d57165caf64e on szokodiakos:master.

Coverage remained the same at 91.228% when pulling c2aa6478d70292bd535459390d0df9b296b4807b on neilime:patch-1 into 287a209ef7d80ccef3ba2866d707d57165caf64e on szokodiakos:master.

@neilime Would you mind adding some tests?

It is still not available in downloaded package. can you please check?

@Ben305, sorry for the delay, I have added some tests

how is this pr different from the virtual properties?
because these are already mongoose get- & set- -ters
-> is this still a problem in the latest master?

this can be closed in favor of a123c490297197fda77e1adedc39205da5e821ba and d357d373ba515cdd37649aabd355c0b9c084de7b
@Ben305


Mac M1 support

On Mac M1 not compile.
Command:
nimble setup

Error:

You are attempting to compile it with arch == "arm64" (you can check it out by typing arch in the cmd line) but unreal 5.1 used x86 to build that PCH (and the dynlibs).
Check out the answer here https://developer.apple.com/forums/thread/718666 and make sure you invoked Nim from Intel


Suggestion for a new familliar

Hello I hope you are doing well, thank you very much for having created this great mod, I suggest you a family to add that would go I think very well with the theme of the mod, Azathoth, it could look like a small round mass with lots of tentacle, thank you very much for reading this have a nice day
To enable them it could be a boost of regeneration or resistance, it is you who choose sorry for my bad english im french

Moved to https://github.com/klikli-dev/occultism/issues/379


pull request for e2e test at 11/14/2016 21:15:12

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

optest/docs/updated_at.md
:white_check_mark:Succeeded

Details

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


No values obtained from fields

Hi! I'm getting started with redux-forms. I have an issue.
My issue: using basic example I was able to get redux-form displayed. But in my submit function I'm making a console.log to check if values which I typed are passed. And when I click submit button all seems to work fine - I see in Redux redux-form/TOUCH and redux-form/SET-SUBMIT-SUCCESS. The issue is that no values are logged, jut an empty object.
redux: 3.7.2
redux-form: 7.1.0
Code:
import React, {Component} from 'react';
import PropTypes from 'prop-types';
import {reduxForm} from 'redux-form';

class LoginForm extends Component {
    submit = (values) => {
        console.log('submit inside form');
        console.log(values);
    };

    render() {
        const {fields: {firstName, lastName, email}} = this.props;
        return (
            <form onSubmit={this.props.handleSubmit(this.submit)}>

                    First Name
                    <input type="text" placeholder="First Name" {...firstName}/>

                    Last Name
                    <input type="text" placeholder="Last Name" {...lastName}/>

                    Email
                    <input type="email" placeholder="Email" {...email}/>

                <button type="submit">Submit</button>
            </form>
        );
    }
}

LoginForm = reduxForm({
    form: 'login',
    fields: ['firstName', 'lastName', 'email']
})(LoginForm);
export default LoginForm;

Hi @stackninigi, you're using Redux Form just like you would on v5 and previous versions, but using v6+. Redux Form v6+ doesn't take a fields configuration.
Please see the examples of your version here:
https://redux-form.com/7.1.0/examples
In case you still need help with these, please ask on StackOverflow.