WIP: EnforceSemiSyncReplicaCount - actively enable/disable semi-sync replicas to match master's wait count

This is a WIP PR that attempts to address https://github.com/openark/orchestrator/issues/1360.
There are tons of open questions and things missing, but this is the idea.
Open questions (all answered in https://github.com/openark/orchestrator/pull/1373#pullrequestreview-693389059):

~EnableSemiSync also manages the master flag. do we really want that? should we not have an EnableSemiSyncReplica?~

~Should there be two modes: EnforceSemiSyncReplicas: exact|enough (exact would handle MasterWithTooManySemiSyncReplicas and LockedSemiSyncMaster, and enough would only handle LockedSemiSyncMaster)?~

~LockedSemiSyncMasterHypothesis waits ReasonableReplicationLagSeconds. I'd like there to be another variable to control the wait time. This seems like it's overloaded.~

TODO:

[x] semi-sync replica priority (or come up with better concept)
[ ] properly succeed failover; currently it kinda retries even though it succeeded, not sure why
[ ] perform sanity checks in checkAndRecover* functions BEFORE enabling/disabling replicas
[ ] add ReasonableLockedSemiSyncSeconds with fallback to ReasonableReplicationLagSeconds
[ ] handle master failover semi-sync enable/disable
[ ] possibly implement MasterWithIncorrectSemiSyncReplicas

You can choose to ignore the comment below if you want to. I think I've convinced myself that this is a good idea and I'll implement it. I can always revert if it doesn't pan out.

Aside from some battles I'm fighting in the code, there's one more open design question regarding LockedSemiSync:
In the current iteration of the PR, I made a config option EnforceSemiSyncReplicas with three possible values:

empty (current behavior; do not handle LockedSemiSyncMaster or MasterWithTooManySemiSyncReplicas)
exact to enable/disable semi sync on replicas to match priority order (for both  LockedSemiSyncMaster and MasterWithTooManySemiSyncReplicas)
enough to handle LockedSemiSyncMaster by enabling replicas in priority order, but never disabling it.

This requires basically an opt-in to handle LockedSemiSyncMaster.
Alternatively we could make a RecoverLockedSemiSyncMaster bool flag (like you did in https://github.com/openark/orchestrator/pull/1332 for RecoverNonWriteableMaster) and make EnforceSemiSyncReplicas a bool too. So we'd have:
RecoverLockedSemiSyncMaster bool
EnforceSemiSyncReplicas bool

Which would behave like this:

For LockedSemiSyncMaster: if EnforceSemiSyncReplicas { recoverExactSemiSyncReplicas() } else { recoverEnoughSemiSyncReplicas() }
For MasterWithTooManySemiSyncReplicas (and later for MasterWithIncorrectSemiSyncReplicas): if EnforceSemiSyncReplicas { recoverExactSemiSyncReplicas() } else { nothing() }

What do you think?

Sounds good and either choice. If you pick using EnforceSemiSyncReplicas, then I'd rename it as EnforceExactSemiSyncReplicas to be more precise.

I think I am making progress. I've not implemented both the "exact" mode (EnforceExactSemiSyncReplicas=true, see allowDisable = true in recoverSemiSyncReplicas), as well as the "enough" mode (RecoverSemiSyncReplicas=true, but EnforceExactSemiSyncReplicas=false, see allowDisable = false). Other than possibly making this more resilient and breaking things into smaller functions I am quite happy with the logic. It works nicely.
I've also implemented ReasonableStaleBinlogCoordinatesSeconds with a fallback to ReasonableReplicationLagSeconds as discussed.
Questions

A scenario this will not detect is if the counts match, but the priority order does not match. I wanted to implement MasterWithIncorrectSemiSyncReplicas, but for that we'd need to read more in GetReplicationAnalysis and I'm not sure if I can get all that I need in that giant query. What are your thoughts on this?
Could you check the TODOs in the PR and give me some feedback as to whether I'm doing the right thing. Specifically, I pretty consistently get repeated failure detection happening after a successful recovery until another round of detection happens. Not sure why that is.

MasterWithTooManySemiSyncReplicas
Jun 30 01:28:48 pheckel-devm-orch-1 orchestrator[591830]: 2021-06-30 01:28:48 INFO executeCheckAndRecoverFunction: proceeding with MasterWithTooManySemiSyncReplicas recovery on pheckel-devm-db-g0-1:3306; isRecoverable?: true; skipProcesses: false
Jun 30 01:28:48 pheckel-devm-orch-1 orchestrator[591830]: 2021-06-30 01:28:48 INFO topology_recovery: master semi-sync wait count is 1, currently we have 2 semi-sync replica(s)
Jun 30 01:28:48 pheckel-devm-orch-1 orchestrator[591830]: 2021-06-30 01:28:48 INFO topology_recovery: possible semi-sync replicas (in priority order):
Jun 30 01:28:48 pheckel-devm-orch-1 orchestrator[591830]: 2021-06-30 01:28:48 INFO topology_recovery: - pheckel-devm-db-g0-2:3306: semi-sync enabled = true, priority = 1, promotion rule = neutral, downtimed = false, last check = true, replicating = true
Jun 30 01:28:48 pheckel-devm-orch-1 orchestrator[591830]: 2021-06-30 01:28:48 INFO topology_recovery: - pheckel-devm-db-g0-3:3306: semi-sync enabled = true, priority = 1, promotion rule = must_not, downtimed = false, last check = true, replicating = true
Jun 30 01:28:48 pheckel-devm-orch-1 orchestrator[591830]: 2021-06-30 01:28:48 INFO topology_recovery: always-async replicas: (none)
Jun 30 01:28:48 pheckel-devm-orch-1 orchestrator[591830]: 2021-06-30 01:28:48 INFO topology_recovery: excluded replicas (downtimed/defunct): (none)
Jun 30 01:28:48 pheckel-devm-orch-1 orchestrator[591830]: 2021-06-30 01:28:48 INFO topology_recovery: taking actions:
Jun 30 01:28:48 pheckel-devm-orch-1 orchestrator[591830]: 2021-06-30 01:28:48 INFO topology_recovery: - pheckel-devm-db-g0-3:3306: setting rpl_semi_sync_slave_enabled=false, restarting slave_io thread
Jun 30 01:28:48 pheckel-devm-orch-1 orchestrator[591830]: 2021-06-30 01:28:48 INFO auditType:register-candidate instance:pheckel-devm-db-g0-3:3306 cluster:pheckel-devm-db-g0-1:3306 message:must_not
Jun 30 01:28:48 pheckel-devm-orch-1 orchestrator[591830]: 2021-06-30 01:28:48 INFO auditType:register-candidate instance:pheckel-devm-db-g0-3:3306 cluster:pheckel-devm-db-g0-1:3306 message:must_not
Jun 30 01:28:48 pheckel-devm-orch-1 orchestrator[591830]: 2021-06-30 01:28:48 INFO auditType:register-candidate instance:pheckel-devm-db-g0-1:3306 cluster:pheckel-devm-db-g0-1:3306 message:neutral
Jun 30 01:28:48 pheckel-devm-orch-1 orchestrator[591830]: 2021-06-30 01:28:48 INFO topology_recovery: recovery complete; success = true

LockedSemiSyncMaster
Both "exact" and "enough" mode look the same in my setup currently until I get another replica going (I think).
Jun 30 01:26:12 pheckel-devm-orch-1 orchestrator[591830]: 2021-06-30 01:26:12 INFO executeCheckAndRecoverFunction: proceeding with LockedSemiSyncMaster recovery on pheckel-devm-db-g0-1:3306; isRecoverable?: true; skipProcesses: false
Jun 30 01:26:12 pheckel-devm-orch-1 orchestrator[591830]: 2021-06-30 01:26:12 INFO topology_recovery: master semi-sync wait count is 1, currently we have 0 semi-sync replica(s)
Jun 30 01:26:12 pheckel-devm-orch-1 orchestrator[591830]: 2021-06-30 01:26:12 INFO topology_recovery: possible semi-sync replicas (in priority order):
Jun 30 01:26:12 pheckel-devm-orch-1 orchestrator[591830]: 2021-06-30 01:26:12 INFO topology_recovery: - pheckel-devm-db-g0-2:3306: semi-sync enabled = false, priority = 1, promotion rule = neutral, downtimed = false, last check = true, replicating = true
Jun 30 01:26:12 pheckel-devm-orch-1 orchestrator[591830]: 2021-06-30 01:26:12 INFO topology_recovery: - pheckel-devm-db-g0-3:3306: semi-sync enabled = false, priority = 1, promotion rule = must_not, downtimed = false, last check = true, replicating = true
Jun 30 01:26:12 pheckel-devm-orch-1 orchestrator[591830]: 2021-06-30 01:26:12 INFO topology_recovery: always-async replicas: (none)
Jun 30 01:26:12 pheckel-devm-orch-1 orchestrator[591830]: 2021-06-30 01:26:12 INFO topology_recovery: excluded replicas (downtimed/defunct): (none)
Jun 30 01:26:12 pheckel-devm-orch-1 orchestrator[591830]: 2021-06-30 01:26:12 INFO topology_recovery: taking actions:
Jun 30 01:26:12 pheckel-devm-orch-1 orchestrator[591830]: 2021-06-30 01:26:12 INFO topology_recovery: - pheckel-devm-db-g0-2:3306: setting rpl_semi_sync_slave_enabled=true, restarting slave_io thread
Jun 30 01:26:12 pheckel-devm-orch-1 orchestrator[591830]: 2021-06-30 01:26:12 INFO auditType:register-candidate instance:pheckel-devm-db-g0-2:3306 cluster:pheckel-devm-db-g0-1:3306 message:neutral
Jun 30 01:26:12 pheckel-devm-orch-1 orchestrator[591830]: 2021-06-30 01:26:12 INFO auditType:register-candidate instance:pheckel-devm-db-g0-2:3306 cluster:pheckel-devm-db-g0-1:3306 message:neutral
Jun 30 01:26:12 pheckel-devm-orch-1 orchestrator[591830]: 2021-06-30 01:26:12 INFO auditType:register-candidate instance:pheckel-devm-db-g0-1:3306 cluster:pheckel-devm-db-g0-1:3306 message:neutral
Jun 30 01:26:12 pheckel-devm-orch-1 orchestrator[591830]: 2021-06-30 01:26:12 INFO topology_recovery: recovery complete; success = true

"enough mode" for LockedSemiSyncMaster
Jun 30 01:08:52 pheckel-devm-orch-1 orchestrator[590627]: 2021-06-30 01:08:52 INFO executeCheckAndRecoverFunction: proceeding with LockedSemiSyncMaster recovery on pheckel-devm-db-g0-1:3306; isRecoverable?: true; skipProcesses: false
Jun 30 01:08:52 pheckel-devm-orch-1 orchestrator[590627]: 2021-06-30 01:08:52 INFO topology_recovery: master semi-sync wait count is 1, currently we have 0 semi-sync replica(s)
Jun 30 01:08:52 pheckel-devm-orch-1 orchestrator[590627]: 2021-06-30 01:08:52 INFO topology_recovery: possible semi-sync replicas (in priority order):
Jun 30 01:08:52 pheckel-devm-orch-1 orchestrator[590627]: 2021-06-30 01:08:52 INFO topology_recovery: - pheckel-devm-db-g0-2:3306: semi-sync enabled = false, priority = 1, promotion rule = neutral, downtimed = false, last check = true, replicating = true
Jun 30 01:08:52 pheckel-devm-orch-1 orchestrator[590627]: 2021-06-30 01:08:52 INFO topology_recovery: - pheckel-devm-db-g0-3:3306: semi-sync enabled = false, priority = 1, promotion rule = must_not, downtimed = false, last check = true, replicating = true
Jun 30 01:08:52 pheckel-devm-orch-1 orchestrator[590627]: 2021-06-30 01:08:52 INFO topology_recovery: always-async replicas: (none)
Jun 30 01:08:52 pheckel-devm-orch-1 orchestrator[590627]: 2021-06-30 01:08:52 INFO topology_recovery: excluded replicas (downtimed/defunct): (none)
Jun 30 01:08:52 pheckel-devm-orch-1 orchestrator[590627]: 2021-06-30 01:08:52 INFO topology_recovery: taking actions:
Jun 30 01:08:53 pheckel-devm-orch-1 orchestrator[590627]: 2021-06-30 01:08:53 INFO topology_recovery: - pheckel-devm-db-g0-2:3306: setting rpl_semi_sync_slave_enabled=true, restarting slave_io thread
Jun 30 01:08:53 pheckel-devm-orch-1 orchestrator[590627]: 2021-06-30 01:08:53 INFO auditType:register-candidate instance:pheckel-devm-db-g0-2:3306 cluster:pheckel-devm-db-g0-1:3306 message:neutral
Jun 30 01:08:53 pheckel-devm-orch-1 orchestrator[590627]: 2021-06-30 01:08:53 INFO auditType:register-candidate instance:pheckel-devm-db-g0-2:3306 cluster:pheckel-devm-db-g0-1:3306 message:neutral
Jun 30 01:08:53 pheckel-devm-orch-1 orchestrator[590627]: 2021-06-30 01:08:53 INFO auditType:register-candidate instance:pheckel-devm-db-g0-1:3306 cluster:pheckel-devm-db-g0-1:3306 message:neutral
Jun 30 01:08:53 pheckel-devm-orch-1 orchestrator[590627]: 2021-06-30 01:08:53 INFO topology_recovery: recovery complete; success = true

A scenario this will not detect is if the counts match, but the priority order does not match. I wanted to implement MasterWithIncorrectSemiSyncReplicas, but for that we'd need to read more in GetReplicationAnalysis and I'm not sure if I can get all that I need in that giant query. What are your thoughts on this?

You're right, this will be hard to detect. Expanding the giant query (BTW it's literally commonly referred to as "orchestrator's giant query") to include the extra information is impractical at this time. Let's not deal with this right now. Detection/recovery will only take place if the number of semi-sync replicas is different/smaller than expected, but not when the number is identical, however distribution may be.
Perhaps, in the future, way may hack around this by tricking orchetrator into a recovery.
I'll review further later today, and please be advised that I'll then be out till mid next week.
allowDisable is a confusing name. Can we make it more descriptive?

MasterWithIncorrectSemiSyncReplicas
I will hold off on this for now.

I'll review further later today, and please be advised that I'll then be out till mid next week.
No rush.

allowDisable is a confusing name. Can we make it more descriptive?
I named it exactReplicaTopology bool now; it's still a little wonky, but okay.

Today I worked on fixing the EnableSemiSyncMaster stuff and now we're correctly enabling the semi-sync flags during a failover. I only tested in a 3-node topology, but I'm hoping that I don't have to change anything to make it work for more replicas.
I left more TODO markers for you to look at. I think I managed to keep everything 100% backwards compatible if none of the new flags are enabled, so I think that's good :-)

I dabbled today with MasterWithIncorrectSemiSyncReplicas -- This works, but it requires changing the giant query and some weird Instance parsing; I don't know if I like it. https://github.com/binwiederhier/orchestrator/pull/1

Even though this is fun, I think I will hold off now and wait for your comments. I think you may want to try it out too, because the behavior is much easier to see when you see it in action.
A few notes/questions (some of which are also in the TODOs):

SemiSyncPriority is a little overloaded, because it currently also controls rpl_semi_sync_slave_enabled (now in MaybeEnableSemiSyncReplica) as well as rpl_semi_sync_master_enabled (now in MaybeDisableSemiSyncMaster and SetReadOnly). During a DeadMaster failover, the master flag is set as a side-effect of SetReadOnly, but in a LockedSemiSyncMaster or MasterWithTooManyReplicas case, it does not toggle the master flag. This is inconsistent. Should we also toggle the master flag when we fix the semi-sync replicas?
ClassifyAndPrioritizeReplicas does this "is excluded check" (~ replica.IsDowntimed || !replica.IsLastCheckValid || !replica.ReplicaRunning()) and classifies these replicas as "do not change anything for these". This leads to odd behavior for "downtimed" replicas that still have semi-sync enabled. Should we maybe just classify downtimed replicas as "always async" and simply disable semi-sync in that case?
Async fallback: During the entire development of this I have not thought about the case of "async fallback" (caused by rpl_semi_sync_master_timeout). Is there anything to consider here?
I was not able to resolve the topology recovery properly at the end of recoverSemiSyncReplicas. For some reason the MasterWithTooManySemiSyncReplicas is retriggered a couple of times and I'm not entirely sure why. Can you help with this?
Please note that the MasterWithIncorrectSemiSyncReplicas PR (https://github.com/binwiederhier/orchestrator/pull/1) is a proof-of-concept that works quite well. It can be hardened to make it less brittle (use a stripped down version of Instance, e.g. ReplicaInstance so it's safer), and it would make MasterWithTooManySemiSyncReplicas obsolete. It is a little awkward though.
If it helps, I can record some videos of failovers or we can do a screenshare. Let me know.

No rush with this. I appreciate all your work. This is a great piece of software.

Looking into

Awesome. Let me know if you have any questions. I have demoed this inside the company twice already and I like this branch more (https://github.com/binwiederhier/orchestrator/pull/1) :-D

SemiSyncPriority is a little overloaded

I agree, and it's confusing. Separate into two distinct variables? Not sure.

This leads to odd behavior for "downtimed" replicas that still have semi-sync enabled. Should we maybe just classify downtimed replicas as "always async" and simply disable semi-sync in that case?

Different users perceive Downtime to mean different things. The official meaning of Downtime is: orchestrator will not failover this instance if it's downtimed. But is a primary crashes, orchestrator may promote a replica that is downtimed (exception is graceful takeover).
So let's treat the downtimed replicas just like any other replica.

Async fallback: During the entire development of this I have not thought about the case of "async fallback" (caused by rpl_semi_sync_master_timeout). Is there anything to consider here?

I think not. That's for MySQL to handle. We will still attempt to set the right number of semi-sync replicas.

I was not able to resolve the topology recovery properly at the end of recoverSemiSyncReplicas. For some reason the MasterWithTooManySemiSyncReplicas is retriggered a couple of times and I'm not entirely sure why. Can you help with this?

I think that may be because you did not return an error (see inline comments). Please let me know if this isn't the solution.

Thank you very much for the comments. I appreciate it!
Before I can start addressing all your comments, since I have two different branches, do you have a preference on whether you'd like MasterWithTooManySemiSyncReplicas (this branch) or MasterWithIncorrectSemiSyncReplicas (https://github.com/binwiederhier/orchestrator/pull/1). I can probably address the comments in both branches, but it'd make it easier if we can make a decision sooner rather than later. My preference is MasterWithIncorrectSemiSyncReplicas because it handles all situations, but I understand if you'd want to go the other way.

I was only reviewing this PR, this branch, so let's go with that.

I've worked on it more but I'm still not done with all your comments; I'll work more on it tomorrow. In particular I'll try to make the "two loop action" thing a little more streamlined and i may drop the logFn stuff and just go with log.Debugf and log.Infof.

I think that may be because you did not return an error (see inline comments). Please let me know if this isn't the solution.

This did not seem to help.

So let's treat the downtimed replicas just like any other replica.

Done, though this is a little odd to me. If we bring in a new replica from a failover and it'll take an hour or two to catch up to replication, this replica being the "semi-sync enabled" one is definitely odd.

I agree, and it's confusing. Separate into two distinct variables? Not sure.

I've left it like it was for now; though I think to make the behavior consistent at least, we need to call MaybeDisableSemiSyncMaster during the recovery of MasterWithTooManySemiSyncReplicas and LockedSemiSyncMaster.

Done, though this is a little odd to me. If we bring in a new replica from a failover and it'll take an hour or two to catch up to replication, this replica being the "semi-sync enabled" one is definitely odd.

I'm open to suggestions, I feel like any suggestion would either be a bit odd or be a bit complex.

I think I'm "done" and quite happy with the result. Please let me know what else you'd want, especially tests and such.
I've uploaded a video showing three cases (with EnforceExactSemiSyncReplicas: true):

Video: https://phil.nopaste.net/jHcg1bB5rN?d=1&f=orch-demo.mp4&a=Pf7hFfWC89
Log output from the demo: https://phil.nopaste.net/BfTtK0RggZ?a=ywx2nLkl8C

Cases covered in video/logs:

Starting at 18:09:39: Failure of our only semi-sync replica (...-db-g0-2), leading to LockedSemiSyncMaster
Starting at 18:12:39: Re-adding of that replica and manually clicking "Start replication", leading to MaybeEnableSemiSyncReplica being called (which does NOT correct the situation) and a subsequent MasterWithTooManySemiSyncReplicas recovery (starting at 18:12:43).
Starting at 18:14:00: Correct semi-sync handling during a DeadMaster

Notable decisions I've made (debatable):

I've re-added the IsDowntimed logic to treat it as an async replica
I am now disabling the semi-sync master flag in the recovery to match the behavior in SetReadOnly

Unresolved issues:

We are still re-triggering recoveries right after they are resolved 4-5 times
I am unsure why the system tests are failing and how to run them locally

Friendly reminder. :grin:

Friendly reminder. grin

Yup! I hope to conclude tomorrow

We are still re-triggering recoveries right after they are resolved 4-5 times

Do you mean: immediately following a recovery there's another recovery running to fix the exact same problem, even though it's already fixed, and this repeats 4-5 times?
If this is the case, then something is missing: we have probably not marked the recovery as successful. Alternatively, anti-flapping doesn't work for this scenario. Also, is there a detection that still claims a semi-sync issue? I can understand if it takes 5-10 seconds for orchestrator to diagnose the resolved scenario. We should ensure that detections during that time are ignored.

Do you mean: immediately following a recovery there's another recovery running to fix the exact same problem, even though it's already fixed, and this repeats 4-5 times?

It's detecting the same situation again a few times after it is resolved. The recovery registration is guarding the repeated execution, so it's fine. It's just ugly:
2021-07-14 18:11:25 ERROR AttemptRecoveryRegistration: instance pheckel-devm-db-g0-1:3306 has recently been promoted (by failover of pheckel-devm-db-g0-1:3306) and is in active period. It will not be failed over. You may acknowledge the failure on

You can see the full output here: https://phil.nopaste.net/BfTtK0RggZ?a=ywx2nLkl8C -- Here's a longer excerpt:
2021-07-14 18:11:24 INFO executeCheckAndRecoverFunction: proceeding with LockedSemiSyncMaster recovery on pheckel-devm-db-g0-1:3306; isRecoverable?: true; skipProcesses: false
....
2021-07-14 18:11:24 INFO topology_recovery: semi-sync: recovery complete; success = true
...
2021-07-14 18:11:25 INFO topology_recovery: done running PostFailoverProcesses hooks
2021-07-14 18:11:25 INFO topology_recovery: Waiting for 0 postponed functions
2021-07-14 18:11:25 DEBUG PostponedFunctionsContainer: waiting on 0 postponed functions
2021-07-14 18:11:25 DEBUG PostponedFunctionsContainer: done waiting
2021-07-14 18:11:25 INFO topology_recovery: Executed 0 postponed functions

2021-07-14 18:11:25 INFO executeCheckAndRecoverFunction: proceeding with LockedSemiSyncMaster detection on pheckel-devm-db-g0-1:3306; isActionable?: true; skipProcesses: false
2021-07-14 18:11:25 INFO checkAndExecuteFailureDetectionProcesses: could not register LockedSemiSyncMaster detection on pheckel-devm-db-g0-1:3306
2021-07-14 18:11:25 INFO executeCheckAndRecoverFunction: proceeding with LockedSemiSyncMaster recovery on pheckel-devm-db-g0-1:3306; isRecoverable?: true; skipProcesses: false
2021-07-14 18:11:25 ERROR AttemptRecoveryRegistration: instance pheckel-devm-db-g0-1:3306 has recently been promoted (by failover of pheckel-devm-db-g0-1:3306) and is in active period. It will not be failed over. You may acknowledge the failure on pheckel-devm-db-g0-1:3306 (-c ack-instance-recoveries) to remove this blockage
2021-07-14 18:11:25 INFO topology_recovery: found an active or recent recovery on pheckel-devm-db-g0-1:3306. Will not issue another RecoverLockedSemiSyncMaster.
2021-07-14 18:11:25 ERROR 2021-07-14 18:11:25 ERROR AttemptRecoveryRegistration: instance pheckel-devm-db-g0-1:3306 has recently been promoted (by failover of pheckel-devm-db-g0-1:3306) and is in active period. It will not be failed over. You may acknowledge the failure on pheckel-devm-db-g0-1:3306 (-c ack-instance-recoveries) to remove this blockage

2021-07-14 18:11:26 INFO executeCheckAndRecoverFunction: proceeding with LockedSemiSyncMaster detection on pheckel-devm-db-g0-1:3306; isActionable?: true; skipProcesses: false
2021-07-14 18:11:26 INFO executeCheckAndRecoverFunction: proceeding with LockedSemiSyncMaster recovery on pheckel-devm-db-g0-1:3306; isRecoverable?: true; skipProcesses: false
2021-07-14 18:11:26 ERROR AttemptRecoveryRegistration: instance pheckel-devm-db-g0-1:3306 has recently been promoted (by failover of pheckel-devm-db-g0-1:3306) and is in active period. It will not be failed over. You may acknowledge the failure on pheckel-devm-db-g0-1:3306 (-c ack-instance-recoveries) to remove this blockage
2021-07-14 18:11:26 INFO topology_recovery: found an active or recent recovery on pheckel-devm-db-g0-1:3306. Will not issue another RecoverLockedSemiSyncMaster.
2021-07-14 18:11:26 ERROR 2021-07-14 18:11:26 ERROR AttemptRecoveryRegistration: instance pheckel-devm-db-g0-1:3306 has recently been promoted (by failover of pheckel-devm-db-g0-1:3306) and is in active period. It will not be failed over. You may acknowledge the failure on pheckel-devm-db-g0-1:3306 (-c ack-instance-recoveries) to remove this blockage

2021-07-14 18:11:27 INFO executeCheckAndRecoverFunction: proceeding with LockedSemiSyncMaster detection on pheckel-devm-db-g0-1:3306; isActionable?: true; skipProcesses: false
2021-07-14 18:11:27 INFO executeCheckAndRecoverFunction: proceeding with LockedSemiSyncMaster recovery on pheckel-devm-db-g0-1:3306; isRecoverable?: true; skipProcesses: false
2021-07-14 18:11:27 ERROR AttemptRecoveryRegistration: instance pheckel-devm-db-g0-1:3306 has recently been promoted (by failover of pheckel-devm-db-g0-1:3306) and is in active period. It will not be failed over. You may acknowledge the failure on pheckel-devm-db-g0-1:3306 (-c ack-instance-recoveries) to remove this blockage
2021-07-14 18:11:27 INFO topology_recovery: found an active or recent recovery on pheckel-devm-db-g0-1:3306. Will not issue another RecoverLockedSemiSyncMaster.
2021-07-14 18:11:27 ERROR 2021-07-14 18:11:27 ERROR AttemptRecoveryRegistration: instance pheckel-devm-db-g0-1:3306 has recently been promoted (by failover of pheckel-devm-db-g0-1:3306) and is in active period. It will not be failed over. You may acknowledge the failure on pheckel-devm-db-g0-1:3306 (-c ack-instance-recoveries) to remove this blockage
2021-07-14 18:11:33 DEBUG analysis: ClusterName: pheckel-devm-db-g0-1:3306, IsMaster: false, LastCheckValid: false, LastCheckPartialSuccess: false, CountReplicas: 0, CountValidReplicas: 0, CountValidReplicatingReplicas: 0, CountLaggingReplicas: 0, CountDelayedReplicas: 0, CountReplicasFailingToConnectToMaster: 0
2021-07-14 18:11:34 ERROR ReadTopologyInstance(pheckel-devm-db-g0-2:3306) show global status like 'Uptime': dial tcp <IP_ADDRESS>:3306: connect: connection refused
2021-07-14 18:11:34 WARNING DiscoverInstance(pheckel-devm-db-g0-2:3306) instance is nil in 0.007s (Backend: 0.006s, Instance: 0.001s), error=dial tcp <IP_ADDRESS>:3306: connect: connection refused
2021-07-14 18:11:39 INFO auditType:forget-unseen-differently-resolved instance::0 cluster: message:Forgotten instances: 0
2021-07-14 18:11:39 INFO auditType:forget-clustr-aliases instance::0 cluster: message:Forgotten aliases: 0

Note how executeCheckAndRecoverFunction is executed multiple times and (luckily) fails with ERROR AttemptRecoveryRegistration. It may just be that it takes this long for everything to fully recover though.

If this is the case, then something is missing: we have probably not marked the recovery as successful. Alternatively, anti-flapping doesn't work for this scenario. Also, is there a detection that still claims a semi-sync issue? I can understand if it takes 5-10 seconds for orchestrator to diagnose the resolved scenario. We should ensure that detections during that time are ignored.

I will investigate this a little more today and tomorrow. If my understanding is correct, the analysis if completely independent of the recovery, so during the recovery, we will trigger and attempt recovery multiple times anyway. So this may in fact be an un-avoidable situation, since it takes a while to restart the replica threads and all that.

I've added two commits to remedy the situation. It is working nicely now. We're now re-reading the master after taking actions until the desired state (based on the count only).
Looks like this in the logs:
17:56:52 DEBUG semi-sync: master = pheckel-devm-db-g0-1:3306, master semi-sync wait count = 1, master semi-sync replica count = 2
17:56:52 DEBUG semi-sync: possible semi-sync replicas (in priority order):
17:56:52 DEBUG semi-sync: - pheckel-devm-db-g0-2:3306: semi-sync enabled = true, priority = 1, promotion rule = neutral, last check = true, replicating = true
17:56:52 DEBUG semi-sync: - pheckel-devm-db-g0-3:3306: semi-sync enabled = true, priority = 1, promotion rule = must_not, last check = true, replicating = true
17:56:52 DEBUG semi-sync: always-async replicas: (none)
17:56:52 DEBUG semi-sync: excluded replicas (defunct): (none)
17:56:52 DEBUG semi-sync: suggested actions:
17:56:52 DEBUG semi-sync: - pheckel-devm-db-g0-3:3306: should set semi-sync enabled = false
17:56:52 INFO topology_recovery: semi-sync: taking actions:
17:56:52 INFO topology_recovery: semi-sync: - pheckel-devm-db-g0-3:3306: setting rpl_semi_sync_slave_enabled=false, restarting slave_io thread
17:56:52 INFO topology_recovery: semi-sync: waiting for desired state:
17:56:52 INFO topology_recovery: semi-sync: - current semi-sync replica count = 2, desired semi-sync replica count = 1
17:56:53 INFO topology_recovery: semi-sync: - current semi-sync replica count = 1, desired semi-sync replica count = 1
17:56:53 INFO topology_recovery: semi-sync: recovery complete; success = true

Note the section waiting for desired state. This waits up to WaitForSemiSyncRecoverySeconds, which defaults to 3 * InstancePollSeconds.
Please find the commits for this here:

Add "wait for desired state": https://github.com/openark/orchestrator/pull/1373/commits/664d3c116582c6335020175e5bd103f04fb28302
Add config option: https://github.com/openark/orchestrator/pull/1373/commits/5382128e7e5f9aa9f5c8d74e8556a3d1b6f537b5

I think I'm happy with this. Let me know if there is anything else you'd like changed.

I'm gonna look at the superfluous detection cycle. I'd like to rollback https://github.com/openark/orchestrator/commit/664d3c116582c6335020175e5bd103f04fb28302 as well as https://github.com/openark/orchestrator/commit/5382128e7e5f9aa9f5c8d74e8556a3d1b6f537b5. Blocking the recovery process while waiting for results does not align with how orchestrator runs things.

It works fine without the two commits, so I'm fine with removing them. Let me know if you'd like to revert them.
The reason why the detection re-triggers is that when we enable/disable replication on the replicas, the Rpl_master_semi_sync_clients variable on the master is only updated after the clients properly connect. So the whole thing typically takes 1-2 seconds before getting into a proper state. If your poll interval is 5 seconds or even 20 seconds, then you'll re-trigger the detection for that long.
In my tests, I never had to wait longer than 1-2 seconds in the detection loop.

Gotcha, thanks.
One last thing: this has to be documented?

Gotcha, thanks.

Does that mean you'd like me to remove the two commits?

One last thing: this has to be documented?

You mean in the Wiki? I can do that if you like.

Does that mean you'd like me to remove the two commits?

Yes please.

You mean in the Wiki?

I'm thinking under https://github.com/openark/orchestrator/blob/master/docs/configuration-discovery-classifying.md and under https://github.com/openark/orchestrator/blob/master/docs/configuration-recovery.md. WDYT?

Sounds good. I'll do it tomorrow.

I added docs. Let me know what you think. They are best viewed on Github:

Semi-sync topology
LockedSemiSyncMaster
MasterWithTooManyReplicas

Two side notes:

I read your blog post regarding your hiatus. Will you be cutting one last release that includes these changes? (Also: It's super sad that you're leaving Orchestrator behind, but totally understandable!)
I will be out of office for 3 weeks starting Wednesday; I'll probably still check in, but I'll mostly be away.

Awesome job on the documentation, much appreciated.
Yes, I'll create one more release with these changes! Won't leave them stranded.
As I mentioned in my post I may be back after some break. I do love and enjoy this product, I'm just unable to keep up.

👍


Error: Driver(UnexpectedPacket { payload: [0, 0, 0, 2, 0, 0, 0] })

I am running this example wrong.
https://docs.rs/mysql_async/0.21.0-alpha.2/mysql_async/
use mysql_async::prelude::*;

#[derive(Debug, PartialEq, Eq, Clone)]
struct Payment {
    customer_id: i32,
    amount: i32,
    account_name: Option<String>,
}

#[tokio::main]
async fn main() -> Result<(), mysql_async::error::Error> {
    let payments = vec![
        Payment { customer_id: 1, amount: 2, account_name: None },
        Payment { customer_id: 3, amount: 4, account_name: Some("foo".into()) },
        Payment { customer_id: 5, amount: 6, account_name: None },
        Payment { customer_id: 7, amount: 8, account_name: None },
        Payment { customer_id: 9, amount: 10, account_name: Some("bar".into()) },
    ];
    let payments_clone = payments.clone();

    let database_url = "mysql://root@<IP_ADDRESS>:3306/test_rust";

    let pool = mysql_async::Pool::new(database_url);
    let conn = pool.get_conn().await?;

    // Create temporary table
    let conn = conn.drop_query(
        r"CREATE TEMPORARY TABLE payment (
            customer_id int not null,
            amount int not null,
            account_name text
        )"
    ).await?;

    // Save payments
    let params = payments_clone.into_iter().map(|payment| {
        params! {
            "customer_id" => payment.customer_id,
            "amount" => payment.amount,
            "account_name" => payment.account_name.clone(),
        }
    });

    let conn = conn.batch_exec(r"INSERT INTO payment (customer_id, amount, account_name)
                    VALUES (:customer_id, :amount, :account_name)", params).await?;

    // Load payments from database.
    let result = conn.prep_exec("SELECT customer_id, amount, account_name FROM payment", ()).await?;

    // Collect payments
    let (_ /* conn */, loaded_payments) = result.map_and_drop(|row| {
        let (customer_id, amount, account_name) = mysql_async::from_row(row);
        Payment {
            customer_id: customer_id,
            amount: amount,
            account_name: account_name,
        }
    }).await?;

    // The destructor of a connection will return it to the pool,
    // but pool should be disconnected explicitly because it's
    // an asynchronous procedure.
    pool.disconnect().await?;

    assert_eq!(loaded_payments, payments);

    // the async fn returns Result, so
    Ok(())
}

mysql server 8.0.18
macos 10.15

Hi! Thanks for report.
Should be fixed in 9cd2a6d and will be published in next release. Please set non-empty password until then.

OK, thanks.


Removes unnecessary mutex on Table/Memory Instance

This was somehow introduced in the past, but the
reality is that the mutex only used during Grow doesn't
solve any concurrent issue at all and still could result in
crash when the module was accessed incorrectly concurrently.
The proper synchronization mechanism must be provided by
threads proposal, and this mutex usage can be removed.

Yes, using this only on Grow makes no sense, everything else would need to take the read lock.
Still, anything that touches memory (tables?) concurrently without the threads proposal is asking for trouble.

@ncruces thank you for the look!


Automate upgrading envoy

There is some build+test+push awesomeness at https://github.com/datawire/envoy/tree/datawire/extauth-build-automation/DATAWIRE that would be awesome to have automated.
Let's do it :tm:

Closing since I think @LukeShu managed to do this...


Parse gcm server response and set device active = false if either NotRegistered or InvalidRegistration errors are returned.  All other errors will continue to throw NotificationError

Your logic is destructive and doesn't check for what kind of error happens. That means if you mistakenly send, for example, an invalid TTL you will be deleting all the IDs it was sent to.

I am checking for the error message.  Only NotRegistered or InvalidRegistration are affected, and the device is de-activated, not deleted.

No? You're only checking in the tests...

I don't understand the test is being run against gcm.py, that is where the checking logic resides.

I will add tests for other error codes to verify.

Added a test to raise GCMError


Check if ligand is in grid and autobox for score and local only

Improvements related to issue #31:

Check if the ligand(s) is(are) in the grid when --score_onlyand local_only requested.
Use argument --autobox in combination --score_only and --local_only for automatically define the grid box around the ligand(s). This does not work with the --batch mode (cannot know beforehand the dimensions of the box without reading all the ligand first).

Miscellaneous:

Moved gd, corner1 and corner2in cache and ad4cache objects, instead of having them floating around...

@jeeberhardt Nice!


Update setup readme

Added some detailed information in the readme to do the first local setup, added additional regtest.js sample for hsm1 use only and a logback file as example to do the logs configuration.

pipeline: run

pipeline: run


small improvements to boilerplate

dockerfile commands are reordered to build faster in general
launchplan exists
metadata is python object not docstring yaml

going to add docs for:
[x] test data
[] remote build
[] exec
[] start/stop


gdal_1_11: remove old and unused version

Motivation for this change
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[ ] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[ ] Fits CONTRIBUTING.md.

cc @MarcWeber

added alias in https://github.com/NixOS/nixpkgs/pull/117593


Investigate replacing binary licenses with license files

Investigate if there's a way to replace the process of encoding Pro licenses into the binary, with encrypted license files.
This would provide several benefits, for instance:

The software and its licenses will be decoupled.
A new license will not require a new SDK version.
Customers can use older versions with the same license file.
Customers don't have to update SDK version to update their license.

I will update the documentation to mention this, and reach out to some yearly Gold and Enterprise customers to validate that it works well, before it becomes the standard method.
The old binary licenses will still work.

This has been added to the v9 branch.


Contribute Solution for LeetCode October Day 29

Day 29:
Contribute a solution for this day's LeetCode problem.
Rules:

Comment on this issue if you're willing to contribute.
Each user can contribute a maximum of 1 solution in Java, C++, or Python.
Each user can contribute a maximum of 2 solutions in other languages like JavaScript, Go, Ruby, etc.
Please ensure that no one has already commented for the language you're planning to contribute in.

Good luck and happy coding! :tada: :rocket:

Kindly assign me this task, I would positively contribute @arya2004

Sorry @ssugam10 but you can contribute 3 solutions as per issue rules.

@arya2004 please assign me this issue . i will solve this diligently and quickly . this will help me in my professional devolopment. please consider this comment!! i have solved more than 100 problems on leetcode and have good command on it

@arya2004 csn you provide the problem link i could not find it or even problem name would do

@roshankraveendrababu it will be available on 29th Oct

@roshankraveendrababu it will be available on 29th Oct

ohhh thanks bruh

Hi! @arya2004, I would like to contribute in C++, I have more than 200 active days in Leetcode, please assign this issue to me.


Tutorial instructs user to create a Windows Server 2016 VM. Should update to 2022.

In the "Create a Windows Server virtual machine" section, step 2 is "From Get started, choose Windows Server 2016 Datacenter." followed by a screenshot highlighting "Windows Server 2016 Datacenter" in the Azure Marketplace.
The tutorial should point users to a Windows Server OS that isn't 8 years old and coming up on end of support.
Nothing in the page besides that line and the Azure Marketplace screenshot immediately below have anything to do with Server 2016 specifically.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 038b3b50-317c-c0c1-cbee-c69f3f828f84
Version Independent ID: 93810ce3-0817-c221-beb4-35de3f40d43e
Content: Join a Windows Server VM to a Microsoft Entra Domain Services managed domain - Microsoft Entra ID
Content Source: docs/identity/domain-services/join-windows-vm.md
Service: entra-id
Sub-service: domain-services
GitHub Login: @Justinha
Microsoft Alias: justinha

@scarson Thanks for raising this! Revised to simply say choose a version of Windows Server. #please-close


why does error occurs in MZTransition.h?

I inherit the LZHCustomeTransition : MZTransition. and set [MZFormSheetController registerTransitionClass:[LZHCustomeTransition class] forTransitionStyle:MZFormSheetTransitionStyleCustom];
but when I run the program it tells me that - (void)entryFormSheetControllerTransition:(UIViewController *)formSheetController completionHandler:(MZTransitionCompletionHandler)completionHandler;
the UIViewController expected a type.? How did that comming?
http://stackoverflow.com/questions/30365086/error-occurs-when-using-cocoapods

You are missing UIKit framework import. You have to @import UIKit;


The oid of the wapip192v1 curve in GMObjectIdentifiers.java is wrong

Hello
GMObjectIdentifiers
According to the definition 《WAPI Certificate Management Part 5: Certificate Format Example》 , the oid of the wapi curve should be <IP_ADDRESS><IP_ADDRESS>.1

Dear zzhouxq,
We greatly appreciate your support for our endeavours. Your inquiry and the provided reference document have not gone unnoticed.
We have observed that the OID <IP_ADDRESS>97.1.301.101 is acknowledged in certain OID databases [1-2]. However, it has not been documented in the official records of the Chinese Cryptography Standardization Technology Committee (CCSTC). Specifically, the standard titled "Information security technology—Cryptographic application identifier criterion specification" [3] does not encompass OIDs ending with 101. Given these considerations, we have decided to retain the OID value.
We extend our gratitude for furnishing the draft standard "WAPI certificate management—Part 5: Example of certificate format (draft version)" [4]. Within this standard, on pages 9 and 10, it is indicated that <IP_ADDRESS><IP_ADDRESS> is designated for the ECDSA-192 algorithm paired with SHA-256, and <IP_ADDRESS><IP_ADDRESS>.1 is allocated to the elliptic curve parameter field. While we have taken note of statements resembling yours in certain documentation [5], asserting that <IP_ADDRESS><IP_ADDRESS>.1 is meant for the WAPI elliptic curve and that <IP_ADDRESS><IP_ADDRESS> signifies the ECDSA signature algorithm with SHA-256, we feel compelled to align ourselves with the official standard's interpretation. Consequently, we will designate <IP_ADDRESS><IP_ADDRESS> for the ECDSA-192 algorithm with SHA-256, and allocate <IP_ADDRESS><IP_ADDRESS>.1 to the elliptic curve parameter field.
Once again, we appreciate your engagement and your valuable input. We are open to any further questions or discussions you may have.
Best regards,
用户zzhouxq 您好，
非常感谢您对我们工作的支持，我们非常感激您提交的问题以及提供的相关文件作为佐证。
我们注意到，在一些OID数据库[1-2]中包含<IP_ADDRESS>97.1.301.101，但在中国密码学标准化技术委员会（CCSTC）的官方标准《信息安全技术—密码应用标识准则规范》[3] 第21页并未记录以101结尾的OID。基于这两个事实，我们决定保留这个OID值。
感谢您提供标准《WAPI证书管理—第5部分：证书格式示例（草案版本）》[4]。该标准在第9页和第10页指出，<IP_ADDRESS><IP_ADDRESS>标识基于SHA-256的ECDSA-192算法，而<IP_ADDRESS><IP_ADDRESS>.1标识椭圆曲线参数字段。我们注意到一些文献[5]与您类似的使用，将<IP_ADDRESS><IP_ADDRESS>.1标识WAPI椭圆曲线算法，<IP_ADDRESS><IP_ADDRESS>标识基于SHA-256的ECDSA签名算法。然而，基于我们对官方标准的理解，我们将采用<IP_ADDRESS><IP_ADDRESS>作为带有SHA-256的ECDSA-192算法，并将<IP_ADDRESS><IP_ADDRESS>.1用于椭圆曲线参数字段。
再次感谢您的参与和宝贵的意见。我们欢迎您提出任何其他问题或进一步的讨论。
顺颂商祺
Reference:
[1] http://oid-info.com/cgi-bin/display?oid=<IP_ADDRESS>97.1.301.101&a=display
[2] https://github.com/IonicDev/ecc-explorer/blob/develop/curveData.json
[3] Information security technology—Cryptographic application identifier criterion specification, CCSTC, published on 1st December 2017, url: http://c.gb688.cn/bzgk/gb/showGb?type=online&hcno=252CF0F72A7BE339A56DEA7D774E8994.
[4] WAPI certificate management—Part 5: Example of certificate format (draft), China Broadband Wireless "IP" Standard Group, published on 20th December 2011, url: http://www.chinabwips.org.cn/doc/101.pdf
[5] http://blackberry.com/developers/docs/6.0.0api/net/rim/device/api/crypto/oid/OIDs.html


feat: Add language support for Go Templ.

#32 the new version was released v1.0.3

Works as expected, tested in VSCodium 1.91.0 (release 24190)

Thanks


type="px" conflicts with margin

Describe the bug
When using type="px" for auro headers, the margin attribute is overridden by :host([type="px") .header-400.
To Reproduce
Steps to reproduce the behavior:

Define type="px" on header attribute
Specify margin="top" (or "bottom") with any size
Inspect element styles and you can see the margin style is overridden.

Expected behavior
Margin attribute should appear with pixels as opposed to rem or being overridden.
Screenshots
Using type="px":

type="px" overriding margin:

Not using type="px":

Margin is configured correctly:

Desktop (please complete the following information):

Browser [Chrome, Edge, Firefox] **Did not check IE or Safari
Version [windows 10]

Smartphone (please complete the following information):
N/A
Additional context
Add any other context about the problem here.

Reviewed issue. In progress for resolution.

:tada: This issue has been resolved in version 1.2.1 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Do not perform any logic for completed BuildRuns

Fixes #338
I am adding a condition to stop the reconciliation trying to work on an already completed BuildRun for the reason outlined in the issue.

/lgtm

/approve


PDFCrowd:  Move to Python 3.+

Hi Jaroslav,
I think I have everything done to get PDFCrowd working with Python 3.  I have tested this with Python 3.4 and so far so good.  When you have time could you take a look and let me know if this works for you?
Thanks,
Derrick

Hi Derrick,
Thank you for your contribution! I created a new repository for the Python 3.x version and applied your changes there: https://github.com/pdfcrowd/pdfcrowd-python3
I also published the 3.x version on pypi so feel free to test it out:
pip install pdfcrowd3


In multi_gpu_model with cpu_relocation the weights of the template model do not change

When using multi_gpu_model with cpu_relocation the weights of the template model do not change when training the model, and are different to the weights of the parallel model, which do change. See below for an example.
This is in contradiction with the documentation, which states:

To save the multi-gpu model, use .save(fname) or .save_weights(fname) with the template model (the argument you passed to multi_gpu_model), rather than the model returned by multi_gpu_model.

But it is useless to save the template model if it does not evolve due to training, and if its weights are different to the parallel model.
See the following minimal example:
from keras import Model, Input
from keras.layers import Dense
from keras.utils import multi_gpu_model
import keras.backend as K
import numpy as np

BATCHSIZE = 8
NITER = 4

# dummy model
x = Input(shape=(4,))
layer = Dense(2, activation='relu')(x)
y = Dense(1)(layer)
model = Model(inputs=x, outputs=y)

try:
    parallel_model = multi_gpu_model(model, cpu_relocation=True)
    print("Training using multiple GPUs..")
except ValueError:
    parallel_model = model
    print("Training using single GPU or CPU..")

parallel_model.compile(optimizer='sgd', loss='mse')

original_weights = K.batch_get_value(model.weights)

# Dummy training
for i in range(NITER):
    x = np.random.randn(BATCHSIZE, 4)
    y = np.random.randn(BATCHSIZE)
    parallel_model.train_on_batch(x, y)

weights = K.batch_get_value(model.weights)
parallel_weights = K.batch_get_value(parallel_model.weights)

if all([np.all(w == ow) for w, ow in zip(weights, original_weights)]):
    print('Weights in the template model have not changed')
else:
    print('Weights in the template model have changed')

if all([np.all(w == pw) for w, pw in zip(weights, parallel_weights)]):
    print('Weights in the template and parallel model are equal')
else:
    print('Weights in the template and parallel model are different')

When executing on a single GPU or CPU, the result is:

Training using single GPU or CPU..
Weights in the template model have changed
Weights in the template and parallel model are equal

When executing on multiple GPUs, the result is:

Training using multiple GPUs..
Weights in the template model have not changed
Weights in the template and parallel model are different

[x] Check that you are up-to-date with the master branch of Keras. You can update with:
pip install git+git://github.com/keras-team/keras.git --upgrade --no-deps

[x] Check that your version of TensorFlow is up-to-date. The installation instructions can be found here.

[x] Provide a link to a GitHub Gist of a Python script that can reproduce your issue (or just copy the script here if it is short).

If I replace
model = Model(inputs=x, outputs=y)

try:
    parallel_model = multi_gpu_model(model, cpu_relocation=True)
    print("Training using multiple GPUs..")
except ValueError:
    parallel_model = model
    print("Training using single GPU or CPU..")

by
try:
    with tf.device('/cpu:0'):
        model = Model(inputs=x, outputs=y)
    parallel_model = multi_gpu_model(model)
    print("Training using multiple GPUs..")
except ValueError:
    model = Model(inputs=x, outputs=y)
    parallel_model = model
    print("Training using single GPU or CPU..")

then, when training with multiple GPUs, I get, as expected:

Training using multiple GPUs..
Weights in the template model have changed
Weights in the template and parallel model are equal

I faced same problem.
My co worker recommend me add one line.
os.environ['CUDA_VISIBLE_DEVICES'] = number
After use this code, I can get same result in different session.
Try this.

@nyjane I tested your additional line and it doesn't change anything. Actually I was already specifying the available gpus via this environment variable.

It's not clear to me if the solution was the wrapper for multi_gpu_model  as explained here https://github.com/keras-team/keras/issues/8123#issuecomment-354857044 or the one proposed here.

[UPDATE]
In my case the solution cited in https://github.com/keras-team/keras/issues/8123#issuecomment-354857044 worked ok. The weights get updated, the model is being saved on multi gpu configuration!


DEVELOPING THE TFRS WEBSITE FRONT-END (BCDX-MEM-004)

DEVELOPING THE TFRS WEBSITE FRONT-END (BCDX-MEM-004)
Code Repo: https://github.com/bcgov/tfrs
Wiki: https://github.com/bcgov/tfrs/wiki
OVERVIEW OF OPPORTUNITY

The Transportation Fuels Reporting System is being designed to streamline compliance reporting for transportation fuel suppliers in accordance with the Renewable & Low Carbon Fuel Requirements Regulation.
We want you to replace the previously developed Django screen templates (Views) with a REACT / REDUX based front end using the Balsamiq screens we have produced. The REACT front end will utilize the REST API that is currently being developed.

BACKEND DEVELOPMENT:

A Python/Django server was developed that implements a REST interface and database design for the specifications of the TFRS application.
The database is implement in POSTGRESQL
Users in the system are authenticated by Siteminder (via an independent process) however roles authorization is going to be developed in a future milestone.

MILESTONES
As a trading partner I need to be able to navigate through a series of self-service screens to:

view information regarding past, proposed and pending credit-related activity including a real-time credit balance and list of activities that may impact that balance.
enter and submit the information required to propose a credit transaction to a trading partner; accept a proposed trade from a trading partner and submit that transaction proposal to the Director for approval.
view the information entered by the user on Transaction Details screen before proposing it to a trading partner or submitting it to the Director for approval.
review before authorising the transaction and submitting it to the Director for approval or saving it for future.
post a credit transfer opportunity. The offer needs to have the same information that a credit transaction proposal would have, without the transfer partner. Offer and credit transfers are separate entities.
respond to a credit transfer opportunity with acceptance of the terms, or respond to the credit transfer opportunity with a proposal that has modified terms

The Low Carbon Fuels Branch staff need to be able to:

input transactions to award credits for a "Part 3 Agreement" or to add validated fuel supply credits or to retire credits used for offsets in annual compliance reporting.
sort and search the list of all transactions and be able to export/download the tabular data for analysis in excel.
add companies, contacts, users, roles and permissions through the Swagger API
recommend approval of a proposed low carbon fuel credit transfer
approve the credit transfers
retire credits from or adjust a company's credit balance

ACCEPTANCE CRITERIA
To be paid for this opportunity, you will:

Participate as a part of our team's sprint planning workshop from 1pm to 4pm on July 18th, 2017 at 3rd Floor - 1012 Douglas Street in Victoria, BC. The planning session can be rescheduled if the selected individual is not available at that time.
Spend a minimum of 2 hours per weekday (Tuesday - Thursday, excluding holidays, between the hours of 8:30 AM and 5:00 PM) working in person with us at our office (1012 Douglas Street in Victoria, BC) for the duration of your work. We want you to work side-by-side with our back-end developer to implement the desired functionality in our dev environment.
Develop the screens in REACT using the balsamiq wireframes with the following directions:
a. Implement a Client REST web service to consume the provided Swagger REST API Specifications,
b. Use the Gov 2.0 Bootstrap Skeleton for look and feel,
c. implement the provided Behavior Driven Development scenarios and system messaging,
d. create the NavUnit templates for automated functional testing,
e. create functions for validation of form data,
Use Trello to document the work completed, to demonstrate completion of the work in this opportunity, and include as appropriate references and links on each trello task to github commits
Conduct knowledge transfer with the Ministry Full Stack Developer and Front-End Developer.

HOW TO APPLY
Please send a proposal by 16:00 PST, 14 July 2017.
Within your email, indicate your acceptance of the Code With Us terms, you must attach a copy of the terms along with any information requested in the "Acceptance" section of the document.
If we are satisfied with the proposals we receive, we will assign this opportunity by 16:00 PST, 17 July 2017 with work proposed to start as soon as possible on/after 18 July 2017.
PROPOSAL EVALUATION CRITERIA
Please email your proposal to<EMAIL_ADDRESS>Please reference the issue name "Developing the TFRS website back-end (BCDX-MEM-004)" in your proposal.
SCORING CRITERIA
Please provide:

Confirmation that you can meet all the Acceptance Criteria by 17 August 2017
Your Approach for meeting each of the Acceptance Criteria. (60 points)
Your Experience working with relevant agile projects (40 points).

NOTE:
In your Experience, include a description of the work you have done recently that demonstrates your capability to deliver similar work and (preferred) a link to examples of your work (e.g. GitHub projects). If needed, explain how your experience is relevant.
In your Approach you should explain how you will work with us. For example, you may identify certain tools, techniques or methodologies that you would use and explain how you might use them.

INFORMATIONAL OPEN HOUSE
On 7 July 2017, we will host an information session from 12pm to 1pm, for interested developers to talk with our team about the work. If you'd like to attend, please email<EMAIL_ADDRESS>Please also feel free to ask questions via the comments section on the issue in GitHub.
QUESTIONS?
Please provide comments below

If selected, is it possible to borrow a Windows machine for on-site development? I have a macbook pro and a windows desktop PC, but no transportable windows machine. Thanks!

We have not yet completed the evaluation of submissions. We have been impacted the past few days with additional workload due to the government response to wildfire in BC.  You patience is appreciated.  I will post an updated timeline today.

The previously planned Sprint Planning meeting will be moved to Friday July 21st from 1pm to 4pm.

Evaluations have completed and the work has been awarded to @Kiesum
Thank you for all who submitted proposals. You can contact me to setup a time to meet if you would like a debrief on your proposal.


Add a Docker compose language

I heard from @bwateratmsft that adding a compose language would be useful for the docker extension and it makes sense to have this in VS Code core. Some of the reasons this could be useful:

Extensions can use dockercompose as the language for activation
Simplifies when clauses for users who what to scope things
Easier to show the proper icon for icon themes
Nice to use in files.associations

Further elaboration: https://github.com/microsoft/vscode/pull/118042#issuecomment-789788225

Everything's working!

🎉

Thank you for verifying!


Adds license and security vulnerabilities checks for Hadoop3 build

Hadoop 2 often causes red security scans on druid builds because of the dependencies it brings. We want to move away from Hadoop 2 and have to provide Hadoop 3 distribution available.
New stages are added in the pipeline to make the license and security-vulnerabilities checks that run with Hadoop2 also run with the Hadoop3 build.
P.S. Not updating Hadoop3 IT tests pipelines since Hadoop tests are not running on Travis currently.

@cryptoe Could you please review this now, analyze dependencies and analyze hadoop3 dependencies seem to be working as expected. Previously a new line crept into the mvn command.


flag

Hello,
is it possible to add flag --zmq-pub tcp://<IP_ADDRESS>:18083, or how to add it while updating container?
Thank you,
Ivo

@isdedina you should be able to just add that into the bitmonero.conf config file mounted at /config .   In unraid, this is in the /appdata/monero-nodeboard folder
zmq-pub=tcp://<IP_ADDRESS>:18083

Thank you, it works.
Ivo


Update illuminate/support

Added support for Laravel 6 and Laravel 7.

You drop support for Laravel 5.x with that patch.

That's true - not sure how to cater for both going forwards. Could you create a new major release? v2.0.0?

So sorry, with everything which has been going on the last month I completely lost track of this, shall take a look after work this evening

It is possible to specify multiple versions: https://getcomposer.org/doc/articles/versions.md#version-range

OK, I'll submit a new PR.

Gah sorry, I ended up working late again on Friday and completely forgot. I've set myself an alarm to look at the PR this evening


Tpwdcee 275

Coverage remained the same at 100.0% when pulling 609c4b0ead87556f8d4466592bc863c0ca1087b3 on TPWDCEE-275 into 1e3308ceb2882a0babac89776e51727377387bd9 on master.

Coverage remained the same at 100.0% when pulling 2e60e97e6bc8cc2e6330e898a05bb0aca6b44c8f on TPWDCEE-275 into 1e3308ceb2882a0babac89776e51727377387bd9 on master.


npm install doesn't work

Node version: 5.0.0
npm version: 3.3.6
When trying to install deps with the above node version, it gets stuck here:
> node install.js

Download already available at /var/folders/zk/0v_5l8190rq57jdt5pxccpxc0000gn/T/phantomjs/phantomjs-1.9.8-macosx.zip
Extracting zip contents
Removing /Users/gideon/Mozilla/brackets/node_modules/phantomjs/lib/phantom
Copying extracted folder /var/folders/zk/0v_5l8190rq57jdt5pxccpxc0000gn/T/phantomjs/phantomjs-1.9.8-macosx.zip-extract-1451943582893/phantomjs-1.9.8-macosx -> /Users/gideon/Mozilla/brackets/node_modules/phantomjs/lib/phantom

When I downgrade my node version to 0.12, it works.

In addition to getting this working well in newer node in general, we should just dump the phantomjs dep, since it's only for running the brackets tests, which we don't do.

This seems to also be an issue with node 4.2

Apparently I can't grunt build-browser anymore either -.-
Ok I'll remove the phantomjs dep


fix: create output directory if it does not exist

Fixes #89

:tada: This PR is included in version 1.0.1 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Support for PUT method in CrudResourceController update()

When used default update() action on resource, the put method have not received data because phalcon getPost() method return only data which are passed through POST method.
This merge request adds possibility to use PUT method properly.

@vitalijssilins Thanks!


Install/activation fails if virtualenvwrapper is installed globally

this happens:
Requirement already satisfied (use --upgrade to upgrade): virtualenv in /usr/local/lib/python2.7/dist-packages (from virtualenvwrapper==4.1.1)
Requirement already satisfied (use --upgrade to upgrade): virtualenv-clone in /usr/local/lib/python2.7/dist-packages (from virtualenvwrapper==4.1.1)
Requirement already satisfied (use --upgrade to upgrade): stevedore in /usr/local/lib/python2.7/dist-packages (from virtualenvwrapper==4.1.1)
Requirement already satisfied (use --upgrade to upgrade): distribute in /home/thomasf/.venvburrito/lib/python/distribute-0.6.49-py2.7.egg (from stevedore->virtualenvwrapper==4.1.1)
Which makes this not happen:
. $HOME/.venvburrito/bin/virtualenvwrapper.sh

I worked around this by just making a symlink to the virtualenvwrapper.sh
cd .virtualenvburrito/bin/
ln -s `which virtualenvwrapper.sh`


chore(project): readme update

Description

nice pics added

Solves #96
Steps completed:

According to our definition of done, I have completed the following steps:

[ ] User stories met
[ ] Storybook stories
[ ] Unit tests added
[ ] Linting passing
[ ] Unit tests passing
[ ] Docs updated (including config and env variables)
[ ] Translations added
[ ] UX tested
[ ] Browsers / platforms tested
[ ] Rebased & ready to merge without conflicts
[ ] Reviewed own code

Waiting for partners approvals


custom ingress classes wont show up in dropdown when created from the ingress

Setup
Rancher version: 2.7-head aa7bea9
Browser type & version: Chrome
Describe the bug
when we create a custom ingress class while creating an ingress , the ingress class won't show up in the UI from drop down of the ingresses
To Reproduce

Create a standard user - u1 and u2
Create an rke1 or rke2 cluster node driver - 1 etcd + cp and 1 worker.
As a cluster owner/project owner
created a cluster IP service from cluster explorer >> Service Discovery >> Services
Create an ingress from the Ingresses page and select the above service in the target Service
Created an ingress class from Ingress class page on the ingress page by typing a custom name

Result
Noticed that once the ingress class is created, it doesn't show up in the drop down if we edit the ingress and try to create a new one.
Notice in the screenshot current ingressclass is custom but its not listed in the kubectl:
kubectl doesn't return the created ingress class:
> k get ingressclass -A
NAME    CONTROLLER             PARAMETERS   AGE
nginx   k8s.io/ingress-nginx   <none>       5h57m

Expected Result
Custom created ingress shows up in the dropdown.
Screenshots

This is working as expected. Ingress classes require spec.controller to be defined so they can't be created on the fly by inputting a name. That field allows the user to manually input an ingress class name so that users who can't get/list ingress classes can still make ingresses that reference a class that they know exists - see comment here

Putting back on QA's plate to either close or then petition for a fix, though if that's the case, it won't be for 2.7.2; it'd be Q2 or later.

Thanks @mantis-toboggan-md . Closing since it is expected.


Add NPM page links to CONTRIBUTING.md

Mostly because I want a short list that I can click through after publishing just to make sure everything is legit.

Trivial change, so merging directly.


README update

You can install plantuml now as debian package.
I can confirm it works under Ubuntu 17.04 and IPlantUML doesn't need any additional configuration.
I also made minor typo fix.

Awesome. Thanks, @igor-mikushkin!


multiple_matching_tokens The cache contains multiple tokens satisfying the requirements. Call AcquireToken again providing more requirements such as authority or account.

Core Library
MSAL Node (@azure/msal-node)
Core Library Version
1.12.0
Wrapper Library
Not Applicable
Wrapper Library Version
None
Public or Confidential Client?
Confidential
Description
After adding 2 new MSAL flows for acquireTokenByRefreshToken and acquireTokenOnBehalfOf we started seeing some errors in existing workflows that had no issues (acquireTokenByClientCredential). The error we are receiving from MSAL is "multiple_matching_tokens". We only see this issue in acquireTokenByClientCredential for our singleton client.
We added authority to all of our client codes as suggested by the error message below but we are still receiving these errors.
We noticed 2 similar incidents that were supposed to fix this issue:
#4486
#4691
Error Message

"stack": "ClientAuthError: multiple_matching_tokens: The cache contains multiple tokens satisfying the requirements. Call AcquireToken again providing more requirements such as authority or account..\n    at ClientAuthError.AuthError [as constructor] (/app/node_modules/@azure/msal-node/node_modules/@azure/msal-common/dist/index.cjs.js:496:24)\n    at new ClientAuthError (/app/node_modules/@azure/msal-node/node_modules/@azure/msal-common/dist/index.cjs.js:790:28)\n    at Function.ClientAuthError.createMultipleMatchingTokensInCacheError (/app/node_modules/@azure/msal-node/node_modules/@azure/msal-common/dist/index.cjs.js:885:16)\n    at ClientCredentialClient.readAccessTokenFromCache (/app/node_modules/@azure/msal-node/node_modules/@azure/msal-common/dist/index.cjs.js:6036:35)\n    at ClientCredentialClient. (/app/node_modules/@azure/msal-node/node_modules/@azure/msal-common/dist/index.cjs.js:5996:50)\n    at step (/app/node_modules/@azure/msal-node/node_modules/@azure/msal-common/dist/index.cjs.js:79:23)\n    at Object.next (/app/node_modules/@azure/msal-node/node_modules/@azure/msal-common/dist/index.cjs.js:60:53)\n    at /app/node_modules/@azure/msal-node/node_modules/@azure/msal-common/dist/index.cjs.js:53:71\n    at new Promise ()\n    at __awaiter (/app/node_modules/@azure/msal-node/node_modules/@azure/msal-common/dist/index.cjs.js:49:12)",
"message": "multiple_matching_tokens: The cache contains multiple tokens satisfying the requirements. Call AcquireToken again providing more requirements such as authority or account..",
"errorCode": "multiple_matching_tokens",
"errorMessage": "The cache contains multiple tokens satisfying the requirements. Call AcquireToken again providing more requirements such as authority or account..",
"subError": "",
"name": "ClientAuthError"
}

Msal Logs
No response
MSAL Configuration
auth: {
                clientId: "9edfcdd9-0bc5-4bd4-b287-c3afc716aac7"
                authority: "https://login.microsoftonline.com/organizations",
                clientCertificate: {
                    thumbprint: pem.thumbprint,
                    privateKey: pem.privateKey,
                    x5c: pem.certificate,
                },
            },

Relevant Code Snippets
this._client.acquireTokenByClientCredential({
  "scopes": [
    "https://management.azure.com/.default"
  ],
  "authority": "https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47"
}),

Reproduction Steps
N/A
Expected Behavior
Expectation would be for acquireTokenBYClientCredential to return a valid access token matching the passed in requirements.
Identity Provider
Azure AD / MSA
Browsers Affected (Select all that apply)
None (Server)
Regression
No response
Source
Internal (Microsoft)

Does this always repro? Or do you have to have a few tokens in the cache already?
Can you try to specify the authority only at the client level, not at the AcquireToken (request) level?

Does this always repro? Or do you have to have a few tokens in the cache already?
Possible workaround: Can you try to specify the authority only at the client level, not at the AcquireToken (request) level?

@bgavrilMS we are seeing this error pretty frequently - this graph shows the number of occurences in our PPE environment:

We need to be able to pass the authority at the request level because the client's authority is set to "https://login.microsoftonline.comorganizations" but when we receive a call when need to get the access/id/refresh tokens for a specific tenant (e.g. "https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47").

@sameerag This seems similar to #4486, and there is comment indicating that particular issue wasn't completely solved: https://github.com/AzureAD/microsoft-authentication-library-for-js/issues/4486#issuecomment-1166832179

After turning off a feature flag that removes the flows for acquireTokenByRefreshToken & acquireTokenOnBehalfOf and performing a service restart, our acquireTokenByClientCredential method is working as expected again. Please let me know if you need anything else from our side. Thanks

@sameerag can you take a look at this and the similar issue that Jason has referenced above.

Just a hunch on this. The internal cache key under which we store access tokens is:
home_account_id-env-AccessToken-client_id-scoppes
However, if client_credentials flow, there is no home_account_id, so the key looks like:
-env-AccessToken-client_id-scopes
I wonder if there is a bug where acquireTokenByClientCredential matches tokens from OBO ?
@r-delgadillo - what are the scopes that you use for OBO / auth_flow and what is the scope for client_credentials?

@bgavrilMS  For OBO flows, we are currently only getting an access token for MS graph (https://graph.windows.net/.default).
For, acquireTokenByRefreshToken we are issuing the following:

["https://graph.microsoft.com/.default"]
["https://graph.windows.net/.default"]
["https://management.azure.com/.default"]

For, acquireTokenByClientCredential we are issuing the following tokens:

["https://management.azure.com/.default"]
["https://azure-devices-provisioning.net/.default"]
["https://iothubs.azure.net/.default"]

Yes @r-delgadillo, I think you can do that. You could:

have 1 application object for acquireTokenByRefreshToken  and 1 for acquireTokenByClientCredential

If explicitly serialize (persist) your cache, e.g. you store the cache data to a file or smth, then just make sure the 2 application objects point to different files. Let me know if this is the case and I can advise further.

This would be a good bug to pick up for @Robbie-Microsoft, maybe @sameerag can act as advisor on this one?

TTD test:

acquire token from ESTS client_credentials to ARM scope, store cache to a file
acquire token from ESTS refresh_token to ARM scope
repeat 1. It might repro the bug.

@bgavrilMS do you know who will be taking point on this and the potential timeline to get a fix out? Thank you

@Robbie-Microsoft will be. We can reach out soon regarding the timeline, need to re-assess some of the other issues which are in the pipeline. Please watch out this space.

Apologies @r-delgadillo. I have begun working on this today.

It occurred to me today fyi..
[Verbose]: [Wed, 21 Jun 2023 19:25:55 GMT]<EMAIL_ADDRESS>: Verbose - Updated an outdated IdToken cache key - []
[Error]: [ClientAuthError] Failed to find account: ClientAuthError: multiple_matching_tokens: The cache contains multiple tokens satisfying the requirements. Call AcquireToken again providing more requirements such as authority or account.. - []
[Error]: MSAL: getToken call failed: [object Object] - []
[Error]: Failed to get subscriptions for account Cheena Malhotra -<EMAIL_ADDRESS>(tenant '380a094b-14c3-4859-857c-7265410f68fa'). Error: multiple_matching_tokens occurred when acquiring token.
The cache contains multiple tokens satisfying the requirements. Call AcquireToken again providing more requirements such as authority or account.. - []

The only change in my execution was I ran application as different user (Windows local user account), and added account there..


pubsub: allow message to modify ack_deadline

The ruby lib has a very useful feature, that I'd like to see on the golang lib. A received message can modify their own ack_deadline: https://github.com/GoogleCloudPlatform/google-cloud-ruby/blob/master/google-cloud-pubsub/lib/google/cloud/pubsub/received_message.rb#L159
The go version only allow a call for Ack/Nack: https://github.com/GoogleCloudPlatform/google-cloud-go/blob/master/pubsub/message.go
Does it make sense? This is something I'm willing to send a PR if you agree.

Ah, right on. For that case I believe you could hold the message locally (all the while the library is extending its ack deadline) and then Nack it after the appropriate amount of time. If it's held less than 10 minutes, it should not expire.
Alternatively and less ideally, you could use the lower-level Pull API (an example here).
Would either of these work for you?

Don't you think that holding the message until the time, consumes space on the Receive flowController, and if one does it to multiple msgs there will be a good amount of locked messages, instead of letting the msgs to flow, with a Delay func the lock release could be done, clearing the flowController, like Ack/Nack.

It depends on your usecase. Is this a once-off kind of thing? Then this would probably work. If you intend on repeating this behavior with frequency, you're right, it would lock your flow controller. It sounds like it's the latter case, so I'll mark this as a feature request.

Modifying the ack deadline wouldn't be enough. You must either ack or nack to release the message's slot in the flow controller.
Instead, I'd recommend increasing the flow control settings so you can accommodate holding on to the extra messages.
We've had requests for this feature before, but PubSub doesn't offer it. See #691 for more information.

@jba  But adding a Delay func to Message will basically work like Nack, but instead of doing a sendMoAck with 0 it will have a different ack_deadline passed to the Delay func, which will also release the flowController.
https://github.com/GoogleCloudPlatform/google-cloud-go/blob/933c4eb7eaa5ce52e869c8448e857044bab33b59/pubsub/iterator.go#L308
https://github.com/GoogleCloudPlatform/google-cloud-go/blob/933c4eb7eaa5ce52e869c8448e857044bab33b59/pubsub/iterator.go#L358

Whatever ModAck you send will be overridden by the ModAck done by our keepalive. Unless Delay also removes the message from our keepalive list.

@jba Is this something that worth adding to the lib? Or the team recommendation is to use the lowlevel api? If you think it worth I can give a try and send a PR, if not we can close this.

I think it makes sense. I like the name "Forget" better than "Delay". In addition to releasing the flow controller and calling ModAck, it should also remove the message from the keepalive map.
Note that the naive implementation will be quite inefficient, generating an RPC for each message. (Normally we batch messages in RPCs to avoid this.) That's a serious concern, because we want and expect this client to be used at scale.
We could just not do the RPC call. The message would then expire whenever its current deadline was up, whenever that was (a function of time-to-ack). That would be much simpler, at the cost of having no control over the redelivery time.
Or, we could have a single configurable value for all Forget calls. Or maybe round the delay up to the nearest second?
Let's figure out the design before you start implementing. And by the way, we use gerrit, not github, for our PRs: see CONTRIBUTING.md.

@jba Considering using the lib at scale, and also giving the user some control over redelivery time, having a global Forget value seems best path here. Rounding to the nearest seconds brings more complexity to the already complex Iterator code. WDYT?

Maybe not global, but per-Subscription.

True, any other consideration? Gonna start to write something around it to gather more feedback.

We should still batch the calls. You can make the call the same time we do nacks (the nackTicker case in the sender method).

Apologies: we are not going to add this feature to the Go client library, despite the fact that Ruby has it. See https://github.com/GoogleCloudPlatform/google-cloud-go/issues/691#issuecomment-425482829 for the rationale.


我push的时候出现错误

Gogs: Key permission denied
Cannot push with deployment key: 7
fatal: Could not read from remote repository.
秘钥我已经添加了

The error is very obvious and said very clear when you add the key, you cannot push with deploy key.


PLV8

Hey @ChakshuGautam , can you please assign me to this ?

Please go ahead!!

can you please assign me to this? @ChakshuGautam

Hi @thesilentline , I have already started working on this, please check the other issues 😃

Referencing in master issue, not tracking separately.


Should FileReaderSync be exposed to service workers?

FileReaderSync is an sync IO API, should service workers be allowed to use it?

There is a non-normative note in the spec:

Any type of synchronous requests must not be initiated inside of a service worker.

I guess we could take away the footgun by making things like FileReaderSync and sync XHR [Exposed=DedicatedWorker,SharedWorker].

XMLHttpRequest/ProgressEvent was changed earlier today. See: https://xhr.spec.whatwg.org/.


⚠ SkipTheCommericals has degraded performance

In d60af79, SkipTheCommericals ($STC) experienced degraded performance:

HTTP code: 200
Response time: 6755 ms

Resolved: SkipTheCommericals performance has improved in a7da043.


nautobot-server runserver ignores -c option

Environment

Nautobot version (Docker tag too if applicable): 2.1.1
Python version: 3.10 on MacOS 13.6.2
Database platform, version: postgres 13
Middleware(s): None

Steps to Reproduce

Setup Nautobot with poetry
Migrate the database
Start Nautobot with the command nautobot-server -c development/nautobot_config.py runserver

Expected Behavior
Running Nautobot

Observed Behavior
Watching for file changes with StatReloader
Performing system checks...

System check identified no issues (0 silenced).
January 17, 2024 - 12:27:25
Django version 3.2.23, using settings 'nautobot_config'
Starting development server at http://<IP_ADDRESS>:8000/
Quit the server with CONTROL-C.
Exception in thread django-main-thread:
Traceback (most recent call last):
  File "/opt/homebrew/Caskroom/miniconda/base/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "/opt/homebrew/Caskroom/miniconda/base/lib/python3.10/threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/USER/Library/Caches/pypoetry/virtualenvs/nautobot-jobs-ilJBC97m-py3.10/lib/python3.10/site-packages/django/utils/autoreload.py", line 64, in wrapper
    fn(*args, **kwargs)
  File "/Users/USER/Library/Caches/pypoetry/virtualenvs/nautobot-jobs-ilJBC97m-py3.10/lib/python3.10/site-packages/django/core/management/commands/runserver.py", line 138, in inner_run
    handler = self.get_handler(*args, **options)
  File "/Users/USER/Library/Caches/pypoetry/virtualenvs/nautobot-jobs-ilJBC97m-py3.10/lib/python3.10/site-packages/django/contrib/staticfiles/management/commands/runserver.py", line 27, in get_handler
    handler = super().get_handler(*args, **options)
  File "/Users/USER/Library/Caches/pypoetry/virtualenvs/nautobot-jobs-ilJBC97m-py3.10/lib/python3.10/site-packages/django/core/management/commands/runserver.py", line 65, in get_handler
    return get_internal_wsgi_application()
  File "/Users/USER/Library/Caches/pypoetry/virtualenvs/nautobot-jobs-ilJBC97m-py3.10/lib/python3.10/site-packages/django/core/servers/basehttp.py", line 45, in get_internal_wsgi_application
    return import_string(app_path)
  File "/Users/USER/Library/Caches/pypoetry/virtualenvs/nautobot-jobs-ilJBC97m-py3.10/lib/python3.10/site-packages/django/utils/module_loading.py", line 17, in import_string
    module = import_module(module_path)
  File "/opt/homebrew/Caskroom/miniconda/base/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/Users/USER/Library/Caches/pypoetry/virtualenvs/nautobot-jobs-ilJBC97m-py3.10/lib/python3.10/site-packages/nautobot/core/wsgi.py", line 15, in <module>
    nautobot.setup()
  File "/Users/USER/Library/Caches/pypoetry/virtualenvs/nautobot-jobs-ilJBC97m-py3.10/lib/python3.10/site-packages/nautobot/__init__.py", line 32, in setup
    configure_app(
  File "/Users/USER/Library/Caches/pypoetry/virtualenvs/nautobot-jobs-ilJBC97m-py3.10/lib/python3.10/site-packages/nautobot/core/runner/runner.py", line 108, in configure_app
    raise ValueError(f"Configuration file does not exist at {config_path}")
ValueError: Configuration file does not exist at /Users/USER/.nautobot/nautobot_config.py

Confirmed. Appears to be specific to the runserver command, as for example nautobot-server --config=development/nautobot_config.py nbshell works just fine.
I suspect the issue is that the configure_app() call in nautobot/__init__.py doesn't know or specify the config_path parameter, unlike the equivalent call in nautobot/core/runner/runner.py which does handle the --config CLI argument. But will need more investigation.


Fix measurement readings and refactor API

This PR addresses several improvements and fixes:

Corrects an issue with reading measurements from the sensor, which previously provided incorrect readings due to endianness bugs in the original code.

Replaces f32 based temperature and humidity settings with scaled u16. This change is aimed at improving convenience and efficiency on small microcontrollers.

I have carefully tested these changes to ensure accurate sensor readings.

Thanks for your pull request. Your changes look very nice. Thanks for detecting the calculation bugs. Tested the calculations not very well.
In addition, could you add a idle() function to switch to idle mode to access all modes.

Added function to go to idle state.


MudNumericField - UI Culture Decimal Seperator only partly taken into account

Hi!
I am trying to use MudNumericField in my current locale (de-DE) to enter Double values.
In default settings the MudNumericField takes the "." as a decimal separator.
When I set the Culture for the MudNumericField to "de-DE" and enter a "." it gets removed automatically when focus is lost.
E.g. "0.5" becomes "5".
However, I cannot enter "," as the correct decimal separator because (by default) the control seems to block this input.
When setting "Step" to "0.1" the outcome is an empty textbox on stepping up or down even though the backing double property gets updated.
Please see this example:
https://try.mudblazor.com/snippet/QucbkpwQbUNNfoyS
Expected behaviour from my side would be:
When setting a culture the decimal seperator of that locale is accepted for entering the number.
When stepping in increments like "0.1" the correct value is shown in the MudNumericField.
Please let me know if you need more information on this.
Best regards,
Sven

Hello, you cannot control how the numbers are displayed in a numeric field, that's dependent on the browser, which displays the number according to the user's locale.

Hi,
understood.
However:
If I set the culture attribute it breaks the rendering. (see extended sample here:
https://try.mudblazor.com/snippet/GYwPapQxvWTllWVr

)
Thus from my understanding there should not be a Culture attribute available for MudNumericField if it breaks things.
Especially because it works as I would expect it to work in a textfield.
The API Doku for the MudNumericField states the availability of a Culture attribute.
Don't get me wrong. If the browser limits here: no problem I will use a textfield instead.
Just got confused by the "different" behaviour.
Best regards,
Sven

The Culture is derived from the base input class, it could be overridden and set to do nothing, explaining that on the docs.
But consider that if the user has its locale set to a culture that uses the comma, like most of the world, the comma will be the primary accepted method.
Unless your goal is to prevent Americans from using the dot, there is really no reason to do this.

My Intention is not to prevent anyone from using anything.
My usecase is that I am faced with users having English OS but German Keyboard Settings.
Forcing me to take care of some UI language/culture stuff myself. In addition this leads to the Numpad Decimal Seperator being "," which leads to the component not seeming to "work" for these users.
In addition, for me as a developer, it is unclear that the documented Culture setting has no effect and even worse when setting it appears to break the UI.
So from my point of view on first sight this looked like a bug.
If it is not: please close the issue.
I understand the reasons for why it is like this and will use textfield instead with custom overrides for up and down keys.

Sure, it's a bug and one workaround is what I described earlier.
As for the culture, I found a Stack Overflow answer that seem to hint the possibility to set the locale on the input, but I cannot make it work.
https://stackoverflow.com/a/32972402/5491086

At worse we could change the base input mode to text, as we implemented the keyboard override anyway...

when you set the input mode to text the drawback is that you will get an unsuitable keyboard on mobiles.

We'd need to overlay the inner field of the input with a click-through div where we render the number or something like that. Not sure if a hack like that is even possible or feasible, but it would give us 100% control over the display of the number.
Would possibly also be a solution for the problem of the immediate input which overwrites itself on server if rendering is not suppressed.

We'd need to overlay the inner field of the input with a click-through div where we render the number or something like that. Not sure if a hack like that is even possible or feasible, but it would give us 100% control over the display of the number.
Would possibly also be a solution for the problem of the immediate input which overwrites itself on server if rendering is not suppressed.
Edit, forget it, you'd have to simulate curser and selection also.

And you wouldn't ever get the focus on the input field.
I tried different approaches with type=text:

Ignore everything, on-screen keyboard will be complete and the user can type non-numeric characters, getting a "Not a number" error by default converter,  and giving that MudBlazor is currently not localized, a custom error could be needed if the application is localized.
ovverride SetCulture, calling base class. If the culture is changed, or it's the first time, populate an HashSet<String> with the characters allowed by the culture (negative sign, dot separator, group separator and digits from 0-9).
OnKeyDown, if the character is not in the list set _keyDownPreventDefault just like arrow up/down down and return (to prevent registering the keystroke). This doesn't work with server-side Blazor.

Either way it's an hack.
I made a branch https://github.com/IIARROWS/MudBlazor/tree/feature/change-numberfield-to-text that changes the docs, and makes small changes on the component, especially overriding Culture property and adding documentation describing the issue.
I'm not ready for a PR because I'm not happy with the result, especially because I cannot find a way to reliably test it: BUnit seems to always render the component as if locale is "US" or invariant.

@IIARROWS does your PR fix this?

It might in some cases, but not all.
For example if the browser locale is US, it shouldn't be able to force the comma as decimal separator.
I would need to test it manually (as I explained, BUnit always renders the US locale), I'm not sure.

Some things can not be tested with BUnit, I know. It's not a problem to test such things only manually.

Yeah, but this is pretty delicate I would really love to do it.
Anyway it doesn't fix the problem, the browser has the final say:

I'm adding this visual test in the next PR I do.

I'm also encountering issues with MudNumericField when using a French culture, but I think it's worth mentioning that while the expected behavior should be the same (both French and German cultures use , as the decimal separator), the actual behavior is different:
When inputting a value with a . as the decimal separator (which happens basically all the time, especially if the user is using their keyboard's numeric keypad), an validation error message is shown: "Not a valid number", although the (default) validation pattern is matched.
See the following snippet and enter a decimal number with a . as a separator (e.g. 0.2) for reproduction:
https://try.mudblazor.com/snippet/wkGmlalhdFExMVRq

According to your use case, it looks like your users have a mismatch of locale and keyboard layout.
The numpad should output a comma.
I've re-read the thread as it's quite old,
TL;DR: there is no definitive solution to this, as it relies on the browser implementation.
It can mitigated if the user locale uses comma as decimal separation, but it's not perfect for US users if the application forces an FR culture (like a user setting).
@henon should we revise the old branch?
https://github.com/IIARROWS/MudBlazor/tree/feature/change-numberfield-to-text

This specific problem might be solvable with a special hand-crafted converter.

The standard French keyboard layout outputs a dot for the numpad . key (standard keycaps also show a dot). It looks like the German layout has a comma instead.
Anyways, a user can use any locale with any keyboard layout, leading to unexpected and unpredictable behavior, as shown by the different observed behaviors when using the MudNumericField with English, French or German locale.
Yes, a hand-made converter can work around the issue, so would a custom component built around the <input type="number"/> HTML element (which handles seamlessly any kind of separator out of the box), but meanwhile, it looks like MudBlazor is unfit for use in a context that might require some internationalization support, at least where numeric fields are required.

Instead of a converter we could just intercept both dot and comma and make it the decimal separator. The component already listens to keyboard events for up/down arrows.

but then some guy will come and complain that we misunderstand thousands separators

Well, we could check the locale... if comma is the thousand separator, don't use it ad decimal.

@Gaazoh To fall back on invariant culture in the default converter might be a good idea and I guess wouldn't be a breaking change, so we can do this. Would you PR this along with a test to fortify the fallback behavior?
@IIARROWS Do you remember why we went away from <input type="number"\> to input type text?

I have no idea... this is the PR:
https://github.com/MudBlazor/MudBlazor/pull/2136

Oh yes I remember now, it was about the keyboard on IPhones :(
https://github.com/MudBlazor/MudBlazor/pull/2136#issuecomment-882078499
Maybe it would be good to just make the input type configurable. All we have to do is keep the default to type "text" in order to avoid breaking changes


Compilation Error in rusoto_core 0.36.0

Compiling worker-fn v0.1.0 (/Users/outcadstgeek/.pp/oss_workspace/kamworks/worker-fn)
Compiling rusoto_core v0.36.0
error[E0599]: no method named http1_read_buf_exact_size found for type hyper::client::Builder in the current scope
--> /Users/outcastgeek/.cargo/registry/src/github.com-1ecc6299db9ec823/rusoto_core-0.36.0/src/request.rs:360:47
|
360 |         config.read_buf_size.map(|sz| builder.http1_read_buf_exact_size(sz));
|                                               ^^^^^^^^^^^^^^^^^^^^^^^^^
error: aborting due to previous error

Adding hyper = "0.12.17" fixes. The issue seems to be with hyper = "0.12.18"

You are right, Cargo.lock was the culprit.

It works with hyper = "0.12.18"
Thank you!


Document how to use SSH port forwarding in a minikube remote setup.

Closes #91

Hi Kamesh, this is a write-up of the setup we discussed earlier. I'd be happy if you could review it. Thanks!


Acronym not expanded in Forward

In SD, ND iTC is spelled out, but the word “Device” isn’t included when defining the acronym

Fixed in editorial-SD-Fixes-v3_0.

Issues resolved with PR #209


Suggest:? Changing Size of champions ?

Hey i have idea to change size of characters in LoL
Like on practice tool when you give +100max hp to bot or dummy
Offset was mscaleskincoef.
Thanks

@ClockworkBN
I do not think to add something like this to the R3nzSkin. If you wish, you can add using this offset.
Offset: GameObj + 0x1820

@R3nzTheCodeGOD
Thanks for replay :) !
I see you use IDA but i can’t remember how to do it i will be grateful if you can show how to do it.
I know its your free time.. to guide ;)
Regards.


2021-06-01 equity dashboard chart data update

Equity dashboard stats updates in this PR may be reviewed on staging - here.
After reviewing, if all looks well, approve and merge this Pull Request.
If there are issues with the data:

Note concerns or issues here by commenting on this PR
Work with Triston directly to resolve data issues
Alert the COVID19 site team in Slack (in the Equity page channel)

Hi everyone - Here are notes from today's data review. @vargovargo, please do review as needed.
:heavy_check_mark: Everything seems normal, and I observe no major red flags!
Thank you!
1. Blue Box Updates:
:heavy_check_mark: All values appear reasonable and stable (Case rate for Pacific Islanders steadily and slowly increasing).
2. Death Rate/100K by R&E:
:heavy_check_mark: Identical values to last three weeks. All values appear reasonable (and stay at 1 or <11).
:heavy_check_mark: % of COVID-19 deaths statewide changed significantly (≥0.5%) for the following categories:
Black: 2.9% increase in % of deaths statewide
Latino: 6% decrease in % of deaths statewide (contrast to last week’s ~4% increase)
White: 3.5% increase in % of deaths statewide (contrast to last week’s ~1% decrease)
Asian American: ~1% decrease in % of deaths statewide (similar to last week’s ~2% decrease)
3. Case Rate/100K by R&E:
:heavy_check_mark: We continue to see consistent decreases all rates. Rates of decrease smaller than last week. Asian American continues to be the lowest ranked category.
:heavy_check_mark: Cases relative to percentage of population in CA did not change significantly (≤0.5% change) except for:
Latino: ~1% increase in % of cases statewide
Asian American: ~.8% decrease in % of cases statewide (similar to last week)
4. Testing Rate/100K by R&E:
:heavy_check_mark: We continue to see (noticeable) decreases in all rates (similar to last two weeks).
:heavy_check_mark: Testing relative to percentage of population in CA did not change significantly did not change significantly (≤0.5% change) except for (same categories we usually see):
Latino: ~1% increase in % of testing statewide
White: ~.6% decrease in % of testing statewide
5. Mortality rate/100K vs. Case rate/100K
:heavy_check_mark: Sanity Check — Mortality rate/100K < Case rate/100K for all categories (as expected)
6. Community case rate/100K by median annual household income bracket:
:heavy_check_mark: As expected, we observe decreases in income bracket cases/100K and statewide case rate, at larger rates of decline than last week — all values appear reasonable. Similar to the past several weeks, $0-40K and $40-60K cases/100K remain close numerically but not identical.
:heavy_check_mark: Like last three weeks, statewide case rate is lower than the $40-60K bracket value and closer to the $60-80K bracket value.
7. HE Metric:
:heavy_check_mark: Line successfully extended; similar to last two weeks, we see 0.2 decreases in both Statewide Positivity and HE Quartile Positivity — values are reasonable, and numerically HE Quartile Positivity is extremely close to Statewide Positivity, as has been the case for the past six weeks.
8. Data completeness (DC):
:heavy_check_mark: DC values predominantly show no significant changes except for the following:
R&E Deaths DC decreased by ~0.9%
SO Cases DC increased by ~2%
SO Deaths DC increased by ~2% (in contrast to last week’s ~1.5% decrease)
GI values remain relatively unchanged for past two weeks