Add a flag to check the number of records

Description
This change adds a flag to the healthcheck to count the number of records in every store in the BrokerDaemon. Since this is considered a dev feature, it is only accessible when using the --json flag as well.
This should allow more visibility into the Broker and help debug issues related to large data stores.
Todos

[x] Tests
[x] Documentation

 Need more visibility into the size of anything DB on the broker


Added missing consonant (j/J) to rövarspråket specification

I noticed this ultra-minor bug during the CTF the other week and couldn't resist making a PR :)

Fantastic, thank you!


PRIO - ENS Timeout? Outdated exposure checks: CWA uses the same diagnosis key package several times in ProvideDiagnosisKeys to ENS

Avoid duplicates

[x] Bug is not mentioned in the FAQ
[x] Bug is specific for Android only, for general issues / questions that apply to iOS and Android please raise them in the documentation repository
[x] Bug is not already reported in another issue

Technical details

Device name: LG-G4 (LGH-815)
Android version: 6.0
App version: 1.15.2 (but also other versions had been affected before)
ENS version:<PHONE_NUMBER>0 (but also other versions had been affected before)

Describe the bug
At some days, CWA is providing the same previously downloaded diagnosis key package to ENS, resulting in outdated exposure checks. This already happened to me last year, and is now occuring again.
My assumption is, that a timeout of ENS and an insufficient handling of the error in CWA is the reason, because I'm experiencing the problem only in times with high incidence rates, and at the moment the DK packages contain 500k+ diagnosis keys.
So, probably many people might be affected currently without knowing it (unless they don't check the logs in ENS UI carefully).
As you can see from my attached ENS logs (see below) this happened on 10 days since April 1st.
Also, on April 18th at around 9:00 am, CWA correctly indicated, that exposure matching failed (for more that 36 hours):

When tapping on "Erneut Starten", the spinning wheel of downloading was visible for <1s, then the green card appeared, but exposure matching didn't start actually (as you can see from the logs).
Steps to reproduce the issue

Have an old, slow phone, preferrably Android 6, as there is only one single exposure check per day currently. Wait for rising incidence rates.
Let CWA do its job for a couple of days.
Go to ENS UI via CWA > Settings > App-Information > tap on ENF version number > select "Überprüfungen auf mögliche Begegenungen" from the 3-dot-menu up right in the screen [ > enter your screen lock pin/password if you have one]
Check the checksums of the matched diagnosis keys packages. They will be the same often.

Expected behaviour

When exposure matching fails, the user should be informed via a notification or via an error reporting card in main screen (like https://github.com/corona-warn-app/cwa-app-android/pull/2481 for incompatibility issues and like it was proposed here in a very raw draft: https://github.com/corona-warn-app/cwa-app-android/pull/864#issuecomment-674185410)
When exposure matching fails, CWA should always download/use the very latest DK package and try again after some time. Is it possible to register a worker for this?

Possible Fix
Re-evaluate and fix error handling and reporting for timeouts.
Additional context
Here are the latest logs of exposure matchings from ENS UI. I tried to separate the "double checks" and marked them with an asterisk for easier finding them. (The logs have been compiled from two exports on different dates!)
20210401-20210518_assembled_all-exposure-checks.txt
(Edit: updated exposure check logs 2021-05-18)

Internal Tracking-ID: EXPOSUREAPP-6654

Again no exposure checks since yesterday:

Tapping "Erneut Starten" -> download wheel for one second -> green card shows up -> but nothing happened.
2 hours later, a new exposure check with a new DK package was registered in the ENS exposure check log, but I don't know, if this was really successful...
There is an increasing number of reports from users in Google Play Store reviews with similar problems.
Would be good, if someone could have a look into it! Thank you in advance!

@vaubaehn
Thanks! We have reported it to the development team as EXPOSUREAPP-6654. Let's see what they can do about this.

Corona-Warn-App Open Source Team

Hi @rugk, inviting you to here from https://github.com/corona-warn-app/cwa-app-android/issues/2941 :
Could you do me a favor and look up, whether in your case also CWA uses outdated diagnosis keys packages? See 'steps to reproduce...' in OP above. If you find same checksums at least twice, I'd say it's the same problem.

Some additional information:
I did some logcatting from Thursday night to Friday. From the logs it looks like that ENS is actually doing its job right! It takes over the downloaded diagnosis keys from CWA and matches them with the recorded RPI scans. After it's finished, it sends an intent to CWA, that CWA is also receiving successfully, because it then starts a worker for risk calculation.
The formerly 'ENS timeout' hypothesis can most likely be discarded, as the matching of diagnosis keys with RPIs actually takes 8 to 9 seconds (sic!) with a package of 500k+ diagnosis keys and around 1300 stored RPIs (all information available in logcat). Actually, most time is spent by ENS in verifying the signature of the downloaded diagnosis keys package (up to one minute). The fast matching seems to be possible by using a two-step approach: first the diagnosis keys are quickly scanned if they could match with RPIs in a process, that is called 'native pre-filter' in logcat. After pre filtering, it looks like a second process is matching pre filtered diagnosis keys with RPIs (JNI matcher).
So it looks like, there is something going wrong with CWA itself. Obviously after ENS notified CWA of having finished, in the following flow the successful completion of diagnosis key matching or the result of risk calculation is not persisted correctly, so that CWA tries again with the previously downloaded diagnosis key package (instead of downloading and providing a new one) at a later time.
Interestingly, this obviously only becomes a problem, when the size of the diagnosis keys package exceeds a certain value (around 450-470k in my case).
So there may be a timing problem or a race condition with the tasks/workers after ProvideDiagnosisKeys.

And some more additional Information: after updating from CWA 1.15.2 to CWA 2.0.3 and an accidently nearly simultanous update of the ENS to v1.8 (211213000), the difference in the timestamps of CWA risk card and the ENS check log was 20 minutes last night! Before it usually was 1-2 minutes. I try to gather new information from logcat, as soon I find time.

Could you do me a favor and look up, whether in your case also CWA uses outdated diagnosis keys packages? See 'steps to reproduce...' in OP above. If you find same checksums at least twice, I'd say it's the same problem.

As the apps now work correctly, I guess I cannot do that.
But thanks, I’ll keep these steps in mind, when I’ll see this later. :smile:

@rugk The question if, if the app indeed runs correctly now. For r1.15.2 it was not doing since around April 1st, but the failing error card came just 2 or 3 times. The ENS log holds entries of the last 16 days,  so it could really help if you looked up the checksums there to verify you have not been affected. Thank you in advance! :)

Short update:
New version 2.0 3 with refactored DiagnosisKeyRetrievalWorker didn't change anything for me: 2 exposure checks today, 1st 3:21am, 2nd 9:10pm - both with package that has hash macdc6RUhFnobBP1Zx3WchR6oJY6Q/5ajUu6uXniYms= .

I'm wondering whether this is related to your #2880 ticket (likely fixed by #2934) :thinking:.
The UNMETERED check is not working, the hourly package sync is being skipped, there will only be one day package per day....
If we provide the ENS with the previous keys again, do we get a calculation with the same checksum?
Within the DownloadDiagnosisKeysTask we have:
https://github.com/corona-warn-app/cwa-app-android/blob/de207de84a795dad78229913458509a3d5da8ae9/Corona-Warn-App/src/main/java/de/rki/coronawarnapp/diagnosiskeys/download/DownloadDiagnosisKeysTask.kt#L171-L185
I would expect keySyncResult.newKeys.isEmpty() to be true, but according to your logs (https://github.com/corona-warn-app/cwa-app-android/files/6343389/20210401-20_assembled_all-exposure-checks.txt), this should also return true, as you have duplicate checks checksums within 24h:
 val nextForcedDetectionAt = lastSuccessfulDetection?.startedAt?.plus(Duration.standardDays(1))
 val hasRecentDetection = nextForcedDetectionAt != null && now.isBefore(nextForcedDetectionAt)

Only if the last provideDiagnosisKeys didn't end up with a successful notification (broadcast) from the ENS? So this will perform better after in 2.1 due to #2934, but the timestamps / notifications from the ENS after providing diagnosis keys still seems suspicious...

Go to ENS UI via CWA > Settings > App-Information > tap on ENF version number > select "Überprüfungen auf mögliche Begegenungen"

Okay, gone there and checked some keys around the error time and the hashes are all different.
I could not compare those with the keys used by CCTG as CCTG does not have the same screen (with their built-in microg version), and I could only export the whole .db there, which would really be too much work.
So my guess is, no it is not related.
And, as said, I can only confirm so far that the issue is gone. I have never seen it again. If I should, I'll make a notice.

@rugk
Great, thanks for your help!

Okay, gone there and checked some keys around the error time and the hashes are all different.
[...] So my guess is, no it is not related.

I now also think so, that the "Risiko-Überprüfung fehlgeschlagen" card is not related to the outdated diagnosis keys that are provided to ENS, and that's a different issue.
The reason why I thought it's related, is, because I found this card after the diagnosis key package was older than 36 hours and CWA might somehow checked that it's outdated without downloading a new package (I still suppose the message on the card is wrong and not necessarily related to a failed download. Anyway, the text of this card is wrong, as the RPIs ("Zufalls-IDs") are not matched with "the server" but with the diagnosis keys).
Also, today my diagnosis keys are outdated for more than 36 hours, but the card is not shown.
So with your help, we could track down the bug a little bit better.

Short update:
The issue is persisting for me. See latest exposure check logs:
20210401-27_assembled_all-exposure-checks.txt
I updated to 2.0.3 on April 23rd, and the problem is still the same.
There was one interesting point:
On April 25th, I pulled CWA to foreground at 2:21am, and downloading DKs and matching worked just fine.
On April 26th, CWA was downloading/checking in background, and it looked like it worked.
But on April 27th (today), the DKs were again checked too early (0:28am instead of after 2:00am), and it was again obviously the same package like on 26th.
@d4rken

I'm wondering whether this is related to your #2880 ticket (likely fixed by #2934) 🤔.
The UNMETERED check is not working, the hourly package sync is being skipped, there will only be one day package per day....

If I'm not wrong, until (including) CWA 1.15.2, there were two workers for downloading and provision of DKs to ENS: the DiagnosisKeyRetrievalOneTimeWorker and the DiagnosisKeyRetrievalPeriodicWorker . Because of #2934 , the DiagnosisKeyRetrievalPeriodicWorker should not have done anything, should it? And the DiagnosisKeyRetrievalOneTimeWorker should have been responsible for falsely providing an outdated DK package to ENS.
Also, it really seems to be related to the package size of DKs, as the problem only occurs with bigger packages. In times with less keys per package, everything worked fine. Or is this some mysterious coincidence?

If we provide the ENS with the previous keys again, do we get a calculation with the same checksum?

I'm quite sure that this is the case.

Within the DownloadDiagnosisKeysTask we have:

Valid from CWA 2.0.3 on

https://github.com/corona-warn-app/cwa-app-android/blob/de207de84a795dad78229913458509a3d5da8ae9/Corona-Warn-App/src/main/java/de/rki/coronawarnapp/diagnosiskeys/download/DownloadDiagnosisKeysTask.kt#L171-L185
I would expect keySyncResult.newKeys.isEmpty() to be true, but according to your logs

now:  20210401-27_assembled_all-exposure-checks.txt

this should also return true, as you have duplicate checks checksums within 24h:
 val nextForcedDetectionAt = lastSuccessfulDetection?.startedAt?.plus(Duration.standardDays(1))
 val hasRecentDetection = nextForcedDetectionAt != null && now.isBefore(nextForcedDetectionAt)

Only if the last provideDiagnosisKeys didn't end up with a successful notification (broadcast) from the ENS? So this will perform better after in 2.1 due to #2934, but the timestamps / notifications from the ENS after providing diagnosis keys still seems suspicious...

I guess there may be unexpected inconsistencies in these vals.
For understanding the bug, it's probably more easy, if we only take the (old) DiagnosisKeyRetrievalOneTimeWorker in combination with the 'hourly bug' into account:
Usually in this set up, only once per day DKs should be downloaded and checked.
But the DiagnosisKeyRetrievalOneTimeWorker was called several times per day in my case, e.g., at 23'rd of April it was called 3 times (whereas the third time, it was already the new DiagnosisKeyRetrievalWorker). This would mean, that the DiagnosisKeyRetrievalOneTimeWorker didn't run successful, because it set a new nextForcedDetectionAt for the same day, but using still the same DK package.
I wonder, if it could be, that the worker was interrupted/stopped unexpectedly, and the persisted vals used in the worker became inconsistent in that case (and should have been reset probably in such a case).
Are now ALL workers started with submitBlocking ? (Didn't check that) Maybe that can already help?

Here's also a logcat from April 23rd, 08:06am - 08:09am (checking exposures again with same keys) with CWA's pid:10658 and uid:10200, related CWA workers, network requests and ENS responses (I tried not to disclose personal data ;). Interestingly, there was obviously another DNS request from CWA that led to nothing, after everything was finished. Wonder, if the workers are a little bit confused.
04-23 08:06:42.054 10658 10658 D WM-SystemJobService: onStartJob for c23cb48f-107f-4a45-a85d-1c8b208d3656
04-23 08:06:42.058 10658 10658 D WM-SystemJobService: onStartJob for 35076de1-16ab-40bc-a5f6-ef78661a71f2
04-23 08:06:42.059 10658 10658 D WM-SystemJobService: Job is already being executed by SystemJobService: c23cb48f-107f-4a45-a85d-1c8b208d3656
04-23 08:06:42.063 10658 10658 D WM-SystemJobService: Job is already being executed by SystemJobService: 35076de1-16ab-40bc-a5f6-ef78661a71f2
04-23 08:06:42.071 10658 10720 D WM-Processor: Processor: processing c23cb48f-107f-4a45-a85d-1c8b208d3656
04-23 08:06:42.072 10658 10720 D WM-Processor: Processor: processing 35076de1-16ab-40bc-a5f6-ef78661a71f2
04-23 08:06:42.113 10658 10658 D WM-WorkerWrapper: Starting work for de.rki.coronawarnapp.worker.DiagnosisKeyRetrievalPeriodicWorker
04-23 08:06:42.133 10658 10658 D WM-WorkerWrapper: Starting work for de.rki.coronawarnapp.deadman.DeadmanNotificationPeriodicWorker
04-23 08:06:42.174 10658 10697 D WM-PackageManagerHelper: androidx.work.impl.background.systemalarm.RescheduleReceiver enabled
04-23 08:06:42.191 10658 10697 D WM-SystemJobScheduler: Scheduling work ID 4e3da071-a1cb-4261-83ab-02077ca6a365 Job ID 49601
04-23 08:06:42.203 10658 10697 D WM-SystemJobScheduler: Scheduling work ID 4e3da071-a1cb-4261-83ab-02077ca6a365 Job ID 49602
04-23 08:06:42.216 10658 10697 D WM-Processor: Processor cancelling ece59a84-d99b-4e5f-a2ca-ef8228ca5003
04-23 08:06:42.216 10658 10697 D WM-Processor: WorkerWrapper could not be found for ece59a84-d99b-4e5f-a2ca-ef8228ca5003
04-23 08:06:42.232 10658 10697 D WM-GreedyScheduler: Cancelling work ID ece59a84-d99b-4e5f-a2ca-ef8228ca5003
04-23 08:06:42.249 10658 10697 D WM-PackageManagerHelper: androidx.work.impl.background.systemalarm.RescheduleReceiver enabled
04-23 08:06:42.258 10658 10697 D WM-SystemJobScheduler: Scheduling work ID b5c72ab9-8fa6-4e2d-94e1-58860de85ab0 Job ID 49603
04-23 08:06:42.264 10658 10697 D WM-SystemJobScheduler: Scheduling work ID b5c72ab9-8fa6-4e2d-94e1-58860de85ab0 Job ID 49604
04-23 08:06:42.273 10658 10697 D WM-WorkerWrapper: de.rki.coronawarnapp.deadman.DeadmanNotificationPeriodicWorker returned a Success {mOutputData=Data {}} result.
04-23 08:06:42.274 10658 10697 I WM-WorkerWrapper: Worker result SUCCESS for Work [ id=35076de1-16ab-40bc-a5f6-ef78661a71f2, tags={ de.rki.coronawarnapp.deadman.DeadmanNotificationPeriodicWorker } ]
04-23 08:06:42.276 10658 10658 D WM-Processor: Processor 35076de1-16ab-40bc-a5f6-ef78661a71f2 executed; reschedule = false
04-23 08:06:42.276 10658 10658 D WM-SystemJobService: 35076de1-16ab-40bc-a5f6-ef78661a71f2 executed on JobScheduler
04-23 08:06:42.282 10658 10697 D WM-GreedyScheduler: Cancelling work ID 35076de1-16ab-40bc-a5f6-ef78661a71f2
04-23 08:06:42.292 10658 10697 D WM-SystemJobScheduler: Scheduling work ID 35076de1-16ab-40bc-a5f6-ef78661a71f2 Job ID 49561
04-23 08:06:42.298 10658 10697 D WM-SystemJobScheduler: Scheduling work ID 35076de1-16ab-40bc-a5f6-ef78661a71f2 Job ID 49605
04-23 08:06:42.305 10658 10697 D WM-WorkerWrapper: de.rki.coronawarnapp.worker.DiagnosisKeyRetrievalPeriodicWorker returned a Success {mOutputData=Data {}} result.
04-23 08:06:42.307 10658 10697 I WM-WorkerWrapper: Worker result SUCCESS for Work [ id=c23cb48f-107f-4a45-a85d-1c8b208d3656, tags={ de.rki.coronawarnapp.worker.DiagnosisKeyRetrievalPeriodicWorker, DIAGNOSIS_KEY_PERIODIC_WORKER } ]
04-23 08:06:42.309 10658 10658 D WM-Processor: Processor c23cb48f-107f-4a45-a85d-1c8b208d3656 executed; reschedule = false
04-23 08:06:42.309 10658 10658 D WM-SystemJobService: c23cb48f-107f-4a45-a85d-1c8b208d3656 executed on JobScheduler
04-23 08:06:42.317 10658 10697 D WM-GreedyScheduler: Cancelling work ID c23cb48f-107f-4a45-a85d-1c8b208d3656
04-23 08:06:42.326 10658 10697 D WM-SystemJobScheduler: Scheduling work ID c23cb48f-107f-4a45-a85d-1c8b208d3656 Job ID 49599
04-23 08:06:42.333 10658 10697 D WM-SystemJobScheduler: Scheduling work ID c23cb48f-107f-4a45-a85d-1c8b208d3656 Job ID 49606
04-23 08:06:42.342 10658 10697 D WM-Processor: Processor stopping background work ece59a84-d99b-4e5f-a2ca-ef8228ca5003
04-23 08:06:42.342 10658 10697 D WM-Processor: WorkerWrapper could not be found for ece59a84-d99b-4e5f-a2ca-ef8228ca5003
04-23 08:06:42.342 10658 10697 D WM-StopWorkRunnable: StopWorkRunnable for ece59a84-d99b-4e5f-a2ca-ef8228ca5003; Processor.stopWork = false
04-23 08:06:42.343 10658 10697 D WM-Processor: Processor stopping background work 35076de1-16ab-40bc-a5f6-ef78661a71f2
04-23 08:06:42.343 10658 10697 D WM-Processor: WorkerWrapper could not be found for 35076de1-16ab-40bc-a5f6-ef78661a71f2
04-23 08:06:42.343 10658 10697 D WM-StopWorkRunnable: StopWorkRunnable for 35076de1-16ab-40bc-a5f6-ef78661a71f2; Processor.stopWork = false
04-23 08:06:42.344 10658 10697 D WM-Processor: Processor stopping background work c23cb48f-107f-4a45-a85d-1c8b208d3656
04-23 08:06:42.344 10658 10697 D WM-Processor: WorkerWrapper could not be found for c23cb48f-107f-4a45-a85d-1c8b208d3656
04-23 08:06:42.344 10658 10697 D WM-StopWorkRunnable: StopWorkRunnable for c23cb48f-107f-4a45-a85d-1c8b208d3656; Processor.stopWork = false
----
Wifi lost: 04-23 08:07:31
LTE connected: 04-23 08:07:32
----
04-23 08:07:47.045 10658 10658 D WM-SystemJobService: onStartJob for 4e3da071-a1cb-4261-83ab-02077ca6a365
04-23 08:07:47.046 10658 10658 D WM-SystemJobService: Job is already being executed by SystemJobService: 4e3da071-a1cb-4261-83ab-02077ca6a365
04-23 08:07:47.052 10658 10721 D WM-Processor: Processor: processing 4e3da071-a1cb-4261-83ab-02077ca6a365
04-23 08:07:47.087 10658 10658 D WM-WorkerWrapper: Starting work for de.rki.coronawarnapp.worker.DiagnosisKeyRetrievalOneTimeWorker

=> Looks like this could be CWA trying to download DKs uid:10200/CWA's pid:10658
04-23 08:07:47.206   437  1310 D BandwidthController: [LG DATA] No such appUid: 10200 /* This happens with all uids of all apps
04-23 08:07:47.206   437  1310 D DnsProxyListener: App 10200 tries DNS query. Accept family:0 protocol:0
04-23 08:07:51.415  1536  1585 D ConnectivityService: getActiveNetwork nai=NetworkAgentInfo{ ni{[type: MOBILE[LTE], state: CONNECTED/CONNECTED, reason: connected, extra: ####### blanked for privacy reasons ####### }
04-23 08:07:51.424  1536  3372 D ConnectivityService: listenForNetwork for Listen from uid/pid:10200/10658 for NetworkRequest [ id=128, legacyType=-1, [ Capabilities: INTERNET&NOT_RESTRICTED&TRUSTED&NOT_VPN] ]
04-23 08:07:51.425  1536  3015 D ConnectivityService: rematching NetworkAgentInfo [MOBILE (LTE) - 123]
04-23 08:07:51.426  1536  3015 D ConnectivityService: handleRegisterNetworkRequest checking NetworkAgentInfo [MOBILE (LTE) - 123]
04-23 08:07:51.426  1536  3015 D ConnectivityService: apparently satisfied.  currentScore=50
04-23 08:07:51.426  1536  3015 E ConnectivityService: BUG: NetworkAgentInfo [MOBILE (LTE) - 123] already has NetworkRequest [ id=128, legacyType=-1, [ Capabilities: INTERNET&NOT_RESTRICTED&TRUSTED&NOT_VPN] ]
04-23 08:07:51.427  1536  1715 I am_wtf  : [0,1536,system_server,-1,ConnectivityService,BUG: NetworkAgentInfo [MOBILE (LTE) - 123] already has NetworkRequest [ id=128, legacyType=-1, [ Capabilities: INTERNET&NOT_RESTRICTED&TRUSTED&NOT_VPN] ]]
04-23 08:07:51.429  1536  3015 D ConnectivityService: releasing NetworkRequest NetworkRequest [ id=128, legacyType=-1, [ Capabilities: INTERNET&NOT_RESTRICTED&TRUSTED&NOT_VPN] ]
04-23 08:07:51.429 10658 26814 D ConnectivityManager.CallbackHandler: CM callback handler got msg 524290
04-23 08:07:51.446 10658 26814 D ConnectivityManager.CallbackHandler: CM callback handler got msg 524290
04-23 08:07:51.449  1536  4190 D ConnectivityService: getActiveNetwork nai=NetworkAgentInfo{ ni{[type: MOBILE[LTE], state: CONNECTED/CONNECTED, reason: connected, extra: ###### }
04-23 08:07:51.451 10658 26814 D ConnectivityManager.CallbackHandler: CM callback handler got msg 524296
04-23 08:07:51.451  1536  4189 D ConnectivityService: getActiveNetwork nai=NetworkAgentInfo{ ni{[type: MOBILE[LTE], state: CONNECTED/CONNECTED, reason: connected, extra: ###### }
04-23 08:07:52.420  3606 26046 I ExposureNotification: Found flag-defined keys de.rki.coronawarnapp:262-v1,MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEc7DEstcUIRcyk35OYDJ95/hTg3UVhsaDXKT0zK7NhHPXoyzipEnOp3GyNXDVpaPi3cAfQmxeuFMZAIX2+6A5Xg== for package de.rki.coronawarnapp [CONTEXT service_id=236 ]
04-23 08:08:13.722 10658 10658 D WM-DelayedWorkTracker: Scheduling work 4e3da071-a1cb-4261-83ab-02077ca6a365
04-23 08:08:13.722 10658 10658 D WM-GreedyScheduler: Starting tracking for [4e3da071-a1cb-4261-83ab-02077ca6a365]
04-23 08:08:13.733 10658 10658 D WM-ConstraintTracker: NetworkStateTracker: initial state = [ Connected=true Validated=true Metered=true NotRoaming=true ]
04-23 08:08:13.733 10658 10658 D WM-NetworkStateTracker: Registering broadcast receiver
04-23 08:08:13.743 10658 10658 D WM-WorkConstraintsTrack: Constraints met for 4e3da071-a1cb-4261-83ab-02077ca6a365
04-23 08:08:13.744 10658 10658 D WM-GreedyScheduler: Constraints met: Scheduling work ID 4e3da071-a1cb-4261-83ab-02077ca6a365
04-23 08:08:13.744 10658 10658 D WM-NetworkStateTracker: Network broadcast received
04-23 08:08:13.744 10658 10697 D WM-Processor: Work 4e3da071-a1cb-4261-83ab-02077ca6a365 is already enqueued for processing
04-23 08:08:14.615  1536  1697 I am_pss  : [10658,10200,de.rki.coronawarnapp,75247616,74264576]
04-23 08:09:02.034  3606 26046 I ExposureNotification: Signature verification succeeded [CONTEXT service_id=236 ]
04-23 08:09:02.182  3606 26892 I ExposureNotification: [MatchingTracer] Starting matching job. [CONTEXT service_id=236 ]
04-23 08:09:02.210  3606 26923 I ExposureNotification: [MatchingTracer] Native pre-filter started. [CONTEXT service_id=236 ]
04-23 08:09:02.229 10658 10720 D WM-WorkerWrapper: de.rki.coronawarnapp.worker.DiagnosisKeyRetrievalOneTimeWorker returned a Success {mOutputData=Data {}} result.
04-23 08:09:02.231 10658 10720 I WM-WorkerWrapper: Worker result SUCCESS for Work [ id=4e3da071-a1cb-4261-83ab-02077ca6a365, tags={ DIAGNOSIS_KEY_ONE_TIME_WORKER, de.rki.coronawarnapp.worker.DiagnosisKeyRetrievalOneTimeWorker } ]
04-23 08:09:02.234 10658 10658 D WM-Processor: Processor 4e3da071-a1cb-4261-83ab-02077ca6a365 executed; reschedule = false
04-23 08:09:02.234 10658 10658 D WM-GreedyScheduler: Stopping tracking for 4e3da071-a1cb-4261-83ab-02077ca6a365
04-23 08:09:02.235 10658 10658 D WM-NetworkStateTracker: Unregistering broadcast receiver
04-23 08:09:02.236 10658 10658 D WM-SystemJobService: 4e3da071-a1cb-4261-83ab-02077ca6a365 executed on JobScheduler
04-23 08:09:02.237  3606 26923 I ExposureNotificationJni: PrefixIdMap loaded 1297 scan records
04-23 08:09:02.238  3606 26923 I ExposureNotification: MatchingJni get native ptr<PHONE_NUMBER>40 [CONTEXT service_id=236 ]
04-23 08:09:02.240  3606 26923 I ExposureNotificationJni: matchingNative get 14 files
04-23 08:09:02.241  3606 26923 I ExposureNotificationJni: Matching with /data/user/0/com.google.android.gms/app_en_diagnosis_keys/1619158071578/export0.bin
04-23 08:09:02.246 10658 10720 D WM-GreedyScheduler: Cancelling work ID 4e3da071-a1cb-4261-83ab-02077ca6a365
04-23 08:09:02.250  3606 26923 I ExposureNotificationJni: Created iterator for /data/user/0/com.google.android.gms/app_en_diagnosis_keys/1619158071578/export0.bin (process_revised_keys = 1)
04-23 08:09:02.251 10658 10721 D WM-Processor: Processor stopping background work 4e3da071-a1cb-4261-83ab-02077ca6a365
04-23 08:09:02.251 10658 10721 D WM-Processor: WorkerWrapper could not be found for 4e3da071-a1cb-4261-83ab-02077ca6a365
04-23 08:09:02.251 10658 10721 D WM-StopWorkRunnable: StopWorkRunnable for 4e3da071-a1cb-4261-83ab-02077ca6a365; Processor.stopWork = false
04-23 08:09:03.029  3606 26923 I ExposureNotificationJni: Matching with /data/user/0/com.google.android.gms/app_en_diagnosis_keys/1619158071578/export1.bin
04-23 08:09:03.030  3606 26923 I ExposureNotificationJni: Created iterator for /data/user/0/com.google.android.gms/app_en_diagnosis_keys/1619158071578/export1.bin (process_revised_keys = 1)
04-23 08:09:03.540  3606 26923 I ExposureNotificationJni: Matching with /data/user/0/com.google.android.gms/app_en_diagnosis_keys/1619158071578/export2.bin
04-23 08:09:03.540  3606 26923 I ExposureNotificationJni: Created iterator for /data/user/0/com.google.android.gms/app_en_diagnosis_keys/1619158071578/export2.bin (process_revised_keys = 1)
04-23 08:09:03.817  3606 26923 I ExposureNotificationJni: Matching with /data/user/0/com.google.android.gms/app_en_diagnosis_keys/1619158071578/export3.bin
04-23 08:09:03.817  3606 26923 I ExposureNotificationJni: Created iterator for /data/user/0/com.google.android.gms/app_en_diagnosis_keys/1619158071578/export3.bin (process_revised_keys = 1)
04-23 08:09:04.207  3606 26923 I ExposureNotificationJni: Matching with /data/user/0/com.google.android.gms/app_en_diagnosis_keys/1619158071578/export4.bin
04-23 08:09:04.207  3606 26923 I ExposureNotificationJni: Created iterator for /data/user/0/com.google.android.gms/app_en_diagnosis_keys/1619158071578/export4.bin (process_revised_keys = 1)
04-23 08:09:04.935  3606 26923 I ExposureNotificationJni: Matching with /data/user/0/com.google.android.gms/app_en_diagnosis_keys/1619158071578/export5.bin
04-23 08:09:04.936  3606 26923 I ExposureNotificationJni: Created iterator for /data/user/0/com.google.android.gms/app_en_diagnosis_keys/1619158071578/export5.bin (process_revised_keys = 1)
04-23 08:09:05.624  3606 26923 I ExposureNotificationJni: Matching with /data/user/0/com.google.android.gms/app_en_diagnosis_keys/1619158071578/export6.bin
04-23 08:09:05.625  3606 26923 I ExposureNotificationJni: Created iterator for /data/user/0/com.google.android.gms/app_en_diagnosis_keys/1619158071578/export6.bin (process_revised_keys = 1)
04-23 08:09:06.228  3606 26923 I ExposureNotificationJni: Matching with /data/user/0/com.google.android.gms/app_en_diagnosis_keys/1619158071578/export7.bin
04-23 08:09:06.229  3606 26923 I ExposureNotificationJni: Created iterator for /data/user/0/com.google.android.gms/app_en_diagnosis_keys/1619158071578/export7.bin (process_revised_keys = 1)
04-23 08:09:06.829  3606 26923 I ExposureNotificationJni: Matching with /data/user/0/com.google.android.gms/app_en_diagnosis_keys/1619158071578/export8.bin
04-23 08:09:06.829  3606 26923 I ExposureNotificationJni: Created iterator for /data/user/0/com.google.android.gms/app_en_diagnosis_keys/1619158071578/export8.bin (process_revised_keys = 1)
04-23 08:09:07.347  3606 26923 I ExposureNotificationJni: Matching with /data/user/0/com.google.android.gms/app_en_diagnosis_keys/1619158071578/export9.bin
04-23 08:09:07.348  3606 26923 I ExposureNotificationJni: Created iterator for /data/user/0/com.google.android.gms/app_en_diagnosis_keys/1619158071578/export9.bin (process_revised_keys = 1)
04-23 08:09:07.635  3606 26923 I ExposureNotificationJni: Matching with /data/user/0/com.google.android.gms/app_en_diagnosis_keys/1619158071578/export10.bin
04-23 08:09:07.635  3606 26923 I ExposureNotificationJni: Created iterator for /data/user/0/com.google.android.gms/app_en_diagnosis_keys/1619158071578/export10.bin (process_revised_keys = 1)
04-23 08:09:08.088  3606 26923 I ExposureNotificationJni: Matching with /data/user/0/com.google.android.gms/app_en_diagnosis_keys/1619158071578/export11.bin
04-23 08:09:08.089  3606 26923 I ExposureNotificationJni: Created iterator for /data/user/0/com.google.android.gms/app_en_diagnosis_keys/1619158071578/export11.bin (process_revised_keys = 1)
04-23 08:09:08.844  3606 26923 I ExposureNotificationJni: Matching with /data/user/0/com.google.android.gms/app_en_diagnosis_keys/1619158071578/export12.bin
04-23 08:09:08.845  3606 26923 I ExposureNotificationJni: Created iterator for /data/user/0/com.google.android.gms/app_en_diagnosis_keys/1619158071578/export12.bin (process_revised_keys = 1)
04-23 08:09:09.612  3606 26923 I ExposureNotificationJni: Matching with /data/user/0/com.google.android.gms/app_en_diagnosis_keys/1619158071578/export13.bin
04-23 08:09:09.613  3606 26923 I ExposureNotificationJni: Created iterator for /data/user/0/com.google.android.gms/app_en_diagnosis_keys/1619158071578/export13.bin (process_revised_keys = 1)
04-23 08:09:10.241  3606 26923 I ExposureNotificationJni: Matching done, total 529032 keys, no key matches
04-23 08:09:10.242  3606 26923 I ExposureNotification: MatchingJni get nullable key set from native. [CONTEXT service_id=236 ]
04-23 08:09:10.253  3606 26923 I ExposureNotification: [MatchingTracer] Native pre-filter found 0 (529032) keys with sightings out of 1297 scan records. Spent time: 8.042s [CONTEXT service_id=236 ]
04-23 08:09:10.259  3606 26923 I ExposureNotification: [MatchingTracer] Java tracing started. [CONTEXT service_id=236 ]
04-23 08:09:10.429  3606 26923 I ExposureNotification: [MatchingTracer] Traced 0 diagnosis keys and found 0 matches. [CONTEXT service_id=236 ]
04-23 08:09:10.429  3606 26923 I ExposureNotification: [MatchingTracer] Persisting matching request record. [CONTEXT service_id=236 ]
04-23 08:09:10.460  3606 26892 I ExposureNotification: [MatchingTracer] Matching job complete. Run time=8.262s. Matches not found. [CONTEXT service_id=236 ]
04-23 08:09:10.460  3606 26892 I ExposureNotification: [MatchingTracer] Sending exposure status update with no new exposures to client. [CONTEXT service_id=236 ]
04-23 08:09:10.473  3606 26892 I ExposureNotification: [MatchingTracer] Successfully removed request. Matching job complete. [CONTEXT service_id=236 ]
04-23 08:09:10.503 10658 10720 D WM-PackageManagerHelper: androidx.work.impl.background.systemalarm.RescheduleReceiver enabled
04-23 08:09:10.515 10658 10720 D WM-SystemJobScheduler: Scheduling work ID 730f60d8-f129-4941-a729-da05465f962e Job ID 49607
04-23 08:09:10.531 10658 10720 D WM-SystemJobScheduler: Scheduling work ID 730f60d8-f129-4941-a729-da05465f962e Job ID 49608
04-23 08:09:10.545 10658 10720 D WM-GreedyScheduler: Starting work for 730f60d8-f129-4941-a729-da05465f962e
04-23 08:09:10.547 10658 10720 D WM-Processor: Processor: processing 730f60d8-f129-4941-a729-da05465f962e
04-23 08:09:10.565 10658 10658 D WM-WorkerWrapper: Starting work for de.rki.coronawarnapp.nearby.ExposureStateUpdateWorker
04-23 08:09:10.569 10658 10704 D WM-WorkerWrapper: de.rki.coronawarnapp.nearby.ExposureStateUpdateWorker returned a Success {mOutputData=Data {}} result.
04-23 08:09:10.570 10658 10704 I WM-WorkerWrapper: Worker result SUCCESS for Work [ id=730f60d8-f129-4941-a729-da05465f962e, tags={ de.rki.coronawarnapp.nearby.ExposureStateUpdateWorker } ]
04-23 08:09:10.572 10658 10658 D WM-Processor: Processor 730f60d8-f129-4941-a729-da05465f962e executed; reschedule = false
04-23 08:09:10.583 10658 10704 D WM-GreedyScheduler: Cancelling work ID 730f60d8-f129-4941-a729-da05465f962e
04-23 08:09:10.589 10658 10704 D WM-Processor: Processor stopping background work 730f60d8-f129-4941-a729-da05465f962e
04-23 08:09:10.589 10658 10704 D WM-Processor: WorkerWrapper could not be found for 730f60d8-f129-4941-a729-da05465f962e
04-23 08:09:10.589 10658 10704 D WM-StopWorkRunnable: StopWorkRunnable for 730f60d8-f129-4941-a729-da05465f962e; Processor.stopWork = false
04-23 08:09:10.601   437  1310 D BandwidthController: [LG DATA] No such appUid: 10200
04-23 08:09:10.601   437  1310 D DnsProxyListener: App 10200 tries DNS query. Accept family:0 protocol:0

@vaubaehn please retest when 2.2 is released.

@harambasicluka , thanks for stopping by!

please retest when 2.2 is released.

Sure, although it won't be as easy as before because the pre-conditions changed:

#2934, the fix for #2881 seems to be successful 🎉 , so I will need to be on mobile/wifi off for some days 😞 .
The error usually occured with large DK packages. Now, incident rates are decreasing 🎉 , but the DK packages are -100k keys compared to mid-April and below the critical point, when I could observe multiple provisions of same DK package to ENS in a single day 😞 . (see explanation above - so probably it's a timing problem somewhere)

Could you give a small hint which PR/change you expect to fix the problem?
Thanks, and enjoy your night!

Hi @d4rken ,
I commented a bit more on https://github.com/corona-warn-app/cwa-app-android/issues/2880#issuecomment-827846585 to illustrate, why I think, it could be that the DiagnosisKeyRetrievalOneTimeWorker had 2 job IDs and was running concurrently twice, even https://github.com/corona-warn-app/cwa-app-android/blob/e1ec946e09600826d16ea389aa8dd96f7c959129/Corona-Warn-App/src/main/java/de/rki/coronawarnapp/task/TaskController.kt#L221-L224
the TaskConrtroller does not allow for running the same task class as a sibling, and
https://github.com/corona-warn-app/cwa-app-android/blob/e1ec946e09600826d16ea389aa8dd96f7c959129/Corona-Warn-App/src/main/java/de/rki/coronawarnapp/diagnosiskeys/download/DownloadDiagnosisKeysTask.kt#L234
the DownloadTask would be skipped, if another instance is already running.
But couldn't that explain the symptoms also seen in the logcat? Two instances are created (falsely), and the collision is detected in DownloadTask. The download is skipped for the SECOND job, and the OLD diagnosis keys are provided to ENS by the second job. Whilst the FIRST job is still running and downloading, but can't provide the fresh keys to ENS anymore.
Does this sound plausible to you? At least a little?

Hi @d4rken , I found another case - I'm not alone: https://github.com/corona-warn-app/cwa-app-android/issues/3093#issuecomment-833567082

I disabled wifi yesterday night, and "hoped" to be able to reproduce the issue again.
I succeeded today - not sure if this is good for a "yeyyyy" or a "noooo"....
@d4rken @harambasicluka
You want me to enable debug logging, as soon as 2.2 is on my device? In case, I would copy the tickt number here?

In case, I would copy the tickt number here?

Yes, please refer to https://www.coronawarn.app/en/faq/#error_log

@Ein-Tim -> there is a typo in your citation ;)

@vaubaehn Don't know what you mean 😇

Dear all, looks like Google stopped logging from ENS to syslog (ExposureNotification) completely in recent days (logcat won't show any useful information anymore). Means, CWA's debug log feature seems to be the only thing we can rely on in the future. Triangulation (using CWA's logs together with ExposureNotification syslog) won't be possible anymore.
I hope, my phone only has a hickup after latest ENS update...

ENS to syslog (ExposureNotification) completely in recent days (logcat won't show any useful information anymore).

JUst for some unrelated background info: This could be related to a vulnerability they had, where they logged too much to that log, see https://www.cvedetails.com/cve/CVE-2021-31815/ (if I got the corect CVE here). It seems they decided to just drop the log completely.

@rugk
It's reasonable to assume that the change in logging is due to the vulnerability response.  Google doesn't seem to have published any direct public response to CVE-2021-31815. Google's release notes are quite high level and don't cover this. So we can only make assumptions.
Google seems to have informed COVID-19 app development partners though, as there is a CWA FAQ article about it under https://www.coronawarn.app/en/faq/#google_security_vulnerability.

UPDATE on this issue
The error is like Heisenberg's Uncertainty Principle: You can either observe the bug, or you log without any bug.
After CWA 2.2.1 arrived on my phone last Friday (🍔 time!), I immediately turned on debug logging. The error did not occur since.
After being on mobile only (wifi turned off) since 12th of May (on which day was the last occurence if the bug), now my quota is going to be exhausted, and I need to turn on wifi again.
This means, after the bug occurs with only one exposure check per day, and with the fix by #2881, it gets quite unlikely to log the error in the near future for me.
I'm adding my compiled exposureLogs here, with last occurence of the bug on May 12th, 14:58 CET with CWA 2.1.:
20210401-20210518_assembled_all-exposure-checks.txt
@d4rken and @harambasicluka I'm offering to upload my debug log so far, and provide the ticket number for further analysis. Please give me a hint, whether you want to look into it. If you don't want to, I'd reset the current log, due to tiny space on my internal storage.
From first self-inspection I found at least on point that might be correlated to the here reported issue:
Duration(now, nextDetectionAt) in DownloadDiagnosisKeysTask.kt is negative most of the time, as nextDetectionAt does not seem to be calculated correctly. NextDetectionAt is lastDetection plus 4 hours, but if only one detection per day takes place (because of metered connection), then this value would become negative, and I don't know what fun wasLastDetectionPerformedRecently is returning in that case...
In general it's the question whether it's worth to further inspect this bug, as it seems to

only appear with large diagnosis key files to be downloaded (during times with high incidence rates)
only appear with one exposure matching per day (for people on metered/mobile connections)

So we can (still) expect an impact of the bug for

people who most of the time need to be on a metered/mobile connection (e.g., refugees, farming helpers from abroad)
times, when incident rates increase again (i.e., 4th wave due to B.1.617 and B.1.618)

The impact would mean that affected people might get warned with "fresh" diagnosis keys with an unwanted long delay (of > 48 hours in my case, see April 5th to April 7th in my exposure logs, or 42 hours on April 17/18).
Please let me know how to proceed, thank you,
V.

@d4rken and @harambasicluka I'm offering to upload my debug log so far, and provide the ticket number for further analysis. Please give me a hint, whether you want to look into it. If you don't want to, I'd reset the current log, due to tiny space on my internal storage.

If you can reproduce the problematic behavior, please upload a log, i.e. unexpectedly skipped exposure checks or duplicate key submissions (according to ENS hashes).
I suspect a relationship with #3093 where the logs show that I checked show various timeouts when communicating with the ENS.
Some kind of combination of the timeout letting the CWA assume that the operation was not successful, but despite the timeout, the ENS still performed the calculation.
Not sure whether the hourly packages make the issue more likely (more provideDiagnosisKeys calls) or less likely (smaller packages) :thinking:.

Hi @dsarkar ,

The error is like Heisenberg's Uncertainty Principle: You can either observe the bug, or you log without any bug.

On a side note: Maybe this helps you to circumvent the uncertainty principle ;-) https://www.eurekalert.org/pub_releases/2021-05/au-etu050521.php and https://www.nist.gov/news-events/news/2021/05/nist-team-directs-and-measures-quantum-drum-duet

thanks for providing these links, they were indeed rather helpful! I was able to turn my old phone into a quantum computing mobile by inserting aluminum membranes that were cut out from some old aluminium foil into the phone. I put the phone into a black box, vacuumed and supra-cooled it. To create quantum states in the aluminium membranes, I simply used our kitchen's microwave oven.
As you guess, this all took a while, that's why I'm repsonding late.
But I was successful: My phone is faster than ever, and I could produce

no error log with no bugs
one error log with no bugs
no error log with bugs
and one error log with bugs

all at the same time! That's fantastic!
But I still couldn't figure out yet, on how to export the (no) logs...
There are also other caveats: Doing phone calls is a bit hard, as I need to shout through the vacuumed box. Also the box is quite cold what hurts on my ears. Then there is my wife complaining about the high energy costs resulting from supra-cooling and that I took her microwave to my cellar hobby room... If you also have some ideas on how to circumvent theses problems, I'd be happy to hear from you!
😉

Hi @d4rken ,

If you can reproduce the problematic behavior, please upload a log, i.e. unexpectedly skipped exposure checks or duplicate key submissions (according to ENS hashes).

Despite from what I wrote above, I was not able to reproduce the issue under the current conditions (low incidence rate and "my" CWA now doing hourly updates also on non-metered connection).

Not sure whether the hourly packages make the issue more likely (more provideDiagnosisKeys calls) or less likely (smaller packages) 🤔.

I'm rather sure that the issue is more likely on bigger packages only, as I only could observe it when incidence rates were high.

I suspect a relationship with #3093 where the logs show that I checked show various timeouts when communicating with the ENS.
Some kind of combination of the timeout letting the CWA assume that the operation was not successful, but despite the timeout, the ENS still performed the calculation.

Yes, that's very likely and fits well together with my previous observations: Most of processor time ENS is consuming is for validating the DK packages against the signature (see logcat: https://github.com/corona-warn-app/cwa-app-android/issues/2880#issuecomment-827846585). In case this blocks ENS for quite a bit of time, then communication timeouts are very likely. It's likely that the time ENS spends on validating signature is proportional to DK package size: higher incidence rates -> more keys -> timeout more likely.
I'm also still not convinced that the issue does not arise because of 2 workers started in parallel, and the one, that could successfully do the job, is cancelled as a sibling.
But if that's the case, then there is still the question, why the times CWA retries on providing DKs again are so weird? It should be one hour later (next worker starting time), but sometimes it's just a couple of minutes later and sometimes 6 hours later...
I assume still some (additional) problems with timestamp calculations/conditions, see also https://github.com/corona-warn-app/cwa-app-android/issues/2880#issuecomment-843065085 (the paragraph with Duration(now, nextDetectionAt)...)
In the last two weeks the number of DKs doubled because of the delta variant in Europe, we're now around 100.000 keys again. Given r-value stays same, in about 5 weeks we will be back at the times, when this issue occured (DKs > 500.000).
Do you feel like waiting until then, or should we look into it again before?

@vaubaehn
Unbelievable what you all do (to your phone), just to improve the Corona-Warn-App 😉
Maybe the issues with your phone you mention in https://github.com/corona-warn-app/cwa-app-android/issues/3692#issuecomment-879879475 don't happen because you switch between languages but because you put aluminum foil into it? 😉

The number of diagnosis keys is increasing much currently. Probably within a few days we'll reach >500k keys.
With hourly update schema there is seemingly no problem at the moment, though I found a 12-minute-difference between risk card timestamp and ENS log timestamp yesterday, which is surprising as the timeout limit is 8 minutes afair...
Somewhere in the next days I will run my phone on mobile data only, then let's see if the issue here is still present.

a 12-minute-difference between risk card timestamp and ENS log timestamp yesterday, which is surprising as the timeout limit is 8 minutes afair...

... not knowing anything about the Google / Android / ENS specifications ...
... just asking stupid questions that come to my mind ...
@vaubaehn
Are you sure that this "8-minute timeout limit" is measured in true time?
An alternative could be CPU time?
So the clock would wait while the thread is suspended?

Real time but if there are device above 8min we can increase the timeout

Hi @thomasaugsten

Real time but if there are device above 8min we can increase the timeout

Thanks for taking care, but after thinking a second time, I'm not sure if increasing the timeout is necessary.
My understanding of the process (simplified) is this:

CWA decides that it's time to download DKs, and does so (->1. relevant timeout: download process. If time out occurs (e.g. poor connectivity), DKs need to be downloaded again).
CWA provides diagnosis keys to ENS and waits for a response (-> 2. relevant timeout: provisioning process. If timout occurs (e.g., heavy system load), CWA would need to provide DKs again).
Then CWA sits back and waits for an intent that ENS finished to check DKs for any risky encounter (-> no timeout involved, afair. If system process is killed while ENS is still doing its work, then updated ExposureWindows should be retrieved/risk calculation should be perfomed with the next instantiating of the app? Or upon next provisioning of DKs? This could be a vulnerable point, if CWA never gets an intent that ENS is finished or actively requests ExposureWindows for risk calculation).
When CWA gets the intent that ENS is finished, it requests the ExposureWindows and performs risk calculation. (-> vulnerable point: see previous bullet point).

So, if CWA can reliably get the Exposure Windows at any time, nothing would need to be changed here. At least the 2 different timeouts that I know would not change anything in the case of my reported time difference of 12 minutes between providing DKs and ENS finishing check.

Hi everybody, a short intermediate result:
There's good news and bad news... Bad news first: we broke the 1M-keys boundary for the full-day diagnosis keys package last night: at 1:20 my phone matched 1.036.393 diagnosis keys.
For the good news: since November 4, I could not reproduce the issue reported anymore while my phone was running on 24/7 mobile data.
My quota is finished for now, I will repeat again with fresh quota in middle December. If the matching works as smooth as now at that time, I'll consider the issue to be solved.
See you then!

I guess https://github.com/corona-warn-app/cwa-app-android/pull/4929 will also drastically improve the situation here, or @vaubaehn?


If LB + 1 = UB then deterministically assign when possible.

Never cut
Never assign to other side when dA / dB are non-INFTY.

Closing, this only works well for LB = #CUT.


[Question/Feedback]: Governance on nat-gw module?

Check for previous/existing GitHub issues

[X] I have checked for previous/existing GitHub issues

Description
Is there AVM governance around this module: https://github.com/Azure/terraform-azurerm-avm-res-network-natgateway ?
issues/pr's seem to go stale, the readme seems to be a standard-generated, there are some oddities like invalid diagnostics resources etc.

I am closing this, handled by @mbilalamjad


[2] Add configuration option to "user api server" status in blackbox exporter

Is your feature request related to a problem? Please describe.
The current configuration assumes that the workload cluster api server is available at https://kube-apiserver.{{ .Values.global.baseDomain }}:/healthz. This is a problem primarily if you are running multiple workload clusters, then blackbox exporter will at best monitor one of them. If the apiserver is not connected to this URL, then the metric will say that the apiserver is failing and the uptime dashboard will also show this (even though the apiserver(s) might work as expected).
Describe the solution you'd like
Add config that allows us to specify any amount of apiservers that blackbox should check.
Also make sure that the uptime dashboard can show the status of multiple apiservers.
Describe alternatives you've considered
The only other alternative I see is to have an option to disable this metric, so that it at least does not give any misinformation.
Additional context
This is how the uptime dashboard looks now in my setup.

Definition of done:
There is config that allows us to specify any number of apiservers to check.
The uptime dashboard can show uptime for multiple apiservers.

As part of this issue, please also make blackbox exporter verify TLS based on the global config.

Will be replaced by: https://github.com/elastisys/ck8s/issues/102


wind convention parameters

oceano / meteo wind

Maybe, in the config, wind_dir_convention or dir_convention are more explicit ?

I took advantage of this PR to change a bit the call of preprocess

I added a function to check that incidence range is included in LUT incidence range ;
in order to prevent user ; if not included, inversion will use wrong inversion angle and so provide wrong wind values

i also renamed  model_vv => model_co et model_vh =>  model_cross


bump js-yaml 3.14 -> 4.0

This bump will break the js-yaml API: safeLoad -> load, safeLoadAll -> loadAll, safeDump -> dump

/remote-lifecycle rotten

/remove-lifecycle rotten


0.11.rc0 version, using all gpu's memory while running on only one

With 0.11.rc0 tf, cudnn 7.5
+------------------------------------------------------+
| NVIDIA-SMI 352.39     Driver Version: 352.39         |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  Tesla K40m          On   | 0000:03:00.0     Off |                    0 |
| N/A   47C    P0   127W / 235W |  10995MiB / 11519MiB |     87%      Default |
+-------------------------------+----------------------+----------------------+
|   1  Tesla K40m          On   | 0000:04:00.0     Off |                    0 |
| N/A   39C    P0    63W / 235W |  10953MiB / 11519MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   2  Tesla K40m          On   | 0000:83:00.0     Off |                    0 |
| N/A   40C    P0    62W / 235W |  10954MiB / 11519MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   3  Tesla K40m          On   | 0000:84:00.0     Off |                    0 |
| N/A   39C    P0    62W / 235W |  10953MiB / 11519MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID  Type  Process name                               Usage      |
|=============================================================================|
|    0      8828    C   ...mpiler/gcc-4.8.2/lib/ld-linux-x86-64.so.2 10964MiB |
|    1      8828    C   ...mpiler/gcc-4.8.2/lib/ld-linux-x86-64.so.2 10923MiB |
|    2      8828    C   ...mpiler/gcc-4.8.2/lib/ld-linux-x86-64.so.2 10923MiB |
|    3      8828    C   ...mpiler/gcc-4.8.2/lib/ld-linux-x86-64.so.2 10923MiB
With 0.10.0 tf, same code running with gpu status below
| NVIDIA-SMI 352.39     Driver Version: 352.39         |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  Tesla K40m          On   | 0000:03:00.0     Off |                    0 |
| N/A   46C    P0   129W / 235W |  10993MiB / 11519MiB |     89%      Default |
+-------------------------------+----------------------+----------------------+
|   1  Tesla K40m          On   | 0000:04:00.0     Off |                    0 |
| N/A   38C    P0    63W / 235W |    108MiB / 11519MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   2  Tesla K40m          On   | 0000:83:00.0     Off |                    0 |
| N/A   39C    P0    62W / 235W |    107MiB / 11519MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   3  Tesla K40m          On   | 0000:84:00.0     Off |                    0 |
| N/A   38C    P0    62W / 235W |    107MiB / 11519MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID  Type  Process name                               Usage      |
|=============================================================================|
|    0     20051    C   ...mpiler/gcc-4.8.2/lib/ld-linux-x86-64.so.2 10959MiB |
|    1     20051    C   ...mpiler/gcc-4.8.2/lib/ld-linux-x86-64.so.2    74MiB |
|    2     20051    C   ...mpiler/gcc-4.8.2/lib/ld-linux-x86-64.so.2    74MiB |
|    3     20051    C   ...mpiler/gcc-4.8.2/lib/ld-linux-x86-64.so.2    74MiB

@yaroslavvb I see thanks, but I don't think it need to grab all gpu memories if only running on one.
Just grap one gpu memory as version 0.10.0 did is fine.
Anyway can force to grab one gpu memory by using CUDA_VISIBLE_DEVICES.

@chenghuige: what do you mean by "only running on one"?  To me, this means setting CUDA_VISIBLE_DEVICES.  If you instead mean that you use with tf.device("/gpu:0") exclusively in your code, then when initializing the TensorFlow runtime, it would have no way to know just one GPU is going to be used.

@concretevitamin  well I think as in version 0.10, it will launch only one GPU, unless your code is like models/image/cifa10/
to explicitly set using more then 1
for i in xrange(FLAGS.num_gpus):
with tf.device('/gpu:%d' % i):

You will get best performance by partitioning your variables manually across GPUs in a way that makes sense.

I think the problem is that there's an Eigen scratch allocator in the GPUDevice constructor that allocates memory on the GPU.  Before r0.11, the call was directly calling cuMalloc instead of going through our allocator, and so although some memory was used on the GPU, it wasn't going through our allocator so the whole thing wasn't being taken.  But to properly track memory usage, we make all of our calls now go through the allocator, so GPUDevice construction now unfortunately allocates all the memory.
Two solutions:

We lazily allocate the scratch allocator space instead of eagerly doing it in the constructor (not sure where / when we would do this though).

You can set the 'allow_growth' option in ConfigProto.GPUOptions so that the initial allocation doesn't cause all of the GPU memory to be allocated.

No response for 10 days.
Also looks like this is working as intended.
Therefore closing the issue.


wait for cache invalidation to complete

Would this have any negative side effects? It's currently tricky to invalidate local cache and reload images since we have no idea whether the invalidation has completed on Cloudfront's end.

https://repost.aws/knowledge-center/cloudfront-serving-outdated-content-s3
"Object invalidations typically take from 10 to 100 seconds to complete. "
Almost 2 minutes is quite a long wait for a user. Is this for the user who has just updated his / her own profile pic ?

@erikolsson you know the id of the media stream for the current user. You can directly fetch the image via /media/{streamId}?key=key&iv=iv that you grab from the users metadata stream.


amplify init says 🛑 Forbidden with no explanation

How did you install the Amplify CLI?
npm
If applicable, what version of Node.js are you using?
v20.0.0
Amplify CLI Version
12.7.0
What operating system are you using?
Mac Sonoma 14.1 Beta
Did you make any manual changes to the cloud resources managed by Amplify? Please describe the changes made.
No manual changes made
Describe the bug
Following the online steps, I installed and setup the amplify CLI
Now when running amplify init I get a Forbidden error, though no information about what is wrong.
dacarson$ amplify init
? Enter a name for the project LynnCommunity
The following configuration will be applied:

Project information
| Name: LynnCommunity
| Environment: dev
| Default editor: Visual Studio Code
| App type: ios

? Initialize the project with the above configuration? No
? Enter a name for the environment dev
? Choose your default editor: Xcode (macOS only)
✔ Choose the type of app that you're building · ios
Using default provider  awscloudformation
? Select the authentication method you want to use: AWS profile

For more information on AWS Profiles, see:
https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-profiles.html

? Please choose the profile you want to use default
🛑 Forbidden

Learn more at: https://docs.amplify.aws/cli/project/troubleshooting/

Session Identifier: 3651cf63-e2df-43bd-b280-a5a00233c602
dacarson$

Expected behavior
No error and I can go onto step (4) of the guide:
https://docs.amplify.aws/lib/project-setup/create-application/q/platform/ios/#2-install-amplify-libraries
Reproduction steps
Follow instructions for project setup for a mac/xCode/Swift as documented here:
https://docs.amplify.aws/lib/project-setup/prereq/q/platform/ios/
Project Identifier
dacarson$ amplify diagnose --send-report
Learn more at https://docs.amplify.aws/cli/reference/diagnose/
⠋ Creating Zip
No Amplify backend project files detected within this folder.
✖ Creating Zip
dacarson$
Log output

2023-10-17T23:26:48.094Z|error : amplify-provider-awscloudformation.amplify-service-permission-check.checkAmplifyServiceIAMPermission.amplifyClient.listApps([])
InvalidSignatureException: Forbidden
2023-10-17T23:26:48.096Z|info : amplify-provider-awscloudformation.amplify-service-manager.init.amplifyClient.createApp([{"name":"[***]munity","environmentVariables":{"_LIVE_PACKAGE_UPDATES":"[{\"pkg\":\"@aws-amplify/cli\",\"type\":\"npm\",\"version\":\"latest\"}]"}}])
2023-10-17T23:26:48.367Z|error : Forbidden
ProjectInitFault: Forbidden
2023-10-17T23:31:00.827Z|info : amplify init core
2023-10-17T23:31:00.879Z|info : @aws-amplify/amplify-cli-core.banner-message/index.ts.fetch banner messages from https://aws-amplify.github.io/amplify-cli/banner-message.json({}
2023-10-17T23:31:26.461Z|info : amplify-provider-awscloudformation.system-config-manager.getProfileConfig(["default"])
2023-10-17T23:31:26.466Z|info : amplify-provider-awscloudformation.system-config-manager.getProfiledAwsConfig.profileConfig([{"region":"us-east-1"}])
2023-10-17T23:31:26.467Z|info : amplify-provider-awscloudformation.system-config-manager.getProfileCredentials(["default"])
2023-10-17T23:31:26.480Z|info : amplify-provider-awscloudformation.amplify-service-permission-check.checkAmplifyServiceIAMPermission.amplifyClient.listApps([])
2023-10-17T23:31:26.783Z|error : amplify-provider-awscloudformation.amplify-service-permission-check.checkAmplifyServiceIAMPermission.amplifyClient.listApps([])
InvalidSignatureException: Forbidden
2023-10-17T23:31:26.785Z|info : amplify-provider-awscloudformation.amplify-service-manager.init.amplifyClient.createApp([{"name":"[***]munity","environmentVariables":{"_LIVE_PACKAGE_UPDATES":"[{\"pkg\":\"@aws-amplify/cli\",\"type\":\"npm\",\"version\":\"latest\"}]"}}])
2023-10-17T23:31:27.091Z|error : Forbidden
ProjectInitFault: Forbidden
2023-10-17T23:35:02.922Z|info : amplify version core  {"version":true,"yes":false}
2023-10-17T23:36:06.020Z|info : amplify init core
2023-10-17T23:36:06.074Z|info : @aws-amplify/amplify-cli-core.banner-message/index.ts.fetch banner messages from https://aws-amplify.github.io/amplify-cli/banner-message.json({}
2023-10-17T23:36:18.960Z|info : amplify init core
2023-10-17T23:36:19.010Z|info : @aws-amplify/amplify-cli-core.banner-message/index.ts.fetch banner messages from https://aws-amplify.github.io/amplify-cli/banner-message.json({}
2023-10-17T23:36:47.537Z|info : amplify-provider-awscloudformation.system-config-manager.getProfileConfig(["default"])
2023-10-17T23:36:47.540Z|info : amplify-provider-awscloudformation.system-config-manager.getProfiledAwsConfig.profileConfig([{"region":"us-east-1"}])
2023-10-17T23:36:47.540Z|info : amplify-provider-awscloudformation.system-config-manager.getProfileCredentials(["default"])
2023-10-17T23:36:47.551Z|info : amplify-provider-awscloudformation.amplify-service-permission-check.checkAmplifyServiceIAMPermission.amplifyClient.listApps([])
2023-10-17T23:36:47.702Z|error : amplify-provider-awscloudformation.amplify-service-permission-check.checkAmplifyServiceIAMPermission.amplifyClient.listApps([])
InvalidSignatureException: Forbidden
2023-10-17T23:36:47.703Z|info : amplify-provider-awscloudformation.amplify-service-manager.init.amplifyClient.createApp([{"name":"[***]munity","environmentVariables":{"_LIVE_PACKAGE_UPDATES":"[{\"pkg\":\"@aws-amplify/cli\",\"type\":\"npm\",\"version\":\"latest\"}]"}}])
2023-10-17T23:36:47.913Z|error : Forbidden
ProjectInitFault: Forbidden
2023-10-17T23:38:16.066Z|info : amplify version core
2023-10-17T23:44:29.725Z|info : amplify diagnose core  {"send-report":true,"yes":false}
2023-10-17T23:44:29.772Z|info : @aws-amplify/amplify-cli-core.banner-message/index.ts.fetch banner messages from https://aws-amplify.github.io/amplify-cli/banner-message.json({}

Additional information
First time going through this.
Before submitting, please confirm:

[X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.
[X] I have removed any sensitive information from my code snippets and submission.

Hey @dacarson :wave: thanks for raising this! As we begin to look at this in more depth I have a few follow-up questions/confirmations:

is the default user set up with the AdministratorAccess-Amplify managed policy?
if you run the same command with --debug does it print a stack trace with more details?

Hey @dacarson 👋 thanks for raising this! As we begin to look at this in more depth I have a few follow-up questions/confirmations:

is the default user set up with the AdministratorAccess-Amplify managed policy?

Yes. I used what was shown in the 'Retrieve access keys' section of: https://docs.amplify.aws/cli/start/install/#configure-the-amplify-cli

if you run the same command with --debug does it print a stack trace with more details?

? Please choose the profile you want to use default
🛑 Forbidden

Learn more at: https://docs.amplify.aws/cli/project/troubleshooting/

ProjectInitFault: Forbidden
    at init (/snapshot/amplify-cli/build/node_modules/@aws-amplify/amplify-provider-awscloudformation/lib/amplify-service-manager.js:163:13)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Object.run (/snapshot/amplify-cli/build/node_modules/@aws-amplify/amplify-provider-awscloudformation/lib/initializer.js:96:9)

Forbidden
InvalidSignatureException: Forbidden
    at Object.extractError (/snapshot/amplify-cli/build/node_modules/aws-sdk/lib/protocol/json.js:80:27)
    at Request.extractError (/snapshot/amplify-cli/build/node_modules/aws-sdk/lib/protocol/rest_json.js:61:8)
    at Request.callListeners (/snapshot/amplify-cli/build/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
    at Request.emit (/snapshot/amplify-cli/build/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
    at Request.emit (/snapshot/amplify-cli/build/node_modules/aws-sdk/lib/request.js:686:14)
    at Request.transition (/snapshot/amplify-cli/build/node_modules/aws-sdk/lib/request.js:22:10)
    at AcceptorStateMachine.runTo (/snapshot/amplify-cli/build/node_modules/aws-sdk/lib/state_machine.js:14:12)
    at /snapshot/amplify-cli/build/node_modules/aws-sdk/lib/state_machine.js:26:10
    at Request.<anonymous> (/snapshot/amplify-cli/build/node_modules/aws-sdk/lib/request.js:38:9)
    at Request.<anonymous> (/snapshot/amplify-cli/build/node_modules/aws-sdk/lib/request.js:688:12)
    at Request.callListeners (/snapshot/amplify-cli/build/node_modules/aws-sdk/lib/sequential_executor.js:116:18)
    at Request.emit (/snapshot/amplify-cli/build/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
    at Request.emit (/snapshot/amplify-cli/build/node_modules/aws-sdk/lib/request.js:686:14)
    at Request.transition (/snapshot/amplify-cli/build/node_modules/aws-sdk/lib/request.js:22:10)
    at AcceptorStateMachine.runTo (/snapshot/amplify-cli/build/node_modules/aws-sdk/lib/state_machine.js:14:12)
    at /snapshot/amplify-cli/build/node_modules/aws-sdk/lib/state_machine.js:26:10
    at Request.<anonymous> (/snapshot/amplify-cli/build/node_modules/aws-sdk/lib/request.js:38:9)
    at Request.<anonymous> (/snapshot/amplify-cli/build/node_modules/aws-sdk/lib/request.js:688:12)
    at Request.callListeners (/snapshot/amplify-cli/build/node_modules/aws-sdk/lib/sequential_executor.js:116:18)
    at callNextListener (/snapshot/amplify-cli/build/node_modules/aws-sdk/lib/sequential_executor.js:96:12)
    at IncomingMessage.onEnd (/snapshot/amplify-cli/build/node_modules/aws-sdk/lib/event_listeners.js:417:13)
    at IncomingMessage.emit (node:events:525:35)
    at IncomingMessage.emit (node:domain:489:12)
    at endReadableNT (node:internal/streams/readable:1359:12)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21)

Is there any other way to build the json files? I am blocked on application development right now.

FWIW, I tried amplify configure again, and this time didn't use the default name for the profile. This time I set it as aws-profile. I then tried amplify init again. It listed both profile and aws-profile, I selected the later. But it still gave me the same error with the same backtrace as above.

Hey @dacarson glad to hear you've found the cause for this and are back up and running! And to your note I agree this error message can be improved. I'll mark this as a feature request to improve the messaging

Moving to bug to improve error messaging.

So I am experiencing this issue right now. From what I read the issue is caused by providng the wrong secret access key for the amplify cli. Also I am not sure if this matters but does being an IAM user have any impact on this?


N20: Bevy Retro

For #531.

I think that the notice about Katharos License should be more prominent/visible. The small link at the end feels too hidden. It is an unusual license and it should be made clear to potential users of the software.

I moved the link to the second paragraph which should be clear enough.
The Katharos License README also has an Open Questions section to make it clear that there were concerns brought up about the license and to point to that discussion.

Hey @zicklag, due to the ethical/moral/religious implications behind the Katharos License, we're going to ask that you make your license sections in the newsletter a lot more explicit about its nature. If you could change the sections out to what is included below, then we should be ok to merge it into the newsletter:
This project was released under the [Katharos License][katharos_license]. This license has moral and ethical implications that you may or may not agree with, so please read it before making use of this project.

Pushed an update: +1:

Merged, thanks for the PR!


Allow creating persistent sessions from client

Fixes #47

Hi @jwittner, I'm your friendly neighborhood Microsoft Pull Request Bot (You can call me MSBOT). Thanks for your contribution!

It looks like you're a Microsoft contributor (Josh Wittner). If you're full-time, we DON'T require a Contribution License Agreement. If you are a vendor, please DO sign the electronic Contribution License Agreement. It will take 2 minutes and there's no faxing! https://cla.microsoft.com.

TTYL, MSBOT;

Once this is in I have a branch for HTK-Unity to absorb the HTK changes.

@jheuvel-ms  Should this up the XTools Version?

If I can get this approved, I could add saving of new persistent sessions into #42

Anyone out there have thoughts on this?

Please update version number and when ready please update HTK-Unity.


Implement a Script to create Train and Test Sets for our Dataset

AIM
Implement a Script to create Train and Test Sets for our Dataset
Details

Download the Kaggle Dataset
Load the Images from the Disk and scale them.
Assign Binary Values to them and dump them to save Pre-Processing Time every time we load the Dataset.
Split the Dataset into Training and Testing Sets.

@HarshCasper can I use pickle to dump?

@HarshCasper I would like to work on this

Yes @Bijay555
I am assigning in to you @Bijay555
@jyrj Issues are assigned on "First Come, First Serve" basis. I will create more Issues in the future nonetheless


Fortigate connect via Paramiko fails as timeout argument is unexpected

As per below, the fix is to add the timeout=None to the command function:
    def command(self, command, args=[], elevate=False, runAs=None):
        self.channel.send("{0}{1}".format(command,"\n"))
        return (0, self.recv(), "")

to

    def command(self, command, args=[], elevate=False, runAs=None, timeout=None):
        self.channel.send("{0}{1}".format(command,"\n"))
        return (0, self.recv(), "")

This bug is related to the remoteCommand action that passes in the timeout argument that is then not supported by the FortiGate class?

Hi,
That's right yes.  It tries to pass the timeout=x to the fortigate.command() function and that function is not expecting the timeout argument so it fails.
Cheers!
Jon.

I have merged in the fix for this. Thanks


Error raising : requests.exceptions.InvalidSchema: No connection adapters were found for "[url]"

Hey,
I tried to run the script.
This error is raising :
Error raising : requests.exceptions.InvalidSchema: No connection adapters were found for "[URL]"
Thanks for you support

Just want to make sure, did you change the [url] part in the snippet with your airtable url ?
for example like this:
airscraper "https://airtable.com/shrSodX4SH7WDXeBS"

You can find detailed explanation on how to create the link on airtable support page

Yes i did, the document is very confidential that's why I replaced it with URL

Tried with your url same error

did you use the package from pip?

I started a new projet
now the error is : ModuleNotFoundError: No module named 'requests'

Python version : 3.9.7
OS : MacOs Monterrey
How can i find my python version manager ?
I'm still a beginner sorry for those easy questions

oh, my bad
I was assuming your python environment already has requests package installed, so you should be fine after installing it with pip as usual
pip install requests

If the problem still persists, I recommend you use python version manager (I usually use conda, but there are many others) and try it again on python 3.6 - 3.8, currently I have only tested this package on those versions.

If those solved the problem then there are 2 issues here:

[ ] Adding documentation on supported python version
[ ] Update setup.py to add requests package as dependency

I opened a jupyter notebook here is the full error :
InvalidSchema                             Traceback (most recent call last)
/var/folders/h5/qcn9lt4953j5p0hr89gbvv3r0000gn/T/ipykernel_90389/1369544519.py in
3 import pandas as pd
4
----> 5 client = AirScraper(["https://airtable.com/shrSodX4SH7WDXeBS"])
6 data = client.get_table().text
7
/opt/anaconda3/lib/python3.9/site-packages/airscraper/airscraper.py in init(self, url, sess, password, locale, tz, cache)
21         self.password = password
22         self.page = None
---> 23         self._is_passwd_protected()
24         if self.is_protected:
25             self.login()
/opt/anaconda3/lib/python3.9/site-packages/airscraper/airscraper.py in _is_passwd_protected(self)
35     def _is_passwd_protected(self) -> None:
36         try:
---> 37             res = self.sess.get(self.url)
38             if res.status_code == 200:
39                 self.is_protected = "passwordProtectedShareFormContainer" in res.text
/opt/anaconda3/lib/python3.9/site-packages/requests/sessions.py in get(self, url, **kwargs)
540
541         kwargs.setdefault('allow_redirects', True)
--> 542         return self.request('GET', url, **kwargs)
543
544     def options(self, url, **kwargs):
/opt/anaconda3/lib/python3.9/site-packages/requests/sessions.py in request(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json)
527         }
528         send_kwargs.update(settings)
--> 529         resp = self.send(prep, **send_kwargs)
530
531         return resp
/opt/anaconda3/lib/python3.9/site-packages/requests/sessions.py in send(self, request, **kwargs)
637
638         # Get the appropriate adapter to use
--> 639         adapter = self.get_adapter(url=request.url)
640
641         # Start time (approximately) of the request
/opt/anaconda3/lib/python3.9/site-packages/requests/sessions.py in get_adapter(self, url)
730
731         # Nothing matches :-/
--> 732         raise InvalidSchema("No connection adapters were found for {!r}".format(url))
733
734     def close(self):
InvalidSchema: No connection adapters were found for "['https://airtable.com/shrSodX4SH7WDXeBS']"

Wait, did you include square bracket around the url?

since there are no more activity, I'll mark it as Close for now


Azure function in image is named as Azure Stack

At point #6, the Azure function is named as Azure Stack. Can you rename to Azure function?
[Enter feedback here]

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: aff2e7d6-9d81-3745-1e2e-c4ce9d9565ca
Version Independent ID: d9b29d19-4325-e4ff-b15b-51dace1c11af
Content: Tier applications and data for analytics - Azure Solution Ideas
Content Source: docs/solution-ideas/articles/tiered-data-for-analytics.yml
Service: architecture-center
Sub-service: solution-idea
GitHub Login: @EdPrice-MSFT
Microsoft Alias: pnp

@OneBlueBird Thanks for the feedback! We are currently investigating and will update you shortly.

We've got this logged. Thank you!

It appears this has been addressed.  If you are still experiencing an issue, please re-open with additional information.


Spirit Hunter's Spirit Enchantment does not function until level 4

Expected behaviour: Spirit Enchantment increases weapon enhancement bonus by 1 in all cases
Actual behaviour: Spirit Enchantment only increases weapon enhancement bonus if the weapon already has a bonus from being at least level 4

Thanks for the report! I have an idea what's going on. I haven't been actively developing but I'm planning to return when the next DLC drops and fix any outsanding issues.

A note to myself: to fix this I need to grant WeaponAttunement at level 1. I think the calculator is already correct so it should be safe to just apply.


fix docs formatting

📚 Documentation
At this moment the docs raise about 140 warnings which may cause some issues in documentation.
To build the docs locally follow these steps:
cd docs
pip install -r requirements.txt
make html

Thanks!

Nice to meet you! I'm Hanbyul Kim, a ML engineer working in South Korea.
I'd like to contribute to ML open source for the first time, so I've been looking around several projects lately. And I found your repo and felt it's cool.
Since I'm a newcomer, I guess it can be a good idea to start doing some "good first issue" things. I'm polishing your code to remove warnings while generating docs. I've just pushed my branch right now, but I think I need more time to do all the things. Is it ok for me to do this job?

@hanbyul-kim super, your help is very welcome! Sure you can take your time to finish the cleaning... The best way would be creating PR with WIP tag so we may help or give an advice...
Moreover we may get in touch on Slack =)

Thank you for welcoming me. I’ll do my best.


Build broken on git master

It seems the culprit is changeset e1ae63dd990e5d74a500d4051f0a8a1dd5fbae9b. See https://travis-ci.org/cython/cython/builds/318500666
@scoder

Hmm, sorry, yes. That changeset should just get reverted. I'll do it tonight.

Maybe the cause of all the wheel builds failing ?
https://travis-ci.org/MacPython/cython-wheels/builds/319238051

Fixed by reverting the commit.


Search is broken. Only return matches for file open on editor.

VSCode Version: Code - Insiders 1.11.0-insider (c995ecf2e94a00aa7eda81ed842e787b999e83ae, 2017-03-27T06:11:09.027Z)
OS Version: Windows_NT ia32 6.1.7601
Extensions:

Extension
Author
Version

cucumberautocomplete
alexkrechik
2.0.1

gitignore
codezombiech
0.5.0

gitlens
eamodio
2.12.2

tslint
eg2
0.8.1

vscode-npm-script
eg2
0.1.8

beautify
HookyQR
1.0.2

debugger-for-chrome
msjsdiag
2.7.0

vscode-docker
PeterJausovec
0.0.13

Steps to Reproduce:
1.ctrl + shift+ F
2. enter a string you know has matches in a file in the current folder but the file is not open in the editor.
3. the search results in no matches

Can you update to the  new insider's build, and try again?

Just did. It's good now. Thanks
On Mar 28, 2017 11:09 AM, "Rob Lourens"<EMAIL_ADDRESS>wrote:

Can you update to the new insider's build, and try again?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/Microsoft/vscode/issues/23430#issuecomment-289801237,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AY7tKSh8G4VY5kZcI1VnSpqNHRSIllqtks5rqSKngaJpZM4Mrql_
.


[engine feature request] Support custom cursor engine API

Hi,
I have tryimg write a custom cursor plugin now(https://github.com/imiskolee/flutter_custom_cursor), its work fine on macOS, but can not work on windows, I thinking it's related to windows message WM_SETCURSOR.
https://docs.microsoft.com/en-us/windows/win32/menurc/wm-setcursor
When I SetCursor from outside, it's worked once, but when we move the pointer, the cursor will be back to basic, theWM_SETCURSOR message will be triggered if the pointer moved,and the engine set cursor again.
https://github.com/flutter/engine/blob/main/shell/platform/windows/window_win32.cc#L364.
so, can we has a public  engine API for plugin to manually set cursor?
somethings like:
void FlutterWindowWin32::UpdateFlutterCursor(HCURSOR cursor) {
  current_cursor_ = cursor;
}

Originally posted by @imiskolee in https://github.com/flutter/flutter/issues/31952#issuecomment-1056115014

@dkwingsmt  already opened the issue for talk about it.

I am thinking engine just handle system mouse cursors is better, and let plugin to supports more cursor behaviors is better.

Hi! I'm glad you're writing a plugin for mouse cursors! (I've been wanting to implement it for a long time...)
I think it's a good change. Can you go ahead and make a PR?

Actually, I'm curious whether and how you managed to create image cursors out of images on Windows, because when I read the documentation on Win32 related to cursors, the only API I find is to define them using .ico files (and only from assets? I don't remember clearly for now.)

current, i used .cur format and SetCursor API, it's easy to convert png to .cur with generate.
I will start the PR now.

any updates on this? it's really a project blocker.

@iamEvara are you still waiting? We are also blocked here. Have you tried to modify the engine?

any updates here? it's really an important bugfix-like feature for uniforming cursors' behaviors on all desktop platforms. See https://github.com/flutter/engine/pull/36143 for details, which gives a complete solution for this.


Viewing angle

When the NPC is idle, don't aggro when the player is not within a certain viewing angle.
Default viewing angle should be 360.0 degrees.
This stat should specifically be used only when the encounter hasn't been started yet.
native FCNPC_SetAggroViewingAngle(npcid, playerid, Float:angle);
native Float:FCNPC_GetAggroViewingAngle(npcid, playerid);

Need a vertical viewing angle as well.


Localize "Images" page to Italian

I would like to have the "Images" page to be translated into Italian
page https://kubernetes.io/docs/concepts/containers/images/
/language it
/priority important-soon

/remove-priority important-soon
/priority important-longterm

/remove-lifecycle stale

/remove-lifecycle stale

/remove-lifecycle stale

Hello guys, may I translate it and auto-assign this issue? @fabriziopandini

/assign

/lifecycle active

/close


Renamed SDK to toolkit

Description
SDK has been renamed toolkit.
Related issues
Addresses [issue #].
Testing
Describe how this change was tested.
Reviewer Checklist

[x] Update the title of the PR to be succinct and less than 50 characters
[x] Tag the PR with the type of update: Bug, New-Sample, Enhancement, or New-Feature
[x] CI is green before merge

Looks great, thanks so much @vmsuthar56


⚠ Fortnite has degraded performance

In 4cf97b9, Fortnite (https://fortnite.com) experienced degraded performance:

HTTP code: 200
Response time: 1487 ms

Resolved: Fortnite performance has improved in c08f02a.


[tool_crash] RPCError

Command
flutter run

Error
[ERROR:flutter/runtime/dart_isolate.cc(147)] Could not prepare isolate.
[ERROR:flutter/runtime/runtime_controller.cc(549)] Could not create root isolate.
[ERROR:flutter/shell/common/shell.cc(690)] Could not launch engine with configuration.

Kildly someone to look into this

Was soo often for the 2 days. Fixed the issue. Though. Thanks.

We would like to investigate and fix the root cause of the issue so if you experience this again, please file a new issue and provide detailed information.
Thank you


Pass cause from ObjectMapper to wrapper clients

Otherwise a developer has to debug to discover that e.g. No serializer found for class .... was the actual cause of The given string value: ... cannot be transformed to Json object.
Usually you want to know why it could not be transformed.

Thanks for the PR. I will review it when I have some time.

Applied. Thanks.

The 2.2.1 version contains your changes and I've just released it. It will soon be available on Maven Central. Thanks and enjoy using it.

Thanks mate!
On Sat, May 12, 2018 at 4:45 PM Vlad Mihalcea<EMAIL_ADDRESS>wrote:

The 2.2.1 version contains your changes and I've just released it. It will
soon be available on Maven Central. Thanks and enjoy using it.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/vladmihalcea/hibernate-types/pull/31#issuecomment-388559972,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAnl_ygh6jU1RJ5FiG1dz_awhSL-QHQZks5txvWRgaJpZM4T7OT2
.


Parent hash isn't validated

Metadata

Hardfork: frontier

What was wrong?
The spec doesn't ensure that the parent_hash of the current block matches the hash of the previous block.
Sources

parentHash: The Keccak 256-bit hash of the parent block’s header, in its entirety; formally Hp.

source

Does this also mean additional test will have to be added, since the existing tests weren’t able to identify the issue? (I see all existing tests pass)

@gurukamath adding a python test would be great!

@SamWilsn Added the relevant asserts to the state transition tests


Unexpected error comes out when everything is ok

This is ok and no error comes out.
    return (

        I'm {this.state.name}

    )

when I add any character after this like:
    return (

        I'm {this.state.name} any string here

    )

and the error comes out:
Expected indentation of 8 space characters but found 1. (react/jsx-indent)

my eslint config :
module.exports = {
  root: true,
  parser: 'babel-eslint',
  parserOptions: {
    sourceType: 'module',
    ecmaFeatures: {
      jsx: true
    }
  },
  env: {
    es6: true,
    commonjs: true,
    browser: true,
  },
  extends: [
      'standard',
      'standard-react'
  ],
  plugins: [
    'react',
    'babel',
    'promise'
  ],
  'rules': {
    'arrow-parens': 0,
    'generator-star-spacing': 0,
    'no-debugger': process.env.NODE_ENV === 'production' ? 2 : 0
  }
}

Is it a bug or something wrong with my config?

There was a bug in eslint-plugin-react (released in a patch version) for the last 2 days. It was just fixed. Try removing your node_modules folder and reinstalling. It should work!


Add support for passing model args via hf hub config

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.


Ansible 2.15.2 and failing to run.

[WARNING]: Collection prometheus.prometheus does not support Ansible version
2.15.2
Running 0.5.1 release of the collection.  Is it looking at the minor version of Ansible and failing?

You are correct there was a secondary syntax issue after cleaning that up I just get the warning message.  Thanks!
I will close this out.


Proposed fix 2/2 for #388

This proposed fix for #388 involves replicating the bare minimum logic from math.js.
Advantages:

Minimal changes to json-editor

Disadvantages:

Uncertain if this method of including derivative code properly follows the licensing restrictions
Copy/Paste Programming is ugly

This proposed fix does not work for e.g. 3.57 multipleOf 0.01. PR #436 now works for it, though.


Bugfix: hasRequestFailed method only works to mark successful requests as failures

In a project, I am using an API call that returns a list of items. If there are no items, then the API call (sadly) returns a 404 response. That means I don't want an exception in that case, but just an empty collection. In my Saloon request, I implemented the hasRequestFailed method and I ensured it returns false when the response has status 404:
class SomeRequest extends Request
{
    public function hasRequestFailed(Response $response): ?bool
    {
        return ($response->serverError() || $response->clientError()) && $response->status() !== 404;
    }
}

When I test this code, Saloon still throws an exception when I get a 404 response. That is because the current implementation of the hasRequestFailed method only allows you to mark successful requests as failures, but not the other way around. This PR fixes that.

Hey @gdebrauwer thank you for posting this - apologies I’ve been really busy this week so I’ll try and look next week for you 👍

Don't worry about the code style workflow

I'm currently on vacation until the beginning of December so I will only have time to update the PR in a couple weeks.

Howdy @gdebrauwer are you still happy to have a look at this PR when you get a chance? Thanks.

Howdy @gdebrauwer are you still happy to have a look at this PR when you get a chance? Thanks.

I added a comment to your PR review question.

Sorry for the delay @gdebrauwer I have blocked out Sunday to go through these things so I will check again then.

I have made a PR #371 which should fix everything proposed - the only thing I didn't implement is the change in the way responses are thrown. I think you may have misunderstood how shouldThrowRequestException works. That method is only used if you call the $response->throw() method or if you are using AlwaysThrowOnError.


Feat/#59 movement feedback

Purpose of this PR:
This PR implements the feedback given by Design.

The used camera system is overhauled to make use of the Cinemachine Camera Orbital Follow preset.
Allows for double jumping in the falling state, and moved the glide key to 'C'.
Transformed the input handling in the FallingState, might be a good idea to implement this system into the Player BaseState instead.
Changed movement's AddForce ForceMode to Acceleration (was Force)

How to test:
All of these changes are contained within the Player prefab.
Therefore this can be tested in any scene that has the Player prefab.
Move around, jump, fall, double jump, have fun
If the player doesnt react to your inputs on first try. You can try replacing the player already in scene with a new instance of the prefab.
links:
closes #59
Design already has access to these changes on their level-design branch, and they are testing with it.
They also know they can change values in the editor.

After further inspection the first might be an issue with laptop trackpad more so than the character logic.
The sercond problem can be addressed later I suppose. Perhaps it's best to merge this now because we need it for the scene so I'll approve it and merge it.


Privacy sniffs

Is your feature request related to a problem?
Currently it is hard for users to find out what plugins and themes are doing with regards to data before installing. There is currently an optional ‘privacy’ section in the readme that gets put in the ‘Privacy policy postbox’. Part of this is currently described in 'Suggesting text for the site privacy policy'.
Describe the solution you’d like
We’d like to check for functions that interact with third parties or load content from external domains. Examples of this would be wp_enqueue_script() with an external domain, or wp_remote_post() which might contain user data. If we detect usage of these functions, we should check if wp_add_privacy_policy_content() is being called, or maybe if the readme contains a privacy policy section.

@NielsdeBlaauw Interesting idea.
I'm not sure PHPCS is the best tooling for something like this as - based on your description - this would require scanning things cross-file and looking for what's missing.
I'd be interested to hear more about how you envision this.

Maybe we can use some of the following (peusdo code down below):
Sniff registers for functions that can call external resources.
External resource call found:
    $this->phpcsFile->config['must_have_privacy_policy'] = true;
    $this->config->reporters->add( PrivacyReport )

Sniff registers for wp_add_privacy_policy_content() or any alternative
Found:
    $this->phpcsFile->config['has_privacy_policy'] = true;

PrivacyReport:
Check config->has_privacy_policy = false  and config->must_have_privacy_policy = true.
    Output privacy policy warning

@NielsdeBlaauw Just checking if I understand you correctly - are you talking about using a custom PHPCS Report (like the existing Full, Info etc reports) ?

Yes, but that is going to be a lot of work to be compatible with other reporting modes. If it is at all possible.
Maybe a better solution would be to add a PrivacyFile extending PHPCS' DummyFile and overwriting the getErrors method to check the config values during report generation.

@NielsdeBlaauw I've done some work on custom reports before and I'm not sure that's the way to go.
Reports get a notification when an error/warning is thrown and can then display those either per file and/or at the end of the run.
So they get triggered on errors/warnings thrown (but need errors/warnings to get triggered) and they can remember that information to summarize it at the end.
For an example of a custom report: https://github.com/jrfnl/QA-WP-Projects/blob/master/Reports/WPQA.php
By default, the Full report is shown and any other report is only shown when specifically requested, so we couldn't enforce the report to be shown to the user unless we overwrite the default report command-line setting which has other implications, especially when the WP standards are combined with other external standards via a custom ruleset.

Maybe a better solution would be to add a PrivacyFile extending PHPCS' DummyFile and overwriting the getErrors method to check the config values during report generation.

That would still require a custom report to be used.
Unless there is an urgent reason you want to work on this now, let's leave this to soak for a while.
There's plenty of other open issues which could do with some attention in the mean time.


Update version number

Updated version number from 1.0.0-beta.6 to 1.0.0-beta.10

@xixian73 : Thanks for your contribution! The author(s) have been notified to review your proposed change. @ddematheu

@xixian73 It looks like the live site has been updated since your pull request was opened; it now shows "version 1.0.0-beta.10", hence the merge conflict. We'll close the pull request as no longer needed. Thanks.
https://docs.microsoft.com/azure/communication-services/quickstarts/voice-video-calling/getting-started-with-calling?pivots=platform-web


wu_ros_tools: 0.2.6-1 in 'kinetic/distribution.yaml' [bloom]

Increasing version of package(s) in repository wu_ros_tools to 0.2.6-1:

upstream repository: https://github.com/DLu/wu_ros_tools
release repository: https://github.com/wu-robotics/wu_ros_tools.git
distro file: kinetic/distribution.yaml
bloom version: 0.9.3
previous version for package: 0.2.5-1

This pull request has been mentioned on ROS Discourse. There might be relevant details there:
https://discourse.ros.org/t/preparing-for-kinetic-sync-2020-04-01/13483/2

CI passed but isn't reporting back https://travis-ci.org/github/ros/rosdistro/builds/670166260?utm_source=github_status&utm_medium=notification


Sql aggregator does not work in native mode

During fixation of https://github.com/apache/camel-quarkus/issues/2623, I found an issue with sql aggregator in native mode.
Firstly, simple Caused by: java.lang.ClassNotFoundException: org.apache.camel.support.DefaultExchangeHolder is thrown.
Fix (by addition of ReflectiveClassBuildItem) uncovers more serious issue. Problem is happening during serialization/deserialization of DefaultExchangeHolder.
Caused by: com.oracle.svm.core.jdk.UnsupportedFeatureError: The offset of private java.lang.Exception org.apache.camel.support.DefaultExchangeHolder.exception is accessed without the field being first registered as unsafe accessed. Please register the field as unsafe accessed. You can do so with a reflection configuration that contains an entry for the field with the attribute "allowUnsafeAccess": true. Such a configuration file can be generated for you. Read BuildConfiguration.md and Reflection.md for details.
	at com.oracle.svm.core.util.VMError.unsupportedFeature(VMError.java:87)
	at jdk.internal.misc.Unsafe.objectFieldOffset(Unsafe.java:50)
	at java.io.ObjectStreamClass$FieldReflector.<init>(ObjectStreamClass.java:2002)
	at java.io.ObjectStreamClass.getReflector(ObjectStreamClass.java:2266)
	at java.io.ObjectStreamClass.<init>(ObjectStreamClass.java:534)
	at java.io.ObjectStreamClass.lookup(ObjectStreamClass.java:381)
	at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1135)
	at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:349)
	at org.apache.camel.processor.aggregate.jdbc.JdbcCamelCodec.encode(JdbcCamelCodec.java:98)

I've tried suggested solution and I've also tried to register class for serialization (using native approach https://github.com/oracle/graal/pull/2730), but it didn't help.

Enable test from https://github.com/apache/camel-quarkus/pull/2694, which is covering this issue.


Country Codes Selector have bad performance...

The list for choosing country codes is slow and sometimes don't receive touchs...
When using;

PhoneInputSelectorType.DROPDOWN, it takes about a second to open the list.
PhoneInputSelectorType.DIALOG, slow launch as above, also takes a while to load the content of list too, pausing while scrolling…
PhoneInputSelectorType.BOTTOM_SHEET have serious problems…

Tested on Samsung Galaxy A10, on Debug.
I think using a different method to load the elements, and using icon glyphs (like emoticons) instead of images will make a huge difference...

I'm experiencing this too.. For example, I think in PhoneInputSelectorType.DIALOG it would be better to use ListView.builder instead of Column.
Also, I think glyphs would be better too.

Hi thanks for raising this up again, I know it took to long to respond but I've made attempts to reduce the load up delay of this package. I tried using Glyphs via custom fonts and icon sets but glyphs in Flutter are monochrome and Flutter does not support SVG out of the box. I am opened to suggestions if you any idea how I can make this happen, then I'll try to work on replacing Column with ListView.builder. Thanks

I'm experiencing this too.. For example, I think in PhoneInputSelectorType.DIALOG it would be better to use ListView.builder instead of Column.

Wow I made a lot of difference

@natintosh Glad I could help :)
Regards the glyphs, you're right, I guess it will require comparing the performance between using current PNG images and using SVG with external library (for example - flutter_svg).

@natintosh You may use flag emoticons in Noto Color Emoji font: https://www.google.com/get/noto/help/emoji/flags/ You will need to translate two letter code of country to regional indicator equivalent, but that must be easy. Also, I was forked this repository, but unfortunately couldn't find time to finish implementing glyphs...

Still I can share my code which uses this fonts flag characters, if you want.

Hi @RVYA this would be great I will very much like to see that code. But not everyone would want to use emoticons. What I will do is to add an optional parameter to use emoticons.

I recently added an option to use emoji flags. Thanks everyone