Start analyze audio only after threshold (decibels)

Hi everybody. Do you know if it's possible starting analyze audio only when that audio exceeds a threshold (decibels)?

Hi @bobosette,
it’s possible. Though you may need to customise this package to do so if you’re willing to tinker a bit with Java and Swift.
you’re more than welcome to contribute to this plugin, if you’re willing to share of course.

I tried to make a sort of merge of the usability between noise_meter package ( it let me check if an audio source gets higher than a threshold) and tfl_audio. It works, but the result doesn't seem perfect to me. But I don't know if that is the right way to implement what I want to do. I have to record/listen audio for a long time (at least 4 hours) and inside this time window i have to check/analyze if the audio is similar to the model from gtm.
How would you do that?

I think it’s fine to use both packages, though I think you may experience a delay as these two may need time to communicate with one another. In my opinion, you could potentially reduce this delay if you directly modify the tflite audio package to detect decibels as well.
As for your next question, it’s possible to extend the recording length by adjusting the parameter numOfInferences in this package. Though, im not sure what you mean by check/analyse if the audio is similar to the model from gtm. Can you clarify for me?

I have to detect a sound (for example you can imagine a song theme (30 sec) which is going to be played a number of times) and i want to detect how many times that sound will be played.
I have already tried to play with numOfInferences parameter but i didn't understand, this parameter is about the time (seconds) of analys?

Explaination for numOfInference
Each model has a maximum recording length. For example, for GTM models, the maximum recording length is around 44032, which is around 1 to 2 seconds if your bufferSize is 22050.
What numOfInference does is that it will extend the recording by looping over the maximum recording length, as well as return multiple inferences. For example:
numOfInference = 3 will repeat the recording three times, so recording length will be (1 to 2 seconds) x 3 = (3 to 6 seconds). Also the model will output 3 results.
If you only want one output, but would also like to increase the length of the recording, you can lower your bufferSize. Though take care not to lower the bufferSize too much as this may negatively influence your inference accuracy.
I hope this answers your question.

Ok now I understood. And if I want to extend the recording/analysis to at
least 4 hours I have to change numOfInference from 1 to almost 14000, is it
correct?
Il Gio 18 Nov 2021, 10:27 Michael Nguyen @.***> ha
scritto:

Explaination for numOfInference
Each model has a maximum recording length. For example, for GTM models,
the maximum recording length is around 44032, which is around 1 to 2
seconds if your bufferSize is 22050.
What numOfInference does is that it will extend the recording by looping
over the maximum recording length, as well as return multiple inferences.
For example:
numOfInference = 3 will repeat the recording three times, so recording
length will be (1 to 2 seconds) x 3 = (3 to 6 seconds). Also the model will
output 3 results.
If you only want one output, but would also like to increase the length of
the recording, you can lower your bufferSize. Though take care not to lower
the bufferSize too much as this may negatively influence your inference
accuracy.
I hope this answers your question.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Caldarie/flutter_tflite_audio/issues/22#issuecomment-972685687,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADEWB4JSEQDRISZX6G2EM4DUMTBGPANCNFSM5IEVZCSA
.

That is correct. But if you want to stop precisely at 4 hours, I suggest that you set numOfInferences to a very high number like 99999. Then set a timer to four hours on flutter and call stopAudioRecognition. from the plugin

Yes. You're right. Thanks man. I will let you know
Il Gio 18 Nov 2021, 10:55 Michael Nguyen @.***> ha
scritto:

That is correct. But if you want to stop precisely at 4 hours, I suggest
that you set numOfInferences to a very high number like 99999. Then set a
timer to four hours on flutter and call stopAudioRecognition. from the
plugin
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Caldarie/flutter_tflite_audio/issues/22#issuecomment-972709062,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADEWB4KZMHJG7WFWSWSSEG3UMTESNANCNFSM5IEVZCSA
.

Just another little thing. Can I get the percentage of the result? I saw that on the console when there is a result it prints Raw Scores: [0.98527735, 0.003569071, 0.011153544]. Where can I get this list?

I have yet to implement that feature yet, but i will provide an update in the very near future.

Hi @Caldarie. I tried to set numOfInferences = 99999, first time I did the recognition everything worked fine, but if I stopped, and then I started again I got this error: java.lang.OutOfMemoryError: Failed to allocate a<PHONE_NUMBER> byte allocation with 6291456 free bytes and 298MB until OOM, target footprint 95760752, growth limit 402653184
E/AndroidRuntime(31894): 	at flutter.tflite_audio.TfliteAudioPlugin.record(TfliteAudioPlugin.java:398)
E/AndroidRuntime(31894): 	at flutter.tflite_audio.TfliteAudioPlugin.access$000(TfliteAudioPlugin.java:59)
E/AndroidRuntime(31894): 	at flutter.tflite_audio.TfliteAudioPlugin$3.run(TfliteAudioPlugin.java:383)
E/AndroidRuntime(31894): 	at java.lang.Thread.run(Thread.java:919)

@bobosette hanks for the detailed bug report.
I’ll investigate it soon and will let you know.

Maybe is that a way to delete the cache or something similar??
Cause it seems that when the service starts it allocates some memory. When
the service stops it should release it. Is it right?
Il Sab 20 Nov 2021, 03:55 Michael Nguyen @.***> ha
scritto:

@bobosette https://github.com/bobosette thanks for the detailed bug
report.
I’ll investigate it soon and will let you know.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Caldarie/flutter_tflite_audio/issues/22#issuecomment-974581718,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADEWB4JYZ675HSCF2GLQELLUM4E37ANCNFSM5IEVZCSA
.

Yes that is correct. It seems this is the offending code:
 int maxRecordingLength = recordingLength * numOfInferences;
 maxRecordingBuffer = new short[maxRecordingLength];

I'm thinking of using two arrays and swap it interchangeably when one array fills up, while resetting the other. Though, it's a bit difficult to implement.

I could try to do something. Where is that part of code??
Il Sab 20 Nov 2021, 09:18 Michael Nguyen @.***> ha
scritto:

Yes that is correct. It seems this is the offending code:
int maxRecordingLength = recordingLength * numOfInferences;
maxRecordingBuffer = new short[maxRecordingLength];
I'm thinking of using two arrays and swap it interchangeably when one
array fills up, while resetting the other. Though, it's a bit difficult to
implement.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Caldarie/flutter_tflite_audio/issues/22#issuecomment-974614567,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADEWB4ITSS2XZYGVUTHL2DDUM5KWTANCNFSM5IEVZCSA
.

Any help would be greatly appreciated.
As you could see, the code below is overly complex and needs to be replaced. (Ive been needing to do so but just havent had time)
Goal is to replace all of the code below, and set two recording buffer arrays. The reason being that should there be a spill over (e.g. 16200/16000 which leaves a remaining 200), the remaining will be appended to the second array. The first recording buffer will be reset once the audio is fed into the model.
 while (shouldContinue) {
            //Reads audio data and records it into redcordFrame
            int numberRead = record.read(recordingFrame, 0, recordingFrame.length);

            recordingBufferLock.lock();
            try {

                //Continue to append frame until it reaches recording length
                //(recordingOffset + numberRead < maxRecordingLength) prevent out of index when buffer exceeds max record length
                if(recordingOffset < recordingEnd && recordingOffset + numberRead < maxRecordingLength){

                    System.arraycopy(recordingFrame, 0, maxRecordingBuffer, recordingOffset, numberRead);
                    recordingOffset += numberRead;
                    Log.v(LOG_TAG, "recordingOffset: " + recordingOffset + "/" + maxRecordingLength);

                //When buffer reaches recording length inference starts. Resets inference loop
                } else if (recordingOffset >= recordingEnd && recordingOffset + numberRead < maxRecordingLength) {
                    Log.v(LOG_TAG, "Recording reached threshold");

                    System.arraycopy(recordingFrame, 0, maxRecordingBuffer, recordingOffset, numberRead);
                    recordingOffset += numberRead;
                    Log.v(LOG_TAG, "recordingOffset: " + recordingOffset + "/" + maxRecordingLength);

                    System.arraycopy(maxRecordingBuffer, recordingStart, recordingBuffer, 0, recordingLength);
                    startRecognition();

                    Log.v(LOG_TAG, "Creating new threshold");
                    recordingStart += recordingLength;
                    recordingEnd += recordingLength;

                //when buffer reaches max record length (recordingLength * numOfInference) start inference and stop recording
                } else if  (recordingOffset >= recordingEnd && recordingOffset + numberRead == maxRecordingLength){
                    Log.v(LOG_TAG, "Recording reached maximum threshold");

                    System.arraycopy(maxRecordingBuffer, recordingStart, recordingBuffer, 0, recordingLength);
                    startRecognition();

                    stopRecording();
                    lastInferenceRun = true;

                //when buffer exeeds max record length, trim and resize the buffer, append, and then start inference
                } else if(recordingOffset + numberRead > maxRecordingLength) {
                    Log.v(LOG_TAG, "Recording exceeded maximum threshold");

                    Log.v(LOG_TAG, "Trimming recording frame and appending to recordingLength");
                    int missingRecordingLength = maxRecordingLength - recordingOffset - 1;
                    short [] resizedRecordingFrame = Arrays.copyOf(recordingFrame, missingRecordingLength);
                    System.arraycopy(resizedRecordingFrame, 0, maxRecordingBuffer, recordingOffset, missingRecordingLength);
                    Log.v(LOG_TAG, "Recording trimmed and appended at length: " + missingRecordingLength + 1);
                    Log.v(LOG_TAG, "recordingOffset: " + (recordingOffset + missingRecordingLength + 1) + "/" + maxRecordingLength);

                    System.arraycopy(maxRecordingBuffer, recordingStart, recordingBuffer, 0, recordingLength);
                    startRecognition();

                    stopRecording();
                    lastInferenceRun = true;

                    //Used when theres only one inference, as the above if statements are used for multiple inferences
                    //Case occurs when buffersize is divisible to recording length
                    //Apends the last recordingFrame to recordingLength
                } else {

                    System.arraycopy(recordingFrame, 0, maxRecordingBuffer, recordingOffset, numberRead);
                    recordingOffset += numberRead;
                    Log.v(LOG_TAG, "recordingOffset: " + recordingOffset + "/" + maxRecordingLength);

                    System.arraycopy(maxRecordingBuffer, recordingStart, recordingBuffer, 0, recordingLength);
                    startRecognition();

                    stopRecording();
                    lastInferenceRun = true;
                }

Mmm ok. I will try to do something. But, check this use case scenario: I
start recording, 4/5 hours, then I stop and I navigate to another page
showing the results and I destroy the recording page. Then I rebuild
recording page and I instantiate again the TflAudio. In this case, the
cache should be free, or not?
Il Sab 20 Nov 2021, 09:36 Michael Nguyen @.***> ha
scritto:

Any help would be greatly appreciated.
As you could see, the code below is overly complex and needs to be
replaced. (Ive been needing to do so but just havent had time)
Goal is to replace all of the code below, and set two recording buffer
arrays. The reason being that should there be a spill over (e.g.
16200/16000 which leaves a remaining 200), the remaining will be appended
to the second array. The first recording buffer will be reset once the
audio is fed into the model.
while (shouldContinue) {
//Reads audio data and records it into redcordFrame
int numberRead = record.read(recordingFrame, 0, recordingFrame.length);
        recordingBufferLock.lock();
        try {

            //Continue to append frame until it reaches recording length
            //(recordingOffset + numberRead < maxRecordingLength) prevent out of index when buffer exceeds max record length
            if(recordingOffset < recordingEnd && recordingOffset + numberRead < maxRecordingLength){

                System.arraycopy(recordingFrame, 0, maxRecordingBuffer, recordingOffset, numberRead);
                recordingOffset += numberRead;
                Log.v(LOG_TAG, "recordingOffset: " + recordingOffset + "/" + maxRecordingLength);

            //When buffer reaches recording length inference starts. Resets inference loop
            } else if (recordingOffset >= recordingEnd && recordingOffset + numberRead < maxRecordingLength) {
                Log.v(LOG_TAG, "Recording reached threshold");

                System.arraycopy(recordingFrame, 0, maxRecordingBuffer, recordingOffset, numberRead);
                recordingOffset += numberRead;
                Log.v(LOG_TAG, "recordingOffset: " + recordingOffset + "/" + maxRecordingLength);

                System.arraycopy(maxRecordingBuffer, recordingStart, recordingBuffer, 0, recordingLength);
                startRecognition();

                Log.v(LOG_TAG, "Creating new threshold");
                recordingStart += recordingLength;
                recordingEnd += recordingLength;

            //when buffer reaches max record length (recordingLength * numOfInference) start inference and stop recording
            } else if  (recordingOffset >= recordingEnd && recordingOffset + numberRead == maxRecordingLength){
                Log.v(LOG_TAG, "Recording reached maximum threshold");

                System.arraycopy(maxRecordingBuffer, recordingStart, recordingBuffer, 0, recordingLength);
                startRecognition();

                stopRecording();
                lastInferenceRun = true;

            //when buffer exeeds max record length, trim and resize the buffer, append, and then start inference
            } else if(recordingOffset + numberRead > maxRecordingLength) {
                Log.v(LOG_TAG, "Recording exceeded maximum threshold");

                Log.v(LOG_TAG, "Trimming recording frame and appending to recordingLength");
                int missingRecordingLength = maxRecordingLength - recordingOffset - 1;
                short [] resizedRecordingFrame = Arrays.copyOf(recordingFrame, missingRecordingLength);
                System.arraycopy(resizedRecordingFrame, 0, maxRecordingBuffer, recordingOffset, missingRecordingLength);
                Log.v(LOG_TAG, "Recording trimmed and appended at length: " + missingRecordingLength + 1);
                Log.v(LOG_TAG, "recordingOffset: " + (recordingOffset + missingRecordingLength + 1) + "/" + maxRecordingLength);

                System.arraycopy(maxRecordingBuffer, recordingStart, recordingBuffer, 0, recordingLength);
                startRecognition();

                stopRecording();
                lastInferenceRun = true;

                //Used when theres only one inference, as the above if statements are used for multiple inferences
                //Case occurs when buffersize is divisible to recording length
                //Apends the last recordingFrame to recordingLength
            } else {

                System.arraycopy(recordingFrame, 0, maxRecordingBuffer, recordingOffset, numberRead);
                recordingOffset += numberRead;
                Log.v(LOG_TAG, "recordingOffset: " + recordingOffset + "/" + maxRecordingLength);

                System.arraycopy(maxRecordingBuffer, recordingStart, recordingBuffer, 0, recordingLength);
                startRecognition();

                stopRecording();
                lastInferenceRun = true;
            }

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Caldarie/flutter_tflite_audio/issues/22#issuecomment-974616349,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADEWB4MNDATYABYI2ODX5LLUM5MYZANCNFSM5IEVZCSA
.

I think cache should be free. I’d imagine cache would accumulate too much

Ok ok. I will try something and I will let you know. Thank you
Il Sab 20 Nov 2021, 09:48 Michael Nguyen @.***> ha
scritto:

I think cache should be free. I’d imagine cache would accumulate too much
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Caldarie/flutter_tflite_audio/issues/22#issuecomment-974617437,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADEWB4OM32CHNWPVTEO2C3LUM5OEHANCNFSM5IEVZCSA
.

here I am. Some updates. I tried to add on the android app manifest this line: android:largeHeap="true" so the application part of the example sounds like this:
<activity
And it works, without losing performance, despite of what this link says https://stackoverflow.com/questions/27396892/what-are-advantages-of-setting-largeheap-to-true. I know it's not a good practice, but till if you don't fix that, I can try.
Now, what can we do with ios???

Hi @bobosette,
many thanks for looking up a solution. Appreciate he assistance.
I had a look at the link you’ve provided and it does indeed help with our problem. As you said, there may be no issues now; but, I feel this may also cause further unexpected problems down the line. Particularly when not everyone has access to high performance devices, or are running multiple apps at the same time.

Yes, you're right, is not the solution. I'm doing many tried and it slows
down the animation and the performance. We have to try something different
Il giorno dom 21 nov 2021 alle ore 02:29 Michael Nguyen <
@.***> ha scritto:

Hi @bobosette https://github.com/bobosette,
many thanks for looking up a solution. Appreciate he assistance.
I had a look at the link you’ve provided and it does indeed help with our
problem. As you said, there may be no issues now; but, I feel this may also
cause further unexpected problems down the line. Particularly when not
everyone has access to high performance devices, or are running multiple
apps at the same time.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Caldarie/flutter_tflite_audio/issues/22#issuecomment-974737811,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADEWB4PD4K2LWE3GPLDQK4DUNBDPLANCNFSM5IEVZCSA
.

Another question: does the service work if the app is in background? I did
some tries but it seems not to work.
Il Dom 21 Nov 2021, 02:47 Roberto Bobo Prandin @.***>
ha scritto:

Yes, you're right, is not the solution. I'm doing many tried and it slows
down the animation and the performance. We have to try something different
Il giorno dom 21 nov 2021 alle ore 02:29 Michael Nguyen <
@.***> ha scritto:

Hi @bobosette https://github.com/bobosette,
many thanks for looking up a solution. Appreciate he assistance.
I had a look at the link you’ve provided and it does indeed help with our
problem. As you said, there may be no issues now; but, I feel this may also
cause further unexpected problems down the line. Particularly when not
everyone has access to high performance devices, or are running multiple
apps at the same time.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Caldarie/flutter_tflite_audio/issues/22#issuecomment-974737811,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADEWB4PD4K2LWE3GPLDQK4DUNBDPLANCNFSM5IEVZCSA
.

Unfortunately, I'm not knowledgeable with running the plugin in the background. Though, i think this is more of a flutter thing than the plugin in itself.
Good news is i found a solution in regards to and have successfully implemented in android. Once i sucesfully implemented the feature with outputting raw scores and upload the update, i will let you know.

Oh perfect. Thank you, please let me know cause I have to implement this feature in a project as soon as possible. But for ios is the service working?
A solution for the background could be tell flutter to force the app to stay in foreground, so the phone will not stanb by

For the iOS, i think there is no problem considering that array size don't need to be fixed.
The problem is only with java since arrays need to be fixed for performance.

Oh perfect. Thank you. Let me know when you update cause I need to
implement this in a project as soon as possible 😅.
For the background I'm reading these articles that maybe can help us:
https://pub.dev/packages/flutter_background
https://medium.com/vrt-digital-studio/flutter-workmanager-81e0cfbd6f6e
Il Dom 21 Nov 2021, 15:04 Michael Nguyen @.***> ha
scritto:

For the iOS, i think there is no problem considering that array size don't
need to be fixed.
The problem is only with java since arrays need to be fixed for
performance.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Caldarie/flutter_tflite_audio/issues/22#issuecomment-974823461,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADEWB4P2LHLJXSSI6XN2QXDUND35HANCNFSM5IEVZCSA
.

Hi @Caldarie . Any update??

I am currently testing for bugs.
Please be patient as I have been busy with other work lately.

Sorry man. I can perfectly understand. I'm also busy with a lot of projects.
Il Mar 23 Nov 2021, 13:02 Michael Nguyen @.***> ha
scritto:

I am currently testing for bugs.
Please be patient as I have been busy with other work lately.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Caldarie/flutter_tflite_audio/issues/22#issuecomment-976445495,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADEWB4N3ZXBOBO3FGSF77OTUNN7GJANCNFSM5IEVZCSA
.

hi @bobosette
Good news, i have published the new update on  dart pub
There should be no problems with memory crash on android.  Seems like memory was issue for iOS, so I also fixed iOS as well.
I also added the ability to output rawScores. Check the example to see how to turn on this feature.
I have done alot of testing, but I may have missed on or two bugs. If you have time, please let me know if you experience any.

Great @Caldarie. I tryed the update and the memory bug is fixed. Thank you. But, question: where can I get the rawScores?

For the background, when I'm debuggin I see from the console that the recognizer keep going also in background but when I come back in foreground it seems to doesn't update the count of relevation

Great @Caldarie. I tryed the update and the memory bug is fixed. Thank you. But, question: where can I get the rawScores?

    TfliteAudio.loadModel(
      model: 'assets/conv_actions_frozen.tflite',
      label: 'assets/conv_actions_label.txt',
      inputType: 'decodedWav',
      outputRawScores: true, //set this parameter to true
      numThreads: 1,
      isAsset: this.isAsset,
    );

For the background, when I'm debuggin I see from the console that the recognizer keep going also in background but when I come back in foreground it seems to doesn't update the count of relevation

I am not sure that i follow you. Are you asking as to how to keep the stream running in the background? If so, unfortunately i am not knowledgable in this area. Perhaps this link could give you some hint?

No I'm saying that the stream seems to running also in background, cause in
the IDE console I can see that it prints (for example rawScores) also when
the app 9s in background
Il Mer 24 Nov 2021, 09:44 Michael Nguyen @.***> ha
scritto:

Great @Caldarie https://github.com/Caldarie. I tryed the update and the
memory bug is fixed. Thank you. But, question: where can I get the
rawScores?
TfliteAudio.loadModel(
  model: 'assets/conv_actions_frozen.tflite',
  label: 'assets/conv_actions_label.txt',
  inputType: 'decodedWav',
  outputRawScores: true, //set this parameter to true
  numThreads: 1,
  isAsset: this.isAsset,
);

For the background, when I'm debuggin I see from the console that the
recognizer keep going also in background but when I come back in foreground
it seems to doesn't update the count of relevation
I am not sure that i follow you. Are you asking as to how to keep the
stream running in the background? If so, unfortunately i am not
knowledgable in this area. Perhaps this link
https://docs.flutter.dev/development/packages-and-plugins/background-processes
could give you some hint?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Caldarie/flutter_tflite_audio/issues/22#issuecomment-977656581,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADEWB4J2QJQPSSLOGWAA4KDUNSQYVANCNFSM5IEVZCSA
.

I see. The issue you are looking at seems to be related to the flutter background background. Honestly speaking, i am unsure how foreground and background processes work.
Can you try opening the issue over there?

Mmm I could try following this article https://medium.com/flutter/executing-dart-in-the-background-with-flutter-plugins-and-geofencing-2b3e40a1a124 . But you should tell me both for android and ios what methods/functions I should run in background

Hi @Caldarie . I tried the update. Works fine, no more memory issues. Thank you. Today I'm working on the background. I will let you know.

I’m glad to hear that.
if you have found that this project has been useful to you, please consider giving your support with a like or a star. Thankyou


Playerlist is not filled on plugin init

When the plugin init is called, the player list is still empty and the plugin init might not do some tasks if it needs that player list.

The player game class should load current players when booting. Will be added soon.


java.lang.IllegalAccessError: failed to access class software.amazon.smithy.model.shapes.NamedMembersShape$Builder

Reproduced in a scala REPL, but the same should happen in a regular java project with a dependency on smithy-model 1.22.0:

scala> :paste
// Entering paste mode (ctrl-D to finish)

import software.amazon.smithy.model.shapes.MemberShape
import software.amazon.smithy.model.shapes.StructureShape
import software.amazon.smithy.model.traits.ErrorTrait

val structMember = MemberShape
    .builder()
    .id("test#struct$testing")
    .target("smithy.api#Integer")
    .build()
val struct =
    StructureShape
    .builder()
    .id("test#struct")
    .addMember(structMember)
    .addTrait(new ErrorTrait("client"))
    .build()

// Exiting paste mode, now interpreting.

java.lang.IllegalAccessError: failed to access class software.amazon.smithy.model.shapes.NamedMembersShape$Builder from class $iw (software.amazon.smithy.model.shapes.NamedMembersShape$Builder is in unnamed module of loader sbt.internal.inc.classpath.ClasspathUtil$$anon$2 @2dacc87b; $iw is in unnamed module of loader scala.tools.nsc.interpreter.IMain$TranslatingClassLoader @30001570)
  ... 67 elided

But if I do:
scala> :paste
// Entering paste mode (ctrl-D to finish)

import software.amazon.smithy.model.shapes.MemberShape
import software.amazon.smithy.model.shapes.StructureShape
import software.amazon.smithy.model.traits.ErrorTrait

val structMember = MemberShape
    .builder()
    .id("test#struct$testing")
    .target("smithy.api#Integer")
    .build()
val struct =
    StructureShape
    .builder()
    .id("test#struct")
    .addTrait(new ErrorTrait("client"))
    .addMember(structMember)
    .build()

// Exiting paste mode, now interpreting.

import software.amazon.smithy.model.shapes.MemberShape
import software.amazon.smithy.model.shapes.StructureShape
import software.amazon.smithy.model.traits.ErrorTrait
val structMember: software.amazon.smithy.model.shapes.MemberShape = (member: `test#struct$testing`)
val struct: software.amazon.smithy.model.shapes.StructureShape = (structure: `test#struct`)

It works fine. The only difference is the order of the calls in the builder. If I move .addTrait(new ErrorTrait("client")) after .addMember(structMember), it fails.

I think this might be a Scala compiler issue based on https://github.com/lampepfl/dotty/issues/13841#issuecomment-1034139570.
This test case works fine in Java (17.0.3) in the smithy-model package:
    @Test
    public void testBuilderOrder() {
        MemberShape structMember = MemberShape
                .builder()
                .id("test#struct$testing")
                .target("smithy.api#Integer")
                .build();
        StructureShape struct = StructureShape
                .builder()
                .id("test#struct")
                .addMember(structMember)
                .addTrait(new ErrorTrait("client"))
                .build();
    }

I suppose we could flatten NamedMemberShape and NamedMemberShape.Builder into StructureShape and UnionShape. I'm not sure if that would work with Scala though.

Wow, I assumed it would not be an issue on scala's side without checking. Sorry for the noise.

Also hit this, for anyone seeing this before the upstream issue in Dotty is fixed, a workaround would be to explicitly upcast the return type of id / addMember to StructureShape.Builder.
For example (with a convenience extension method to avoid extra parentheses nesting:
extension [T](it: T) def upcast[U >: T]: U = it

def mkStruct(shapeId: ShapeId, members: List[MemberShape]) = StructureShape
  .builder()
  .members(members.asJava)
  .upcast[StructureShape.Builder]
  .id(shapeId)
  .upcast[StructureShape.Builder]
  .build()


FIX: History module fixes

Minor refactor of the alarm structure used by the history module. This mostly fixes the issues below.  We still have an "off by one error" with the monthly frequency.  Perhaps @yixinmao can fix that 😉.
closes #596, #606, #607
cc @yixinmao for review

@jhamman - please update branch

@jhamman : branch out of date. In addition, can you clarify whether the "off-by-one" error still exists for the months?

What's the difference between set_alarm and reset_alarm?

@jhamman: Minor changes - fix and merge

Thanks @bartnijssen.

What's the difference between set_alarm and reset_alarm?'

set_alarm is only called at initialization and sets the alarm settings (e.g. frequency).  The reset_alarm, resets the count to zero and determines when the alarm will go off again.


How to properly close a background child process in a shutdown event?

First check

[x] I added a very descriptive title to this issue.
[x] I used the GitHub search to find a similar issue and didn't find it.
[x] I searched the FastAPI documentation, with the integrated search.
[x] I already searched in Google "How to X in FastAPI" and didn't find any information.
[x] I already read and followed all the tutorial in the docs and didn't find an answer.
[x] I already checked if it is not related to FastAPI but to Pydantic.
[x] I already checked if it is not related to FastAPI but to Swagger UI.
[x] I already checked if it is not related to FastAPI but to ReDoc.
[x] After submitting this, I commit to one of:

Read open issues with questions until I find 2 issues where I can help someone and add a comment to help there.
I already hit the "watch" button in this repository to receive notifications and I commit to help at least 2 people that ask questions in the future.
Implement a Pull Request for a confirmed bug.

Example
Here's a self-contained, minimal, reproducible, example with my use case:

from multiprocessing import Process, Value

from fastapi import FastAPI

app = FastAPI()

terminate_flag = Value('h', 0)

def work_hard(terminate_flag: Value):
    while terminate_flag.value == 0:
        # CPU-bound calculations
        pass

@app.get('/hello-world')
def hello_world():
    return {
        'message': 'hello world'
    }

process = Process(target=work_hard, args=(terminate_flag,))

@app.on_event('startup')
def startup_event():
    process.start()

@app.on_event('shutdown')
def shutdown_event():
    terminate_flag.value = 1
    process.join()

Description
I have some CPU-bound calculations known as work_hard, which contains a flagged loop to control its flow.
The function is designed to be executed in a child process that starts within FastAPI startup event, and joins within FastAPI shutdown event.
However, it seems that the FastAPI shutdown event does not get executed while uvicorn is shutting down by the user pressing Ctrl + C.
It stuck at the log "Finished server process" and never really terminates.
I did try to look up for the standard practice of using multiprocessing module in FastAPI, but there are no concrete examples about it.
Concurrency and async/await section mentioned that "To see how to achieve this parallelism in production see the section about Deployment.", yet Deployment section has no such examples as well.
Thank you in advance!
Environment

OS: Ubuntu 18.04.4 TLS
FastAPI Version : 0.61.1
Python version: 3.6.8

Additional context
cliff@cliff-pc:~/repo/fastapi-multiprocessing$ uvicorn app.main:app
INFO:     Started server process [24032]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://<IP_ADDRESS>:8000 (Press CTRL+C to quit)
^CINFO:     Shutting down
INFO:     Finished server process [24032]

Looks very similar to this: https://stackoverflow.com/questions/58133694/graceful-shutdown-of-uvicorn-starlette-app-with-websockets.
I've tried both methods and neither seem to work, maybe you will have better luck:rocket:

How do you guys handle parallelism in your FastAPI application by the way?
Using multiprocessing is not a must for me, I just wanted an elegant, maintainable way to implement it.

I believe the way you do it is by having asynchronous code within FastAPI, and then you deploy with gunicorn workers which provide parallelism.
I am not sure about running multiprocessing within FastAPI itself.

You can scale using gunicorn and set a number of workers
Or wrap your code in a docker container , running a unique uvicorn process and let your docker-swarm| kubernetes | Any docker scheduler scale the number of containers|pods to run.

Also to respond to your question :
 FastAPI shutdown event does not get executed while uvicorn is shutting down by the user pressing Ctrl + C.
No it does

from fastapi import FastAPI
import uvicorn

async def on_shutdown():
    print("HERE")

if __name__ == '__main__':  # local dev
    web_app = FastAPI(docs_url="/", on_shutdown=[on_shutdown])
    uvicorn.run(web_app, host="<IP_ADDRESS>", port=8000)

give
INFO:     Started server process [10579]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://<IP_ADDRESS>:8000 (Press CTRL+C to quit)
^CINFO:     Shutting down
INFO:     Waiting for application shutdown.
HERE
INFO:     Application shutdown complete.
INFO:     Finished server process [10579]

@clifflau1120 You can apparently close this issue. Thank you

@raphaelauv . I have created a reproducible example where the same issue persists. This is a mini app which creates a central server where your webcam or other IP cameras can be visualized. The requirements are imutils, imagezmq, fastapi, opencv-python. I apologize for not being able to create a more standalone example due to lack of time. However, I think this should help in highlighting the issues raised in the ongoing thread. In order to reproduce kindly run the following :
# Start the server program
uvicorn fastapp:app --reload
# Visit the URL so that the server initializes itself for the client process.
# Run the client to send images to the server
python camera_client_0.py
# The mini app should be up and running and your webcam would be live streaming in the browser url.
# Try Ctrl + C ( twice)
# The terminal would get stuck at Finished server proces

SourceCode : exp.tar.gz
Thanks

@clifflau1120
Could you close this issue , thank you.

Hello, I have the terminal issue as well. I don't know whether this issue should be reopened or a new one created, let me know either way.
I can replicate this issue, and it's pretty annoying! The shutdown hook appears to fire intermittently, and only in the workers, but not in the parent process. I'm not sure whether it should fire in the parent, but it certainly shouldn't block the terminal. It only seems to occur for multiple workers though. See the code snippet below (adapted from the above example). Tweak workers from 1 to 2 to see it not working.
I think this only occurs on Windows, though I've tried cmd and git bash with the same effect (screenshot from git bash):
import sys

from fastapi import FastAPI
import uvicorn

async def on_shutdown():
    print("HERE")

app = FastAPI(docs_url="/", on_shutdown=[on_shutdown])

if __name__ == '__main__':
    workers = int(sys.argv[1])
    print(f"Running with {workers} worker(s)")
    uvicorn.run("main2:app", host="<IP_ADDRESS>", port=8001, workers=workers)

Hello, I have the terminal issue as well. I don't know whether this issue should be reopened or a new one created, let me know either way.
I can replicate this issue, and it's pretty annoying! The shutdown hook appears to fire intermittently, and only in the workers, but not in the parent process. I'm not sure whether it should fire in the parent, but it certainly shouldn't block the terminal. It only seems to occur for multiple workers though. See the code snippet below (adapted from the above example). Tweak workers from 1 to 2 to see it not working.
I think this only occurs on Windows, though I've tried cmd and git bash with the same effect (screenshot from git bash):
import sys

from fastapi import FastAPI
import uvicorn

async def on_shutdown():
    print("HERE")

app = FastAPI(docs_url="/", on_shutdown=[on_shutdown])

if __name__ == '__main__':
    workers = int(sys.argv[1])
    print(f"Running with {workers} worker(s)")
    uvicorn.run("main2:app", host="<IP_ADDRESS>", port=8001, workers=workers)

It seems that there is no simple solution to this.
At the end, I broke the workers into standalone programs and communicate with them via sockets.
Certainly not the best way doing it, but it solves my problem.

Thanks for the help and discussion here everyone and thanks @clifflau1120 for coming back to close the issue. ☕

Sorry for the long delay! 🙈  I wanted to personally address each issue/PR and they piled up through time, but now I'm checking each one in order.


GULP ERROR IS THROWN WHEN EXECUTE COMMAND GULP

/home/ubuntu/.nvm/versions/node/v8.4.0/lib/node_modules/gulp/bin/gulp.js:129
gulpInst.start.apply(gulpInst, toRun);
^
TypeError: Cannot read property 'apply' of undefined
at /home/ubuntu/.nvm/versions/node/v8.4.0/lib/node_modules/gulp/bin/gulp.js:129:19
at _combinedTickCallback (internal/process/next_tick.js:131:7)
at process._tickCallback (internal/process/next_tick.js:180:9)
at Function.Module.runMain (module.js:611:11)
at startup (bootstrap_node.js:158:16)
at bootstrap_node.js:598:3

This project is using gulp 4, which is not compatible with gulp 3, which is what you have installed globally. After you install the dependencies, you can use our npm scripts to run gulp. instead of running it directly, which will run the local install. npm run develop is equivalent to gulp. there is also npm run build and npm run test.
If you are curious, here is a blog post that discusses the specifics of this setup and the reasoning behind it.


Error: A slave with the same server_uuid/server_id as this slave has connected to the master;

Hi @rodrigogs,
Has been working great!
But looking at the log error I saw a new one:

'A slave with the same server_uuid/server_id as this slave has connected to the master; the first event 'mysql-bin.000089' at 43535345, the last event read from '/mysql/binlog/mysql-bin.000089' at 56994447, the last byte read from '/mysql/binlog/mysql-bin.000089' at 56994447.',
sqlState: 'HY000' }

I have 2 databases (Production, Development )
And have 2 listeners that gives this error
My Code Dev
const mysql = require('mysql');
var MySQLEvents = require("@rodrigogs/mysql-events");
 const schemaName ='nms';

My Code Prod
const mysql = require('mysql');
var MySQLEvents = require("@rodrigogs/mysql-events");
 const schemaName ='nmsProd';

Same code for both
const db_config = { host: '<IP_ADDRESS>', user:XXX', password: XXX', database: schemaName, };

let connectDB = mysql.createConnection(db_config);
let instance = new MySQLEvents(connectDB, {
    startAtEnd: true,
    excludedSchemas: {
        mysql: true
    },
});
startTrigers();

function handleDisconnect() {
    console.log('handleDisconnect');
    connectDB.destroy();
    connectDB = mysql.createConnection(db_config);
    connectDB.connect(function (err) {
        if (err) {
            console.log(' Error when connecting to db  (DBERR001):', err);
            setTimeout(handleDisconnect, 1000);
        }
        else{
            // startTrigers();
        }
    });
}

instance.on(MySQLEvents.EVENTS.CONNECTION_ERROR, (err) => {
    console.log('Connection is asleep (time to wake it up): ', err);
    setTimeout(handleDisconnect, 1000);
    handleDisconnect();
});

instance.on(MySQLEvents.EVENTS.ZONGJI_ERROR, console.error);

function startTrigers() {

    instance.start();

    instance.addTrigger({
        name: 'ALERT_INSERTED',
        expression: schemaName + '.nms_alert',
        statement: MySQLEvents.STATEMENTS.INSERT,
        onEvent: (event) => {
            console.log("", new Date());
            console.log("==== ALERT INSERTED  ===");
            sendAlert(event);
        },
    });

Do I have to change something?

This is probably a MySQL error related to the binlog state. I don't think it will stop the app from working properly tho.
Do you think you missed any event due to this?

Yes, I think is working ok.
I am restarting the app every day …
So not sure ...

On 29 Aug 2019, at 19:44, Rodrigo Gomes da Silva<EMAIL_ADDRESS>wrote:
This is probably a MySQL error related to the binlog state. I don't think it will stop the app from working properly tho.
Do you think you missed any event due to this?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/rodrigogs/mysql-events/issues/21?email_source=notifications&email_token=AAYZAHWMO3X63V6BYOBOGBLQHADI5A5CNFSM4ISBEJI2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5PIZ2Y#issuecomment-526290155, or mute the thread https://github.com/notifications/unsubscribe-auth/AAYZAHQZR2XFM7MU4R3O3CLQHADI5ANCNFSM4ISBEJIQ.

Same problem for me.

In my case, the error occurred when restart the triggers (your "startTrigers" method). For a moment it create a conflict with the precendet instance (or something of similar...). But the real problem is a disconnection from the server that, in my case, happens every 8 hours (my server is an amazonAWS one). If I force to restart the entire process when an CONNECTION_ERROR occurred, I don't get the topic error.
Try to invoke instance.stop() before call startTrigers:
instance.stop().then(()=>{
     startTrigers()
})

Sorry for not being able to help. I'm deeply involved in some COVID-19 urgent projects and it's sucking my time to the bone.

Hopefully someone has a solution, otherwise this library is useless after the first time it disconnects.  Looks like the control connection gets disconnected after the wait_timeout but the binlog connection remains.  Try to pass it a new connection as above returns the "slave with same server_id" error.

Unfortunately, I've not managed to get instance.stop() to work in any iteration, I always end up with 'UnhandledPromiseRejectionWarning: Error: Cannot enqueue Quit after fatal error.'

please don't flame me, i'm really new with nodejs, i don't know exactly what i'm doing.
But to solve this problem I assigned a random value, (more precisely the current date) so I know it will never have the same value.

  const instance = new MySQLEvents(connection, {
    startAtEnd: true,
    serverId: Date.now()
  });

Hi, I have this error before, my environment was a master and some slave database, there is replication from slave-db to master-db and if I using mysql-events plugin to connect from another vm, it get error (A slave with the same server_uuid/server_id as this slave has connected....)
So finally I am creating another replicate user on master-db with REPLICATION_SLAVE privileges, so both from slave-db and mysql-events can connect to the same master-db (but using different mysql user)


LineChartRenderer context bug

In the LineChartRenderers drawLinear method the Context gets saved before checking for existing data.
This leads to never restoring the context, which causes other chart-components to disappear.
This commit fixes this by saving the context after the check for data.
The bug described above can be seen in the two GIFs below:
Behaviour with bug:

Behaviour with fix:

Relates to #740

This is the correct fix :-) (Also note that there could have been use of defer which avoids the need to  avoid return calls...

defer would have executed the code after the fill function gets called, which could lead to other problems.


⚠ Processors has degraded performance

In 26b41ed, Processors ($STATUS_URL) experienced degraded performance:

HTTP code: 200
Response time: 93 ms

Resolved: Processors performance has improved in 8927016.


suite: move installer configuration to balenaOS configuration section

Change-type: patch

Rebased into https://github.com/balena-os/meta-balena/pull/3126/commits

lgtm


Bump JavaScriptKit dependency to 0.13.0

@MaxDesiatov this is the release that contains the fix for --debug-info param, isn't it?

Yes, all the latest code from the main branch of the JSKit repo.

Shouldn't we tag this release so the docker image and the binaries are available for users as version 0.13.0? I don't know if you do this as part of this PR or you wait until is merged. Asking just in case @MaxDesiatov 😃  Thanks for the release btw.

Sure, the release will be tagged after https://github.com/swiftwasm/carton/pull/305 is merged. #306 is just a dependency update PR.

Great thanks for the update! I see it is already approved so I guess it is a matter of time 😃

Merged and tagged now. 0.13.0 Docker image should be ready as well. Due to some incompatibilities with Tokamak I'm seeing, I think I'll wait for SwiftWasm 5.6 release to fix this before distributing latest carton on Homebrew and widely announcing it. But I hope it works well for you in the meantime.

Thanks @MaxDesiatov 😃


Paket restore fails with .netcoreapp3.1 on macOS

Description
paket restore and paket update fails with .NET Core 3.1 on macOS. It has previously worked with .NET Core 3.0.
Output from paket restore:
> paket restore
Paket version 5.215.0
Performance:
 - Runtime: 184 milliseconds
Paket failed with
-> Error in paket.dependencies line 4
     Could not parse framework 'netcoreapp3.1'. Try to update or install again or report a paket bug.

Output from paket update:
> paket update
Paket version 5.215.0
Performance:
 - Runtime: 478 milliseconds
Paket failed with
-> Error in paket.dependencies line 3
     Could not parse framework 'netcoreapp3.1'. Try to update or install again or report a paket bug.

Output from paket install -v:
Paket version 5.215.0
found: /Users/Repos/psint-catalog-api/paket.dependencies
Parsing /Users/Repos/psint-catalog-api/paket.dependencies
Performance:
 - Runtime: 444 milliseconds
Paket failed with
-> Error in paket.dependencies line 4
     Could not parse framework 'netcoreapp3.1'. Try to update or install again or report a paket bug.
   StackTrace:
       at<EMAIL_ADDRESS>(System.String message) [0x00000] in <34a07b69c40c4371a1c422a46fffa0ff>:0
       at Paket.DependenciesFileParser.parseLine$cont@493 (System.String fileName, System.Boolean checkDuplicates, Microsoft.FSharp.Collections.FSharpList`1[T] state, System.Int32 lineNo, System.String line, Microsoft.FSharp.Core.Unit unitVar) [0x00102] in <34a07b69c40c4371a1c422a46fffa0ff>:0
       at Paket.DependenciesFileParser.parseLine (System.String fileName, System.Boolean checkDuplicates, System.Int32 lineNo, Microsoft.FSharp.Collections.FSharpList`1[T] state, System.String line) [0x00021] in <34a07b69c40c4371a1c422a46fffa0ff>:0
       at Microsoft.FSharp.Collections.ArrayModule.Fold[T,TState] (Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] folder, TState state, T[] array) [0x00023] in <34a07b69c40c4371a1c422a46fffa0ff>:0
       at Paket.DependenciesFileParser.parseDependenciesFile (System.String fileName, System.Boolean checkDuplicates, System.String[] lines) [0x00023] in <34a07b69c40c4371a1c422a46fffa0ff>:0
       at Paket.DependenciesFile.ReadFromFile (System.String fileName) [0x0002e] in <34a07b69c40c4371a1c422a46fffa0ff>:0
       at<EMAIL_ADDRESS>(Microsoft.FSharp.Core.Unit unitVar0) [0x00001] in <34a07b69c40c4371a1c422a46fffa0ff>:0
       at Paket.Utils.RunInLockedAccessMode[a] (System.String lockedFolder, Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] action) [0x00077] in <34a07b69c40c4371a1c422a46fffa0ff>:0
       at Paket.Dependencies.Install (Paket.InstallerOptions options) [0x0001d] in <34a07b69c40c4371a1c422a46fffa0ff>:0
       at Paket.Dependencies.Install (System.Boolean force, System.Boolean withBindingRedirects, System.Boolean cleanBindingRedirects, System.Boolean createNewBindingFiles, System.Boolean onlyReferenced, Paket.SemVerUpdateMode semVerUpdateMode, System.Boolean touchAffectedRefs, System.Boolean generateLoadScripts, Microsoft.FSharp.Collections.FSharpList`1[T] providedFrameworks, Microsoft.FSharp.Collections.FSharpList`1[T] providedScriptTypes, Microsoft.FSharp.Core.FSharpOption`1[T] alternativeProjectRoot) [0x0006e] in <34a07b69c40c4371a1c422a46fffa0ff>:0
       at Paket.Program.install (Argu.ParseResults`1[Template] results) [0x00a38] in <34a07b69c40c4371a1c422a46fffa0ff>:0
       at<EMAIL_ADDRESS>(Argu.ParseResults`1[Template] results) [0x00001] in <34a07b69c40c4371a1c422a46fffa0ff>:0
       at Paket.Program.processWithValidationEx$cont@42[a] (System.Boolean silent, Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] commandF, a result, Microsoft.FSharp.Core.Unit unitVar) [0x00002] in <34a07b69c40c4371a1c422a46fffa0ff>:0
       at Paket.Program.processWithValidationEx[a] (Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] printUsage, System.Boolean silent, Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] validateF, Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] commandF, a result) [0x00067] in <34a07b69c40c4371a1c422a46fffa0ff>:0
       at Paket.Program.processWithValidation[T] (System.Boolean silent, Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] validateF, Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] commandF, Argu.ParseResults`1[Template] result) [0x00006] in <34a07b69c40c4371a1c422a46fffa0ff>:0
       at Paket.Program.processCommand[a] (System.Boolean silent, Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] commandF, Argu.ParseResults`1[Template] result) [0x00007] in <34a07b69c40c4371a1c422a46fffa0ff>:0
       at Paket.Program.handleCommand (System.Boolean silent, Paket.Commands+Command command) [0x001a0] in <34a07b69c40c4371a1c422a46fffa0ff>:0
       at Paket.Program.main () [0x0041e] in <34a07b69c40c4371a1c422a46fffa0ff>:0

paket.dependecies looking like this;
source https://www.myget.org/...
source https://api.nuget.org/v3/index.json

framework: netcoreapp3.1, netstandard2.0
storage: none

nuget Microsoft.ApplicationInsights.AspNetCore
nuget Microsoft.AspNetCore.TestHost
nuget Microsoft.AspNetCore.Cors
nuget Microsoft.AspNetCore.HttpsPolicy
nuget Microsoft.AspNetCore.Server.Kestrel
github battermann/fsharp-hypermedia src/Hypermedia/Hypermedia.fs
nuget Microsoft.Azure.Storage.Blob
nuget xunit
nuget xunit.runner.visualstudio
nuget Giraffe
nuget Microsoft.Extensions.Http
nuget Microsoft.Extensions.Hosting

dotnet --info is looking like this:
.NET Core SDK (reflecting any global.json):
 Version:   3.1.100
 Commit:    cd82f021f4

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  10.14
 OS Platform: Darwin
 RID:         osx.10.14-x64
 Base Path:   /usr/local/share/dotnet/sdk/3.1.100/

Host (useful for support):
  Version: 3.1.0
  Commit:  65f04fb6db

.NET Core SDKs installed:
  2.2.300 [/usr/local/share/dotnet/sdk]
  2.2.401 [/usr/local/share/dotnet/sdk]
  2.2.402 [/usr/local/share/dotnet/sdk]
  3.0.100 [/usr/local/share/dotnet/sdk]
  3.1.100 [/usr/local/share/dotnet/sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.All 2.2.5 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.6 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.7 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.2.5 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.6 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.7 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.0.0 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.0 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.1 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.2.5 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.6 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.7 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.0.0 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.0 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]

To install additional .NET Core runtimes or SDKs:
  https://aka.ms/dotnet-download

I tried to install and uninstall the runtime and SDK for version 3.1.0, but with no luck. I get this error on another project with .NET Core 3.1 as well. My colleagues with Windows have successfully updated there. I have not had the opportunity to try this on another Mac.
Repro steps
Get a project with .NET Core App 3.1 with SDK and Runtime 3.1. Get paket and run paket update or paket restore on a Mac.

Not sure about Op, but I was having the same problem and updating to 5.242.2 fixed this error for me.


Expression for energy gradient in hf/dft

Hello there,
I'm trying to read the code to determine which analytical expression is used for the energy gradients for dft. I have the impression that the bulk of the work happens in GradientsBase. But I don't know all the methods well enough to work out the formulas that have been programmed (in reasonable time; say in less than a few days).
So it would be helpful if you could provide the equation that is being used to calculate the forces for dft. I'd like to use pyscf for my project, but I need to understand how the forces are calculated.
Thanks for reading

you may refer to this paper https://doi.org/10.1063/1.464906

Thanks for your message. Is it equation 9 that you have programmed in there for the dft energy gradient?

The program implementation does not strictly follow a single literature source. It has been extensively optimized based on a slower initial implementation, which you can find https://github.com/pyscf/pyscf/blob/3ca5a34288fca6217177a094000cfc1eb51dd67f/pyscf/grad/test/test_tdrhf_grad.py#L31. You can refer to the paper to understand the equations, and then follow the slow implementation to see how it is translated into the PySCF code.


Update MinimalLib with a version that exposes get_mcs() through MolIterator and returns a JSMol as MCS

update MinimalLib with a version that exposes get_mcs() through MolIterator and returns a JSMol as MCS
update RDKit API TS definitions

@MariaDolotova This PR fixes the issue with TS definitions.
Regarding the issue with molblock, that's expected. Molblock are not a good format for query atoms/bonds - many RDKit queries do not exactly translate to MDL queries.
A better format to serialize RDKit query is SMARTS (through RDMol.get_smarts()), but also that is not guaranteed to exactly recover all query features in the original molecule. If you need to exactly serialize an RDKit molecule to recover it later, use RDMol.get_as_uint8array() (to serialize) and then RDMol.get_mol_from_uint8array() (to deserialize).

Hi @ptosco ! Can you please check the issue with AtomCompare and BondCompare parameters. Changing the parameters doesn't affect the final result. For example if you run the following code you will get the same mcs for all cases - [#8]=[#6]1-[#6]-[#7]=#6-[#6]1-[#6]-[#6]-[#6]-[#6]-[#6]-1
const csv = smiles
O=C1CN=C(c2ccccc2N1)C3CCCCC3
CN1C(=O)CN=C(c2ccccc12)C3CCCCC3
CCCCN1C(=O)CN=C(c2ccccc12)C3CCCCC3
CC(C)CCN1C(=O)CN=C(c2ccccc12)C3CCCCC3
O=C1CN=C(c2ccccc2N1CC3CCCCC3)C4CCCCC4
CN1C(=O)CN=C(c2cc(Cl)ccc12)C3CCCCC3`
const t = DG.DataFrame.fromCsv(csv);
console.log((await getMCS(t.col('smiles')!, true ,true))?.get_smarts());
console.log((await getMCS(t.col('smiles')!, false, false))?.get_smarts());
console.log((await getMCS(t.col('smiles')!, true, false))?.get_smarts());
console.log((await getMCS(t.col('smiles')!, false, true))?.get_smarts());`
It seems that get_mcs always uses the default values for atoms and bond search types no matter which parameters are passed. I tried to look through C++ implementation and couldn't find what could go wrong there. Looks like everything should work as expexted but the results prove otherwise.
And one more question - how do you think, maybe it makes sense to return smarts string instead of RDMol object from get_mcs? If I understood correctly from your previous comment, neither get_molblock() nor get_smiles() are not expected to return correct results. But in case we return RDMol object we need to additionaly handle it in typescript code (I mean delete). Or are there reasons why it is better to return RDMol instead of just smarts string?
@StLeonidas , @skalkin also FYI.


HDDS-11711. Add SCM metrics for delete commands sent and response received per datanode

What changes were proposed in this pull request?
When there are many DNs, and if any DN is slow in processing. It is difficult to identify which DN is having issue unless we check the log deeply and can take long time. For fast debug in this PR, metrics are added in SCM per datanode to show how many delete command sent to datanode and how many response received or timedout. This will help in understand which DN is slow or having problem in much efficient way.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-11711
How was this patch tested?
Updated test in TestBlockDeletion

@sumitagrawl @ashishkumar50 could you please help in reviewing this patch?

@adoroszlai thanks for the review, I have addressed all the comments

Thanks @Tejaskriya for updating the patch.
findbugs complains:
M M LI: Incorrect lazy initialization of static field org.apache.hadoop.hdds.scm.block.ScmBlockDeletingServiceMetrics.instance in org.apache.hadoop.hdds.scm.block.ScmBlockDeletingServiceMetrics.create()  At ScmBlockDeletingServiceMetrics.java:[lines 110-112]

but create() is not being changed in this PR, so this is a flaky check in findbugs.  (Not the first one we encounter.)  We still need to fix it, otherwise it will keep complaining on master if this is merged.

thanks @adoroszlai , I have fixed it with making the method synchronized

@Tejaskriya I have a question: The number of blocks deleted by each DN when processing a delete command is inconsistent, which can be observed by enabling the DN’s debug logs. If there is no performance difference between DNs, the varying number of blocks to be deleted could still result in differences in deletion speed. So, how do we determine if there is a difference in deletion speed of a DN, based solely on whether the delete command succeeds or not?

@slfan1989 Although we can't have a direct comparison of the values for the different datanodes due to the reasons you have mentioned, we can still get useful information from the comparison between how many commands are being sent and how many are successful. If deletion is indeed stuck at the datanode level due to some issues with DN not responding/executing the commands, It can be identified using these metrics.
Moreover, we can get a picture of how the deletion commands are being distributed against the different datanodes

@ashishkumar50 @sumitagrawl thank you for the reviews, I have addressed the comments


fix(protractor): Shows locator used when a timeout occur.

Fixes #2996

Signed it!

@sjelin modified the parameter sent to waitForAngular() and now the description is only added to error message when is a locator.
Failed: Timed out waiting for Protractor to synchronize with the page after 11 seconds. Please see https://github.com/angular/protractor/blob/master/docs/faq.md      - Locator: by.binding("slowAngularTimeoutPromiseStatus").      The following tasks were pending:      - $timeout: function () {           // intentionally empty         }

closed with b6f1a63da77ec88a3f487e5a099df8febe3742aa

For the future - changes like this are not a 'fix', since it's not fixing a bug. I would call it a 'feat' or 'chore'.

got it @juliemr , thanks.


Better-looking UI for viewer

This is being put under our Q4 goal to create an easily installable FF extension.

@vingtetun @andreas or I may be able to cajole a FF UI person into making us a mockup.  Remind one of us when this gets near the front of someone's priority queue.

we got a UX team! we should be hearing from @clarkbw and his team soon


Add docker for Shears

Reordered the working directory call to simplify the other lines and added --rm to the example docker run command to prevent excessive container creation. PTAL


Typo

Please add example: how to use this module with create-react-app-ts?
(Needed install PostCSS?)


Add Visual Separation Between Header,Feature, and Pricing Sections

I would like to improve the website's design by adding clear visual separation between different sections like the header, features, and pricing sections. Kindly assign this issue to me under Hacktoberfest and GSSoC Ext 24.

Go ahead and include screenshots make sure to keep it responsive and in similar design style


IOS  component not work

hello,
All is in the title :)
Thx

It maybe react-native or react version problem.
Works fine in "react": "16.0.0-beta.5", "react-native": "~0.49.3".
Try to check example project's package.json and compare your project version.


Limitations due to code not being on the core

Recently started some attempts with boot hacking to setup a probe for the various modes of use, because running the code through 0: is limited by the connection and bad that way.  Launch services is okay, but beyond that this needs fixed.
Thinking next step, will be to reduce the number of boot scripts, and instead have a boot script that compiles required and dependent files to the core.  Then overwrites the boot file.  lv scripts will need to be overhauled to list dependencies.  Mission files will become mission directories with an init.ks listing the files to compile.

Resolved with the spacemanspiff update


makefile breakout of components

Currently the k8s step can be ran seperatly as make aws_k8s, let the CP components also be able to be brought in one at a time if needed.
This allows to expand/modify parts of the setup without needing to touch other infra thats already in place

I think this is possible with the current makefile. Reopen if that's not the case.


StorageCluster: make readability enhancement

Signed-off-by: Raghavendra Talur<EMAIL_ADDRESS>

/hold

/retest

/retest

@raghavendra-talur could you please add more information about the changes in the description  or Commit message ?

/unhold

@rohantmp  @jarrpa @anubhav-here I have rebased on master now. If you believe this change is not worth it, I am willing to close the PR.

/test images


Updated get started documentation

Category

[x] Content fix
[ ] New article

Related issues
What's in this Pull Request?
Updated docs referencing /sharepoint/debug which doesn't exist. Actual directory created after gulp package-solution is /sharepoint/solution/debug.

No files submitted in this PR.


PR: Correct indentation folding cases

Description of Changes

[ ] Wrote at least one-line docstrings (for any new functions)
[ ] Added unit test(s) covering the changes (if testable)

[x] Included a screenshot or animation (if affecting the UI, see Licecap)

Add cases for the identation folding using a regular expression
Issue(s) Resolved

Fixes #9434
Affirmation
By submitting this Pull Request or typing my (user)name below,
I affirm the Developer Certificate of Origin
with respect to all commits and content included in this PR,
and understand I am releasing the same under Spyder's MIT (Expat) license.

I certify the above statement is true and correct: steff456

Hello @steff456! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

In the file spyder/plugins/editor/api/folding.py:

Line 32:69: W291 trailing whitespace


dropdown popup is hidden by section title in docs

[x] I have searched the Issues to see if this bug has already been reported
[x] I have tested the latest version

Steps to reproduce

Go to https://flowbite-react.com/docs/components/dropdown#dropdown-items-with-icon
Scroll up and make the button is close to the end of the page
Click on the dropdown button

Current behavior

Describe what is currently happening and why it's a problem.
Expected behavior
Dropdwon menu shouldn't be hidden by the title
Context
What are you trying to accomplish? Does this only happen on a specific browser, screen size, or operating system?
If possible, provide a live example URL, screenshot, video, or a repository with the minimal reproduction of the issue.

nice catch!
we could either solve this by isolating the code demo example in an iframe (which is already implemented mechanism) and give it fixed height, or try to figure out further why it is behind.

@SutuSebastian, can't we just increase the z-index?

@SutuSebastian, can't we just increase the z-index?

Or we can use a nice little tailwindcss class called isolate which it does exactly what it says, it isolates references

@SutuSebastian, can't we just increase the z-index?

Or we can use a nice little tailwindcss class called isolate which it does exactly what it says, it isolates references

isolate only works with static position, and H2 components use relative position.


Support for Date type

$lt, $lte, $gt and $gte work on numbers and strings

Does it mean these operators will not work with Date field?
Also sort on field with type Date does not seem to work.

I am new using this library. I have an example where dates are working as expected:
This filters the objects properly:
db.find({ date: { $gte: new Date('2004-05-21T09:00:00.000+0000') } }).exec(function (err, docs) {}
This sorts all objects as expected:
db.find({}).sort({ date: -1 }).exec(function (err, docs) {}

@nuvolarmobileteam not sure, I tried same and it did not sort properly. I resorted to saving date in unix timestamp as integer.

For now I would advise you to look for this problem in the nedb library, this package is a clone of it.

Looks like Date() is the way to go: https://github.com/louischatriot/nedb/issues/383


GH-2344: AggReplyingKT Support Custom Correlation

Resolves https://github.com/spring-projects/spring-kafka/issues/2344
The ReplyingKafkaTemplate supports the use of a custom header name
for correlation, but the subclass AggregatingReplyingKafkaTemplate
was hard coded to use the default header.
Support the use of a custom header name in the subclass.
cherry-pick to 2.9.x, 2.8.x

... and cherry-picked to 2.9.x & 2.8.x


use the term "bundling" instead of "batching"

Coverage remained the same at 83.661% when pulling 82283ba38a36d65e86d1f67d97b39743a1406f04 on pongad:batch-to-bundle into daaf0617b2905b6a610e1f28085e73d7bb8603bc on GoogleCloudPlatform:pubsub-hp.

LGTM - let @davidtorres take a look too .


Event model is broken and poorly documented

How to reproduce
Consider the leaflet 1.0 map of https://playground-leaflet.rhcloud.com/qal/edit?html,console,output
Below is a table of the console output when clicking the circle marker for two different versions of the playground code. The first version runs in the playground itself, the second version is the code of the playground page as an isolated html file. The developer tools of both Chromium and Firefox include a mobile device simulator ("responsive design" with simulated touch events in Firefox).

.
playground
standalone

Chromium
mousedown
mousedown

Chromium - device simulator
mousedown
nothing

Firefox
mousedown
mousedown

Firefox - responsive design
mousedown twice
mousedown twice

Expected behavior
The double mousedown event in Firefox when simulating touch is probably a result of simulation, but still undesirable. The lack of any output with the Chromium device simulator in standalone mode surely must be a bug.
The events in the Leaflet Evented class are synthetic. Regardless of the input method, events will be Mouse Events (mousemove, mousedown, mouseup, click). A more method agnostic scheme is available in the form of pointer events. Conceptually, pointermove, pointerdown, pointerup would be better names for synthetic events since they do not imply a specific input method.
Documentation
Apart from the synthetic events fired by the Evented class, there is the DomEvent class which operates on Dom events. Although Mouse Events can be captured through this interface too, an important distinction is that these events are not (or at least less) synthetic. It is not documented that a mousedown event in the Evented class is aggregated from multiple input methods, whereas the event by the same name in the DomEvent class is not aggregated, i.e. strictly separated from, say, touchdown.
Resolution
The 'correct' solution will break backward compatibility and is therefore not ideal. It consists of renaming all Evented events to pointer events, fixing the behavior in the above table and discouraging the use of the DomEvent class for end users. The DomEvent class is not flexible as, for example, it is not easy to obtain the container of a Vector layer to bind a DomEvent to. Additionally, more detailed documentation of event processing in Leaflet would be welcome.
Playground Code
<!DOCTYPE html>

<link<EMAIL_ADDRESS>rel="stylesheet" type="text/css" />

So I've read through this, and my takeaway is:

Mouse clicks or touches in Chromium while using the "device toolbar" are not recognised
Mouse clicks or touches in Firefox in "responsive design mode"
Reporter did not specify OS of browser versions where this might be reproduced (a piece of information which is incredibly important to have since Chromium 55 implemented native PointerEvents)
Reporter is annoyed at the L.Evented logic
Reporter hasn't read the bit of the documentation that says that a click event might happen because of a touchscreen tap, not a mouse click and thinks that all L.Eventeds can fire clicks.
Reporter thinks that PointerEvents are a silver bullet to the problems previously exposed.

There are problems with this bug report.
Besides the lack of browser versions, there is no good analysis. What native DOM events happen in the device simulators? Sure you can throw a L.DomEvent.on(map.getcontainer(), 'touchdown pointerdown mousedown', ev=>console.log(ev)) in a playground and provide useful information.
This should be a separate bug report, providing as much information as possible.
Instead of an analysis, we've got a leap of faith. How can the reporter be so sure that calling the Leaflet synthetic event pointerdown instead of mousedown will solve the problem???
Making the interactive layers and the map fire pointer* events should be a separate feature request, specifying the pros and cons of polyfilling mouse/touch events with the missing info when appropriate, and pros/cons of existing pointer polyfills.
Also:

Additionally, more detailed documentation of event processing in Leaflet would be welcome.

Documentation is wrong? No problem, just create a separate issue or, better even, create a pull request to improve it (after reading the contributing guide).
You've taken the time to write a lengthy bug report with your ideas, but unfortunately we cannot accept this in its current form. Please take a bit of time to read "How To Ask Questions The Smart Way" and "Reporting Bugs Effectively", make a better bug report, and consider sending PRs for documentation improvements (or even in-length tutorials).

I am well aware of all the documentation you have linked to but thought a tracking bug for the state of event handling, which is known to be near impossible to get right, would be nice. I would have liked to have found something such as this report before having to find things out through reading code. I apologize for the title of the report, which may be a bit too opinionated.
I am disappointed by the ad hominem response and will probably not pursue this any further with PRs or whatnot. Environment is Firefox 50 / Chromium 55 on Linux.


Poor performance on image and video

We are trying to use AdaptiveCards intergrating bot service on app, but when displaying images or videos is too slow to render. And will these images be cached? How can I use AdaptiveCards on UITableViewCell to dynamically change the height?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: cf1d3ae5-2446-137e-3684-a5a34413d4a7
Version Independent ID: c81bd83a-1064-3bb9-ccc1-c1354431f7fd
Content: iOS SDK - Adaptive Cards
Content Source: AdaptiveCards/sdk/rendering-cards/ios/getting-started.md
Product: adaptive-cards
GitHub Login: @matthidinger
Microsoft Alias: mahiding

Hi there, I assume you're using the iOS renderer? Can you include an exam JSON paypoad to help us debig? What size are the images and videos, and does the performance hit happen as they are appearing/disappearing (perhaps in a scroll view of some kind) or something else?

Hi @matthidinger , I'm using sample JSON file on AdaptiveCards Samples, I try to intergrate all samples json to a single page not in a scrollview, but it takes too long to render ImageGallery, Input Form, etc...
But I aslo try to render AdaptiveCards on UITableViewCell, actually rendering performance is worse. How to solve the problem of slow rendering, and how to change the style of some of the controls(such as video).


[Feature] Add 'TAG' options

Add 'TAG' options

Thank you for your time contributing to this repo!
Would your use case be served by this suggestion: https://github.com/leisim/logger/issues/68#issuecomment-786637080

OK，but I think 'PrefixPrinter' is global. In fact, I only add some FLAGs for some modules.

The idea is to have a separate logger for each 'tag' or module.

Thanks,  I misunderstand.


location: Add location subsystem and API

The rational behind this API and accompanying subsystem
is to define a generic format and API for getting data
from any driver which is capable of supplying such data,
whether it is as a primary or secondary function of the
device the driver is written for.
A device with the primary purpose of providing location
data could be a GPS or GNSS sensor/modem. These devices
should implement the location API as part of their own
API, then register themselves to the location subsystem
as well.
A device with the secondary purpose of providing location
data could be a cellular modem, using triangulation, or
WIFI location. These devices only implement the API in
their drivers, and register to the location subsystem.
The application can then get a list of all registered
location providers and iterate through them, looking for
the best fitting location data. They can also get a ptr
to the underlying device instance to perform other
functions like resume/suspend etc.
The specific data in question is:
The position of the device in latitude and longitude, in
the unit microdegrees, from, -180000000 to 18000000.
A single microdegree is approx. 11 cm along the earths surface.
The bearing of the device movement relative to True North in
millidegrees.
The speed of the device relative to ground in millimeters pr
second
The altitude relative to the mean sea level in millimeters.
The 4 different parameters are all accompanied by the time
the data was created, along with a well defined accuracy
measurement. This allows the application to directly compare
the parameters, even though they may be from different device
types.
The overarching goal of adding this subsystem is to create a
base from which to create an API for GNSS devices, and add
cellular triangulation to existing and future modem drivers.
Signed-off-by: Bjarki AA<EMAIL_ADDRESS>

I only have experience with GNSS module that outputs NMEA sentences and Quectel cellular location, some initial suggestions that I have, based on my custom location interface in my application:
1. Sometimes it takes a while for the GNSS to get a fix, so I had a timeout arg for my `get_location` API call (When the GNSS location provider took too long to get a fix, I will try to get a fix from the cellular location provider)

2. Probably something to notify the app that a fix is available asynchronously, though callback/signal?

There will definitely be a callback the application can register to, and an event providers can call to indicate they have data ready. This callback would most likely return a pointer to the location provider with the updated data.
To indicate that the data is not ready, the driver implementing the location API can simply return an error, like -ENODATA, or setting the uptime to 0 or -1, but i prefer the error code :)

There will definitely be a callback the application can register to, and an event providers can call to indicate they have data ready. This callback would most likely return a pointer to the location provider with the updated data.

This sounds good

To indicate that the data is not ready, the driver implementing the location API can simply return an error, like -ENODATA, or setting the uptime to 0 or -1, but i prefer the error code :)

IMO async notification/ get data with timeout is better than polling

cc @sslupsky @jukkar @allen-k-liu @jtguggedal

There will definitely be a callback the application can register to, and an event providers can call to indicate they have data ready. This callback would most likely return a pointer to the location provider with the updated data.

This sounds good

To indicate that the data is not ready, the driver implementing the location API can simply return an error, like -ENODATA, or setting the uptime to 0 or -1, but i prefer the error code :)

IMO async notification/ get data with timeout is better than polling
I think both direct calls and callbacks should be available. I can begin work on the callback upon new data available from specific provider today, should hopefully only take hours to implement and test. And i will add returning -ENODATA in case data is not available, but the API is supported to the location API descriptions

To be clear This is not a GNSS API :D It is a foundation for it. The GNSS API should use this subsystem and implement the API. The GNSS driver will provide a lot more data specific to the GNSS itself, alongside the location data, like satelite count etc. and possibly a NMEA passthrough.

Would this be an alternative to GH-30564 or orthogonal? Also would this open a door to a A-GPS/SUPL client?

Would this be an alternative to GH-30564 or orthogonal? Also would this open a door to a A-GPS/SUPL client?

This comment could answer the first question https://github.com/zephyrproject-rtos/zephyr/pull/46660#issuecomment-1159939607
Regarding A-GPS/SUPL
The following code snipped shows how a driver could implement A-GPS/SUPL like functionality (naively) using the location subsys.
/* Location event handler */
void gnss_on_location_event(const struct location_provider *provider, union location_event event,
            void *user_data)
{
    /* Variables */
    struct location_position pos;
    const struct device *dev = (const struct device *)user_data;

    /* Get position from first provider */
    location_position_get(provider, &pos);

    /* Send A-GNSS data to gnss*/
    gnss_send_a_gnss_data(dev, &pos);
}

/* GNSS Init */
int gnss_init(const struct device *dev)
{
    /* Register to location events */
    location_event_handler_register(gnss_on_location_event, (void*)dev);

    /* Start looking for fix */
    gnss_pm_resume(dev);
}

The location subsystem is deliberately designed to offload processing of location data to the application and drivers, which allows both for drivers to implement SUPL like functionality directly like in this example, and for a compliant SUPL server/client to be created on the application side. In an embedded system however, it seems unnecessarily complicated to create a compliant SUPL server/client if the location subsys is sufficient.

The Location API and subsystem is now... the Location API
The design using a subsystem, which was inspired by the
socket offloading and network iface APIs/subsystems has
been rejected.
The new and improved method for giving devices multiple
APIs, like a cellular modem, which primarily uses
the net_if API but often also has a built in flash file
system, and location services. To enable these secondary
services, and interact with them from the application,
the supported services will be exposed as child nodes in
the device tree, represented by struct devices.
The primary node, which could be the cellular modem,
handles the interactions between the sub nodes.
These interactions could be power management and
synchronizing interactions with the sub nodes.
An example for a cellular modem which provides three
APIs, one for the net_if API, one for the UART itself
for things like firmware updating, or application side
drivers, and one for the location API
&uart0: uart {
    compatible = "uart,driver";
    modem: modem {
        compatible = "modem,parent";

        modem_net_if: modem_net_if {
            compatible = "modem,sub,netif";
        };

        modem_serial_if: modem_serial_if {
            compatible = "modem,sub,serialif";
        };

        modem_ffs: modem_ffs {
            compatible = "modem,sub,ffs";
        };

        modem_location: modem_location {
            compatible = "modem,sub,location";
        };
    };
};

A GNSS modem would be very similar, it would also
include some API for real time for example. The
main takeaway is that the possibilites are endless.
Modems like cellular and GNSS provide are complete
subsystems, providing an ever expanding set of
features, this approach allows for dynamically
including these features without creating new
ever expanding APIs.

I am not sure if the API should make a reference to WGS-84. Yes, the GNSS position from any receiver is typically output in WGS-84, but this is not true if some kind of correction (SBAS, RTCM) is used. In such case, the correction provider essentially dictates the used reference frame. This is maybe somewhat nitpicking since the different coordinate systems are anyway quite close to each other, but it's something to be aware of.

Would you suggest we use another standard? By stating that WGS-84 is the reference, any driver not using this reference would have to offset its own reference to fit with the WGS-84 reference.

True, but I am not sure if this is what is going to happen in practice because the coordinate transformation is not trivial. Other option would be to document it like "the coordinates will be output in whatever reference system your GNSS device uses". This would of course require the user to have some knowledge about what kind of GNSS receiver there is (assuming he is interested or even knowledgeable about the reference frames in the first place).

I suggest we discuss this proposal in the dev-review meeting with @mbolivar-nordic and @galak.

@bjarki-trackunit @carlescufi If its of any help to you, I posted various out-of-tree things I built for zephyr here:  https://github.com/sslupsky/zephyr-out-of-tree
Included there is my gnss api and a driver for the ublox_m8.  The approach I used, as you pointed out earlier, is to treat the GNSS receiver as a sensor.  There was already some "plumbing" in zephyr for this and unfortunately there was very little feedback when I posted the RFC a couple years ago.  So this is how I proceeded.

I updated the description to state that the reference is device specific. I fail to find any better solution than the one you proposed :)

Well, I have made a mistake it seems, closing this pull request and starting fresh... Sorry for the inconvenience

Do you mind putting a link to your next iteration here when it is ready? I would like to keep up to date with this

Do you mind putting a link to your next iteration here when it is started? I would like to keep up to date with this

Absolutely :)

There should be no need to create a new PR, just force push you changes to this PR.

Here is a link to the new PR https://github.com/zephyrproject-rtos/zephyr/pull/48536


Merge all config files into pyproject.toml

cc @AntonyMilneQB

Partially addressed by gh-8, only tox.ini left.

Actually, tox support for pyproject.toml requires you dumping the whole thing to a multiline string... https://tox.wiki/en/latest/config.html#pyproject-toml
Won't do. Closing!


LightClientIniter should store checkpoints not finalized checkpoint

Describe the bug
LightClientIniter assumes the finalized state is in the cache. That's a big strech and will break in long non-finality
https://github.com/ChainSafe/lodestar/blob/6cf7c2c3f3df6012bd4ce773ab3b5310c347e884/packages/lodestar/src/chain/lightClient/LightClientIniter.ts#L97
Intead, store all checkpoints, or all checkpoints after N slots (i.e. 32) to guard against short lived forks. Then store by checkpoint (epoch | block-root) in reverse for fast pruning and update the init proof route to have the full checkpoint as parameter

Fixed with https://github.com/ChainSafe/lodestar/pull/3461


Feature/format contracts prettier plugin solidity

Resolves #839

enforce prettier-plugin-solidity for packages/contracts solidity files
improve format:all
upgrade prettier-plugin-solidity to 1.2.0

Plus fix prettier output for IUbiquityGovernance.sol

Please, provide QA, which pattern is following?

Please, provide QA, which pattern is following?

@molecula451 sure, please check the QA demo below: I am changing one contract, one test file, two places each and prettier is fixing the layout during pre-commit stage.
https://github.com/ubiquity/ubiquity-dollar/assets/88761781/26869f36-93a0-44c5-9361-0663c1437ea6

@molecula451 sure, please check the QA demo below: I am changing one contract, one test file, two places each and prettier is fixing the layout during pre-commit stage.
prettier_qa_demo.mp4

That's crazy to me that you're using vim without syntax highlighting.

Merging with 2 approvals.

All right, though this one was already closed. Fortunately you should be able to easily rebase your branch on top of development as it does not affect existing code much and should not change anything in your pull request.

no worries, but the best ideal is to have everyones feedback before it gets merged, unless! and unless! the implementation is absolutely clear

no worries, but the best ideal is to have everyones feedback before it gets merged, unless! and unless! the implementation is absolutely clear to everyones eyes


[Battery Optimizer] Failure to set charging limit MacOS 15.0

Extension
https://www.raycast.com/Qetesh/battery-optimizer
Raycast Version
1.82.5
macOS Version
15.0
Description
The extension does not work on MacOS 15.0
I do not want to pressure anyone or say that it is not working on the release day of a new OS version, i thought, that may my screenshot and report could help others, that they're not alone with the problem i encountered.

Steps To Reproduce

update to latest MacOs version
set battery charging limit

Current Behaviour

Expected Behaviour
it should set the pre set charging max amout to 80%

Yes, I was able to reproduce this problem. 😢
I could have a look unless if @Qetesh already has an idea how to do it? Unfortunately, since this is a side thing, I must find time to do it... :(

@raycastbot assign me
@raycastbot keep this issue open

@raycastbot rename this issue to "[Battery Optimizer] MacOS 15 compatibility"
as this is not a one off problem with setting the charge limit. None of the three commands are working...

+1 having the same issue here with MacOS 15.

Unfortunately the developer can't fix this easily, because it relies on BCLM, where the issue originates: https://github.com/zackelia/bclm/issues/49. Until that tool gets updated, it won't be fixed here...

+1  having the same issue .


Respond with authentication scheme when client authentication fails

As per section <IP_ADDRESS>.  Error Response:

"invalid_client":  Client authentication failed (e.g., unknown
client, no client authentication included, or unsupported
authentication method).  The authorization server MAY return an
HTTP 401 (Unauthorized) status code to indicate which HTTP
authentication schemes are supported.  If the client attempted
to authenticate via the "Authorization" request header field,
the authorization server MUST respond with an HTTP 401
(Unauthorized) status code and include the "WWW-Authenticate"
response header field matching the authentication scheme used
by the client.

We should respond with the required authentication scheme when a client fails authentication.

Hi @jgrandja , the latest version of OAuth 2.1 is draft-ietf-oauth-v2-1-04, so perhaps you could update the link address in the original post to this: <IP_ADDRESS>. Error Response

Hi @jgrandja i can work on it, in which version is it planned? Thank you!


mv: Show 'skipped' when a file isn't overwriten

Should fix tests/mv/mv-n.sh

@cakebaker @tertsdiepraam
Any idea what is going on?
--- TRY 3 STDERR:        coreutils::tests test_mv::test_mv_arg_interactive_skipped_vin ---
thread 'test_mv::test_mv_arg_interactive_skipped_vin' panicked at 'called `Result::unwrap()` on an `Err` value: Custom { kind: Other, error: "failed to write to stdin of child: Broken pipe (os error 32)" }', tests/common/util.rs:2004:18
stack backtrace:
   0: rust_begin_unwind
             at /rustc/84c898d65adf2f39a5a98507f1fe0ce10a2b8dbc/library/std/src/panicking.rs:579:5
   1: core::panicking::panic_fmt
             at /rustc/84c898d65adf2f39a5a98507f1fe0ce10a2b8dbc/library/core/src/panicking.rs:64:14
   2: core::result::unwrap_failed
             at /rustc/84c898d65adf2f39a5a98507f1fe0ce10a2b8dbc/library/core/src/result.rs:1750:5
   3: core::result::Result<T,E>::unwrap
             at /rustc/84c898d65adf2f39a5a98507f1fe0ce10a2b8dbc/library/core/src/result.rs:1090:23
   4: tests::common::util::UChild::wait_with_output
             at ./tests/common/util.rs:2001:13
   5: tests::common::util::UChild::wait
             at ./tests/common/util.rs:1948:22
   6: tests::common::util::UCommand::run
             at ./tests/common/util.rs:1543:9
   7: tests::common::util::UCommand::fails
             at ./tests/common/util.rs:1567:26
   8: tests::test_mv::test_mv_arg_interactive_skipped_vin
             at ./tests/by-util/test_mv.rs:1263:5
   9: tests::test_mv::test_mv_arg_interactive_skipped_vin::{{closure}}
             at ./tests/by-util/test_mv.rs:1259:42
  10: core::ops::function::FnOnce::call_once
             at /rustc/84c898d65adf2f39a5a98507f1fe0ce10a2b8dbc/library/core/src/ops/function.rs:250:5
  11: core::ops::function::FnOnce::call_once
             at /rustc/84c898d65adf2f39a5a98507f1fe0ce10a2b8dbc/library/core/src/ops/function.rs:250:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

works locally

Hm, the code looks fine and works locally, too. Maybe it's the unnecessary pipe_in call?

I used ignore_stdin_write_error to ignore the stdin

Any idea what is going on?

Not really, but I have some vague recollection of this happening due to nextest in some other PR?


Transfer and three-way-call function

Hi,
Can you please let me know when "Transfer" and "three-way" function be available?
Thanks

Hello,
The 'Transfer' and 'Three-way' functions have been implemented a while ago.
( https://github.com/brekekesoftware/operator-console/tree/dai-1_3_0- )
Thank you

Hi Dai,
Could you please clarify how to utilize the three-way button?
I'm encountering a situation where I need to transfer a call between UserA from DepartmentA and a client, who subsequently requests to be connected to DepartmentB, set up as a Group extension. Our current method using the SIP phone involves the following steps:

UserA places the client on hold.
UserA initiates a call to DepartmentB (Group Ext).
Upon someone answering at DepartmentB, we merge the calls into a conference, effectively connecting all three parties.
Once introductions are made, UserA disconnects, leaving the Client and DepartmentB connected.

I'd appreciate your guidance on how to accomplish this seamlessly or a more effective way using operator console.
Thank you

Hello,
Instructions on how to make a three-way call using the Transfer button and Three-way button are as follows.
1.User B presses the transfer button to transfer the active call Call A to User C.
2.User C answers the forwarded call.
3.User B presses the Three-way button.
Now you can three-way call the call A.
Thank you

I might be missing something:

In the above image, I first initiated a call to the number ending in 0203 (the client), then I called user 203. So now, I have two active calls. I put both on hold and then tried to "Conference" which is three-way button but nothing happens. Below the browser console..

Technically, once I press "conference" all three parties should be on the same call, no?
Am I doing something wrong here?
Thanks!

Hello,
Unfortunately, Operator Console do not support that case.
Thank you


Create groups based on user's groups.

They are available as course::{course_id}::group::{name}. This also changes the format of course-based groups from canvas::{course_id}::{enrollment_type} to course::{course_id}::enrollment_type::{enrollment_type}.

@yuvipanda This is meant to address #1 and berkeley-dsep-infra/datahub#4364. Your thoughts are welcome, though I understand if you're busy! (not sure if this is used outside Berkeley)

This has been tested locally, so I'll go ahead and merge.


[common] The abstraction of Incremental Snapshot Framework

Currently, only MySQL CDC implemented the Incremental Snapshot algorithm which makes users enjoying lock-free, parallel-reading and restoration-at-anytime .
For other CDC Connectors, users still suffer  from the single read task and lock issue, we can abstract a common Incremental Snapshot framework to make other Connectors take flight easily.

@leonardBang  Are you working on it?  I want to complete this feature.

@leonardBang Are you working on it? I want to complete this feature.

I'm woking on the design phase, Cooperation is pretty welcome, @YesOrNo828 .  Could you share your design/API document?

@leonardBang  Sorry for the slow response, this is my API document, and I‘m doing some POC.

@leonardBang Sorry for the slow response, this is my API document, and I‘m doing some POC.

Nice work ! @YesOrNo828 I think we can discuss the API document first, and then we can continue the POC. It's a big challenge that we need to think more for other datasource like MongoDB, TiDB which don't depend on Debezium.
I‘ll give some feedback for your API document soon.

I post the design document here,  API design

What stage is the API design at, I want to contribute to this that use incremental snapshot framework for oracle cdc connector


Is second test vector valid?

I ran the test vectors with Libsodium (1.0.16) and Monocypher (2.0.5)
For test  in section A.3.1. AEAD_XCHACHA20_POLY1305, I got the expected result.
For test in section A.3.2. XChaCha20 (plaintext: "The dhole...", I got a different result (same result with Libsodium and Monocypher):
7d 0a 2e 6b 7f 7c 65 a2 36 54 26 30 29 4e 06 3b
7a b9 b5 55 a5 d5 14 9a a2 1e 4a e1 e4 fb ce 87
ec c8 e0 8a 8b 5e 35 0a be 62 2b 2f fa 61 7b 20
2c fa d7 20 32 a3 03 7e 76 ff dc dc 43 76 ee 05
3a 19 0d 7e 46 ca 1d e0 41 44 85 03 81 b9 cb 29
f0 51 91 53 86 b8 a7 10 b8 ac 4d 02 7b 8b 05 0f
7c ba 58 54 e0 28 d5 64 e4 53 b8 a9 68 82 41 73
fc 16 48 8b 89 70 ca c8 28 f1 1a e5 3c ab d2 01
12 f8 71 07 df 24 ee 61 83 d2 27 4f e4 c8 b1 48
55 34 ef 2c 5f bc 1e c2 4b fc 36 63 ef aa 08 bc
04 7d 29 d2 50 43 53 2d b8 39 1a 8a 3d 77 6b f4
37 2a 69 55 82 7c cb 0c dd 4a f4 03 a7 ce 4c 63
d5 95 c7 5a 43 e0 45 f0 cc e1 f2 9c 8b 93 bd 65
af c5 97 49 22 f2 14 a4 0b 7c 40 2c db 91 ae 73
c0 b6 36 15 cd ad 04 80 68 0f 16 51 5a 7a ce 9d
39 23 64 64 32 8a 37 74 3f fc 28 f4 dd b3 24 f4
d0 f5 bb dc 27 0c 65 b1 74 9a 6e ff f1 fb aa 09
53 61 75 cc d2 9f b9 e6 05 7b 30 73 20 d3 16 83
8a 9c 71 f7 0b 5b 59 07 a6 6f 7e a4 9a ad c4 09

I see the problem. You're calling the AEAD interface in both places.
The second test vector is for XChaCha20 itself (rypto_stream_xchacha20_xor_ic), not the AEAD construction.

I wonder if including the raw keystream prior to encryption would make this easier to troubleshoot.

Sounds like a good idea to me.

https://github.com/bikeshedders/xchacha-rfc/commit/57a8ce29287f63446397194a9e025f46daf6e511 should make it easier to prevent this in the future. Anyone who is about to retrace the same steps will have to notice that the keystream seems to be offset from what the test vectors provide, which should be a dead giveaway that they're using the AEAD interface for both test vectors.

https://github.com/bikeshedders/xchacha-rfc/blob/0e3b53d0beadaff6dd6aca12dce86670753064dd/draft-arciszewski-xchacha-rfc-03.txt#L565-L569 makes it explicit, too.
Thanks @philanc for reporting this issue. 👍