AndroidX support

Steps to Reproduce
I wanted to add Flutter to my existing Android app. I've found this tutorial. I've created flutter module and my next step was to create FlutterFragment. Unfortunately my app is using AndroidX dependencies while Flutter is still using Android Support Libraries.
I think this is quite major issue because a lot of Android apps are using AndroidX already, moreover with the release of Android 9.0 (API level 28) all new library development will occur only in the AndroidX library.
Android Support artifacts are no longer supported.

really need this change! Any updates on this?

Working on making this easier to migrate to in the Engine - we just recently rolled to a newer Android support library, but haven't yet rolled to AndroidX.
Related to #17780

I cant migrate to AndroidX with AndroidStudio --> Refactor --> Migrate to AndroidX
I have this errors

How i can migrate?

I have the same problem, but my migration has not been successful.

@arieslee did you add the android directory in Android Studio?

@arieslee did you add the android directory in Android Studio before executing the command?

thank you, i solved it.

@arieslee did you add the android directory in Android Studio before executing the command?

What did you mean? please, describe step by step

When support androidx? we need integrate flutter to our project

any updates?

@leagmain see the linked issue and a related PR from that.  Right now there's some work we're doing to make sure that plugins and projects using or not using Android X will be able to work together properly.

HI,
I recently started to work on a flutter project which has barcode scanner.
To use this plugin I had to upgrade the project to androidx settings.
This new settings cause my sms plugin to crash.
It's a chicken and an egg thing.
It seems to be really unstable platform.
Any solution?
Can you add an android x compatability sign for each plugin in the dartpub.

Does this still happen if you use compileSdkVersion 28?

I got a strange exception.
When i accept the SMS permission the app fails on the scanner which is not part of the current widget screen.
The scanner  is initialzie in the next screen after  finishing the sign up process.
So it may not be an issue of the AndroidX.
I think that i'll go back to native code.
I gave flutter a chance but its really unstable
E/AndroidRuntime( 5893): Process: il.co.ylm.inspector_native, PID: 5893
E/AndroidRuntime( 5893): java.lang.RuntimeException: Failure delivering result ResultInfo{who=@android:requestPermissions:, request=1, result=-1, data=Intent { act=android.content.pm.action.REQUEST_PERMISSIONS (has extras) }} to activity {il.co.ylm.inspector_native/il.co.ylm.inspector_native.MainActivity}: java.lang.NullPointerException: Attempt to read from field 'com.github.rmtmckenzie.qrmobilevision.QrReader com.github.rmtmckenzie.qrmobilevision.QrMobileVisionPlugin$ReadingInstance.reader' on a null object reference
E/AndroidRuntime( 5893): 	at android.app.ActivityThread.deliverResults(ActivityThread.java:4491)
E/AndroidRuntime( 5893): 	at android.app.ActivityThread.handleSendResult(ActivityThread.java:4534)
E/AndroidRuntime( 5893): 	at android.app.ActivityThread.-wrap20(Unknown Source:0)
E/AndroidRuntime( 5893): 	at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1752)
E/AndroidRuntime( 5893): 	at android.os.Handler.dispatchMessage(Handler.java:105)
E/AndroidRuntime( 5893): 	at android.os.Looper.loop(Looper.java:164)
E/AndroidRuntime( 5893): 	at android.app.ActivityThread.main(ActivityThread.java:6944)
E/AndroidRuntime( 5893): 	at java.lang.reflect.Method.invoke(Native Method)
E/AndroidRuntime( 5893): 	at com.android.internal.os.Zygote$MethodAndArgsCaller.run(Zygote.java:327)
E/AndroidRuntime( 5893): 	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1374)
E/AndroidRuntime( 5893): Caused by: java.lang.NullPointerException: Attempt to read from field 'com.github.rmtmckenzie.qrmobilevision.QrReader com.github.rmtmckenzie.qrmobilevision.QrMobileVisionPlugin$ReadingInstance.reader' on a null object reference
E/AndroidRuntime( 5893): 	at com.github.rmtmckenzie.qrmobilevision.QrMobileVisionPlugin.stopReader(QrMobileVisionPlugin.java:73)
E/AndroidRuntime( 5893): 	at com.github.rmtmckenzie.qrmobilevision.QrMobileVisionPlugin.onRequestPermissionsResult(QrMobileVisionPlugin.java:60)
E/AndroidRuntime( 5893): 	at io.flutter.app.FlutterPluginRegistry.onRequestPermissionsResult(FlutterPluginRegistry.java:203)
E/AndroidRuntime( 5893): 	at io.flutter.app.FlutterActivityDelegate.onRequestPermissionsResult(FlutterActivityDelegate.java:129)
E/AndroidRuntime( 5893): 	at io.flutter.app.FlutterActivity.onRequestPermissionsResult(FlutterActivity.java:137)
E/AndroidRuntime( 5893): 	at android.app.Activity.dispatchRequestPermissionsResult(Activity.java:7737)
E/AndroidRuntime( 5893): 	at android.app.Activity.dispatchActivityResult(Activity.java:7560)
E/AndroidRuntime( 5893): 	at android.app.ActivityThread.deliverResults(ActivityThread.java:4487)

P.S
Thanks for the quick response!

@Bartosz-Kozajda I encountered the same difficulties (and some more, such as product flavor, data communication, etc.), and wrote an instruction here
https://hackernoon.com/how-to-add-flutter-to-your-production-android-app-4b5150de9e44
Hope this help.

Is this being worked on? This is completely blocking my development with Flutter.

Should only support androidx, don't support old package

I've got a branch which adds support for AndroidX that you can try out: https://github.com/flutter/flutter/pull/31028

Is this going to be merged? Can I not just checkout this branch in the existing repo?

Yes you can check out this branch and the Flutter tool will rebuild when you next run the command.
I've been using this as my working branch for the last month.

I cant migrate to AndroidX with AndroidStudio --> Refactor --> Migrate to AndroidX
I have this errors

How i can migrate?

open android folder with android studio, and then migrate to android x.

I cant migrate to AndroidX with AndroidStudio --> Refactor --> Migrate to AndroidX
I have this errors

How i can migrate?

Open android folder with Android Studio, then migrate to androidx.

@mklim is this distinct from https://github.com/flutter/flutter/issues/32714?

@tvolkert no, this is the same root issue.

I wonder if I'm getting a bit closer.  It's throwing
2019-06-03 11:15:49.707 18177-18177/com.example.t130_leaf_flutter_android E/AndroidRuntime: FATAL EXCEPTION: main
    Process: com.example.t130_leaf_flutter_android, PID: 18177
    java.lang.UnsatisfiedLinkError: No implementation found for long io.flutter.embedding.engine.FlutterJNI.nativeAttach(io.flutter.embedding.engine.FlutterJNI, boolean) (tried Java_io_flutter_embedding_engine_FlutterJNI_nativeAttach and Java_io_flutter_embedding_engine_FlutterJNI_nativeAttach__Lio_flutter_embedding_engine_FlutterJNI_2Z)
        at io.flutter.embedding.engine.FlutterJNI.nativeAttach(Native Method)
        at io.flutter.embedding.engine.FlutterJNI.attachToNative(FlutterJNI.java:455)
        at io.flutter.embedding.engine.FlutterEngine.attachToJni(FlutterEngine.java:147)
        at io.flutter.embedding.engine.FlutterEngine.<init>(FlutterEngine.java:119)
        at com.example.t130_leaf_flutter_android.MyApp$flutterEngine$2.invoke(MyApp.kt:10)
        at com.example.t130_leaf_flutter_android.MyApp$flutterEngine$2.invoke(MyApp.kt:8)
        at kotlin.SynchronizedLazyImpl.getValue(LazyJVM.kt:74)
        at com.example.t130_leaf_flutter_android.MyApp.getFlutterEngine(Unknown Source:25)
        at com.example.t130_leaf_flutter_android.MainActivity$onCreate$2.run(MainActivity.kt:33)
        at android.os.Handler.handleCallback(Handler.java:873)
        at android.os.Handler.dispatchMessage(Handler.java:99)
        at android.os.Looper.loop(Looper.java:193)
        at android.app.ActivityThread.main(ActivityThread.java:6718)
        at java.lang.reflect.Method.invoke(Native Method)
        at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:493)
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:858)

I wonder if the jetifier is messing with JNI

Google Play services and Firebase are migrating to AndroidX
https://android-developers.googleblog.com/2019/06/google-play-services-and-firebase.html

Repros are https://github.com/xster/flutter-test/tree/master/t130_leaf_flutter and https://github.com/xster/flutter-test/tree/master/t130_leaf_flutter_android

@xster The flutter module in that repro just needed to have androidx enabled: https://github.com/xster/flutter-test/pull/3

To close the loop on https://github.com/flutter/flutter/issues/23586#issuecomment-498369718. I had some suspicion though it was related to https://github.com/flutter/flutter/issues/34444 and https://github.com/flutter/flutter/issues/34401 but it wasn't.
Trying again with the new androidX: true tag in the Flutter module's pubspec and with the outer project bringing in implementation 'androidx.lifecycle:lifecycle-common-java8:2.0.0', I'm still getting the same JNI runtime issue.

@xster I was able to build your sample app: https://github.com/xster/flutter-test/tree/master/t130_leaf_flutter and https://github.com/xster/flutter-test/tree/master/t130_leaf_flutter_android with the host app using AndroidX and Jetifier and the Flutter module using support libraries. In order words, letting Jetifier do the translation.
Here's how:

Remove this patch: https://github.com/xster/flutter-test/pull/3/files
In the Flutter module, build an AAR: cd into your .android and run ./gradlew app:assembleReleaseAar
In the host app, settings.gradle comment out:

// evaluate(new File(                                                      // new
//   'flutter/.android/include_flutter.groovy'                          // new
// ))

Also, in build.gradle (Module) comment out  implementation project(':flutter')

Import the AAR produced in step 2. This can be done in Android Studio,  New Module -> import .JAR/.AAR package.

Add the dependency to  build.gradle (Module)

dependencies {
  implementation project(':flutter-release')
}

Build the host app APK from Android Studio.

This only includes the release AAR, but you can include the debug AAR as well.
This may work as well for solving AndroidX incompatibilities in plugins since Jetifier is designed to operate on AAR/JARs only.

The root cause for the AndroidX failures is that we include  io.flutter.facade.* java classes  that aren't precompiled. https://github.com/flutter/flutter/tree/master/packages/flutter_tools/templates/module/android/library/Flutter.tmpl/src/main/java/io/flutter/facade.  Jetifier can't  translate those since it only operates on JARs or AARs. Also, using the AndroidX migration tool from Android Studio won't work since the files are auto generated.  The current solution is to do this https://github.com/xster/flutter-test/pull/3/files, however this is a general issue. Plugins face this challenge as well, as a result I propose:

Move  the io.flutter.facade.* classes to flutter.jar or remove them if they aren't needed.
Support build aar for module and plugin Flutter projects.

For add2app, ideally, developers can just use include_flutter.groovy since this mechanism provides better devX:  no need to build separate AARs for debug, release, flavors, etc...  I still need to evaluate the tradeoffs, but that's what I observed so far.
Any comments? @dnfield @mklim @xster

@blasten:

Good find. That makes sense to me. I noticed that those classes have been patched to work with the --androidx template though, so theoretically they should be able to be generated without any conflicts. Are there still failures with that template activated?

Yeah, I agree with that solution. For the plugins the issue is like this but it's also the native Java sources in the plugins themselves. flutter/flutter#29328 has a lot of the context and prior work on this.

Thanks for the explorations! Glad to know we have a workaround, that's a great first step. I'm not sure I understood the details of why failing to convert the non-aar io.flutter.facade.* java source causes the manifestation of the java.lang.UnsatisfiedLinkError: No implementation found for long io.flutter.embedding.engine.FlutterJNI.nativeAttach(io.flutter.embedding.engine.FlutterJNI, boolean) (tried Java_io_flutter_embedding_engine_FlutterJNI_nativeAttach and Java_io_flutter_embedding_engine_FlutterJNI_nativeAttach__Lio_flutter_embedding_engine_FlutterJNI_2Z) symptoms.
I think io.flutter.facade.* isn't referenced at all in the example repro code. Can you elaborate?
Point 2 totally SG.

@xster I was unable to repro the java.lang.UnsatisfiedLinkError issue. Maybe, the issue was fixed in the embedding? Maybe @matthew-carroll knows more about it.

From the flutter docs:
https://flutter.dev/docs/development/packages-and-plugins/androidx-compatibility

An update on this issue:
If you encounter this issue, then run flutter config --enable-build-plugin-as-aar.
If you are building a full Flutter app
Run flutter build <apk/appbundle>. This fixes the AndroidX compatibility issue.  The app must be using AndroidX, which is the result of creating the app flutter create -t app --androidx <app-directory>.
If you are building a Flutter module (Add to app)
Make sure the project is using AndroidX. e.g. flutter create -t module --androix <module-directory>.  If the current project isn't using AndroidX, then edit the <module-directory>/pubspec.yaml, and add
module:
    androidX: true // This line

Next, remove  the <module-directory>/.ios and <module-directory>/.android directories, so they get regenerated next time you use the Flutter tool.

The remaining issue is discussed on https://github.com/flutter/flutter/issues/38988

Sounds good @Hixie. For anyone reading this thread, see https://github.com/flutter/flutter/issues/23586#issuecomment-518886736

An update on this issue:
If you encounter this issue, then run flutter config --enable-build-plugin-as-aar.
If you are building a full Flutter app
Run flutter build <apk/appbundle>. This fixes the AndroidX compatibility issue. The app must be using AndroidX, which is the result of creating the app flutter create -t app --androidx <app-directory>.
If you are building a Flutter module (Add to app)
Make sure the project is using AndroidX. e.g. flutter create -t module --androix <module-directory>. If the current project isn't using AndroidX, then edit the <module-directory>/pubspec.yaml, and add
module:
    androidX: true // This line

Next, remove the <module-directory>/.ios and <module-directory>/.android directories, so they get regenerated next time you use the Flutter tool.

It's not work for me. I remove the .android and .ios directory, but when I regenerated them, it's just the same as before.


feat(rust): add opentelemetry tracing and logging support

This PR introduces a way to produce OpenTelemetry traces and logs. This feature is not yet fully operational but this PR allows to us to start experimenting with such a support.
The following sections describe the main modifications.
OpenTelemetry tracing
The tracing configuration has been extended with additional layers in order to send spans and log records to an OpenTelemetry collector.
Locally an OpenTelemetry collector can be started by:

Checking out the project https://github.com/open-telemetry/opentelemetry-rust.git
Running the command docker run -v $(pwd)/collector.yaml:/etc/otelcol-contrib/config.yaml -p 4317:4317 otel/opentelemetry-collector-contrib:latest with the appropriate collector.yaml file.

A minimal configuration for the collector.yaml file is:
receivers:
  otlp:
    protocols:
      grpc:

exporters:
  debug:
    verbosity: detailed

service:
  pipelines:
    traces:
      receivers: [otlp]
      exporters: [debug]
    logs:
       receivers: [otlp]
       exporters: [debug]

This allows traces to appear in the CLI. If you want to export traces to Honeycomb, please refer to the corresponding documentation.
Instrumentation
Command names
In order to produce spans, a root span has been added to the run function inside the ockam_command crate.
Every time a command is executed, a new trace is started with the command name as the root span name.
Many files are modified in the PR, only to give a textual name to all our commands.
API functions
Some functions have been instrumented using the #instrument macro, but this PR does not cover the entirety of a node API.
Trace context propagation
We want to produce distributed traces, across different components:

The CLI
A background node
The Controller
A project node
An authority node

In order to do this we need to send, on every remote call, some tracing context data. This PR implements some support for this:

During node creation, so that the command ockam node create starting a background node can have tracing data from the original ockam node create command started by a user. (Note: there is currently a race condition and some spans might be missing)
During a RPC call, using the node RPC support. For example when getting the status of a background node

There is no tracing context propagation yet for the Ockam protocol itself.
Initialization
The initialization of a Tracer requires an async runtime. In order to configure tracing even before an internal node Context is created, I had to change access to the tokio runtime and change all the node_rpc functions to accept a runtime parameter. This is also why many of the ockam_command files had to be touched in this PR.
Journeys
The concept of a HostJourney and a ProjectJourney have been introduced. Those are pseudo-traces that we generate with major events for a given host, or a given project:

When a node has been created
When a user has just enrolled
When a TCP outlet has been created
etc...

The Host journey collects events for a given host, and has a random trace id, while the Project journey collects events for a given project and uses a trace id derived from the project id.
There are pros and cons to this approach:

pros: we can easily visualize the main commands (and possibly the main errors) happening for a given installation or project

cons: those are not real traces and will hit a size limit for a given trace (32000 spans for Honeycomb). This is not necessarily a problem for a Host journey because we could roll another trace but this could be a problem for a project journey since there might not be a way to drop old spans from a given trace.

This approach is still experimental and might be replaced in the future. A better option would be to have better querying and visualization tools across several traces. At the moment such a query in Honeycomb is not fantastic:

For example, it is not possible to sort events by their application timestamp.
Application database
In order to store data which is not delete when running ockam reset, an additional database has been added.
That database is initialized with its own migration file (see ApplicationMigration). This opens the door to creating additional Vault databases with their own migration file, instead of creating the full node database schema.
Standard attributes
The attribute names set on spans and log records are generally used to query events, and possibly to set metrics and alerts. One consequence is that we need to make sure that the names we use are stable. This PR follows the OpenTelemetry semantic conventions where possible and defines Ockam-specific constants otherwise.
Testing traces / logs export
The test test_log_and_traces shows how we can use some in-memory SpansExporter and LogsExporter to capture the spans and logs that are created when we run a command.
This test support is also used to make sure we create the right events for a host/project journey.
Environment variables
There are now more ways to configure logging and tracing for an Ockam application:

OCKAM_LOG_LEVEL sets the log level to apply for both logging and tracing (there is no distinction at the moment)
OCKAM_LOGGING must be set (to any value) in order to enable logging
OCKAM_TRACING must be set (to any value) in order to enable tracing
OTEL_EXPORTER_OTLP_ENDPOINT must be set to the URL of the OpenTelemetry collector (http://localhost:4317 by default)
OTEL_EXPORTER_OTLP_HEADERS if the OTLP endpoint is directly set to the Honeycomb API. Then that header can be used to set the Honeycomb API key

The old OCKAM_LOG environment variable is now deprecated, because it controls both the level filtering that is used for logs and traces and if tracing/logging is on. That env. variable is still available but does not set the tracing on if OCKAM_TRACING is not set.

@adrianbenavides I rebased and did the change that you suggested (not creating unnecessary exporters).


Remove unnecessary Map.delete

I may be missing something subtle (or obvious), but the Map.delete call is only made when the key does not exist in the map, so the function should always return the map unchanged. Is there a scenario where Map.delete(changes, key) will have an effect on changes?

:heart: :green_heart: :blue_heart: :yellow_heart: :purple_heart:


Bug: new tiles are added even when the board hasn't changed

Description:
In the original game, keypresses that do not result in a board changing — do not produce new tiles. gxt-game-2048 does.
Steps to reproduce:

Keep pressing the same direction again and again.

Expected behavior:
When all tiles reach the edge and cannot move or merge, no tiles are being added with subsequent keypresses.
Actual behavior:
When all tiles reach the edge and cannot move or merge, tiles continue being added with subsequent keypresses.

Fix added, @lolmaus thank you for report!


Segfault when using option g:syntastic_auto_loc_list

Vim crashes (segfault) when I do the following:

open a file such that all errors reported by the checker are in another file.
:SyntasticCheck (this opens a loc list)
move on the first line of the loc list
press enter

If I comment the line 'let g:syntastic_auto_loc_list = 1' in my .vimrc and use :Errors to open the loc list the problem does not occur.
Remark: I am using a custom checker.

This looks like the bug described at #1127.  It's a bug in Vim, not in syntastic.  I'm no longer following Vim development that closely these days, but I think there has been some work done recently in the  quickfix code that might be relevant here.  So you might try installing the latest version of Vim, and, if the problem persists, post a stack trace to vim_dev.

I get the same pb with vim 8.1.0279.
I also tried neovim and got the error 'E926 Current location list was changed' instead of the crash.
Anyway thanks for your answer. I'll read the thread you mentioned and see if I report an issue to vim.


Request for OpenAI API Proxy Design

Hello,
I would like to request the design of an OpenAI API proxy that would allow users to access the ChatGPT API through a third-party proxy. This could help facilitate smoother access to the API in various regions.
You might consider referencing the design from Bin-Huang/chatbox as a potential starting point for this proxy.
I hope that international tensions do not hinder our ability to use these convenient tools. I also want to extend my gratitude to the developers of Smart Composer for creating such an excellent tool.
Thank you!

Can you elaborate more on third-party proxies? Are they unavailable when using the custom OpenAI-compatible model option?

Thank you for your response. I just checked, and it meets the requirements. Thank you once again!

 CORS issues can indeed occur when using custom OpenAI-compatible models. If you encounter this issue, please comment on #158 for further assistance.


Release 1.1 with KSQL 5.3.0

Coverage decreased (-0.3%) to 98.519% when pulling 5cd325f1bdd56ea9951a2b6d3e0350d513476049 on develop into 644ab5ddb8b9465cf88e27de5a74cf2ca1f3d7d8 on master.


Sonarr list endpoints

This will add the endpoints to the client for import lists. The starr library looks like it already has everything
TODO

[x] Build it local and test
[x] Verify API docs updated

sigh i rebased on all the new stuff and now it is all attached to this one


Fix inappropriate chars for quotation.

My fault. Had the wrong node version and it threw errors.

Good point though - those characters shouldn't be there. If you want to get the PR submitted though, which I'll accept, you'll have to change those strings like this:
`>> File ${display_path} changed.`

Turns into:
">> File " + display_path + " changed."


Add check if the file extension is null before calling the to toLowerCase() - Fixes gh-20302

Add check if the file extension is null before calling the to toLowerCase() - Fixes gh-20302

Can you please add a test. Also which issue does it fix since this project doesn't have that number issue

Yes sure. I’ll add test for it.
The issue is: https://github.com/spring-projects/spring-boot/issues/20302

@spencergibb & @ryanjbaxter test added.

@ryanjbaxter I did a rebase with force push just to change the author email to avoid the use of my work email. Sorry for the disturbance :-)

@ryanjbaxter @spencergibb I can't find the fix in 2.2.2.RELEASE although it is mentioned here https://github.com/spring-cloud/spring-cloud-config/milestone/75?closed=1
Any explanation for it, please?

That was my mistake.  I didn't realize the PR was made against master and not the 2.2.x branch.  I cherry picked the commit to the 2.2.x branch so it will be part of Hoxton.SR4 (2.2.3.RELEASE). Sorry for the confusion (and the missed opportunity).

@ryanjbaxter No problem, thanks for the clarification.


Authentication documentation error

The signup section of https://aws.github.io/aws-amplify/media/authentication_guide#working-with-user-attributes has attributes that look like standard attributes, but are not. I believe according to this
first_name
last_name
nick_name

should be
given_name
family_name
nickname

Possibly fixed:
Auth.signUp({
    'username': 'jdoe',
    'password': 'mysecurerandompassword#123',
    'attributes': {
        'email'<EMAIL_ADDRESS>        'phone_number': '+12128601234', // E.164 number convention
        'given_name': 'Jane',
        'family_name': 'Doe',
        'nickname': 'Jane'
    }
});

I assume that the aws.github.io site is getting generated from the markdown file in the repo? If so, should I fix?

@funkfinger yes you can do that with a pull request. Thanks!

Thanks @funkfinger, your correction has been approved.


Removed always true PHP_VERSION_ID condition

because of raised min-php version
identified via https://github.com/phpstan/phpstan-src/pull/3584

Thank you!


Sample for async conversation transcription without voice samples

Hi,
I'm trying to use Azure Speech to transcribe conversations (in my case it's mostly recorded interviews) that have:

Multiple participants (anywhere between 1 and 10)
No voice samples of the participants

I implemented the first prototype using the batch transcription REST API 3.0, but there only two speakers are support.
My second approach was to use async conversation transcription, since in the documentation it says that you don't need samples of the speakers to get diarization (the second Note box here https://docs.microsoft.com/en-us/azure/cognitive-services/speech-service/conversation-transcription clearly says "User voice samples are optional. Without this input, the transcription will show different speakers, but shown as "Speaker1", "Speaker2", etc. instead of recognizing as pre-enrolled specific speaker names."). However, I'm at a loss as to how to get it to work. I tried simply removing the "addParticipantAsync" from the samples, but that just produces empty output.
So my question/feature request: is there an example that shows how to do async conversation transcription with speaker diarization without speaker samples? And if there isn't, it would be great to have one (any language, though I personally would prefer JS).
Thanks :)

@matej-svejda Are your recorded interviews in 8-channel audio? If so, could you share one so that I can see what happening and give a code sample that works for me?

@glharper Ah, so you having an 8-channel audio is hard requirement for using the conversation transcription service? What I need transcriptions for are mostly recordings of online meetings (Zoom, Teams etc.). So if the conversation transcription service isn't the way to go, is there any other way to transcribe those with speaker diarisation? It would really help a lot, because accuracy wise your transcription service beats the other ones by a lot :)

@matej-svejda could you send this question to<EMAIL_ADDRESS>that I will be happy to facilitate you further.

Closed as further support is handled by CTSFeatureCrew.


Bot is unable to receive attached files in Teams

I have a bot running in Teams (Bot Framework SDK v4) but when I'm trying to upload an attachment (other than an image) from a Microsoft Teams channel to my bot, the file doesn't show up anywhere in the message that the bot receives.
In bot's session.message.attachments i receive only its mention, as below.
{
  "contentType": "text/html",
  "content": "

bot_name
\n
"
}

I want to mention that I set the supportsFiles property in the manifest to true.
Is there a way for the bot to receive files in a Teams channel?
Reproduction Steps

Attach file using paperclip in Teams
@-mention bot and send message

Hi @glucan1 ,
Bot can only send and receive file in personal chat. It does not work in the channel or group chat context.
Here is the link

Please re-open if you still have any queries.


fixed cmdlet typos

@boecskoer : Thanks for your contribution! The author(s) have been notified to review your proposed change.

@boecskoer : Thanks for your contribution! The author(s) have been notified to review your proposed change.

@boecskoer Thank you for catching these! We will get it merged right away. Thanks again for your contributions. #sign-off

@boecskoer Thank you for catching these! We will get it merged right away. Thanks again for your contributions. #sign-off


limit allow Infinity

Consider Infinity as valid value for limit.
It should set no limit at all.

Sounds like a hack to me for trying to circumvent limitation that knex doesn't support removing parts from builder... Passing infinity to limit is usually done by error so I would vote no for this one.
Other than that, documentation and tests are missing.

Adding info about related issue https://github.com/tgriesser/knex/pull/1907 that has better API for clearing stuff. This should be with same kind of implementation + tests and docuementation.


Implement BOLT1 warning messages

Related: https://github.com/lightningnetwork/lnd/issues/5593#issuecomment-891787008, Acinq sent a new warning message type in response to a ChannelReestablish.
See https://github.com/lightningnetwork/lightning-rfc/pull/834, which currently isn't merged yet but apparently already implemented by Eclair.

We have a very WIP here: https://github.com/lightningnetwork/lnd/pull/5560
But as you mention the spec PR isn't really finalized yet, and we also don't advertise any few feature bits, so I wouldn't expect them to send a new message to us.

New coop close proposal PR mentions warnings should be sent, not a hard requirement from my reading though: https://github.com/lightningnetwork/lightning-rfc/pull/847

Fixed by #6546, we also start to send them now.


call vs execute _pathogen#infect()

My Question
Can you comment on the subtly (that I assume I am not grasping yet) that led you to prefer execute over call as reflected in the README.
You can see some details about my setup under the  [scenario][#scenario] heading further down.
Totally Extra Questions / Discussion Topics
I don't expect any response on these questions, but I would really appreciate any discussion (from anyone in the community here).

As I am new to Pathogen can you comment on your first one or two troubleshooting steps when something does not seem to be functioning.  I will be digging deeper myself after I post this, but I am hoping you might share: some insight about how pathogen works or an article or maybe the first two or three things you verify when troubleshooting a fresh setup of pathogen.  (I don't think Pathogen is at fault here, not by any means.)

Assuming that I have followed your advice for file/folder naming conventions and choose "not to get crazy".  As far as I can tell I have setup exactly what you describe as your configuration (if not truly your preferred then the simplest).  With the exception of the using call; I am getting expected and reliable behavior as I add packages like vim-scriptease, vim-sensible, and a few others.
Then, I came across a [plugin][vim-ipython] that is not loading like the previous plugins.

I installed the plugin from a repo in the same way that I did vim-sensible and vim-scriptease
I can source the plugin file and then I get the functionality I am expecting
Based on other plugins that are functioning, the plugin folder structure appears to be correct
The fact that I can source the plugin manually, but it doesn't autoload seems to indicate either a path issue or a flag / feature that pathogen looks for in each plugin as an indication that "it wants" to be automatically loaded by Pathogen.
Obviously, any errors with the [vim-ipython] should be directed to the repo for that plugin; that's not what I'm getting at here.  I'm interested in what your first 2/3 steps to troubleshoot / debug when plugins mis-behave.

The Scenario
I notice that the Pathogen README differs from a majority of other instructions on setting up Pathogen.  In fact, it appears that early last year, tpope/vim-pathogen/532f0ca7d936f1229b926b73e84012a78c4f9800, the README itself changed from call to what it now prescribes as the approach to invoke pathogen through one's .vimrc file.
I was following this repo's README at first, but my initial attempts to verify that bundles like [vim-scriptease] and [vim-sensible] were in fact functioning did not seem to be working (I was still fumbling around a bit, so I'm not positive, but I can go back through and capture the steps-to-reproduce if you would like me to).
I had already stripped down my vim installation to a blank-slate, starting fresh with compiled VIM installation and Pathogen.  Then I changed, in my .vimrc file exectue to call and I was able to verify that the bundles were indeed loading.
Before I changed call to execute, I checked the VIM help documentation and it seemed to me that call would have been the correct choice, or at least more correct in the case that they both will work with some idiosyncrasies.
execute
Executes the string that results from the evaluation of [the expression] as an Ex command.  Multiple arguments are concatenated, with a space in between.  To avoid the extra space use the "." operator to concatenate strings into one argument. {expr1} is used as the processed command, command line editing keys are not recognized. Cannot be followed by a comment. |
call
Call function {func} with the items in List {arglist} as arguments. {func} can either be a Funcref or the name of a function. a:firstline and a:lastline are set to the cursor line. Returns the return value of the called function. {dict} is for functions with the "dict
VIM per-user configuration
My .vimrc file
call pathogen#infect()
syntax on
filetype plugin indent on

The above file, along with the folder structure described below, reside in my user's home directory (i.e. ~ or $HOME)

.vimrc
.vim

autoload

pathogen.vim
(pathogen has this directory all to itself)

bundle

vim-sensible
vim-scriptease
vim-coffee-script
vim-ipython

From a fresh start, execute didn't seem to work for me but rather call:
call pathogen#infect()

I'm not sure why (maybe something broke in most recent vim version?)
It seems changing to call works and gives expected behaviour, but I'm ignorant of details. I just thought I would mention as feedback; maybe a mention of this issue should be mentioned in the docs?
thanks for great plugin!


This helped me thanks

my problem is solved by doing this thanks @Gork7
To solve the problem, instead of using -- "from kivy.properties import NumericProperty"  try using
-->  "import kivy.properties as any_name" -- then call it by saying for example --> "button_width = any_name.NumericProperty(int_button_width) "
Originally posted by @Gorky7 in https://github.com/kivy/kivy/issues/6059#issuecomment-541312799

for which issue is this for ?
did u want to make a comment on a issue ?
i think this can be closed


Full opacity when sig is active/focused

Just found this extension and I think it'll help with readability. One issue is that the editing experience is a bit difficult with the sigs hidden. Thoughts on making it so that if my cursor is within a sig then it becomes "unhidden" temporarily until it loses focus?

That's a great idea. I'm not working on this plugin actively however if anyone willing to submit a PR I'm more than happy to help.


Adding WebP support

I like using webp and as #2673 shows I'm not the only one.
And since the implementation was basically just adding the respective feature of the image crate,
so I went ahead and made a PR.
If there is an Issue with this PR (I've never done this before) please let me know.

I think this introduces a c-dependence

As far as I can see the c-dependence comes from the encoder feature

My bad then!

As I see it, this feature has a few pros and cons:
Pros:

Much smaller binary size increase than for JPEG XL
Widely used
Would be the only format with lossy encoding and a (usable cf. GIF) alpha mask
Future-proofing for HTML export

Cons:

Encourages use of WebP in print documents despite often quite visible compression artifacts
The compression advantage for print-sized images at similar quality to JPEG references is low
PDF does not support WebP. We either have to lossily transcode to JPEG or embed a deflated pixel buffer (PNG-like, makes the file larger)
It does proliferate image formats in Typst, which are currently limited to a standard set

What do y'all think about this?

I'm leaning towards instead using the plugin system to add more image formats.

Slightly off topic but according to https://github.com/pdf-association/pdf-issues/discussions/248#discussioncomment-4244510 the most likely format to be adopted in a future PDF standard is AVIF or JPEG-XT.

I'm leaning towards instead using the plugin system to add more image formats.

Since there were no further arguments in favor of having it built-in and I agree with this, I'm gonna go ahead and close this.

One thing we can think about in the future is to make it easy to use a custom decoder with the existing image function.

In terms of custom decoders, it would be nice to be able to convert raw bytes to an image based on a size and format. Like image.from-raw or image.from-pixels or something

@mattfbacon See here: https://github.com/typst/typst/issues/2575


[Refactor] Refactor error log when create primary table using ORDER BY

What type of PR is this：

[ ] BugFix
[ ] Feature
[ ] Enhancement
[x] Refactor
[ ] UT
[ ] Doc
[ ] Tool

Which issues of this PR fixes ：

Fixes https://github.com/StarRocks/starrocks/issues/21416
Problem Summary(Required) ：

The OLAP table must has at least one short key and the float and double should not be short key, so the float and double could not be the first sort column in OLAP table.
If we separate primary keys and sort keys and the first sort key column type is float/double, the error log is inaccurate.
i.e.
we try to create a table with the following sql
CREATE TABLE `primary_table1` (
    `k1` date NOT NULL,
    `k2` datetime NOT NULL,
    `k3` varchar(20) NOT NULL,
    `k4` varchar(20) NOT NULL,
    `k5` boolean NOT NULL,
    `v1` tinyint NULL,
    `v2` smallint NULL,
    `v3` int NULL,
    `v4` bigint NULL,
    `v5` largeint NULL,
    `v6` float NULL,
    `v7` double NULL,
    `v8` decimal(27,9) NULL
)
PRIMARY KEY(`k1`, `k2`, `k3`, `k4`, `k5`)
COMMENT "OLAP"
DISTRIBUTED BY HASH(`k1`, `k2`, `k3`) BUCKETS 3
ORDER BY(`v7`, `v5`, `v6`, `v8`, `v1`, `v2`, `v3`, `v4`, `k1`, `k2`, `k3`, `k4`, `k5`)
PROPERTIES (
    "replication_num" = "3",
    "enable_persistent_index" = "true",
    "storage_format" = "v2"
);

we will get the error log
ERROR 1064 (HY000): Unexpected exception: Data type of first column cannot be DATE

However, the true reason is the column v7 type is double which cannot be the first column of short key but not the k1 column type is error, so the error log after this pr is
ERROR 1064 (HY000): Unexpected exception: Data type of first sort column cannot be DOUBLE

Checklist:

[ ] I have added test cases for my bug fix or my new feature
[ ] This pr will affect users' behaviors
[ ] This pr needs user documentation (for new or modified features or behaviors)

[ ] I have added documentation for my new feature or new function

Bugfix cherry-pick branch check:

[x] I have checked the version labels which the pr will be auto backported to target branch

[x] 3.0
[x] 2.5
[ ] 2.4
[ ] 2.3

@mergify rebase

Duplicate with https://github.com/StarRocks/starrocks/pull/21595, close.


Use $schema and $id in the right way

Description

Closes #282
Checklist - did you ...

[x] Add a file to the news directory (using the template) for the next release's release notes?

[x] Add / update necessary tests?
[ ] Add / update outdated documentation?

xref https://github.com/pydantic/pydantic/issues/1478

I'm not sure about the migration path for this change. Need to take a look at the validation code in conda-build, but with this change it looks like we could just use JSON schema directly without having menuinst around.
We should also publish the schema to jsonstore.
Questions to address:

How do we version the schema? I've seen several options:

Independent versioning with SchemaVer or regular SemVer. Right now we just have the schema, but it will keep evolving as we are seeing in https://github.com/conda/ceps/pull/98.
Same as menuinst? I don't like this, should be tied to a tool agnostic method.

How do we name the schema filename once we decide on a version scheme?
Do we keep all schema versions around in the repo?
Think of runtime validation.

I'm not sure about the migration path for this change. Need to take a look at the validation code in conda-build, but with this change it looks like we could just use JSON schema directly without having menuinst around.
We should also publish the schema to jsonstore.
Questions to address:
* How do we version the schema? I've seen several options:

  * Independent versioning with SchemaVer or regular SemVer. Right now we just have _the_ schema, but it will keep evolving as we are seeing in [CEP-11: Update schema with menuinst 2.1.x and 2.2.x changes ceps#98](https://github.com/conda/ceps/pull/98).
  * Same as menuinst? I don't like this, should be tied to a tool agnostic method.

* How do we name the schema filename once we decide on a version scheme?

* Do we keep all schema versions around in the repo?

* Think of runtime validation.

I'm not sure about the migration path for this change. Need to take a look at the validation code in conda-build, but with this change it looks like we could just use JSON schema directly without having menuinst around.
We should also publish the schema to jsonstore.
Questions to address:
* How do we version the schema? I've seen several options:

  * Independent versioning with SchemaVer or regular SemVer. Right now we just have _the_ schema, but it will keep evolving as we are seeing in [CEP-11: Update schema with menuinst 2.1.x and 2.2.x changes ceps#98](https://github.com/conda/ceps/pull/98).
  * Same as menuinst? I don't like this, should be tied to a tool agnostic method.

* How do we name the schema filename once we decide on a version scheme?

* Do we keep all schema versions around in the repo?

* Think of runtime validation.

This should probably be an issue to discuss this more, but I think schema version and menuinst version should be independent. The major versions will probably coincide though.
I don't know much about SchemaVer, but if this is commonly used, I don't see why we wouldn't use that one for the schema. For the file name, I would then prefer something like menuinst.schema.<SchemaVer>.json since SchemaVer uses dashes.

@wolfv, any thoughts about this?

I just removed $id from parsing in the rust version, too.


Let check for updates from dev channel

It will be good to have possibility to check for updates inside FD IDE from dev channel


feat/implement-ice: Styling

The styling of the in-game window needs upgrading. Currently, there are three colours used: a blue background, golden text and purple buttons. I suggest:

Different colour buttons for bet, call fold.
Background colours for "status" window (the window which shows text).
The three colours I mentioned above can be changed. I chose them because I started off with the logo colour and used that for the text, then picked 2 other colours that I thought would suit.

I think this ticket should be completed after #4 .

Code for implementing styling: https://github.com/TelluricDeckay/telluricdeckay/blob/feat/implement-ice/src/gui/style.rs


feat: Improve IntelliJ support for json fields

These usages in particular are useful when writing tests.
Pull request checklist

[x] Please read our contributor guide
[ ] Consider creating a discussion on the discussion forum
first
[x] Make sure the PR doesn't introduce backward compatibility issues
[x] Make sure to have sufficient test cases

Pull Request type

[ ] Bugfix
[x] Feature
[ ] Refactoring (no functional changes, no api changes)
[ ] Build related changes
[ ] Other (please describe):

Changes in this PR
The two constructors in GraphQLResponse have the json field annotated as containing JSON.
Alternatives considered
N/A

@srinivasankavitha Please let me know if other changes are needed.


Algorithmic pictures

Consider:
dynamicPicture :: (Point -> Color, Number, Number) -> Picture

then dynamicPicture(f, w, h) is a rectangle of width w and height h, the color of each pixel being taken by sampling the function f with the appropriate coordinates.  For examples of some of the pretty amazing things that can be done in this model, see https://www.shadertoy.com.
The major open question is whether this would ever be efficient enough to work.  It appears that shadertoy.com works by compiling user programs to the GPU, which probably isn't an option with simple Haskell code.

This is almost surely not feasible; you need a GCU to get reasonable performance from this model.  So I'll just close it.
(@nomeata did something at https://www.joachim-breitner.de/blog/742-WebGL%2C_Fragment_Shader%2C_GHCJS_and_reflex-dom, but it's just using GHCJS to build the machinery for injecting imperative code.)

You could have
dynamicPicture :: (Point -> Color, Number, Number) -> Picture

if you use a compiler plugin a la  Conal’s compiling to categories to compile the function passed to dynamicPicture to Shader :-)


[Enhancement] Remove db lock when planning olap table query

What type of PR is this：

[ ] BugFix
[ ] Feature
[x] Enhancement
[ ] Refactor
[ ] UT
[ ] Doc
[ ] Tool

Which issues of this PR fixes ：

Fixes https://github.com/StarRocks/starrocks/issues/15627
Problem Summary(Required) ：

Remove the db lock when planning olap table query
deep copy the olap table meta when query analyze
when query optimizer, use the copied olap table meta all the time.

[FE PR Coverage Check]
:heart_eyes: pass : 27 / 33 (81.82%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/catalog/Table.java
0
1
00.00%
[561]

:large_blue_circle:
com/starrocks/qe/StmtExecutor.java
0
2
00.00%
[394, 396]

:large_blue_circle:
com/starrocks/sql/StatementPlanner.java
23
26
88.46%
[67, 98, 99]

:large_blue_circle:
com/starrocks/catalog/OlapTable.java
3
3
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/rule/tree/AddDecodeNodeForDictStringRule.java
1
1
100.00%
[]

[FE PR Coverage Check]
:heart_eyes: pass : 27 / 27 (100.00%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/sql/optimizer/MvRewritePreprocessor.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/analyzer/QueryAnalyzer.java
5
5
100.00%
[]

:large_blue_circle:
com/starrocks/sql/analyzer/AnalyzerUtils.java
9
9
100.00%
[]

:large_blue_circle:
com/starrocks/sql/StatementPlanner.java
12
12
100.00%
[]

run starrocks_admit_test


Bazel Build all_protos proto_library Support

I would like to build a proto_library for all the protobuf files in (core) tensorflow.
Specifically, I need to build the GraphDef protobuf message (and later target it for Golang) and include it in a custom proto definition. However, it seems only cc and py are visible/supported. It does not seem like the general proto_library is exposed by the tensorflow repo. Is there a simple way to do this? Ideally there would be a proto_library definition that is available in the core/BUILD file.
When I try to execute "tf_additional_all_protos" after successfully loading it from build_config.bzl, I get an error saying no such package 'tensorflow/core'.
Any suggestions?
Have I written custom code
Bazel Build
# bazel build :my_tf_proto
ERROR: /Users/zane/dev0/piran/test/BUILD:4:1: no such package 'tensorflow/core': BUILD file not found on package path and referenced by '//:_tensorflow_all_protos'
ERROR: Analysis of target '//:my_tf_proto' failed; build aborted: Analysis failed
INFO: Elapsed time: 0.346s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (1 packages loaded)

Environment
OS Platform and Distribution
# bazel version
Build label: 0.15.2
Build target: bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Tue Jul 17 12:22:40 2018<PHONE_NUMBER>)
Build timestamp:<PHONE_NUMBER>
Build timestamp as int:<PHONE_NUMBER>
#
# lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 16.04.5 LTS
Release:	16.04
Codename:	xenial

tf.proto
syntax = "proto3";

package mytf;

import "tensorflow/core/framework/graph.proto";

message MyMessage {
  tensorflow.GraphDef myGraph = 1;
}

WORKSPACE

git_repository(
    name = "io_bazel_rules_closure",
    remote = "https://github.com/bazelbuild/rules_closure.git",
    tag = "0.8.0",
)

git_repository(
    name = "com_github_tensorflow_tensorflow",
    remote = "https://github.com/tensorflow/tensorflow.git",
    commit = "0c8a8289da120ee353c4fba5decb0bea9014e0a7"  # Sep 18, 2018
)
load("@com_github_tensorflow_tensorflow//tensorflow:workspace.bzl", "tf_workspace")
tf_workspace()

BUILD
package(default_visibility = ["//visibility:public"])
load("@com_github_tensorflow_tensorflow//tensorflow/core:platform/default/build_config.bzl", "tf_additional_all_protos")

proto_library(
  name = '_tensorflow_all_protos',
  srcs = tf_additional_all_protos(),
)

proto_library(
	name = 'my_tf_proto',
	srcs = ['tf.proto'],
	deps = [':_tensorflow_all_protos'],
)

TensorFlow installed from: N/A
TensorFlow version: N/A
CUDA/cuDNN version: N/A
GPU model and memory: N/A
Exact command to reproduce: bazel build :my_tf_proto
Mobile device: N/A

Thank you for your post. We noticed you have not filled out the following field in the issue template. Could you update them if they are relevant in your case, or leave them as N/A? Thanks.
Have I written custom code
OS Platform and Distribution
TensorFlow installed from
TensorFlow version
CUDA/cuDNN version
GPU model and memory
Exact command to reproduce
Mobile device

Any suggestions or feasible workarounds?

@yongtang Any inputs ?

Really not familiar with bazel setup with respect to protobuf. Wondering if @gunan has any insight with this issue?

Nagging Assignee @yongtang: It has been 14 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @yongtang: It has been 29 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @yongtang: It has been 44 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

what the hell is going on

Hi @zanes2016 ! Can you check with below command once ? Thank you!
bazel build //tmp:my_tf_proto


some people may prefer use css

This was done in https://github.com/select2/select2/pull/3350 about a year ago.


Opening some SVG files consumes 30+ GB RAM

When opening the following file in the demo app it consumes 30+ Gb RAM. I had to kill it because it has exhausted both my RAM and swap, so I don't know if it will render successfully if given enough memory.
https://inkscape.org/~inkscape/★about-screen-inkscape-092
Rendering the same file with resvg and cairo backend takes only 25Mb RAM.
Tested on commit 5d5d30346bf4703ba49244e85bcb8c6ed7df48e9

Here is a simpler reproducing example from openclipart:
turtle_jurgen_gaeremyn_01.svg.zip

Stumbled on the same issue.

The turtle picture is fixed with 40377d657aaff537a9869698690779111d564716 and previous commits (though the gradients don't show up because the SVG frontend is incomplete).

The original test case no longer consumes a ton of RAM, though it doesn't render properly (which may just be because the SVG frontend is incomplete). Closing. Thanks for the bug report!

A quick smoke tests on a small subset of OpenClipArt shows no other anomalous behavior. Nice!


add place, geo coordinates to structured data for each track

Thank you for contributing!Preview these changesLink expires: Sun Oct  7 06:48:39 UTC 2018Note: it may take a moment for the branch to deploy. Check all pending travis builds have completed


Datetime filter issue

Date filter is not working properly for collection LC8_30_16D_STK-1 (not tested for other collections).
An interval of 14-days is informed but no item is returned.
For the examples bellow, the date parameters of the requests have the same start date but different end dates (highlighted in bold).
--> requested datetime: 2018-08-30/2018-09-12 (interval of 14-days)
http://brazildatacube.dpi.inpe.br/stac/search?collections=LC8_30_16D_STK-1&datetime=2018-08-30/2018-09-12
<-- (no result)
--> requested datetime: 2018-08-30/2018-09-13 (interval of 15-days)
http://brazildatacube.dpi.inpe.br/stac/search?collections=LC8_30_16D_STK-1&datetime=2018-08-30/2018-09-13
<-- items from 2018-08-29/2018-09-13 interval
--> datetime: 2018-08-30/2018-09-14 (interval of 16-days)
http://brazildatacube.dpi.inpe.br/stac/search?collections=LC8_30_16D_STK-1&datetime=2018-08-30/2018-09-14
<-- items from 2018-08-29/2018-09-13 (same as of above) and 2018-09-14/2018-09-29 intervals

Moved to bdc-stac

Moved to bdc-stac


feat(ssa refactor): mem2reg opt pass

Related issue(s)

Resolves #1353
Description
Summary of changes

Dependency additions / changes

Test additions / changes

Checklist

[ ] I have tested the changes locally.
[ ] I have formatted the changes with Prettier and/or cargo fmt with default settings.
[ ] I have linked this PR to the issue(s) that it resolves.
[ ] I have reviewed the changes on GitHub, line by line.
[ ] I have ensured all changes are covered in the description.

Documentation needs

[ ] This PR requires documentation updates when merged.

Additional context

fyi, I removed the .rev() from BasicBlock::remove_instruction  as it was removing the element at the mirrored position.


Error while generating virtual frame product CPF_L1VFRA

I'm trying to simulate the processor generating the CPF_L1VFRA product. The scenario configuration is the following:
{
    "mission": "biomass",
    "scenarios": [
        {
            "name": "Framer",
            "file_name": "framer",
            "processor_name": "Framer",
            "processor_version": "01.00",
            "task_name": "Framer",
            "task_version": "01.00",
            "log_level": "debug",
            "begin_end_position_from_toi": true,
            "toi_start_offset": 0,
            "toi_stop_offset": 0,
            "anx": [
                "2017-02-25T05:00:18.883013Z",
                "2017-02-25T06:38:29.864514Z",
                "2017-02-25T08:16:40.846015Z",
                "2017-02-25T09:54:51.827516Z",
                "2017-02-25T11:33:02.809016Z",
                "2017-02-25T13:11:13.790517Z"
            ],
            "outputs": [
                {
                    "type": "CPF_L1VFRA",
                    "output_path": "outputs/CPF_L1VFRA"
                }
            ]
        }
    ]
}

The TOI in the job order corresponds to the sensing start/stop of the input slice S3_RAW__0S product.
Running procsim I got the following messages:
(pnode)  % procsim -t framer -j JobOrder.xml framer-scenario.json
2022-05-25T08:35:11.132838+00:00 ESR01104747 Framer 01.00 Framer [0000032228]: [I] Procsim v1.7.0 processor stub simulator
2022-05-25T08:35:11.133052+00:00 ESR01104747 Framer 01.00 Framer [0000032228]: [I] Simulate scenario Framer
2022-05-25T08:35:11.133064+00:00 ESR01104747 Framer 01.00 Framer [0000032228]: [I] Read JobOrder JobOrder.xml
2022-05-25T08:35:11.133073+00:00 ESR01104747 Framer 01.00 Framer [0000032228]: [D] JobOrder validation against schema: OK
2022-05-25T08:35:11.133081+00:00 ESR01104747 Framer 01.00 Framer [0000032228]: [I] Read task Framer from the JobOrder
2022-05-25T08:35:11.133094+00:00 ESR01104747 Framer 01.00 Framer [0000032228]: [I] Input type S3_RAW__0S: BIO_S3_RAW__0S_20170225T094604_20170225T094703_I_G01_M02_C01_T000_F057_01_BOSSJH
2022-05-25T08:35:11.133104+00:00 ESR01104747 Framer 01.00 Framer [0000032228]: [I] Input type S3_RAW__0M: BIO_S3_RAW__0M_20170225T094604_20170225T095834_I_G01_M02_C01_T000_F____01_BOSSJH
2022-05-25T08:35:11.133112+00:00 ESR01104747 Framer 01.00 Framer [0000032228]: [I] Input type AUX_ORB___: BIO_AUX_ORB____20170225T094548_20170225T095834_01_BOSSJH
2022-05-25T08:35:11.157583+00:00 ESR01104747 Framer 01.00 Framer [0000032228]: [D] Overwrite begin_end_position_from_toi from False to True
2022-05-25T08:35:11.157623+00:00 ESR01104747 Framer 01.00 Framer [0000032228]: [D] Overwrite toi_start_offset from 0.0 to 0.0
2022-05-25T08:35:11.157637+00:00 ESR01104747 Framer 01.00 Framer [0000032228]: [D] Overwrite toi_stop_offset from 0.0 to 0.0
2022-05-25T08:35:11.157653+00:00 ESR01104747 Framer 01.00 Framer [0000032228]: [D] Overwrite output_path from outputs/CPF_L1VFRA to outputs/CPF_L1VFRA
2022-05-25T08:35:11.157726+00:00 ESR01104747 Framer 01.00 Framer [0000032228]: [D] Output directory is /opt/pnode/data/wdir-s57/outputs/CPF_L1VFRA
2022-05-25T08:35:11.157745+00:00 ESR01104747 Framer 01.00 Framer [0000032228]: [D] Use begin_position as input for validity start time
2022-05-25T08:35:11.157760+00:00 ESR01104747 Framer 01.00 Framer [0000032228]: [D] Use end_position as input for validity stop time
2022-05-25T08:35:11.157789+00:00 ESR01104747 Framer 01.00 Framer [0000032228]: [E] Cannot perform framing, slice nr. is not known
2022-05-25T08:35:11.157804+00:00 ESR01104747 Framer 01.00 Framer [0000032228]: [I] Terminate with code 1

Please find attached the working directory including the scenario configuration:
wdir-s57.zip

Thank you for the extensive description and attached working directory. It really helps to resolve these issues.
We were able to reproduce the issue and solve it. Previously, a slice number was supposed to be supplied ('slice_frame_nr' in the scenario) so that the frame numbers could be determined from it. That was not changed with the implementation of these virtual frames.
You can now specify just the slice bounds and ANX times, as you have done in your attached scenario, and procsim will automatically determine the slice number.
Let us know at what point you would like us to make another release. I assume you would like to do more testing soon, so perhaps it is best to hold off until you have verified the new feature works as expected.

Hello, I tried again. Now procsim generates 3 outputs in the time interval [2017-02-25T09:46:04,  2017-02-25T09:46:58]:

BIO_OPER_CPF_L1VFRA_20170225T094604_20170225T094620_01_BOUPMV.EOF
BIO_OPER_CPF_L1VFRA_20170225T094618_20170225T094639_01_BOUPMV.EOF
BIO_OPER_CPF_L1VFRA_20170225T094637_20170225T094658_01_BOUPMV.EOF

But the input L0 slice (S3_RAW__0S) has sensing interval [2017-02-25T09:46:04.620, 2017-02-25T09:47:03.747]. So, I was expecting an additional short product with sensing stop = 2017-02-25T09:47:03.747.
Can you please check?

It seems in this case there is a slice boundary before the end of the sensing time, as determined from ANX information. In previous implementations as well as in the current one, framing happens on a single slice, causing the last part of this particular dataset to be cut off. The philosophy behind this is that the input product comprises a slice and should be within slice bounds itself.
We can change this behaviour, if you like, so that frames are constructed from any range of sensing time, no matter where they fall in the slice grid. Unfortunately today is a holiday in the Netherlands and the S&T team will be away till Tuesday. I hope that the current behaviour is not too obstructive to your testing till then.

According to the ANX information:
"anx": [
    "2017-02-25T05:00:18.883013Z",
    "2017-02-25T06:38:29.864514Z",
    "2017-02-25T08:16:40.846015Z",
    "2017-02-25T09:54:51.827516Z",
    "2017-02-25T11:33:02.809016Z",
    "2017-02-25T13:11:13.790517Z"
]

The input L0 slice (S3_RAW__0S), with sensing interval [2017-02-25T09:46:04.620, 2017-02-25T09:47:03.747] falls in the orbit [2017-02-25T08:16:40.846015Z, 2017-02-25T09:54:51.827516Z] and it is partial (i.e. it is the first slice of the data take), because the theoretical slice (number 57) should have sensing interval [2017-02-25T09:45:16.731248440, 2017-02-25 09:47:03.747056180]:
start = anx + (57 - 1) * slice_duration - 5
stop = start + slice_duration + 7

with slice_duration = 95.01580774 and anx = 2017-02-25T08:16:40.846015 but the data take starts at 2017-02-25T09:46:04.620, so the actual slice starts later than the theoretic one but ends at the same time. Why do you say that there is a "slice boundary before the end of the sensing time"?

Hello, I have tried again. Now procsim generates 4 products as expected. The start/stop sensing times seem correct:

#
Frame ID
File
Start
Stop

1
285
BIO_OPER_CPF_L1VFRA_20170225T094604_20170225T094620_01_BPF2FO.EOF
2017-02-25T09:46:04.620000
2017-02-25T09:46:20.740861

2
286
BIO_OPER_CPF_L1VFRA_20170225T094618_20170225T094639_01_BPF2FO.EOF
2017-02-25T09:46:18.740861
2017-02-25T09:46:39.744023

3
287
BIO_OPER_CPF_L1VFRA_20170225T094637_20170225T094658_01_BPF2FO.EOF
2017-02-25T09:46:37.744023
2017-02-25T09:46:58.747185

4
288
BIO_OPER_CPF_L1VFRA_20170225T094656_20170225T094703_01_BPF2FO.EOF
2017-02-25T09:46:56.747185
2017-02-25T09:47:03.747000

but I don't have the same values of the Frame ID, i.e. all the frame_id in the products seem to be greater then the expected one. As explained above, the orbit interval is [2017-02-25T08:16:40.846015Z, 2017-02-25T09:54:51.827516Z], thus, if you consider frame id 285:
anx = datetime(2017, 2, 25, 8, 16, 40, 846015)
frame_id = 285
frame_duration = 21.00316155  # according to production model
frame_final_overlap = 2
frame_grid_duration = frame_duration - frame_final_overlap
frame_start_time = anx + timedelta(seconds=(frame_id - 1) * frame_grid_duration)
frame_stop_time = anx + timedelta(seconds=frame_id  * frame_grid_duration + frame_final_overlap)

this gives:
(datetime.datetime(2017, 2, 25, 9, 46, 37, 743895), datetime.datetime(2017, 2, 25, 9, 46, 58, 747057))

which is the frame start/stop values of file #4. It seems that the frame IDs in the products are all "shifted" by two, i.e.: product #1 should have frame id 283, product #2 should have frame id 284, etc.

You are completely right. In this particular case an error occurred where the frame ID was shifted by two due to a mistake in the calculation of the first frame ID in a slice. This was resolved in PR #40 and merged into the master branch.

According to the discussion at #42, the last generated frame should be even shorter than the one indicated above (see last line):

n
start
stop
duration

283
2017-02-25 09:46:04.620000
2017-02-25 09:46:20.740733084
16.1207

284
2017-02-25 09:46:18.740733084
2017-02-25 09:46:39.743894632
21.0032

285
2017-02-25 09:46:37.743894632
2017-02-25 09:46:58.747056180
21.0032

286
2017-02-25 09:46:56.747056180
2017-02-25 09:46:58.747056180
2

Do you agree?

Hi Milo, in light of https://github.com/stcorp/procsim/issues/42#issuecomment-1152274311, the last product #4 (see list above) should not be generated (because will be generated by processing the nest slice). Please, let me know your opinion.

Indeed, in your very helpful and thorough description of the situation in https://github.com/stcorp/procsim/issues/42#issuecomment-1152274311, the last part of this particular slice is not transferred to a frame. I agree with this approach in broad lines, although there are subtleties for which I might need your input. In order to make issue tracking a little easier I recommend that we move further discussion on this topic to issue #42.

I think we can close this issue in favor of #42 .

Closed in favor of #42 .


Problem of maxCache

When I use Chart.SetMaxCache() for line chart. I found that the data will not change dynamically after reaching the value set by maxcache.How can I deal with it. Thank u.

How did you add the data? Could you show the relevant code?

How did you add the data? Could you show the relevant code?
Just like this. Implemented in a loop within a thread.

eyeQueue.TryDequeue(out eyeData);
dataChart.AddXAxisData("x" + cnt);
dataChart.AddData(0, eyeData);

The code seems fine. If you called dataChart.SetMaxCache before StartServer, it should work as expected. Could you provide more details? What version of XCharts are you using?

代码似乎很好。如果您之前致电，它应该会按预期工作。你能否介绍有关情况？您使用的是哪个版本的 XCharts？dataChart.SetMaxCache``StartServer

I've just realized that it may have been an issue due to the ignore break I had set, I kept updating the index of the x-axis causing the array to go out of bounds. It's fixed now, thanks.

The code seems fine. If you called dataChart.SetMaxCache before StartServer, it should work as expected. Could you provide more details? What version of XCharts are you using?

I've just realized that it may have been an issue due to the ignore break I had set, I kept updating the index of the x-axis causing the array to go out of bounds. It's fixed now, thanks.


How to train my own dataset?

I just want to detect the fish and capture it from a image.  One picture only contain one kind of fish. My dataset has 7 kinds of fishes image.   How to train my dataset and i can capture fish from a  image.   Thanks.

Please, see SSD_training.ipynb.

@zhanghan328 Hi, I suppose that you are working on the Kaggle competition. How is your progress with this SSD algorithm? From my experiment, the model suffered a lot from over-fitting and it could not detect fishes in the validation/testing dataset.