Can not find strtod_l function on Android device

1. System information

OS Platform and Distribution: Android 5.1, OPPO device
TensorFlow library : Tensorflow lite 2.13.0

When I try to load model and get error:
E/art: dlopen("/data/app/com.app.demo-1/lib/arm64/libtensorflowlite_jni.so", RTLD_LAZY) failed: dlopen failed: cannot locate symbol "strtod_l" referenced by "/data/app/com.app.demo-1/lib/arm64/libtensorflowlite_jni.so"...

@yen-dang-ts
In order to expedite the trouble-shooting process, please provide a code snippet to reproduce the issue reported here. Thank you!

Hi @sushreebarsa ,

Load model part:

val options = Interpreter.Options()
            val tfLite = Interpreter(
                TFLiteUtils.loadModelFile(context.assets, modelFileName),
                options
            )

loadModelFile function

fun loadModelFile(assets: AssetManager, modelFilename: String?): MappedByteBuffer {
        val fileDescriptor = assets.openFd(modelFilename!!)
        val inputStream = FileInputStream(fileDescriptor.fileDescriptor)
        val fileChannel = inputStream.channel
        val startOffset = fileDescriptor.startOffset
        val declaredLength = fileDescriptor.declaredLength
        return fileChannel.map(FileChannel.MapMode.READ_ONLY, startOffset, declaredLength)
    }

Please have a look on that

Hi @yen-dang-ts
What is the SDK and NDK version that are being used?
As per the documentation,
NDK_API_LEVEL="26"
ANDROID_BUILD_TOOLS_VERSION="30.0.3"
ANDROID_SDK_API_LEVEL="30"

are recommended for the latest versions.
Thanks.

Here is my config:
  ndkVersion "26.0.10792818"
  compileSdkVersion 33
  buildToolsVersion "30.0.3"

And issue still happen

Hi @yen-dang-ts
Thanks for the information. I believe there is a Android version compatibility issue with latest TF versions.
Are you using tflite-flutter for the task?
Have you tried compiling the libtensorflowlite_jni.so for amr64 with the the latest TF 2.13 and nightly pull and still observe the same error?
Thanks.

This problem has been reported in May for tflite-flutter: https://github.com/tensorflow/flutter-tflite/issues/73
The problem happens with Android 7.1.1 too. Not sure if Android 8.x/9.x are impacted (Android 13 is not impacted).
Where are the instructions to build libtensorflowlite_jni.so?

Hi @yen-dang-ts
Thanks for the information. I believe there is a Android version compatibility issue with latest TF versions.
Are you using tflite-flutter for the task?
Have you tried compiling the libtensorflowlite_jni.so for amr64 with the the latest TF 2.13 and nightly pull and still observe the same error?
Thanks.

@pjpratik The problem was introduced in Tensorflow Lite 2.12.0. All versions >= 2.12.0 seem to be affected. Version 2.11.0 is not affected by this issue. I also tested version 2.10.0, not affected either.

Hi @andynewman10, can you give me more context to help reproduce your issue. Do you have a toy version of your model that reproduces this issue on load? Does it run into this on any model? If you can export/share a toy project that'll be easiest, or at least tell me your imports that contextualizes your code snippets:
val options = Interpreter.Options()
            val tfLite = Interpreter(
                TFLiteUtils.loadModelFile(context.assets, modelFileName),
                options
            )

fun loadModelFile(assets: AssetManager, modelFilename: String?): MappedByteBuffer {
        val fileDescriptor = assets.openFd(modelFilename!!)
        val inputStream = FileInputStream(fileDescriptor.fileDescriptor)
        val fileChannel = inputStream.channel
        val startOffset = fileDescriptor.startOffset
        val declaredLength = fileDescriptor.declaredLength
        return fileChannel.map(FileChannel.MapMode.READ_ONLY, startOffset, declaredLength)
    }

I should note that currently we only support NDK up to 25b and not 26, you might want to see if that resolves your issues.

https://github.com/tensorflow/tensorflow/blob/1bd1b7f4b0311575ead19f14e74488a97b524555/tensorflow/lite/g3doc/android/lite_build.md?plain=1#L126
Apparently , the minimum supported version of the official build is Android 8.0

Hi @yen-dang-ts, @andynewman10, can you give me more context to help reproduce your issue. Do you have a toy version of your model that reproduces this issue on load? Does it run into this on any model? If you can export/share a toy project that'll be easiest, or at least tell me your imports that contextualizes your code snippets:

@pkgoogle You can use any model, for instance, this one (that I confirm works flawlessly):
https://tfhub.dev/tensorflow/lite-model/efficientnet/lite4/uint8/2
Click on the Download link on the right (download size: 14.34MB), and load this model in your code.
Your code does not have to do anything meaningful. Just import the TF Lite dependencies (>= 2.12.0) in your Gradle file, and make sure that in your program, you execute the lines of code you mentionned in your post above
Then run your the code on Android 7.1.1 (real device or emulator) and this will clearly replicate the issue (libtensorflowlite_jni.so issue with 'strtod_l')
Now switch to Tensorflow Lite 2.11.0 dependencies in your Gradle file : the issue will not show up.

https://github.com/tensorflow/tensorflow/blob/1bd1b7f4b0311575ead19f14e74488a97b524555/tensorflow/lite/g3doc/android/lite_build.md?plain=1#L126
Apparently , the minimum supported version of the official build is Android 8.0

Nice find. Apparently, this commit (dated August, 7) is the culprit:
https://github.com/tensorflow/tensorflow/commit/6d53a5aefdecaba99e93c0933d6cc32050c108d7
As I am writing these lines, this is the latest commit affecting this file.

@fiberflow, my bad I thought the previous comment referred to the line above that
https://github.com/tensorflow/tensorflow/blob/1bd1b7f4b0311575ead19f14e74488a97b524555/tensorflow/lite/g3doc/android/lite_build.md?plain=1#L125C1-L125C74
build --action_env ANDROID_NDK_HOME="/usr/local/android/android-ndk-r21e"

which is the NDK version not the NDK_API_LEVEL.

Hi @yen-dang-ts, @andynewman10 I was able to replicate here (with SDK API level = 22, i.e. Android 5.1): Test61951_min22.zip. I was able to get around the error by using a higher SDK level, I tried with SDK API 34. If you can update your Android version and thus SDK API level you can possibly get around this.
Hi @miaout17, can you please take a look? Thanks.

@pkgoogle Here is what I did:

I reinstalled Android Studio Giraffe from scratch using latest version
imported your project (Test61951_min22.zip) that has minSdkVersion 22 specified in build.gradle, as well as org.tensorflow:tensorflow-lite:2.13.0
ran the program on my Android 7.1.1 (API level 25) device

As expected, I am getting the exception:
Process: com.example.test61951_min22, PID: 17748
java.lang.UnsatisfiedLinkError: Failed to load native TensorFlow Lite methods. Check that the correct native libraries are present, and, if using a custom native library, have been properly loaded via System.loadLibrary():
  java.lang.UnsatisfiedLinkError: dlopen failed: cannot locate symbol "strtod_l" referenced by "/data/app/com.example.test61951_min22-1/lib/arm64/libtensorflowlite_jni.so"...
	at org.tensorflow.lite.TensorFlowLite.init(TensorFlowLite.java:137)
	at org.tensorflow.lite.NativeInterpreterWrapper.<init>(NativeInterpreterWrapper.java:62)
	at org.tensorflow.lite.NativeInterpreterWrapperExperimental.<init>(NativeInterpreterWrapperExperimental.java:36)
	at org.tensorflow.lite.Interpreter.<init>(Interpreter.java:232)

After that, I switched to org.tensorflow:tensorflow-lite:2.11.0: the exception disappears.
Conclusion: Android < 8 compatibility was suddenly lost with Tensorflow Lite >= 2.12.0.
Is that expected?

@andynewman10 Your conclusion statement doesn't seem to match your test outcome. Based on what you shared, it seems you are still having the issue. Do you mind revising your reply?
Context: I am also having exactly the same issue and opened a separate issue here. I am interested in resolution of this issue asap.

Hi @pkgoogle I have tried Test61951_min22.zip as per your instructions and updated following versions to 34. I can confirm that I am still getting the same crash on Android Emulators using Android 7 (API level 25) and below.
compileSdk 34
targetSdk 34

Conclusion: Android < 8 compatibility was suddenly lost with Tensorflow Lite >= 2.12.0.
Is that expected?

The minimum API level was increased to API 26 in order to provide guaranteed support for AHardwareBuffer,
which is used by the GPU delegate for the asynchronous API.
However, I don't think the consequences of this were fully anticipated, understood, and weighed off
when that change was made.  Breaking compatibility with ~6% of existing devices (based on the numbers on https://apilevels.com/) seems undesirable.  I think TF Lite should strive to support ~99% of existing devices.

@fergushenderson IMHO, it always feels really good when you know a piece of software works on a "best endavor" basis, targeting the widest possible audience. (on a side note, I would point out the fact that Android OS versions may sometimes be upgraded for just a few years only, but it should be noted that many apps can target Android 5 - Android 5 has plenty to offer already).
The dealbreaker for me is that the Gpu delegate is extremely restrictive. Very few models work with it. None of my models do actually.

Hi @pkgoogle Is there any update on this?
The thing is that the official documentation still says minSdkVersion is 19 for tensorflow-lite library. Increasing it to 26 is a massive device coverage drop and should be openly communicated.

Hi @pkgoogle Is there an update on this?
Please also note that the official documentation still indicates that the supported minSdkVersion is 19.

We are going to fix this.
I have been discussing this with the TF Lite team, and we have consensus that this is a problem that should be fixed.
I have analyzed the cause and possible solutions, and produced a prototype solution that allows use of AHardwareBuffer on versions of Android that support it while still remaining compatible with earlier versions of Android back to Android API level 21.  A TF Lite team member (@turbotoribio) has taken on the remaining tasks to get this fix landed.
The main part of the fix has landed:
https://github.com/tensorflow/tensorflow/commit/c7c3135b2802f3b8886f2ecefc39de49799b1e9b
The remaining parts are to change ANDROID_NDK_API_LEVEL from 26 back to 21.
The fix will be included in TF Lite 2.16.

Thanks for the reply @fergushenderson. That is a great news!
Are you also considering patching the previous versions, namely 2.12.0, 2.13.0, 2.14.0?

We are considering patching 2.15, since that could be shipped a lot sooner than 2.16.
There doesn't seem to be much benefit from patching 2.12-2.14, since users who are affected by this issue could just use 2.15 instead.  (But please correct me if I am wrong about that.)

@fergushenderson Do we know when the version 2.15 can be released on Maven central?

@fergushenderson Our project needs version 2.15 please help me to answer the release date

@fergushenderson I have just tried tensorflow-lite version 2.15.0. Now I am getting the following compilation error:
> Failed to transform guice-5.1.0.jar (com.google.inject:guice:5.1.0) to match attributes {artifactType=android-dex, asm-transformed-variant=NONE, dexing-enable-desugaring=true, dexing-enable-jacoco-instrumentation=false, dexing-is-debuggable=true, dexing-min-sdk=21, org.gradle.category=library, org.gradle.libraryelements=jar, org.gradle.status=release, org.gradle.usage=java-runtime}.
      > Execution failed for DexingWithClasspathTransform: /Users/nijatahmadli/.gradle/caches/modules-2/files-2.1/com.google.inject/guice/5.1.0/da25056c694c54ba16e78e4fc35f17fc60f0d1b4/guice-5.1.0.jar.
         > Error while dexing.
           Increase the minSdkVersion to 26 or above.

Looks like guice was introduced in 2.15.0 and requires minSdkVersion 26 which I guess stems from the use of Java 8 language features. Unfortunately, upgrading to 26 is not an option for our project.

@fergushenderson I have the same issue with @nijat-ahmadli. Please support us

@fergushenderson I have just tried tensorflow-lite version 2.15.0. Now I am getting the following compilation error:
> Failed to transform guice-5.1.0.jar (com.google.inject:guice:5.1.0) to match attributes {artifactType=android-dex, asm-transformed-variant=NONE, dexing-enable-desugaring=true, dexing-enable-jacoco-instrumentation=false, dexing-is-debuggable=true, dexing-min-sdk=21, org.gradle.category=library, org.gradle.libraryelements=jar, org.gradle.status=release, org.gradle.usage=java-runtime}.
      > Execution failed for DexingWithClasspathTransform: /Users/nijatahmadli/.gradle/caches/modules-2/files-2.1/com.google.inject/guice/5.1.0/da25056c694c54ba16e78e4fc35f17fc60f0d1b4/guice-5.1.0.jar.
         > Error while dexing.
           Increase the minSdkVersion to 26 or above.

Looks like guice was introduced in 2.15.0 and requires minSdkVersion 26 which I guess stems from the use of Java 8 language features. Unfortunately, upgrading to 26 is not an option for our project.

Same problem here.

Thanks for raising that issue with the guice dependency.  I have alerted the TensorFlow Lite team of this new issue.
The tensorflow 2.16.0 preview release is out.  Is the issue still reproducible in 2.16.0?
In general it is possible to use (most) Java 8 language features while depending only on API level 21, rather than 26.
See https://developer.android.com/studio/write/java8-support for details.
So if guice is using Java 8 language features, that doesn't necessarily mean that it needs to be incompatible with API level 21.

I have a problem too, I think it's the same problem
ERROR:C:\Users\hp.gradle\caches\transforms-3\0e1dbadb4b8b682380e0f29f2a20be93\transformed\jetified-guice-5.1.0.jar: D8: com.android.tools.r8.internal.a2: MethodHandle.invoke and MethodHandle.invokeExact are only supported starting with Android O (--min-api 26)
FAILURE: Build failed with an exception.

What went wrong:
Execution failed for task ':app:mergeExtDexDebug'.

Could not resolve all files for configuration ':app:debugRuntimeClasspath'.
Failed to transform guice-5.1.0.jar (com.google.inject:guice:5.1.0) to match attributes {artifactType=android-dex, asm-transformed-variant=NONE, dexing-enable-desugaring=true, dexing-is-debuggable=true, dexing-min-sdk=21, org.gradle.category=library, org.gradle.libraryelements=jar, org.gradle.status=release, org.gradle.usage=java-runtime}.
Execution failed for DexingWithClasspathTransform: C:\Users\hp.gradle\caches\transforms-3\0e1dbadb4b8b682380e0f29f2a20be93\transformed\jetified-guice-5.1.0.jar.
Error while dexing.

According to the error messages, guice-5.1.0.jar is not compatible with Android 21. The Java support 8 of Android by desugaring seems not working with guice-5.1.0.jar, could you please consider removing it from the dependency list of tensorflow-lite?

@fergushenderson I have just tried tensorflow-lite version 2.15.0. Now I am getting the following compilation error:
> Failed to transform guice-5.1.0.jar (com.google.inject:guice:5.1.0) to match attributes {artifactType=android-dex, asm-transformed-variant=NONE, dexing-enable-desugaring=true, dexing-enable-jacoco-instrumentation=false, dexing-is-debuggable=true, dexing-min-sdk=21, org.gradle.category=library, org.gradle.libraryelements=jar, org.gradle.status=release, org.gradle.usage=java-runtime}.
      > Execution failed for DexingWithClasspathTransform: /Users/nijatahmadli/.gradle/caches/modules-2/files-2.1/com.google.inject/guice/5.1.0/da25056c694c54ba16e78e4fc35f17fc60f0d1b4/guice-5.1.0.jar.
         > Error while dexing.
           Increase the minSdkVersion to 26 or above.

Looks like guice was introduced in 2.15.0 and requires minSdkVersion 26 which I guess stems from the use of Java 8 language features. Unfortunately, upgrading to 26 is not an option for our project.

Hi @fergushenderson, we would like to inquire about the release date of version 2.16.0 on Maven Central, as it has been released for two weeks. Additionally, do you have any further information regarding the incompatibility of Guice on Android 7 or below?

It looks like the dependency on guice is probably not needed.  So a possible work-around is to simply exclude that in your Gradle config file (e.g. build.gradle):
implementation ("org.tensorflow:tensorflow-lite:2.15.0") {
  exclude group: 'com.google.inject', module: 'guice'
}

[Credit to Khan LeViet for this suggestion.]
We have not tested this work-around yet, but I wanted to share it as soon as possible. Please let us know if this work-around works for you.

It looks like the dependency on guice is probably not needed. So a possible work-around is to simply exclude that in your Gradle config file (e.g. build.gradle):
implementation ("org.tensorflow:tensorflow-lite:2.15.0") {
  exclude group: 'com.google.inject', module: 'guice'
}

[Credit to Khan LeViet for this suggestion.]
We have not tested this work-around yet, but I wanted to share it as soon as possible. Please let us know if this work-around works for you.

I tried this, but I get the following exception when initializing the Interpreter on SDK 23:
val assetManager = context.assets
val model = loadModelFile(assetManager, "mnist.tflite")
val options = Interpreter.Options()
val interpreter = Interpreter(model, options)

java.lang.UnsatisfiedLinkError: Failed to load native TensorFlow Lite methods. Check that the correct native libraries are present, and, if using a custom native library, have been properly loaded via System.loadLibrary():
	java.lang.UnsatisfiedLinkError: dlopen failed: cannot locate symbol "strtod_l" referenced by "/data/app/com.abhi.minibrainage-2/base.apk"...
	at org.tensorflow.lite.TensorFlowLite.init(TensorFlowLite.java:137)
	at org.tensorflow.lite.NativeInterpreterWrapper.<init>(NativeInterpreterWrapper.java:62)
	at org.tensorflow.lite.NativeInterpreterWrapperExperimental.<init>(NativeInterpreterWrapperExperimental.java:36)
	at org.tensorflow.lite.Interpreter.<init>(Interpreter.java:232)
	at com.abhi.minibrainage.DigitClassifier.initializeInterpreter(DigitClassifier.kt:59)
	at com.abhi.minibrainage.DigitClassifier.initialize$lambda$0(DigitClassifier.kt:36)
	at com.abhi.minibrainage.DigitClassifier.$r8$lambda$YB-smHSmZMC0H3c5D98bqcY-rb8(DigitClassifier.kt)
	at com.abhi.minibrainage.DigitClassifier$$ExternalSyntheticLambda0.run(D8$$SyntheticClass:0)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1113)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:588)
	at java.lang.Thread.run(Thread.java:818)
Caused by: java.lang.UnsatisfiedLinkError: No implementation found for void org.tensorflow.lite.TensorFlowLite.nativeDoNothing() (tried Java_org_tensorflow_lite_TensorFlowLite_nativeDoNothing and Java_org_tensorflow_lite_TensorFlowLite_nativeDoNothing__)
	at org.tensorflow.lite.TensorFlowLite.nativeDoNothing(Native Method)
	at org.tensorflow.lite.TensorFlowLite.init(TensorFlowLite.java:132)
	at org.tensorflow.lite.NativeInterpreterWrapper.<init>(NativeInterpreterWrapper.java:62)
	at org.tensorflow.lite.NativeInterpreterWrapperExperimental.<init>(NativeInterpreterWrapperExperimental.java:36)
	at org.tensorflow.lite.Interpreter.<init>(Interpreter.java:232)
	at com.abhi.minibrainage.DigitClassifier.initializeInterpreter(DigitClassifier.kt:59)
	at com.abhi.minibrainage.DigitClassifier.initialize$lambda$0(DigitClassifier.kt:36)
	at com.abhi.minibrainage.DigitClassifier.$r8$lambda$YB-smHSmZMC0H3c5D98bqcY-rb8(DigitClassifier.kt)
	at com.abhi.minibrainage.DigitClassifier$$ExternalSyntheticLambda0.run(D8$$SyntheticClass:0)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1113)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:588)
	at java.lang.Thread.run(Thread.java:818)

The symptom that you observe (cannot locate symbol "strtod_l") indicates that the code has been built with the Android NDK (minimum) API level set to a value that is higher than the API level of the system that the code is run on.
I have sent https://github.com/tensorflow/tensorflow/pull/64451 which I suspect will fix this.  But that will require an updated build of TF Lite, if you are using a binary distribution of TF Lite.
If you are building TF Lite from source, I think it may be enough to just make sure you configure with ANDROID_NDK_API_LEVEL=21 rather than ANDROID_NDK_API_LEVEL=26 (see https://www.tensorflow.org/lite/android/lite_build#configure_workspace_and_bazelrc.)

Hi @fergushenderson Thanks for you reply. Just wanted to share that I was also able to compile and run the project after excluding guice as instructed. However, I am also getting cannot locate symbol "strtod_l" for Android SDK versions 21-25. Will look forward to the next release for the fix mentioned above.
If guice is not needed, are you also planning to remove it from the project for the next release?

@nijat-ahmadli @Abhiek187 @shawn-lin-013 @Ahmed82232 @mamahd12 @andynewman10
Fergus's change has made it to the TFLite nightly build. Could you test it out if it works in your app by change the TFLite dependency in your Gradle file to use the nightly build?
implementation ("org.tensorflow:tensorflow-lite:0.0.0-nightly-SNAPSHOT")

If everything is fine, we'll work on releasing a stable TFLite release with this fix.

Hi @fergushenderson Thanks for you reply. Just wanted to share that I was also able to compile and run the project after excluding guice as instructed. However, I am also getting cannot locate symbol "strtod_l" for Android SDK versions 21-25.

Try using the nightly build, as Khanh suggested.
For more detailed instructions on how to do that, see
https://www.tensorflow.org/lite/android/lite_build#use_nightly_snapshots.
Or alternatively, build from source, and make sure to configure with ANDROID_NDK_API_LEVEL=21 rather than ANDROID_NDK_API_LEVEL=26 (see https://www.tensorflow.org/lite/android/lite_build#configure_workspace_and_bazelrc).

If guice is not needed, are you also planning to remove it from the project for the next release?

Yes, the dependency on guice has been removed in the latest nightly builds, and should no longer be present in the next release.

The nightly build works for me!

Hi @fergushenderson Thanks for you reply. Just wanted to share that I was also able to compile and run the project after excluding guice as instructed. However, I am also getting cannot locate symbol "strtod_l" for Android SDK versions 21-25.

Try using the nightly build, as Khanh suggested. For more detailed instructions on how to do that, see https://www.tensorflow.org/lite/android/lite_build#use_nightly_snapshots.
Or alternatively, build from source, and make sure to configure with ANDROID_NDK_API_LEVEL=21 rather than ANDROID_NDK_API_LEVEL=26 (see https://www.tensorflow.org/lite/android/lite_build#configure_workspace_and_bazelrc).

If guice is not needed, are you also planning to remove it from the project for the next release?

Yes, the dependency on guice has been removed in the latest nightly builds, and should no longer be present in the next release.

Hi @fergushenderson, unfortunately, this nightly build does not work for me on Android 5.
java.lang.UnsatisfiedLinkError: Failed to load native TensorFlow Lite methods. Check that the correct native libraries are present, and, if using a custom native library, have been properly loaded via System.loadLibrary():
	java.lang.UnsatisfiedLinkError: dlopen failed: cannot locate symbol "__register_atfork" referenced by "libtensorflowlite_jni.so"...
	at org.tensorflow.lite.TensorFlowLite.init(TensorFlowLite.java:137)
	at org.tensorflow.lite.NativeInterpreterWrapper.<init>(NativeInterpreterWrapper.java:62)
	at org.tensorflow.lite.NativeInterpreterWrapperExperimental.<init>(NativeInterpreterWrapperExperimental.java:36)
	at org.tensorflow.lite.Interpreter.<init>(Interpreter.java:232)
Caused by: java.lang.UnsatisfiedLinkError: No implementation found for void org.tensorflow.lite.TensorFlowLite.nativeDoNothing() (tried Java_org_tensorflow_lite_TensorFlowLite_nativeDoNothing and Java_org_tensorflow_lite_TensorFlowLite_nativeDoNothing__)
	at org.tensorflow.lite.TensorFlowLite.nativeDoNothing(Native Method)
	at org.tensorflow.lite.TensorFlowLite.init(TensorFlowLite.java:132)
	at org.tensorflow.lite.NativeInterpreterWrapper.<init>(NativeInterpreterWrapper.java:62)
	at org.tensorflow.lite.NativeInterpreterWrapperExperimental.<init>(NativeInterpreterWrapperExperimental.java:36)
	at org.tensorflow.lite.Interpreter.<init>(Interpreter.java:232)

Hi @fergushenderson, @khanhlvg, We have reconfirmed that the nightly build performs well on Android 5 and above. Could you kindly assist us in releasing the new SDK version to Maven Central? Additionally, we would like to know about the schedule for the next release. Thank you once again for all your assistance.

Hi @fergushenderson @khanhlvg the nightly build also works for me. I have tried on Android 5 with the following dependencies. Thanks!
    implementation 'org.tensorflow:tensorflow-lite:0.0.0-nightly-SNAPSHOT'
    implementation 'org.tensorflow:tensorflow-lite-support:0.0.0-nightly-SNAPSHOT'

Hi @fergushenderson, @khanhlvg, any updates for the next release? 🙏

The fix has been cherry picked onto the r2.16 release branch,
and TF Lite 2.16.1 has been released on Maven Central now.
I have verified that the libtensorflowlite_jni.so files from TF Lite 2.16.1 do not contain any references to strtod_l.
So this issue is fixed now.

Hi @yen-dang-ts, this issue should be fixed now, do you agree? Thanks.


feat: 867 mobile buttons header

What type of PR is this? (check all applicable)

[x] 🍕 Feature
[ ] 🐛 Bug Fix
[ ] 📝 Documentation Update
[ ] 🎨 Style
[ ] 🧑💻 Code Refactor
[ ] 🔥 Performance Improvements
[ ] ✅ Test
[ ] 🤖 Build
[ ] 🔁 CI
[ ] 📦 Chore (Release)
[ ] ⏩ Revert

Description

This is still a WIP. I am attempting to only show an Octocat Mark when in mobile, but the fact there are a ton of ! in the SupabaseButton, it is preventing me from doing the simplest thing.
Going to clean that up, but might need eyes on if this breaks other things when removed.
Related Tickets & Documents

Fixes #898
Mobile & Desktop Screenshots/Recordings

Added tests?

[ ] 👍 yes
[ ] 🙅 no, because they aren't needed
[ ] 🙋 no, because I need help

Added to documentation?

[ ] 📜 README.md
[ ] 📓 docs.opensauced.pizza
[ ] 🍕 dev.to/opensauced
[ ] 📕 storybook
[ ] 🙅 no documentation needed

[optional] Are there any post-deployment tasks we need to perform?
[optional] What gif best describes this PR or how it makes you feel?

Talking about Supabase UI limitations, I'll strongly suggest we switch to Radix before this is implemented... Too many ! with the current supabase components. I can work on this after the feeds shipped

Talking about Supabase UI limitations, I'll strongly suggest we switch to Radix before this is implemented... Too many ! with the current supabase components. I can work on this after the feeds shipped

👍 on this, the amount of technical debt from these primitives is bothering everyone 😅

Forgot to add this before getting merged, invisible should still be replaced with hidden because otherwise it keeps "hogging" the space in the layout; invisible elements are still there, just transparent 😅


Convert js files to ts

Build tooling are configured. Closes #486.
TODO: Set-up eslint for coding style constraint in js/ts files. Add unit tests ?

Great work @GerkinDev! I'll check it out a soon as I can...

Now that the source code is in Typescript, I highly suggest a rewrite of some parts of the code to follow eslint recommended lint settings, or custom ones.
For now, if I can suggest some changes, I would personally replace all var keywords with const or let, & use more arrow functions to help making this keyword more easily readable, since it can only be redefined in standard functions. Also, some DOM queries are re-ran every time some functions are executed. So selecting them out of the scope of the function would be a bit faster.
There's a lot more that can be done, but I need your feedbacks to know what to do.
Happy new year :clinking_glasses:

@GerkinDev excuse me for the delay. The problem is that I am not into frontend and I currently don't have a reviewer for this on my side. But I am willing to merge this if you still want to make this change happen and are willing to help fix any fallout that might occur. Please give me a heads up! Thanks!

No activity

Hi,
I'm quite busy right now too. But changes are mostly non-functional, mainly syntax upgrades from JS to TS. So yeah, I guess I could give some minutes to fix (improbable) regressions, but there should be none.
What else would you like ?

Hi,
I'm quite busy right now too. But changes are mostly non-functional, mainly syntax upgrades from JS to TS. So yeah, I guess I could give some minutes to fix (improbable) regressions, but there should be none.
What else would you like ?


How spec complete/compliant is the library currently?

Hi there, first and foremost I just wanted to say how much I appreciate the work that's gone into this library already. I'm relatively new to the ins and outs of the FIDO2 specifications but a long time fan of the concept of Passwordless auth.
My query is: how spec compliant is the FIDO2-NET library currently? For example, I was looking at the VerifyAsync method in the AuthenticatorAttestationResponse class. It looks like possibly some of the steps aren't yet implemented from the spec, but I can't quite tell as the linked resource in the comments to the WebAuthN spec (https://www.w3.org/TR/webauthn/#registering-a-new-credential) does not appear to exist anymore (I'm guessing they rejigged the spec). I had a look around the spec but couldn't see the 19 step registration outline, but it could just be my unfamiliarity with the spec/FIDO2.
Following on, would this library be considered production ready?

@aseigler Might have more to say about the nitty-gritty regarding spec compliant (if there are any recent changes). I think the upcoming changes of MDS is a gray area, perhaps not part of the spec but should be treated as "compliance"  and it will be supported by the library.
With that said, on a larger picture this library is and aims to be spec compliant. We are also testing running FidoAlliance compliance testing and I believe were one of the first (at least OSS) projects to reach 100% compliance score.
The definition of "Production ready" differs from company to company, but I and multiple others run this library in production. However, no paid support plans are available (at the time of writing).

To add to what Anders said, there are a few things in play here, compliance to WebAuthn spec and compliance to FIDO Alliance certified server.  We tried to write the library to closely match the WebAuthn spec, sometimes you'll find links to the spec in the source comments, but it is a living spec that has had significant changes over the past few years.  We didn't plan the library from day one to align perfectly with anything, we kind of built it as we went.
In terms of compliance, the best we've got at the moment is the passing results from the conformance tools.  If some individual or organization wants to sponsor the library for server certification, I'd be willing to go through the process.

Thanks @aseigler for weighing in, and thanks again to you both for your comments. That has given me a good idea of the current state 👍


vs code debug problem

If you are reporting any crash or any potential security issue, do not
open an issue in this repo. Please report the issue via emailing
<EMAIL_ADDRESS>where the issue will be triaged appropriately.
Title: One line description
Description:

Describe the issue.
/tmp/Microsoft-MIEngine-Cmd-winez2b3.5jl: line 2: 11365 Aborted                 (core dumped) "/opt/rh/gcc-toolset-11/root/usr/bin/gdb" --interpreter=mi --tty=$DbgTerm < "/tmp/Microsoft-MIEngine-In-jfdfwozk.ewx" > "/tmp/Microsoft-MIEngine-Out-fda3xgga.2tp"

[admin@localhost envoy]$ /opt/rh/gcc-toolset-11/root/usr/bin/gdb -v
GNU gdb (GDB) Red Hat Enterprise Linux 10.2-5.el8
Copyright (C) 2021 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
[admin@localhost envoy]$

launch.json
{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "(gdb) 启动",
            "type": "cppdbg",
            "request": "launch",
            "program": "${workspaceFolder}/envoy-debug",
            "args": [ "-c", "/home/admin/envoy/examples/brotli/brotli-envoy.yaml", "--service-cluster", "brotli"],
            "stopAtEntry": false,
            "cwd": "${workspaceFolder}",
            "environment": [],
            "externalConsole": false,
            "MIMode": "gdb",
            "miDebuggerPath":"/opt/rh/gcc-toolset-11/root/usr/bin/gdb",
            "setupCommands": [
                {
                    "description": "为 gdb 启用整齐打印",
                    "text": "-enable-pretty-printing",
                    "ignoreFailures": true
                }
            ]
        }
    ]
}

envoy debug binary was build in ubuntu system,I am using centos8 to debug ,will this a prolblem?
envoy was compiled by ENVOY_DOCKER_BUILD_DIR=~/build ./ci/run_envoy_docker.sh './ci/do_ci.sh bazel.debug.server_only'
envoy version is release/v1.23
[optional Relevant Links:]

Any extra documentation required to understand the issue.

1: (2471) ->1006^error,msg="Dwarf Error: bad DWP hash table, lookup didn't terminate [in module /build/envoy/source/exe/envoy/envoy.dwp]"

cross with https://github.com/envoyproxy/envoy/issues/22788


Documentation: links are broken for implementors of traits in gtk::prelude

When you go to any trait in gtk::prelude (e.g. ButtonExt) and then click on one of the implementors (e.g. Button) you get a 404 error.

I think this was resolved in the latest docs: https://gtk-rs.org/gtk-rs/git/docs/gtk/prelude/trait.ButtonExt.html

Can you confirm that?

Yes, indeed :)


0.15 regression - Support negative numbers in the timestamp regular expression

This solves https://github.com/crsmithdev/arrow/issues/662
Currently arrow does not allow you to pass in negative numbers for seconds relative to epoch when using strings. This patch is an attempt to fix this.

@jadchaar if this is a-ok could you cut a release for this?

Codecov Report

Merging #663 into master will not change coverage.
The diff coverage is 100%.

@@          Coverage Diff          @@
##           master   #663   +/-   ##
=====================================
  Coverage     100%   100%
=====================================
  Files           8      8
  Lines        1536   1536
  Branches      244    244
=====================================
  Hits         1536   1536

Impacted Files
Coverage Δ

arrow/parser.py
100% <100%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9b05216...1974316. Read the comment docs.

Thanks @JBKahn, we'll look this over then merge if it's all good. We're probably going to wait a little before releasing 0.15.1 in case any more obvious bugs arise. Is this one really messing you guys up?

Yeah, especially since a lot of apps don't lock down the version and we have lots of historical dates being passed into arrow.

Alright we'll speed things along then.

Thanks, already had some issues with some places automatically picking up 0.15.0 if they din't lock the version and it was a sub-dependancy.


Publish Source and Docs

In IntelliJ if you check to include SBT sources it will fail to import a project with the following:
[warn]     ::::::::::::::::::::::::::::::::::::::::::::::
[warn]     ::              FAILED DOWNLOADS            ::
[warn]     :: ^ see resolution messages for details  ^ ::
[warn]     ::::::::::::::::::::::::::::::::::::::::::::::
[warn]     :: org.portable-scala#sbt-platform-deps;1.0.0-M2!sbt-platform-deps.jar(doc)
[warn]     :: org.portable-scala#sbt-platform-deps;1.0.0-M2!sbt-platform-deps.jar(src)
[warn]     ::::::::::::::::::::::::::::::::::::::::::::::

The sources and docs are definitely published on Bintray, as can be seen here. Moreover, updateClassifiers in sbt can find them. So I'm going to close this. There might be something funny going on with IntelliJ's way of resolving sources and docs artifacts.


Compare load()-ed type and values against ImageMagick load()

I was just about to propose the Int(bit_detph) change.:-)
Other thing I noticed is that ImageMagick treats some Grays are RGBs, which is valid, but I prefer what PNG.jl curently does.
E.g.:
UInt8_2: ImageMagick read type equality: Test Failed at /tmp/cirrus-ci-build/test/runtests.jl:134
  Expression: eltype(read_in) == eltype(read_in_immag)
   Evaluated: GrayA{Normed{UInt8,0x08}} == RGBA{Normed{UInt8,8}}

I agree. I generally prefer what PNG.jl does.
Perhaps some exceptions where ImageMagick seems better:
BitArray_1: ImageMagick read type equality: Test Failed at /Users/ian/Documents/GitHub/PNG.jl/test/runtests.jl:168
  Expression: eltype(read_in) == eltype(read_in_immag)
   Evaluated: Gray{Normed{UInt8,8}} == Gray{Bool}

Bool_1: ImageMagick read type equality: Test Failed at /Users/ian/Documents/GitHub/PNG.jl/test/runtests.jl:134
  Expression: eltype(read_in) == eltype(read_in_immag)
   Evaluated: Gray{Normed{UInt8,8}} == Gray{Bool}

I'm not sure how ImageMagick knows how many unique values were in the written image -- these two were saved as 8 bit depth images. In any case sizeof(Gray{Normed{UInt8, 8}}) == sizeof(Gray{Bool}) == 1 so this should not be a big problem. I would like to eventually have a Gray{Bool} for images with bit depth of 1, but for other bit depths checking whether Gray{Bool} is good eltype is not free (AFAIK).

I also noticed that I might not handle the PNGSuite background cases correctly, I'll look into it.

True, I forgot that Bool is 8-bit.
I invited you as a collaborator. Feel free to work  directly on this branch if you like. Or PR into it, whatever you prefer.
It seems like it's logical to have cases where we're not type identical to ImageMagick, so we should figure out a strategy for allowing those inequalities in the looped tests.
Also, once the types are set figuring out the value inequalities would be good, although that can be a bit of a rabbit hole

Just FYI, I've renamed to PNGFiles given the discussion on https://github.com/JuliaRegistries/General/pull/9508
Travis hasn't updated the repo name yet https://travis-ci.com/JuliaIO/PNG.jl

@Drvi I worked on master a little. Do you want to resolve the conflicts, or should I?

This was/is a bit more complicated than I expected. The load function now supports files with custom gamma values and backgrounds + cases like and RGB (with no alpha) mixed with a transparency chunk produce an RGBA.
However, making the ImageMagick tests pass is quite a challenge. I've skipped tests with sub 8 bit depth as ImageMagick doesn't handle them. IM seems to only apply gamma correction to colored images and not to grayscale images, which is problematic (PNGFiles handles colored and grayscale images in the same way wrt gamma corrections). And I think applies the gamma correction slightly differently, because on some images I get nearly identical results, but like 10% pixels are out of \approx range.
So 80 tests are failing mainly because of IM not applying the same gamma corrections to grayscale, some of these close results and in a couple of cases because the way I guess the element type from PngSuite problem name is not perfect.
Not sure what is the best way to proceed, what do you think @ianshmean?

Thanks for the resolve! :+1:

I messed it up a little. Let me fix it

On all platforms we see:
LoadError: Some tests did not pass: 1073 passed, 80 failed, 0 errored, 0 broken.

Except for MacOS which locally and on CI gives:
LoadError: Some tests did not pass: 748 passed, 405 failed, 0 errored, 0 broken.

@Drvi thinks it's a difference in how ImageMagick is handling gamma on MacOS, which I assume could just be a case of finding thee gamma correction curve for mac and isapple()-ing that.
Beyond fixing the macos issue, I guess we could:

Consider ImageMagick to be ground truth, and not accept victory until this matches exactly
Ignore ImageMagick entirely and just infer the types from the PngSuite tests to check against
Find some middle ground, which would require setting up a lookup table for whether we're allowing failures for each of the 1153 tests..

cc. @timholy What do you reckon?

Ok, now we should have the same 80 errors on all platforms.

I've ignored gamma correction for the Gray PngSuite tests, so we're now at 63 failures.
Also, I moved to a sum of absolute difference (sad) image comparison approach that returns a scalar value so we can assess the magnitude of mismatch in tests directly. (tests currently check for < 0.01 difference i.e. < 1%)
Finally skipped png_set_expand_gray_1_2_4_to_8 during load, when it was already 8-bit.

To dig deeper into the differences in
basi2c16.png: ImageMagick read values equality: Test Failed at /home/travis/build/JuliaIO/PNGFiles.jl/test/runtests.jl:246
  Expression: imdiff(read_in_pngf, read_in_immag) < 0.01
   Evaluated: 2.0768463416494907 < 0.01

(The error is roughly two times as big as for the basi2c08.png case)
original image:

PNGFiles.jl

ImageMagick.jl

Both saved files appear much brighter than the original and are pretty similar to my eye (in my files explorer at least; here is a screenshot:
). Here are channelwise differences (as in e.g. Gray.(red.(x) .- red.(y))) between the image read in by PNGFiles.jl and ImageMagick.jl:
red:

green:

blue:

So... pretty substantial differences.
Here are some chunk metadata from pngcheck
Original:
➜  test git:(compare_against_imagemagick) ✗ pngcheck -cvp ./basi2c16.png
File: ./basi2c16.png (595 bytes)
  chunk IHDR at offset 0x0000c, length 13
    32 x 32 image, 48-bit RGB, interlaced
  chunk gAMA at offset 0x00025, length 4: 1.0000
  chunk IDAT at offset 0x00035, length 522
    zlib: deflated, 32K window, default compression
  chunk IEND at offset 0x0024b, length 0
No errors detected in ./basi2c16.png (4 chunks, 90.3% compression).

PNGFiles.jl
➜  test git:(compare_against_imagemagick) ✗ pngcheck -cvp ./basi2c16_new.png
File: ./basi2c16_new.png (6244 bytes)
  chunk IHDR at offset 0x0000c, length 13
    32 x 32 image, 48-bit RGB, non-interlaced
  chunk IDAT at offset 0x00025, length 6187
    zlib: deflated, 8K window, superfast compression
  chunk IEND at offset 0x0185c, length 0
No errors detected in ./basi2c16_new.png (3 chunks, -1.6% compression).

ImageMagick.jl:
➜  test git:(compare_against_imagemagick) ✗ pngcheck -cvp ./basi2c16_new_im.png
File: ./basi2c16_new_im.png (531 bytes)
  chunk IHDR at offset 0x0000c, length 13
    32 x 32 image, 48-bit RGB, non-interlaced
  chunk gAMA at offset 0x00025, length 4: 0.45455
  chunk cHRM at offset 0x00035, length 32
    White x = 0.3127 y = 0.329,  Red x = 0.64 y = 0.33
    Green x = 0.3 y = 0.6,  Blue x = 0.15 y = 0.06
  chunk bKGD at offset 0x00061, length 6
    red = 0xffff, green = 0xffff, blue = 0xffff
  chunk IDAT at offset 0x00073, length 396
    zlib: deflated, 8K window, maximum compression
  chunk IEND at offset 0x0020b, length 0
No errors detected in ./basi2c16_new_im.png (6 chunks, 91.4% compression).

And here are metadata info from ImageMagics cli tool identify:
Original:
original_image.txt
PNGFiles.jl:
pngfiles_image.txt
ImageMagick.jl:
imagemagick_image.txt

When I manually set the gamma in GIMP to 2.2 (~ 1 / 0.454545...) and save the image without any further gamma chunk, ImageMagick and PNGFiles agree on the read in image.

When I then compare this image that is gamma adjusted manually to those read in by PNGFiles.jl and ImageMagick.jl, I can see that this manual adjustment is equal to what PNGFiles.jl does, but differs from what ImageMagick.jl does.

As another datapoint, it seems that pythons OpenCV wrapper ignores gamma chunk when reading in an image
julia> using PNGFiles, PyCall, Images
julia> cv = pyimport("cv2");
julia> x = reinterpret(N0f16, cv.imread("basi2c16.png", cv.IMREAD_UNCHANGED)[:,:,[3,2,1]]);
julia> opencv = colorview(RGB, x[:,:,1], x[:,:,2], x[:,:,3]);
julia> all(PNGFiles.load("basi2c16.png", true) .≈ opencv)
true

Just a couple of notes for myself: It seems that ImageMagick has some rules/heuristics that governs whether an image is read in in a linear/non-linear Gray/RGB colorspace. Most of this is set in ReadOnePNGImage with the more interesting parts happening here (often using SetImageColorspace. Here are some older notes about colorspaces in ImageMagick. Using blame one can see comments like that Set linear colorspace when gamma is over 0.75, not when exactly 1.0.
One can see a little bit into the process when using the IM CLI with -debug all.

I've not looked at this repo, but just in general terms I am aware of cases where IM messes with the actual values saved to disk. One of the reasons this and many other tests spans the full range of intensities is to try to keep IM from messing with the histogram.

Good to know. Maybe we should port the relevant ImageMagick tests over to here

Though I should say I haven't looked for this since the JLL transition, it's possible our own compiled version of IM may be more consistent. I'm not seeing it with the simple examples I expected to show this. But still worth knowing that at least historically this was an issue.

I've been having a great time switching out using FileIO>ImageMagick with PNGFiles directly.. A huge speed improvement. I'm worried that we're chasing a difficult goal/perfect here.
We could:

go ahead and merge https://github.com/JuliaIO/FileIO.jl/pull/257 and keep working on this PR, with the caveat that some returned types might be weird (although ImageIO is opt-in, so less of a concern)
merge this PR with failures, release a new version, and then merge the FileIO PR
keep going on this until all green, release, then merge FileIO PR

What do you think @drvi ? If time is limited or a clear solution to fix failures is missing, I'm tempted to say we do option  1

@ianshmean This was a very crazy week for me workwise, sadly I didn't have the time work on this.
Question about FileIO.jl: let's say that we do add PNGFiles.jl to FileIO.jl, but we put it behind IM (so IM is still the default png handler), is there a way the user can choose the backend they want? If yes, people who know about PNGFiles.jl could opt-in to speed up standard cases with sRGB pngs and other users won't be bitten by these edge-cases I still am not sure we handle sensibly.

That's not quite how we have it set up. I think we have a nice arrangement now:
FileIO > png format > ImageIO, QuartzImageIO (Mac only), ImageMagick (in that order)
ImageIO

Installs PNGFiles, (XXFiles, once we make them)
Doesn't load them until their specific load/save methods are requested

That means that when we merge the FileIO PR, any package that already uses FileIO for png's that must have had ImageMagick and/or QuartzImageIO as a dep won't break because FileIO will silently see that ImageIO isn't installed and look for the others. Anyone who installs FileIO in a fresh environment and tries to load a png would then get a suggestion to install, in order: ImageIO, QuartzImageIO (Mac only), ImageMagick.
i.e. we won't break established packages or environments, but will likely be used in new projects where people are less likely to depend on returning specific types

We're down to 20 errors, a good couple of them is just slighly above 1 % error, and about the around 2 %. For some images, I don't understand what IM does and actually find our results preferable: look at Images with different gamma chunks -- my browser gamma-corrects them such that they seem nearly identical, which is the same as output as we produce; IM actually created versions with different levels of brightness. I'll look into the other cases later.

If you've inspected all the failures (which is certainly essential), then given my experience with IM I'd be strongly predisposed to believing you when you say

actually find our results preferable

Here I enclose a little analysis of the 20 erroring test cases. The columns of the images represent:
1: PNGFile
2: ImageMagick
3: absolute differences
4: scaled differences
The first row contains all channels, each subsequent rows is a per channel view.
I think that the problem is that IM has some heuristics wrt application gamma correction. In the html, all cases are non sRGB images with a gAMA chunk (i.e. having gamma value). The cases starting with g are explicitly varying gamma values and one can see that here the differences are the largest. IM is actually doing gamma correction here, but the results do not look very good (in theory, the g pictures should look very similar after gamma correction).
I don't know if there is a deep reason behing what IM does. At this point I think that we should apply gamma correction consistently (i.e. every time there is a sRGB chunk or gAMA chunk) even for Gray images (IM doesn't seem to gamma correct these), but we should provide the user the option to skip gamma correction or provide a custom value.
If no one is against, I'd clean this MR, add a gamma kwarg and ignore these 20 errors (increase the imdiff tolerance with a comment).

I agree with Tim, if you’re satisfied with the result, go for it. QuartzImageIO and ImageMagick still don’t agree for some test images, and we were never able to figure out which one was “right”.
I haven’t looked through any of this, but I wouldn’t be surprised if it’s a good bit faster than IM and even QIIO. OMETIFF.jl is a pure Julia Tiff reader, and it’s faster than libtiff (which QuartzImageIO uses behind the scenes)

I think we're in a good place here. Thanks @Drvi for being so thorough!
The windows travis build failures seems to be a travis issue with a recent update https://travis-ci.community/t/an-operating-system-not-supported-suddenly-appears/7525/2
I'll merge and we can push forward with the FileIO roll-out!

Thanks indeed @Drvi and @ianshmean!


[MINOR][PYTHON][TEST] Use collect() instead of show() to make the output silent

What changes were proposed in this pull request?
This PR replace an effective show() to collect() to make the output silent.
Before:
test_simple_udt_in_df (pyspark.sql.tests.SQLTests) ... +---+----------+
|key|       val|
+---+----------+
|  0|[0.0, 0.0]|
|  1|[1.0, 1.0]|
|  2|[2.0, 2.0]|
|  0|[3.0, 3.0]|
|  1|[4.0, 4.0]|
|  2|[5.0, 5.0]|
|  0|[6.0, 6.0]|
|  1|[7.0, 7.0]|
|  2|[8.0, 8.0]|
|  0|[9.0, 9.0]|
+---+----------+

After:
test_simple_udt_in_df (pyspark.sql.tests.SQLTests) ... ok

How was this patch tested?
Manually tested.

cc @BryanCutler

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3272/
Test PASSed.

Test build #96311 has started for PR 22479 at commit ceef555.

Thanks @HyukjinKwon. LGTM

Merged to master and branch-2.4.

Thanks, @dongjoon-hyun and @viirya


Opensource version?

Quick analysis give dependency from COM mshtml. Is it possible to build this application without it ?
 Microsoft.mshtml, Version=7.0.3300.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
 Microsoft.VisualStudio.QualityTools.UnitTestFramework, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a, processorArchitecture=MSIL
 PresentationCore
 PresentationFramework
 UIAutomationClient
 UIAutomationTypes
[~] Accessibility
[~] Microsoft.CSharp
[~] System
[~] System.Core
[~] System.Data
[~] System.Data.DataSetExtensions
[~] System.Deployment
[~] System.Drawing
[~] System.Windows.Forms
[~] System.Xaml
[~] System.Xml
[~] System.XML
[~] System.Xml.Linq
[~] WindowsBase

The XsltViewer uses mshtml to render HTML.  So you would have to comment out that feature or find another browser to use instead of that.


RFC: [META] Improve our release process

Problem
Recently releases that we have been putting together for core have not been very easy. There's a couple of things that we come across that makes it hard:
When security patches are involved the release process becomes a lot harder
Security patches means that RCs cannot be public - stable release must be the point of disclosure. Quoting @unclecheese:

This complicates things. We have to release from the security fork and that involves setting up a satis server and arm wrestling with cow a bit. It doesn't set well with me when releases are so non-deterministic, so to speak, and require a lot of thinking and different paths to take depending on varying conditions. It seems very error prone, and hitting the big red button lacks the confidence and predictability it should have.

It becomes difficult to isolate critical fixes from general fixes on a bugfix branch
This was made abundantly clear on the recent 4.3.0 release. The RC was prepared and then tested - finding a few critical issues. Although you could argue that testing should be done before an RC it doesn't detract the fact that critical issues can be found after an RC is created. Over the ~4 weeks it takes for an RC to go through a security audit (if applicable) and further user testing we fix these issues but we also still accept contributions and other minor fixes into the 4.3 branch.
The issue arises when we decide what exactly we want to apply to the actual 4.3.0 stable release and what can wait for 4.3.1. Again, you could argue that all bugfixes applied between RC and stable release should go into the stable but that will mean all these fixes go un-audited in any security audit (unless there's a follow up audit). In the end it's practical to have the ability to "pick and choose" so to speak.
Current state
This is from what I know that might not be 100% - I'm not incredibly intimate with this process yet. The release process is a long (4-6 weeks!) process that involves one (or more) RC(s) that are usually security audited when bundled with a CWP release
Currently when it's time to begin the release process we will:

Release an RC1:

We use cow to make a plan and generate a changelog.
In core, all modules are tagged with an RC1 suffix.
In CWP (ie. all other supported modules) we will release at this point unless we expect security patches or critical hotfixes (rare)

Perform sanity checks
If there are security issues to be audited we pull those in manually to a zip into package up for auditing
Ship the RC for auditing
When critical bugs arise we fix them on the minor branch (ie. 4.3 for 4.3.0-rc1)
When non critical bugs arise it's the same process - target the minor branch
When security issues are raised:

We prep a patch in a private fork of the affected repo
The patch is discussed, reviewed and approved but cannot be merged
The PR sits open until stable release time

When it's time to release we must retrospectively identify issues that should be included into stable (that were merged into minor release branches)
We create a "release branch" from the commit that RC1 was tagged at
We cherry pick fixes into that release branch
We merge in security fixes from the private repo
We release with cow using that release branch
We have to cherry pick or merge back the security fix after the release into the patch branches

Proposed state
@sminnee @unclecheese @chillu and I took a moment to sit down and work through this in a whiteboard session. Originally the core contributors were discussing having a private "releases" fork of all our modules to assist with releases. The initial concern is we'll be overcomplicating the process because a third repo is involved. Our discussion concluded that we should not create new forks on a new repo but instead use the existing forks on our current private repo.
"It's time to start a release":

Create release branches on our private repo for the modules we will be tagging with RC1 (eg. 4.3.0-release) - preferably have a tool to automate this (cow)
Create a Satis configuration to point to this repo with the packages that have been branched. Satis can be configured to only provide some of the packages from a repo. - This could also be part of the previous step if automated
Tag an RC1 tag on the private repo, sanity check, and push to origin (release publicly) always.
When critical bugs arise those are fixed in the public minor branch (ie. 4.3 for 4.3.0-rc1) - same as before

A release manager may then cherry pick this into the private release branch if this needs to be released in the stable release

Minor bugfixes are the same as before - they target the minor release branch.
If there are security PRs in flight we can re-target them to our release branch as they already exist on our private repo. These PRs are merged when ready
Release an RC2 if applicable. This and subsequent RCs should be made public so long as there are no security patches included.
Release an RC before auditing and ship that to the auditor
Continue with bugfixes - we can now get a solid list of changes that are not audited with something like git log 4.3.0-rc2..HEAD because we're providing a tag (and nothing additional) to the auditor
Responses from the auditor are actioned and merged into the release branch
Tag stable from the release branch and push the stable tag to origin
Merge the release branch back into the minor branch
Delete the release branch - publish any previously private RCs (if applicable)

How is this different
Some major advantages from my point of view:

We always have a known state of the release - just look at the release branches
We are removing effort at stable release time and spreading it through the whole process (= less stress at stable release time)
We're relying on having a solid release process because we're tagging RCs more often
We can safely "roll-back" a tag because they will be created on a private repo and then pushed to origin

And disadvantages:

Cow updates
Satis is always part of our release process

Please let me know @chillu, @unclecheese, and @sminnee if there's something you understood differently from our chat - or anyone else if there's something I've gotten confused about.
It would be great if @dhensby, @robbieaverill, and @tractorcow had any feedback, suggestions, or advice as you are all quite familiar with the release process!

This does make repo management difficult. We don't need repos for every module - in CWP we usually just release modules at RC stage. The difference is that we will still need private repos for any module that may have a security issue. This means manually sorting out private repos in those cases.

we don't include security fixes in RC releases anyway, only stable

To be clear this flow suggests changing this. RCs (after RC1) may contain security fixes. Whether they do or not will indicate whether they're made public.

Another option in my mind is that core releases are tagged as stable when we think they're stable, we include a stable release in the CWP audit releases (rc1) and then any security issues arising are fixed, revalidated and released as a patch release.

I am still also a fan of this approach, but it hasn't seemed to gain much traction. It'll be nice to outline why this is but I'm not sure.

I am a fan of the idea of making security and regular, silverstripe-installer and cwp-installer releases follow the same release process every time - at a high level

Another option in my mind is that core releases are tagged as stable when we think they're stable, we include a stable release in the CWP audit releases (rc1) and then any security issues arising are fixed, revalidated and released as a patch release.

I don't think we've explicitly discarded this option in our whiteboarding session. As you say, it means we're more likely to perform two patch releases in quick succession if there's pending security issues. But more patch releases is a good thing, and helps get non-security bugfixes out faster.
RCs are most useful on minor releases, not patch releases. They allow more disruptive changes to bed in before we call that release line stable, with the assumption that there's early adopters who are willing to help us test in RC stage, but aren't willing to run dev release branches. I think that subset is small enough to be ignored, most early adopters are happy running dev release branches. The situation changes once we prepare a new major release (5.0.0), but let's not worry about that right now.
Can someone clarify in which of these proposed cases we need more than 20 repos on github-security? That's the main point of contention for me at the moment (💰). Could we cover this by making the first step in the release process "identify which repos have pending security issues, then point satis at those repos only"? Or would that be too much inconsistency @unclecheese? That could also save us a lot of cow automation, because typically you have pending security issues on one or two repos, which we could handled with a well documented manual process. Any process we need to run for ~15 core repos (or ~90 supported modules) needs to be automated.

Having done a couple of these new "private" releases, my perception is that for core releases, the private repos need to be all or nothing. We can't just whitelist a few affected repos to be behind the curtain for a a given release.
The reason for that is that core releases tag everything, regardless of whether it contains new commits since the previous release. This is done to keep semantic versioning parity across the whole release. If we were to only put some modules in private, we'd end up with public tags for<EMAIL_ADDRESS>when<EMAIL_ADDRESS>doesn't even exist.
Two ways around that:

We release more like CWP and stop tightly coupling the version numbers
We don't care if there's a new release floating around publicly that isn't yet tied to a stable recipe.

Assuming those things remain unchanged, we need to have > 10 repos in the private forks, which pushes us into a higher priced plan with Github.
Other options:

We use the security repos on an as-needed basis (i.e. has a pending security PR), and create a new workspace for silverstripe-releases that is all inclusive. Could be the USD $200 fee by having only one contributor (cow).
We move the private repos to something other than Github, like Gitlab, or Bitbutcket.

Having done a couple of these, I can say that it's really refreshing to work with an entirely private release, and not have to worry about what repo pairs with which remote. It feels much more automation friendly and less error prone.
I wonder how this could be baked into cow, as well, so we don't have to manually remind it to publish to a custom repo.
$ cow release:create 4.3.2 silverstripe/installer
$ cow release:stage https://security-repo.git http://my-satis.local
$ cow release:publish <-- assumes stage
$ cow release:publish --public

But that's probably a separate PR/RFC for cow.

If we were to only put some modules in private, we'd end up with public tags for<EMAIL_ADDRESS>when<EMAIL_ADDRESS>doesn't even exist.

That's not entirely accurate. We'd end up with public tags for<EMAIL_ADDRESS>and private tags for<EMAIL_ADDRESS>For the final release, both of them are public. This mix means that anyone pinning to @rc dependencies would run a mixed set of patch releases - which should really work. We could reduce the potential impact of this by mandating that any cross-module bugfixes need to go into the next minor release branch. I think people either pin to @stable or @dev anyway, so I'm not too worried about that edge case. If we accept that, would it make it easier?

Could beat the USD $200 fee by having only one contributor (cow).

I'm not a fan of systems which break our audit trail. cow creates commits (e.g. merge commits) and tags with the user's Github account. If we make this a generic, shared account, you can't tell who's performed those actions. And shared accounts with this level of access are too much of a single point of failure. We mandate MFA, which would get difficult with a shared account (although we could pass around a Yubikey).

We move the private repos to something other than Github, like Gitlab, or Bitbutcket, which may have cheaper plans.

I'm veto'ing that, we already have enough trouble with user management on the various Github organisations (and potential tooling around that).
@robbieaverill @ScopeyNZ If we went for the $100/month plan and 50 repos, we'd cover core, and you'd still have a few repos spare in case security issues in supported modules come up. Does that complicate your cwp release process? Assuming that CWP pre-releases often need to include unreleased core versions with security fixes, this means you need to work with Satis as part of the release process as well. It sounds like that's acceptable? I think increasing our release management costs by $50USDx12 per year is a bit more palatable than an increase of $150USDx12 per year.

We stop caring if there's a new release floating around publicly that isn't yet tied to a stable recipe.

There's no problem with doing this as long as they're not on recipes - they wouldn't be installable without all of their child dependencies having public tags. Modules like assets for example could be tagged individually and it wouldn't really matter.

We release more like CWP and stop tightly coupling the version numbers

+1 for this too. There's really no reason to do this other than historic policy and it being easy to work out which version goes with which version. It's the recipes that matter, not the module versions.
In some cases someone might be running assets 1.2.3 against framework 4.2.2, which may have a fix for a bug in assets and not yet have a corresponding framework patch - that's really the only downside I can think of though.

@robbieaverill @ScopeyNZ If we went for the $100/month plan and 50 repos, we'd cover core, and you'd still have a few repos spare in case security issues in supported modules come up. Does that complicate your cwp release process?

We typically don't need that many private repositories for CWP releases since (as @unclecheese noted) we don't tag everything every time we do a release, only what needs to be released. There are some parts which usually do get tagged together though: kitchen sink, installer, cwp-recipe-cms, cwp-recipe-core, cwp and cwp-core. If any of these recipes/modules have dependencies (like framework) that require a security release then we'd need a private repository for it.
If we're going to go down the road suggested in this issue and make every release process the same as a security release, I think we'd probably need around 6-8 private repositories for our side. Most of our recipes and modules get tagged individually whenever a worthy bugfix gets merged anyway.

Assuming that CWP pre-releases often need to include unreleased core versions with security fixes, this means you need to work with Satis as part of the release process as well. It sounds like that's acceptable?

Yep, no problem as long as it's standardised.

@chillu Since we're locking versions across core modules, we can't just push a public<EMAIL_ADDRESS>because it doensn't contain an undisclosed security issue. It relies on a private<EMAIL_ADDRESS>with an undisclosed security issue.

This is not entirely true, the modules never have tight dependency constraints, only recipes do

Thinking about this more, with a bit of a post-mortem on the 3.5-4.3 patch releases. The satis server approach showed a number of limitations and pitfalls:

It relies on a lot of manual syncing. First, you have to sync upstream with security, which is a tedious process, especially when the security branches may have merged PRs that you don't want to overwrite.

You have to rebuild the satis server frequently, depending on many mistakes you make along the way. This is a long-running task, and it isn't always obvious that you have to do so.

Installations from the satis server fail silently with a fallback on packagist, which means that your cow processes are running with the public repos as their upstream, unbeknownst to you. This resulted in the publication of tags on the public repos that had to be immediately deleted.

My solution for the next set of releases was a much more manual approach:

Edit the remotes of each cloned repo and set them to silverstripe-security.
Force-push the branch to ensure the private repo was up to date
Cherry-pick the commits off any security PRs.

This worked much better, but again, there are massive windows for error at every step along the way, and I benefitted tremendously from some one-off automation scripts.
Chatted to both @ScopeyNZ and had the following thoughts:
A disposable silverstripe-releases workspace
tl;dr As much as I think this would benefit us, I don't think this is going to work out.
I know I keep coming back to this. It just feels really bullet-proof to me. It really comes down to the issue of managing state. Syncing is a pain, and the easiest antidote to syncing is an architecture that allows for short-lived, disposable state. This would be a space with many repositories and zero pull requests and issues. A cow command would strip them bare on release:stage (TBD), and rebuild them, pushing up the appropriate branches and tags. Security issues would live on the security repo, which would host all of the discussions and PRs to keep an audit trail, and commits would be cherry-picked across.
Given that only a few repos at a time are affected by security patches, the security workspace could be dialled back to its old price point. The releases workspace, however, would need more money thrown at it, and for that reason, it seems like a non-starter. Guy and I discussed the possibility of hoisting up a git repo on an internal server, but that seems antithetical to the spirit of open source and inviting collaboration.
Later chatted to @robbieaverill and had the following thoughts:
Remove the rigidity around early tagging and version-syncing
As discussed above, unlike the CWP recipes, the core recipes tag each module regardless of whether there are changes or not. This is basically an artefact of a time when the CMS was much more tightly coupled. We're past that now, and if you look at packages like silverstripe-graphql, silverstripe-config, and versioned-admin, we're already breaking the boundaries of those conventions, and it maybe time to just officially move on.
The idea would be:

If a module has new public tag ahead of a recipe that uses it, that's okay. It's a pre-release tag. While it shouldn't be considered a "staged" tag like in the security repos, it does mean that the costs of stuffing it up are much less, because it only affects users on @dev releases. If anything goes horribly wrong, we could just issue another tag, and we wouldn't have to worry about getting a whole new recipe together.
Only recipes and repos with security patches coming are released in darkness for internal preview. This would limit the number of private repos significantly, and reduce our Github subscription cost.
If a release is an incremental patch only (e.g. previous tag + 1 commit), it doesn't matter if other modules have newer tags. We can still pin the recipe to earlier tags, and allow the newer tags to be available in the next recipe release.
If a module has no new commits since the last recipe release, don't tag it!
New cow command would let you opt into privacy: cow release:stage silverstripe/framework my-security-repo

This process is much more similar to the CWP release process and would bring some consistency across the teams, and it would save us some money to boot.

Remove the rigidity around early tagging and version-syncing

I think it probably makes sense to at least keep framework, cms and admin in lockstep for minor versions - although for admin obviously three majors behind. But any other repo I think it's a great solution.
Maybe we could work cow into a state where we flag a security release. cow release:security silverstripe/framework --org silverstripe-security. This would change the remote of that, and it's dependency tree, to silverstripe-security (or whatever). So for that command, silverstripe/framework, silverstripe/recipe-core, and silverstripe/installer would have updated remotes. cow would need to go and update forks at that point too, but you already have an appropriate base branch to do so. Basically pseudocode of cow:
repos = getDependancies('silverstripe/framework')
foreach repos as repo:
  cd repo
  git pull
  git remote add security 'silverstripe/security'
  git push security --delete {current-branch}
  git push -u security {current-branch}
  git remote remove origin // Safety

If we're going to break the lockstep in our release, testing the lowest dependency set probably becomes more important.
https://github.com/silverstripe/silverstripe-framework/issues/8990

Separated out some billing concerns into an internal discussion at https://github.com/silverstripeltd/product-issues/issues/122

Since we last discussed this, I think a few things changed for the better:

The Github pricing model is based on users, with unlimited private repos (see announcement in January). My earlier comment referred to the legacy plans only.
Github has introduced Security Advisories with free private forks per advisory - I don't think that'll work for us though
We now have a clear policy for which security issues need to be disclosed through a quarterly recipe release (or a hotfix recipe release): Anything with CVSS 9.0 or higher. Anything else can be released separately (either as an individual module, or a core recipe release), to be bundled into a quarterly recipe release later (and audited at that stage).
We have broken up lockstep releases in core on a patch level, but retained it on a minor level. This should reduce release complexity here.
Github

FYI We've been doing some related work related to this issue on a private issue on the CMS Squad board. We've put out some info about we'll approach releases on the forum.

Can anyone break down specific next steps to close this RFC? @dnsl48 @maxime-rainville Are we OK now in terms of how we handle security releases on private repos? Is is documented, understood and possibly automated enough that we can perform these releases while minimising human error? Do we have sufficient Github private repo allocation without driving up costs? I'm assuming we've come up with a process that does not rely on every recipe repo running on a private fork for a security release candidate, otherwise I would've heard about it because we would've run out of Github private repos ;)

What I ended up doing for the 4.4.5/4.5.1 release was merged the security patches locally. That was suggested as simpler than setting up a satis server and appropriate when doing a simple patch release.
To be honest, I don't think we have this process fully nailed down. The next release will have a lot more security patches and will probably be a minor. So it's going to be a lot more involved.

OK @brynwhyman this might be an issue to clarify before the next minor release, how it relates to the team's backlog. We've closed out the "improve release process" epic, but this was actually the first big issue which triggered this epic in the first place back in the days heh.

Yeah, I think opting to include more work in this area to the late 'Product Security Tooling and Practices' epic may have kicked this can too far down the road.
Still, there's been a lot of new process change that's been documented but not necessarily linked back to this issue. We'll regroup to pull together a succinct summary in this issue and see what actions are thought to be remaining - there's got to be some in-between actions before getting Satis in the dockerised Cow environment.

I think we should consider that before the next Security release, not a minor. I don't think that needs to be done separately from the release epic itself though.
Also, my main assumption is that performing the whole Satis routine would take about the same time as to do it in a reusable way (through adding to Cow).
Otherwise, we could perhaps keep releasing without Satis (as we currently do) and with somewhat more manual and rigid approach. I guess we can review this RFC once more when it comes closer to making the next security release.

@brynwhyman @Cheddam Are you planning to include the three current security issues in the upcoming 4.7.0 release? This might be a good opportunity to revisit the security process. Which I'm also keen to sort out because we need to clarify billing with the github.com/silverstripe-security org (see https://github.com/silverstripeltd/product-issues/issues/122)


Add baselines and comparisons

Closes #9, #8, and #7.
Still to do:

[ ] document added functions and API
[ ] clean up the API
[ ] add tests

This also closes #15 and does part of #13.

I'm going to merge this without docs or tests, for now.


Changed HashMap to TreeMap to fix flaky test

Fixes
Fixed a flaky test where collection.sort is not sorting the arraylist-

I will get back with detailed description


User, from local to global

You have to convert the local user state to a global state with Redux.

Done!


Swaggert docs

Just a short implementation of swagger for API definition.

That's cool. It would be awesome to extend it by the data format notes as explained in the https://github.com/DivanteLtd/vue-storefront-integration-boilerplate/blob/master/1. Expose the API endpoints required by VS/Required API specification.md

Closing due to long inactivity time


icon battery

Ciao 1of11 sul tuo firmware, l'icone della batteria rimane ferma ? le tacche non scendono?

they work as they should so display battery level..

and strange on my radio they stay still they don't go down

So when the battery percent falls to say 35%, the bars are still full ?

Yes today I reached 56% and they were still all full

I don't think the percentage is quite right, we know that, but the bars will start to fall when the percent reaches down to about 40%.

okok


Print font-face does not work properly

Printing works pretty well unless your page uses @font-face.
Have a look at the "dive into html5" front page inside node-webkit (on the left)
and the print result (on the right)

See how paragraph's characters have been totally messed up ?
It seems related to chromium, but latest Chrome handle web font perfectly.
Any idea ?

Want to back this issue? Place a bounty on it! We accept bounties via Bountysource.

On the Windows 32bit version, printing does not do anything. No crash, no error in console, but no output on printer.

Also seeing the no print issue on Windows 7 64 bit with nwv0.11.5 (32bit), this is a massive problem for me as my app really needs to print reports.

Yes.

Should be working with latest 0.13.0-beta2.


Add support for commands as well as events.

Description
I propose we add support for commands in the control and not just hook onto events. We can still keep the events for backwards compatibility.
The Why
This would allow users to further decouple their views from their models. It allows for easy integration into projects that use the MVVM design pattern.
The How
It's something I'll have to a little further into but from what I know it should be fairly straightforward; just add a few ICommand properties to the control and call the Execute function when they need to be executed.

To be honest, thinking about this more - there is always EventBinder to accomplish this as a workaround. Happy to close if no one really wants this feature.

This can be done easily with the Reactive Linq extensions, or ReactiveMarbles code generator libs.
// Reactive Linq extensions
using ReactiveUI;
using static System.Reactive.Linq.Observable;

<snip>

this.WhenActivated(d => {
    FromEvent<TimeSpan>(handler => GlWpfControl.Render += handler,
                        handler => GlWpfControl.Render -= handler)
        .WhereNotNull()
        .Select(_ =>Unit.Default)
        .InvokeCommand(ViewModel.DrawCmd)
        .DisposeWith(d);
});


bug: Dropdown affects height while hidden

What version of daisyUI are you using?
v3.3.5
Which browsers are you seeing the problem on?
Chrome
Reproduction URL
https://play.tailwindcss.com/7X98V0vXQg
Describe your issue
The height of a dropdown created using method #2 from the documentation on dropdown can overflow the parent elements, which breaks scrolling. This might be the same problem encountered in #1276.

Hello @eulertour 😄
If I understood your initial request correctly, I think the problem came from this line of code :
<label tabindex="0" class="btn m-1">open/close dropdown</summary>
By replacing the correct tag closure, it works. I've also tested replacing label with summary and it's the same result.

I'm still seeing the problem after matching either  or  tags. I was trying to prevent the collapsed menu from increasing the scroll height.

I understand better after retesting. I'll look into it right away !

Could this please be merged? I confirm bug is still present and adding dropdown stretches parent container because "visibility" is used not hide/block approach.  Thank you

@darklow I understand but this is a breaking change, any site that is relying on the current visibility and opacity will break so I can't change it in a patch version or minor version. It must be major version.
We should wait for daisyUI 5. Until then, add some custom styles to get the desired result.


PCQM4Mv2 code instruction missing

Hi Weihua!
All training and inference instructions are now detailed, including a link to download a pretrained model. See #17
Best,
Ladislav

Thanks! Updated the leaderboards!


API

What works, what's broken, what's wrong?
Does Postman add anything for us?
What does Recogito 3 want?

Instead of Postman, I'd recommend Swagger Open Source, which has no cost implications.
The Editor and UI applications can be integrated as separate services into the Docker environment. When exposed in production, the UI can provide users with documentation and testing services.
I've looked at classifying APIs as either public or private so that we might use the system for testing internal APIs that are not intended for public use. The simplest way to implement this might be to create a View for serving the swagger.json which would modify it depending on the user's logged-in privileges.


withLocalCompose(false) not working for version 3 of local-compose.yml

I am trying to figure out why DockerComposeContainer configured as withLocalCompose(false) is not working.
I have a very simple docker-compose.yml:
version: "3"
services:
  postgresql:
    image: "postgres:10"
    ports:
      - 5432

I start as following:
new DockerComposeContainer(...).withLocalCompose(false).withPull(false).starting(null)
I get the following generic error:
20:45:31.061 [main] ERROR 🐳 [docker/compose:1.8.0] - Could not start container
java.lang.IllegalStateException: Container did not start correctly.
	at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:231)
	at org.testcontainers.containers.GenericContainer.lambda$start$0(GenericContainer.java:184)
	at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:76)
	at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:182)
	at org.testcontainers.containers.ContainerisedDockerCompose.invoke(DockerComposeContainer.java:432)
	at org.testcontainers.containers.DockerComposeContainer.runWithCompose(DockerComposeContainer.java:152)
	at org.testcontainers.containers.DockerComposeContainer.createServices(DockerComposeContainer.java:128)
	at org.testcontainers.containers.DockerComposeContainer.starting(DockerComposeContainer.java:112)

When using withLocalCompose(true), it works fine.
Docker version: Docker version 17.09.0-ce, build afdb6d4
Any idea what is wrong or what I can do to investigate this? The strange this is that a few days ago, something similar was working. I am 100% sure that I was testing with withLocalCompose(false), as I remember intentionally deleting docker-compose on my host to verify that everything was running in a container, as expected. I remember being amazed (as this would avoid installing docker-compose on CI servers) ;). However, I cannot really see what is different now.
Thanks!

Interesting update! It seems to be working fine when I set version to 2 instead of 3 in docker-compose.yml. Is this a known issue or as expected?

@rrrrrrrrramba I assume docker/compose:1.8.0 didn't support version 3. And yes, we need to update the image we use because it's a bit outdated :D

Oh, I did not realize the version is fixed in TestcontainersConfiguration. I thought that latest was always used. But using a fixed version is a good idea, so you will have consistent behavior.
Maybe it is also a good ideato allow to configure the desired docker-compose (and other containers, as mentioned in TestcontainersConfiguration to use, instead of 'hardcoding' it?

I created tescontainers.properties with compose.container.image=docker/compose:1.17.1 and now it works great! Thanks for the tip.
I am now seeing the following error though:
22:06:36.368 [dockerjava-netty-1-4] ERROR o.t.s.i.n.util.ResourceLeakDetector - LEAK: ByteBuf.release() was not called before it's garbage-collected. Enable advanced leak reporting to find out where the leak occurred. To enable advanced leak reporting, specify the JVM option '-Dorg.testcontainers.shaded.io.netty.leakDetection.level=advanced' or call ResourceLeakDetector.setLevel() See http://netty.io/wiki/reference-counted-objects.html for more information.

FYI, there also is no :latest for docker-compose it seems: https://hub.docker.com/r/docker/compose/tags/

Let's close this and open a new ticket for that resource leak detection.
Thanks for helping me!

@rrrrrrrrramba this "leak detection" thing is something known and comes from docker-java I think, we're not using Netty directly in TestContainers

Oops, did not realize that. I'll close the other one :).


Print Preview and Print Totally Not Work

Describe the bug
Print Function, includes Print Preview Dialog are Totally Not Work.
To Reproduce
Steps to reproduce the behavior:

Open any webpage.
Ctrl + P
See error: Print Preview will fail, and print settings(options) and print-button grayed-out.

Expected behavior
normal print preview and print job send.
Screenshots

Desktop (please complete the following information):

OS: Windows 10 Build 10.0.19045
Supermium: 126.0.6478.249 (64-Bit)

Additional context
I tried:
・run Supermium with no command parameter & a new data profile: Same Bug.
・set default printer to another: Same Bug.
・do with latest Chrome: Everything goes well.
I also tried to collect log:

[3584:6096:1001/135042.022:VERBOSE1:print_preview_utils.cc(78)] Found printer Xxxxxxxxx with device name Xxxxxxxxx
[3584:6096:1001/135042.022:VERBOSE1:print_preview_utils.cc(257)] Enumerate printers finished, found 5 printers
[3584:6096:1001/135042.028:VERBOSE1:device_event_log_impl.cc(198)] [13:50:42.026] Printer: EVENT: local_printer_handler_default.cc:249 Getting printer capabilities in-process for Xxxxxxxxx
[3584:6096:1001/135042.028:VERBOSE1:device_event_log_impl.cc(198)] [13:50:42.030] Printer: EVENT: local_printer_handler_default.cc:123 Got basic info for Xxxxxxxxx
[3584:6096:1001/135042.028:VERBOSE1:device_event_log_impl.cc(198)] [13:50:42.030] Printer: EVENT: printer_capabilities.cc:218 Get printer capabilities start for Xxxxxxxxx
[3584:6096:1001/135042.028:VERBOSE1:device_event_log_impl.cc(198)] [13:50:42.032] Printer: EVENT: printer_capabilities.cc:221 Driver info: FX ApeosPort-V C5576;<IP_ADDRESS>;10.0.19041.4597 (WinBuild.160101.0800);Microsoft® Windows® Operating System
[16056:3740:1001/135042.263:INFO:cpu_info.cc(53)] Available number of cores: 12
[3584:6096:1001/135042.274:VERBOSE1:device_event_log_impl.cc(198)] [13:50:42.272] Printer: EVENT: printer_capabilities.cc:229 Got printer capabilities for Xxxxxxxxx
[3584:6096:1001/135042.276:VERBOSE1:print_preview_handler.cc(1025)] Get printer capabilities finished
[3584:6096:1001/135042.293:VERBOSE1:print_preview_handler.cc(695)] Print preview request start
[3584:6096:1001/135042.297:VERBOSE1:device_event_log_impl.cc(198)] [13:50:42.295] Printer: EVENT: printer_query.cc:274 Updating paper printable area in-process for Xxxxxxxxx
[3584:6096:1001/135042.398:VERBOSE1:device_event_log_impl.cc(198)] [13:50:42.395] Printer: EVENT: print_view_manager_base.cc:343 Paper printable area updated for vendor id 9
[3584:6096:1001/135042.461:VERBOSE1:device_event_log_impl.cc(198)] [13:50:42.459] Printer: EVENT: print_preview_ui.cc:976 Compositing for document type kPDF
[3584:16908:1001/135042.481:ERROR:sandbox_win.cc(936)] Sandbox cannot access executable. Check filesystem permissions are valid. See https://bit.ly/31yqMJR.: アクセスが拒否されました。 (0x5)
......

A workaround is to use the browser with command line switch --no-sandbox. Anyway, this is the App Container Sandbox, which is enabled on Windows 10 1809/17763 and up (1809 is used to build Supermium and there are no problems there).
But I haven't tested Supermium on Windows 10 22H2/19045. I'll try that and see what happens. All the sandbox hacks and limitations should be gated to Windows 8 or below, so Windows 10/11 should be OK, but this is something I haven't seen before.
Is GDI font rendering enabled?


When multiple file types supplied, it should "ignore" space

If you have fileTypes as js, jsx it wont catch jsx file
If you have fileTypes as js,jsx it works as expected
Either strip the space when user defines (option two) or simply correctly parse the fileTypes to ignore space (it should work as ext, ext and ext,ext etc)

This will be fixed in the next (version >=1.3) release

Fixed in 1.3.0.
Sorry for the delay


Updating rate calculation

Description
Thanks to Christian Forfang for discovering bitwise math issues in the shading dynamic rate sample.  This PR is meant to fix it.
General Checklist:
Please ensure the following points are checked:

[x] My code follows the coding style
[x] I have reviewed file licenses
[x] I have updated the header Copyright to reflect the current year (CI build will fail if Copyright is out of date)
[x] I have commented any added functions (in line with Doxygen)
[x] I have commented any code that could be hard to understand
[x] My changes do not add any new compiler warnings
[x] My changes do not add any new validation layer errors or warnings
[x] I have used existing framework/helper functions where possible
[x] My changes build on Windows, Linux, macOS and Android. Otherwise I have documented any exceptions
[x] My changes do not add any regressions
[x] I have tested every sample to ensure everything runs correctly
[x] This PR describes the scope and expected impact of the changes I am making

Sample Checklist
If your PR contains a new or modified sample, these further checks must be carried out in addition to the General Checklist:

[ ] I have tested the sample on at least one compliant Vulkan implementation
[ ] If the sample is vendor-specific, I have tagged it appropriately
[ ] I have stated on what implementation the sample has been tested so that others can test on different implementations and platforms
[ ] Any dependent assets have been merged and published in downstream modules
[ ] For new samples, I have added a paragraph with a summary to the appropriate chapter in the samples readme

Looks good to me :)


EVerest Config Option

Hello again! In case anyone didn't see my last PR, I am on a team that is working on setting up software simulated demos for the charging software EVerest.
In some of those demos, we have patch files that we use to get MaEVe to work with EVerest. Our idea was that we could create some type of config within MaEVe for future use by whomever wants to setup a MaEVe and EVerest environment. This PR is our attempt at that, creating a single script that would be ran before running docker compose up, and would apply all the patches.
We would greatly appreciate suggestions against this to see if there are any better, more eloquent ways to create this config option.

Thank you @shankari for the feedback on all of this. I just pushed out two commits that addresses the following issues.
LB Patch

config/everest/maeve-csms-no-lb.patch
I think that this is already in the maeve docker compose.
We may want to just move to a newer commit on maeve to pick up the change.
We will then need to re-create the patches that fail and re-test, but we can remove this patch.

You were totally right as per my response to the comment above. In the first commit, I removed the lb patch from the directory and script.
WSS Patch

config/everest/maeve-csms-no-wss.patch
Instead of applying this patch, we should change the maeve code so that instead of crashing the entire program if no certs are defined, it only stops listening to wss https://github.com/US-JOET/base-camp/issues/14#issuecomment-1980213695

To work around this issue, I added in a check before the WSS initialization that determines whether or not any certs were provided. If at least one was provided, we can go through to the initialization, but if none were provided, then we skip over the initialization.
When approaching this problem, the main issue I struggled with was how to determine whether or not the user wanted to use SP1 or SP2/3. In my implementation, I am assuming that if the user provided no certificates at all, they didn't plan on using SP2/3. On the flip side, if they supplied at least one certificate, then we know they at least tried to provide something, implying they would want to use SP2/3. From there, the error handling in place already would pick up the slack and determine if they supplied all the correct ones or not.
Testing Done
For the WSS change, I ran our everest-demo/demo-iso15118-2-ac-plus-ocpp.sh script twice, once on SP1 and once on SP3. The gateway container successfully started both times and they both successfully connected to EVerest.

LGTM, let's see what the MaEVe community thinks!


Can you pls add Ctrl+Tab for service switching along with existing Ctrl+1,2,3,etc. ? It'll be helpful.

Can you pls add Ctrl+Tab for service switching along with existing Ctrl+1,2,3,etc. ? It'll be helpful.

Duplicate of #850


using a wrapper component in MDX

i'm working with the layout part of MDX and this isn't working. my endgoal is to have the component in pages/index.tsx automatically be wrapped in the layout styled-component.
and this setup isn't working:
// components/layout.tsx

import styled from 'styled-components';

export default styled.div`
  padding: 16px;
  margin-bottom: 16px;
  font-family: system-ui;
  background: #f2f2f2;
  color: #4f4f4f;
`;

// docs/typography.mdx

import Layout from '../components/layout'

export default Layout

## Overview

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Vestibulum massa ex, vestibulum vitae urna quis, maximus faucibus nisi. Nunc aliquet nec ligula sit amet dignissim. Fusce sed auctor urna. Nulla facilisi. Curabitur sed maximus tellus. Nulla a turpis euismod quam iaculis hendrerit. Morbi nec eros pharetra, pellentesque tortor vitae, efficitur est.

// pages/index.tsx
import Typography from '../docs/typography.mdx'

export default () => (<Typography />)

// next.config.js
const withTypescript = require('@zeit/next-typescript');

const withMDX = require('@zeit/next-mdx')({
  extension: /\.mdx?$/
});

module.exports = withTypescript(withMDX());

Is there something I'm missing between the spec and my implementation? The react dev tools show that the component Layout={StyledComponent} attached to the MDX component.

Maybe it's not rendering it's children? I'm not sure 🤔 Should work fine if it's passed into mdxtag like you're saying. Could you make a GitHub repository for this code so I can have a look.

@timneutkens sure thing! https://github.com/charliewilco/typescript-next-mdx

okay i resolved this on my end, explicitly adding @mdx-js/tag@latest resolved this, maybe there's a version mismatch somewhere in the various dependencies
mdx + next.js is just awesome BTW

Hello anybody is there anyway we can use next.js without importing and exporting layout on every file. I saw you can do "transclusion.", is it possible to do something like:
// some-api-endpoint/[slug].jsx
import Slug from `./${slug}.md`
export default <Slug>

@vixalien I think this could help you: https://nextjs.org/docs/advanced-features/custom-app
If you create a _app.js file with this content:
import Layout from '../components/layout';

export default function MyApp({ Component, pageProps }) {
  return (
    <Layout>
      <Component {...pageProps} />
    </Layout>
  )
}

All of your pages will be wrapped in your Layout

sorry but I was meaning a layout only for some specific files maybe like a layout for pages in some folder.