Add support for searching for jars within archives

Today we search for jars as well as jars within jars. This PR additionally adds support for looking for jars within archives (zip, tar, and gzipped/bzip2/brotli/lz4/sz/xz/zst compressed tars). This is done by explicitly looking for archives with supported extensions and looking at the file listing for each archive to determine if there are any java archives that should be extracted (jar, war, ear, hpi, jpi, etc...) and extracts them if they exist.
Performance note: zip files have a central header which can be used to do this file-listing search efficiently, however, tar does not support this feature. In order to search the file-listing we must decompress the entire archive contents. This will lead to longer scanning times for images or directories that have many small archives (or a few large archives).
Sample performance difference:
# latest release of syft
syft ~/images/gitlab-latest.tar -vv  49.05s user 8.09s system 161% cpu 35.340 total

# this PR
snapshot/syft-macos_darwin_amd64/syft ~/images/gitlab-latest.tar -vv  78.16s user 7.63s system 159% cpu 53.941 total

For the sample image there is an additional 30 seconds needed for the java cataloger alone
Note for future implementation: Upon implementing https://github.com/anchore/syft/issues/246 we should remove these parser functions entirely and lean on the archive cataloger to discover packages of any kind within archives.

FWIW I ran this in comparison to the log4j-sniffer and here are the results, showing this PR does make Syft find the same things (there are some expected misses, like the java_projects/light_shading Gradle shade build (which the sniffer also does not identify).
Here's the output from each:

log4j-sniffer output
kzantow@KZANTOW log4j-sniffer % for d in examples/*/ ; do echo ; echo scan $d ; go run main.go crawl $d ; done

scan examples/archived_fat_jar/
No files affected by CVE-2021-45046 or CVE-2021-45105 detected
1 total files scanned, skipped 0 paths due to permission denied errors, encountered 0 errors processing paths

scan examples/cve-2021-45105-versions/
[INFO] Found archive with name matching vulnerable log4j-core format at examples/cve-2021-45105-versions/log4j-core-2.12.2.jar
[INFO] Found JndiLookup class in the log4j package at org/apache/logging/log4j/core/lookup/JndiLookup.class
[INFO] Found JndiManager class that was an exact md5 match for a known version at org/apache/logging/log4j/core/net/JndiManager.class
[MATCH] CVE-2021-45105 detected in file examples/cve-2021-45105-versions/log4j-core-2.12.2.jar. log4j versions: 2.12.2. Reasons: JndiLookup class and package name matched, jar name matched, JndiManager class and package name matched, class file MD5 matched
[INFO] Found archive with name matching vulnerable log4j-core format at examples/cve-2021-45105-versions/log4j-core-2.16.0.jar
[INFO] Found JndiManager class that was an exact md5 match for a known version at org/apache/logging/log4j/core/net/JndiManager.class
[INFO] Found JndiLookup class in the log4j package at org/apache/logging/log4j/core/lookup/JndiLookup.class
[MATCH] CVE-2021-45105 detected in file examples/cve-2021-45105-versions/log4j-core-2.16.0.jar. log4j versions: 2.16.0. Reasons: JndiLookup class and package name matched, jar name matched, JndiManager class and package name matched, class file MD5 matched
Files affected by CVE-2021-45046 or CVE-2021-45105 detected: 2 file(s) impacted by CVE-2021-45046 or CVE-2021-45105
2 total files scanned, skipped 0 paths due to permission denied errors, encountered 0 errors processing paths

scan examples/fat_jar/
[INFO] Found JndiManager class that was an exact md5 match for a known version at org/apache/logging/log4j/core/net/JndiManager.class
[INFO] Found JndiLookup class in the log4j package at org/apache/logging/log4j/core/lookup/JndiLookup.class
[MATCH] CVE-2021-45046 and CVE-2021-45105 detected in file examples/fat_jar/fat_jar.jar. log4j versions: 2.14.0-2.14.1. Reasons: JndiLookup class and package name matched, JndiManager class and package name matched, class file MD5 matched
Files affected by CVE-2021-45046 or CVE-2021-45105 detected: 1 file(s) impacted by CVE-2021-45046 or CVE-2021-45105
1 total files scanned, skipped 0 paths due to permission denied errors, encountered 0 errors processing paths

scan examples/good_version/
[INFO] Found JndiLookup class in the log4j package at org/apache/logging/log4j/core/lookup/JndiLookup.class
[INFO] Found JndiManager class that was an exact md5 match for a known version at org/apache/logging/log4j/core/net/JndiManager.class
No files affected by CVE-2021-45046 or CVE-2021-45105 detected
1 total files scanned, skipped 0 paths due to permission denied errors, encountered 0 errors processing paths

scan examples/inside_a_dist/
[INFO] Found nesting archive matching the log4j-core jar name at log4j-core-2.14.1.jar
[MATCH] CVE-2021-45046 and CVE-2021-45105 detected in file examples/inside_a_dist/wrapped_log4j.tar. log4j versions: 2.14.1. Reasons: jar name inside archive matched
[INFO] Found nesting archive matching the log4j-core jar name at log4j-core-2.14.1.jar
[MATCH] CVE-2021-45046 and CVE-2021-45105 detected in file examples/inside_a_dist/wrapped_log4j.tar.bz2. log4j versions: 2.14.1. Reasons: jar name inside archive matched
[INFO] Found nesting archive matching the log4j-core jar name at log4j-core-2.14.1.jar
[MATCH] CVE-2021-45046 and CVE-2021-45105 detected in file examples/inside_a_dist/wrapped_log4j.tar.gz. log4j versions: 2.14.1. Reasons: jar name inside archive matched
[INFO] Found nesting archive matching the log4j-core jar name at log4j-core-2.14.1.jar
[MATCH] CVE-2021-45046 and CVE-2021-45105 detected in file examples/inside_a_dist/wrapped_log4j.zip. log4j versions: 2.14.1. Reasons: jar name inside archive matched
Files affected by CVE-2021-45046 or CVE-2021-45105 detected: 4 file(s) impacted by CVE-2021-45046 or CVE-2021-45105
4 total files scanned, skipped 0 paths due to permission denied errors, encountered 0 errors processing paths

scan examples/inside_a_par/
[INFO] Found nesting archive matching the log4j-core jar name at lib/log4j-core-2.14.1.jar
[MATCH] CVE-2021-45046 and CVE-2021-45105 detected in file examples/inside_a_par/wrapped_in_a_par.par. log4j versions: 2.14.1. Reasons: jar name inside archive matched
Files affected by CVE-2021-45046 or CVE-2021-45105 detected: 1 file(s) impacted by CVE-2021-45046 or CVE-2021-45105
1 total files scanned, skipped 0 paths due to permission denied errors, encountered 0 errors processing paths

scan examples/java_projects/
No files affected by CVE-2021-45046 or CVE-2021-45105 detected
10 total files scanned, skipped 0 paths due to permission denied errors, encountered 0 errors processing paths

scan examples/light_shading/
[INFO] Found JndiManager class not in the log4j package at relocated/core/net/JndiManager.class
[INFO] Found JndiManager class that had identical bytecode instruction as a known version at relocated/core/net/JndiManager.class
[INFO] Found JndiLookup class not in the log4j package at relocated/core/lookup/JndiLookup.class
[MATCH] CVE-2021-45046 and CVE-2021-45105 detected in file examples/light_shading/shadow-all.jar. log4j versions: 2.12.0-2.14.1. Reasons: JndiLookup class name matched, JndiManager class name matched, byte code instruction MD5 matched
Files affected by CVE-2021-45046 or CVE-2021-45105 detected: 1 file(s) impacted by CVE-2021-45046 or CVE-2021-45105
1 total files scanned, skipped 0 paths due to permission denied errors, encountered 0 errors processing paths

scan examples/multiple_bad_versions/
[INFO] Found archive with name matching vulnerable log4j-core format at examples/multiple_bad_versions/log4j-core-2.10.0.jar
[INFO] Found JndiLookup class in the log4j package at org/apache/logging/log4j/core/lookup/JndiLookup.class
[INFO] Found JndiManager class that was an exact md5 match for a known version at org/apache/logging/log4j/core/net/JndiManager.class
[MATCH] CVE-2021-45046 and CVE-2021-45105 detected in file examples/multiple_bad_versions/log4j-core-2.10.0.jar. log4j versions: 2.10.0, 2.9.0-2.11.2. Reasons: JndiLookup class and package name matched, jar name matched, JndiManager class and package name matched, class file MD5 matched
[INFO] Found archive with name matching vulnerable log4j-core format at examples/multiple_bad_versions/log4j-core-2.11.0.jar
[INFO] Found JndiLookup class in the log4j package at org/apache/logging/log4j/core/lookup/JndiLookup.class
[INFO] Found JndiManager class that was an exact md5 match for a known version at org/apache/logging/log4j/core/net/JndiManager.class
[MATCH] CVE-2021-45046 and CVE-2021-45105 detected in file examples/multiple_bad_versions/log4j-core-2.11.0.jar. log4j versions: 2.11.0, 2.9.0-2.11.2. Reasons: JndiLookup class and package name matched, jar name matched, JndiManager class and package name matched, class file MD5 matched
[INFO] Found archive with name matching vulnerable log4j-core format at examples/multiple_bad_versions/log4j-core-2.11.1.jar
[INFO] Found JndiManager class that was an exact md5 match for a known version at org/apache/logging/log4j/core/net/JndiManager.class
[INFO] Found JndiLookup class in the log4j package at org/apache/logging/log4j/core/lookup/JndiLookup.class
[MATCH] CVE-2021-45046 and CVE-2021-45105 detected in file examples/multiple_bad_versions/log4j-core-2.11.1.jar. log4j versions: 2.11.1, 2.9.0-2.11.2. Reasons: JndiLookup class and package name matched, jar name matched, JndiManager class and package name matched, class file MD5 matched
[INFO] Found archive with name matching vulnerable log4j-core format at examples/multiple_bad_versions/log4j-core-2.11.2.jar
[INFO] Found JndiLookup class in the log4j package at org/apache/logging/log4j/core/lookup/JndiLookup.class
[INFO] Found JndiManager class that was an exact md5 match for a known version at org/apache/logging/log4j/core/net/JndiManager.class
[MATCH] CVE-2021-45046 and CVE-2021-45105 detected in file examples/multiple_bad_versions/log4j-core-2.11.2.jar. log4j versions: 2.11.2, 2.9.0-2.11.2. Reasons: JndiLookup class and package name matched, jar name matched, JndiManager class and package name matched, class file MD5 matched
[INFO] Found archive with name matching vulnerable log4j-core format at examples/multiple_bad_versions/log4j-core-2.12.0.jar
[INFO] Found JndiLookup class in the log4j package at org/apache/logging/log4j/core/lookup/JndiLookup.class
[INFO] Found JndiManager class that was an exact md5 match for a known version at org/apache/logging/log4j/core/net/JndiManager.class
[MATCH] CVE-2021-45046 and CVE-2021-45105 detected in file examples/multiple_bad_versions/log4j-core-2.12.0.jar. log4j versions: 2.12.0. Reasons: JndiLookup class and package name matched, jar name matched, JndiManager class and package name matched, class file MD5 matched
[INFO] Found archive with name matching vulnerable log4j-core format at examples/multiple_bad_versions/log4j-core-2.12.1.jar
[INFO] Found JndiLookup class in the log4j package at org/apache/logging/log4j/core/lookup/JndiLookup.class
[INFO] Found JndiManager class that was an exact md5 match for a known version at org/apache/logging/log4j/core/net/JndiManager.class
[MATCH] CVE-2021-45046 and CVE-2021-45105 detected in file examples/multiple_bad_versions/log4j-core-2.12.1.jar. log4j versions: 2.12.0, 2.12.1. Reasons: JndiLookup class and package name matched, jar name matched, JndiManager class and package name matched, class file MD5 matched
[INFO] Found archive with name matching vulnerable log4j-core format at examples/multiple_bad_versions/log4j-core-2.13.0.jar
[INFO] Found JndiManager class that was an exact md5 match for a known version at org/apache/logging/log4j/core/net/JndiManager.class
[INFO] Found JndiLookup class in the log4j package at org/apache/logging/log4j/core/lookup/JndiLookup.class
[MATCH] CVE-2021-45046 and CVE-2021-45105 detected in file examples/multiple_bad_versions/log4j-core-2.13.0.jar. log4j versions: 2.13.0, 2.13.0-2.13.3. Reasons: JndiLookup class and package name matched, jar name matched, JndiManager class and package name matched, class file MD5 matched
[INFO] Found archive with name matching vulnerable log4j-core format at examples/multiple_bad_versions/log4j-core-2.13.1.jar
[INFO] Found JndiLookup class in the log4j package at org/apache/logging/log4j/core/lookup/JndiLookup.class
[INFO] Found JndiManager class that was an exact md5 match for a known version at org/apache/logging/log4j/core/net/JndiManager.class
[MATCH] CVE-2021-45046 and CVE-2021-45105 detected in file examples/multiple_bad_versions/log4j-core-2.13.1.jar. log4j versions: 2.13.0-2.13.3, 2.13.1. Reasons: JndiLookup class and package name matched, jar name matched, JndiManager class and package name matched, class file MD5 matched
[INFO] Found archive with name matching vulnerable log4j-core format at examples/multiple_bad_versions/log4j-core-2.13.2.jar
[INFO] Found JndiLookup class in the log4j package at org/apache/logging/log4j/core/lookup/JndiLookup.class
[INFO] Found JndiManager class that was an exact md5 match for a known version at org/apache/logging/log4j/core/net/JndiManager.class
[MATCH] CVE-2021-45046 and CVE-2021-45105 detected in file examples/multiple_bad_versions/log4j-core-2.13.2.jar. log4j versions: 2.13.0-2.13.3, 2.13.2. Reasons: JndiLookup class and package name matched, jar name matched, JndiManager class and package name matched, class file MD5 matched
[INFO] Found archive with name matching vulnerable log4j-core format at examples/multiple_bad_versions/log4j-core-2.13.3.jar
[INFO] Found JndiLookup class in the log4j package at org/apache/logging/log4j/core/lookup/JndiLookup.class
[INFO] Found JndiManager class that was an exact md5 match for a known version at org/apache/logging/log4j/core/net/JndiManager.class
[MATCH] CVE-2021-45046 and CVE-2021-45105 detected in file examples/multiple_bad_versions/log4j-core-2.13.3.jar. log4j versions: 2.13.0-2.13.3, 2.13.3. Reasons: JndiLookup class and package name matched, jar name matched, JndiManager class and package name matched, class file MD5 matched
[INFO] Found archive with name matching vulnerable log4j-core format at examples/multiple_bad_versions/log4j-core-2.14.0.jar
[INFO] Found JndiManager class that was an exact md5 match for a known version at org/apache/logging/log4j/core/net/JndiManager.class
[INFO] Found JndiLookup class in the log4j package at org/apache/logging/log4j/core/lookup/JndiLookup.class
[MATCH] CVE-2021-45046 and CVE-2021-45105 detected in file examples/multiple_bad_versions/log4j-core-2.14.0.jar. log4j versions: 2.14.0, 2.14.0-2.14.1. Reasons: JndiLookup class and package name matched, jar name matched, JndiManager class and package name matched, class file MD5 matched
[INFO] Found archive with name matching vulnerable log4j-core format at examples/multiple_bad_versions/log4j-core-2.14.1.jar
[INFO] Found JndiManager class that was an exact md5 match for a known version at org/apache/logging/log4j/core/net/JndiManager.class
[INFO] Found JndiLookup class in the log4j package at org/apache/logging/log4j/core/lookup/JndiLookup.class
[MATCH] CVE-2021-45046 and CVE-2021-45105 detected in file examples/multiple_bad_versions/log4j-core-2.14.1.jar. log4j versions: 2.14.0-2.14.1, 2.14.1. Reasons: JndiLookup class and package name matched, jar name matched, JndiManager class and package name matched, class file MD5 matched
[INFO] Found archive with name matching vulnerable log4j-core format at examples/multiple_bad_versions/log4j-core-2.15.0.jar
[INFO] Found JndiManager class that was an exact md5 match for a known version at org/apache/logging/log4j/core/net/JndiManager.class
[INFO] Found JndiLookup class in the log4j package at org/apache/logging/log4j/core/lookup/JndiLookup.class
[MATCH] CVE-2021-45046 and CVE-2021-45105 detected in file examples/multiple_bad_versions/log4j-core-2.15.0.jar. log4j versions: 2.15.0. Reasons: JndiLookup class and package name matched, jar name matched, JndiManager class and package name matched, class file MD5 matched
Files affected by CVE-2021-45046 or CVE-2021-45105 detected: 13 file(s) impacted by CVE-2021-45046 or CVE-2021-45105
13 total files scanned, skipped 0 paths due to permission denied errors, encountered 0 errors processing paths

scan examples/obfuscated/
[INFO] Found JndiManager class that partially matched the bytecode of a known version at org/a/a/a/a/g/b.class
[INFO] Found finding in what appeared to be an obfuscated jar at examples/obfuscated/2.14.1-aaaagb.jar
[MATCH] CVE-2021-45105 detected in file examples/obfuscated/2.14.1-aaaagb.jar. log4j versions: 2.12.2. Reasons: jar file appeared obfuscated, byte code partially matched known version
Files affected by CVE-2021-45046 or CVE-2021-45105 detected: 1 file(s) impacted by CVE-2021-45046 or CVE-2021-45105
1 total files scanned, skipped 0 paths due to permission denied errors, encountered 0 errors processing paths

scan examples/par_in_a_dist/
No files affected by CVE-2021-45046 or CVE-2021-45105 detected
1 total files scanned, skipped 0 paths due to permission denied errors, encountered 0 errors processing paths

scan examples/single_bad_version/
[INFO] Found archive with name matching vulnerable log4j-core format at examples/single_bad_version/log4j-core-2.14.1.jar
[INFO] Found JndiManager class that was an exact md5 match for a known version at org/apache/logging/log4j/core/net/JndiManager.class
[INFO] Found JndiLookup class in the log4j package at org/apache/logging/log4j/core/lookup/JndiLookup.class
[MATCH] CVE-2021-45046 and CVE-2021-45105 detected in file examples/single_bad_version/log4j-core-2.14.1.jar. log4j versions: 2.14.0-2.14.1, 2.14.1. Reasons: JndiLookup class and package name matched, jar name matched, JndiManager class and package name matched, class file MD5 matched
Files affected by CVE-2021-45046 or CVE-2021-45105 detected: 1 file(s) impacted by CVE-2021-45046 or CVE-2021-45105
1 total files scanned, skipped 0 paths due to permission denied errors, encountered 0 errors processing paths
kzantow@KZANTOW log4j-sniffer %

 Syft using this PR
kzantow@KZANTOW syft % for d in ../log4j-sniffer/examples/*/; do echo ; echo scan $d ; go run main.go dir:$d; done
scan ../log4j-sniffer/examples/archived_fat_jar/
 ✔ Indexed ../log4j-sniffer/examples/archived_fat_jar/
 ✔ Cataloged packages      [2 packages]
NAME        VERSION  TYPE
fat_jar     2.14.1   java-archive
log4j-core  2.14.1   java-archive

scan ../log4j-sniffer/examples/cve-2021-45105-versions/
 ✔ Indexed ../log4j-sniffer/examples/cve-2021-45105-versions/
 ✔ Cataloged packages      [2 packages]

NAME        VERSION  TYPE
log4j-core  2.12.2   java-archive
log4j-core  2.16.0   java-archive

scan ../log4j-sniffer/examples/fat_jar/
 ✔ Indexed ../log4j-sniffer/examples/fat_jar/
 ✔ Cataloged packages      [2 packages]

NAME        VERSION  TYPE
fat_jar     2.14.1   java-archive
log4j-core  2.14.1   java-archive

scan ../log4j-sniffer/examples/good_version/
 ✔ Indexed ../log4j-sniffer/examples/good_version/
 ✔ Cataloged packages      [1 packages]

NAME        VERSION  TYPE
log4j-core  2.17.0   java-archive

scan ../log4j-sniffer/examples/inside_a_dist/
 ✔ Indexed ../log4j-sniffer/examples/inside_a_dist/
 ✔ Cataloged packages      [4 packages]

NAME        VERSION  TYPE
log4j-core  2.14.1   java-archive

scan ../log4j-sniffer/examples/inside_a_par/
 ✔ Indexed ../log4j-sniffer/examples/inside_a_par/
 ✔ Cataloged packages      [1 packages]

NAME        VERSION  TYPE
log4j-core  2.14.1   java-archive

scan ../log4j-sniffer/examples/java_projects/
 ✔ Indexed ../log4j-sniffer/examples/java_projects/
 ✔ Cataloged packages      [1 packages]

NAME            VERSION  TYPE
gradle-wrapper           java-archive

scan ../log4j-sniffer/examples/light_shading/
 ✔ Indexed ../log4j-sniffer/examples/light_shading/
 ✔ Cataloged packages      [9 packages]

NAME                     VERSION                                    TYPE
error_prone_annotations  2.5.1                                      java-archive
failureaccess            1.0.1                                      java-archive
guava                    30.1.1-jre                                 java-archive
j2objc-annotations       1.3                                        java-archive
jsr305                   3.0.2                                      java-archive
listenablefuture         9999.0-empty-to-avoid-conflict-with-guava  java-archive
log4j-api                2.14.1                                     java-archive
log4j-core               2.14.1                                     java-archive
shadow-all                                                          java-archive

scan ../log4j-sniffer/examples/multiple_bad_versions/
 ✔ Indexed ../log4j-sniffer/examples/multiple_bad_versions/
 ✔ Cataloged packages      [13 packages]

NAME        VERSION  TYPE
log4j-core  2.10.0   java-archive
log4j-core  2.11.0   java-archive
log4j-core  2.11.1   java-archive
log4j-core  2.11.2   java-archive
log4j-core  2.12.0   java-archive
log4j-core  2.12.1   java-archive
log4j-core  2.13.0   java-archive
log4j-core  2.13.1   java-archive
log4j-core  2.13.2   java-archive
log4j-core  2.13.3   java-archive
log4j-core  2.14.0   java-archive
log4j-core  2.14.1   java-archive
log4j-core  2.15.0   java-archive

scan ../log4j-sniffer/examples/obfuscated/
 ✔ Indexed ../log4j-sniffer/examples/obfuscated/
 ✔ Cataloged packages      [8 packages]
NAME                     VERSION                                    TYPE
error_prone_annotations  2.5.1                                      java-archive
failureaccess            1.0.1                                      java-archive
guava                    30.1.1-jre                                 java-archive
j2objc-annotations       1.3                                        java-archive
jsr305                   3.0.2                                      java-archive
listenablefuture         9999.0-empty-to-avoid-conflict-with-guava  java-archive
log4j-api                2.14.1                                     java-archive
log4j-core               2.14.1                                     java-archive

scan ../log4j-sniffer/examples/par_in_a_dist/
 ✔ Indexed ../log4j-sniffer/examples/par_in_a_dist/
 ✔ Cataloged packages      [1 packages]

NAME        VERSION  TYPE
log4j-core  2.14.1   java-archive

scan ../log4j-sniffer/examples/single_bad_version/
 ✔ Indexed ../log4j-sniffer/examples/single_bad_version/
 ✔ Cataloged packages      [1 packages]
NAME        VERSION  TYPE
log4j-core  2.14.1   java-archive
kzantow@KZANTOW syft %

@wagoodman Approved! Great changes and really awesome work here


SortByPrimaryKey option

An option to prevent unconditional ThenBy if PrimaryKey is pecified:
https://github.com/DevExpress/DevExtreme.AspNet.Data/blob/3167f39e86d4d7bfd5da35e25eff6605bb8be017/net/DevExtreme.AspNet.Data/DataSourceLoadContext.cs#L192-L195
Motivation:

T631814
T657002
T686411
T826066 (Blazor)

The T835414 case looks pretty real. Let's do this :)


Create addon and MidiManager in C#

Currently, all the core logic should be available both to GDScript and C#, but the actual addon is written in GDScript.
In other words, the contents of https://github.com/nlaha/godot-midi-4.0/tree/main/game/addons/godot_midi
Should be written in C# alongside the GDScript implementation.

I might take a shot at this; just to be clear, the contents of /addons/godot_midi/ need to be recreated in C#?

Also, should this be noted? (Godotengine.org - C# basics)[https://docs.godotengine.org/en/stable/tutorials/scripting/c_sharp/c_sharp_basics.html#current-gotchas-and-known-issues:~:text=Writing editor plugins is possible%2C but it is currently quite convoluted.]

I've started working on a rewrite of how the MIDI parser interfaces with Godot. The new version no longer relies on the animation system. For that reason, I suggest you wait until I have that in a working state, as much of the Godot addon logic will be changed to accommodate the new system.
I'm closing this issue for now, but will re-open it once the new version (written in GDScript) is merged into main.

Let me note real quick, ChatGPT worked phenomenally for this task.  I did one script myself, and after debugging for an hour or so, got it working.  I gave GPT the script and this prompt: Translate the following code from GDScript to C#:
It worked flawlessly, and in ~2 seconds.

Let me note real quick, ChatGPT worked phenomenally for this task. I did one script myself, and after debugging for an hour or so, got it working. I gave ChatGPT the script and this prompt: Translate the following code from GDScript to C#:
It produced the exact same working result I created, and in ~2 seconds.

I'm going to keep this issue closed as I was able to write a MIDI playback node ("MidiPlayer") in C++, thus no GDScript or C# version is required.


Migrate build to use docker compose

The docker compose build will build all services on all platforms

[ ] Create a docker-compose.yml at the root of the repository
[ ] Adapt all services Dockerfile files

[ ] Remove duplication of demo.proto file
[ ] Compile protobuf source files during build

[ ] Testing

References

https://docs.docker.com/compose/

Hey @mic-max, what's the status of this? Do you have any key blockers?

Hello @mic-max, I've tried to reach out to you via Slack, but I guess here it is better, because we can have it documented.
I'm taken a look at the PR #33 and I saw that there was a lot of changes regarding env vars, dockerfiles and proto files.
Don't you think it would be better to split them up into small PRs?
I've tried to set it up locally with docker compose and I couldn't make it work.
I have tried to make some changes, but still the services are not talking with each other and the traces are not arriving at Jaeger. It seems that the proto changes have affected the behavior of the services.
I've taken your docker compose as start and created a simpler one, just to make the app run.
I've pushed into a branch:
https://github.com/open-telemetry/opentelemetry-demo-webstore/blob/feature/docker-compose/compose.yml
If you agree I can open a PR, and then we can do the changes you have done in your PR gradually.
What do you think?
5:05
I don't want to step on your toes, please don't get me wrong.
If you don't agree I can remove the branch and we continue on yours.


No filter class

Removed core.Filter() class and put it into a separate module named svo_filters located here.

I completely pulled the Filter class out of the package and made svo_filters a dependency. I also fixed the ModelGrid interpolation in core.py so that the pre-loaded flux grid is reloaded after ModelGrid.customize() is run.

Are you planning on adding svo_filters to PyPI?  I don't think we should introduce dependencies unless they can be installed with a package manager.

Just added it to PyPI. pip install svo_filters

Just make sure it gets added to all of the dependency lists.
.rtd-environment.yaml
.travis.yaml
setup.cfg

Everytime I read "no filter" I think about the Instagram "no filter" bs an the meme's associated with it

The pr test is showing fail because I canceled the Mac builds, but everything else passed so I'm merging.


Language Functionality

Spurtcms Admin allows to set a default language.could you extend the functionality to download the language file for user to his system?

Thank you for raising this requirement. We have developed this feature shortly.

This Feature was implemented. Please check and let us know your comments.


No RAPL zones found!

On a AMD new CPU I got this info:
energymon_init_rapl: No RAPL zones found!
energymon:finit: No such device
source: Intel RAPL
exclusive: false
interval (usec): 1000
precision (uJ): 0
reading (uJ): 0
but from turbostat:
turbostat
turbostat version 2024.05.10 - Len Brown<EMAIL_ADDRESS>Kernel command line: BOOT_IMAGE=/boot/vmlinuz-6.9.9-060909-generic root=UUID=99b2cc4b-d814-457a-be83-92e966ce048d ro text amd_pstate=active
CPUID(0): AuthenticAMD 0x10 CPUID levels
CPUID(1): family:model:stepping 0x1a:44:0 (26:68:0) microcode 0x0
CPUID(0x80000000): max_extended_levels: 0x80000028
CPUID(1): SSE3 MONITOR - - - TSC MSR - HT -
CPUID(6): APERF, No-TURBO, No-DTS, No-PTM, No-HWP, No-HWPnotify, No-HWPwindow, No-HWPepp, No-HWPpkg, No-EPB
CPUID(7): No-SGX No-Hybrid
cpu0: cpufreq driver: amd-pstate-epp
cpu0: cpufreq governor: powersave
/dev/cpu_dma_latency:<PHONE_NUMBER> usec (default)
current_driver: acpi_idle
current_governor: menu
current_governor_ro: menu
cpu0: POLL: CPUIDLE CORE POLL IDLE
cpu0: C1: ACPI FFH MWAIT 0x0
cpu0: C2: ACPI IOPORT 0x414
cpu0: C3: ACPI IOPORT 0x415
RAPL: 234 sec. Joule Counter Range, at 280 Watts
cpu0: MSR_RAPL_PWR_UNIT: 0x000a1000 (1.000000 Watts, 0.000015 Joules, 0.000977 sec.)
Core    CPU     Avg_MHz Busy%   Bzy_MHz TSC_MHz IPC     IRQ     POLL    C1      C2      C3      POLL%   C1%     C2%     C3%     CorWatt PkgWatt

  -       1       0.03    3738    3793    0.26    343     6       24      79      222     0.00    0.01    0.12    99.84   0.03    22.24

0       0       3       0.07    3876    3793    0.16    137     4       22      26      87      0.00    0.12    0.39    99.44   0.01    22.24
how can I fix it?

The energymon-rapl implementation is designed for Intel processors, but may work with some AMD processors.  This energymon backend parses the sysfs filesystem at /sys/class/powercap/ looking for intel-rapl nodes.
First, are you running with sudo/root privileges?  Unfortunately this is needed to read powercap energy values.
If so and it's still not working, I'll point you to a different tool that's specifically meant for working with powercap (full disclosure: I'm also the developer).
If you're using a Debian-based Linux like Ubuntu, just:
sudo apt install powercap-utils

Otherwise you can compile/install from the its repository: https://github.com/powercap/powercap
Then run:
sudo powercap-info

and please share the output here.

I have ran sudo chmod u+s /usr/local/bin/energymon-* before run energymon-info.
I have also try powercap-info before, but nothing output after run it.
sudo modprobe intel_rapl_msr
sudo rapl-info
Zone does not exist
Considerations for common errors:

Ensure that the intel_rapl kernel module is loaded
Some files may simply not exist
On some systems, the kernel always returns an error when reading constraint max power (-U/--c-max-power)

If powercap-info (not just rapl-info) doesn't show any output, then your system hasn't loaded any powercap drivers from which you can read RAPL energy counters.  This usually means that either your CPU doesn't support RAPL or the Linux kernel version on your system isn't detecting the RAPL support (it looks like you're using a recent kernel, but you also said your CPU is new).
If you're confident your CPU supports RAPL, you can try to read the MSRs directly.  The energymon-msr backend does this, but I caution that (1) it's only for the "package" domain, and (2) MSR configurations can vary across CPUs and the msr energymon backend only implements the default configuration.

I got error with the energymon-msr (git master):
edison@u24:~/Downloads/energymon/build/msr$ sudo ./energymon-info
/dev/cpu/0/msr: Input/output error
energymon:finit: Input/output error
source: X86 MSR
exclusive: false
interval (usec): 1000
precision (uJ): 0
reading (uJ): 0
I think that maybe caused by the msr bits?

energymon-info only uses the MSR_RAPL_POWER_UNIT and MSR_PKG_ENERGY_STATUS registers, and IIRC you'll get an I/O error like you saw if either MSR doesn't exist.  Like I said, that code was written for Intel CPUs and only for the "package" domain.  If AMD implements RAPL in different MSRs, the energymon code would need to be modified.  If you can determine what turbostat is doing, you can make those changes yourself (I have no way to test them).
FYI you can test reading MSRs directly from the terminal using rdmsr (from the msr-tools Debian package I think).

I'd be curious if/how you end up getting any RAPL results from your AMD CPU, but AFAICT this isn't an issue in energymon.  Cheers.

I'd be curious if/how you end up getting any RAPL results from your AMD CPU, but AFAICT this isn't an issue in energymon. Cheers.

I can now use the simplest rdmsr to read the energy state of the processor in C code without any other libraries, and because the RAPL MSR of the processor is 64-bit, there is no need for additional code and thread code to wrap around for accumulation.

It looks like that repository has support for (at least some) AMD CPUs.  I can see that they specify different MSRs for those chips.  Whereas energymon-msr expects Intel CPUs and reads from MSRs 0x606 (for units) and 0x611 (for package energy), it looks like AMD requires MSRs 0xc0010299 and 0xc001029B, respectively.  The registers might also use different encodings, so even if you read them you'll need to know how to make sense of the bits.  I haven't looked myself.
Also, while I expect the registers on both architectures to be 64 bit, (1) not all the bits are necessarily used for the energy value and (2) the values have some unit of measurement that is not likely to be a standard unit (e.g., Joule or microJoule---hence the need for the power units register).  The energy register values can still overflow, but depending on how long your experiments run for, it may not be necessary for you to try and detect this in code as the likelihood if it happening in the middle of your experiments might be exceptionally small, esp. if you're system isn't drawing a lot of power.  In contrast, energymon-msr is designed to support long-running experiments on servers that can draw significantly more power and thus the energy counter registers would overflow frequently enough that it was a problem that needed to be addressed.


MJ_RAW in grapesJS MJML

is mj-raw will works for grapesJS MJML? If yes then how can I achieve this? Basically I want to add the html code inside my MJML. Is it possible in grapesJS MJML editor??
Thanks. Waiting for feedback and suggestions.

You could add the sourcedialog plugin for CKeditor

mj-raw is not included, but I think you could even use this plugin: https://github.com/artf/grapesjs-custom-code Just remember that your HTML will be rendered in email clients


no output on nova 2.2

the card is not showing up, sourcecode view in browser shows:
<html-card card="[object Object]" lens="" class="card-panel"></html-card>

was just a composer issue


Don't know how to / Can't assign UID when uploading

Sorry if this something simple I'm missing:
set -x
set -v

VIEWCONF='{}'
VIEWCONF_UID=empty-view
VIEWCONFS_URL=http://localhost:8888/api/v1/viewconfs/

# This doesn't work: Trying to curl in two different incompatible ways.
curl --globoff -H "Content-Type: application/json" -X POST -d "$VIEWCONF" $VIEWCONFS_URL -F "uid=$VIEWCONF_UID"

# Not recognized as a parameter in the url: random UID still generated. (GET will work.)
curl --globoff -H "Content-Type: application/json" -X POST -d "$VIEWCONF" $VIEWCONFS_URL'?uid='$VIEWCONF_UID

# UID is not recognized, and the data stored in the database has HTTP cruft. (GET fails because JSON can't be parsed.)
curl --globoff -F "fileupload=$VIEWCONF" -F "uid=$VIEWCONF_UID" $VIEWCONFS_URL

echo

curl -u `cat ~/.higlass-server-login` \
     -F<EMAIL_ADDRESS>\
     -F 'filetype=beddb' \
     -F 'datatype=gene-annotation' \
     -F 'uid=OHJakQICQD6gTD7skx4EWA' \
     -F 'name=Gene Annotations' \
     -F 'coordSystem=hg19' \
     localhost:80/api/v1/tilesets/

For comparison, here's tilesets from the google doc. Notice that the examples above are missing credentials, and have a different field for the file. Where does this come from? Are they actually being handled differently?

Viewconfs don't require credentials and can't be assigned a uuid (at the moment).
Preventing unauthorized file uploads is more important because those are user-visible (e.g. one can click 'add track' and see the list of tracks that are present). Viewconfs are largely invisible. Unless you know the UUID of a particular viewconf, you won't be aware of its existence.
We can make the viewconfs method accept uids. The function viewconfs in tilesets/views.py is responsible for storing saved viewconfs.
To see an example of where we store 'uids', take a look at perform_create in tilesets/views.py.

I'll take a swing at this. Being able to give the ID when creating a viewconfig would make it a lot easier to describe reproducers.

This is implemented and tested: https://github.com/hms-dbmi/higlass-server/blob/7e9519d36b1b91d69c5a5e7755bd43cb28a2762f/tilesets/tests.py#L65-L66


Update README.md

typo

Django Security Check identified issues:
System check identified some issues:
WARNINGS:
?: (security.W005) You have not set the SECURE_HSTS_INCLUDE_SUBDOMAINS setting to True. Without this, your site is potentially vulnerable to attack via an insecure connection to a subdomain. Only set this to True if you are certain that all subdomains of your domain should be served exclusively via SSL.
?: (security.W008) Your SECURE_SSL_REDIRECT setting is not set to True. Unless your site should be available over both SSL and non-SSL connections, you may want to either set this setting True or configure a load balancer or reverse-proxy server to redirect all connections to HTTPS.
?: (security.W018) You should not have DEBUG set to True in deployment.
?: (security.W021) You have not set the SECURE_HSTS_PRELOAD setting to True. Without this, your site cannot be submitted to the browser preload list.
System check identified 4 issues (0 silenced).


inject environment root router into new root from sheet & present

inject environment root router into new root from sheet & present transitions

Codecov Report

Merging #8 (58e246c) into main (d18a986) will decrease coverage by 2.12%.
The diff coverage is 42.30%.

@@            Coverage Diff             @@
##             main       #8      +/-   ##
==========================================
- Coverage   84.30%   82.17%   -2.13%
==========================================
  Files          12       12
  Lines         344      359      +15
==========================================
+ Hits          290      295       +5
- Misses         54       64      +10

Impacted Files
Coverage Δ

Sources/sRouting/Viewing/NavigatorView.swift
29.11% <0.00%> (-4.22%)
:arrow_down:

Sources/sRouting/Viewing/ScreenView.swift
88.88% <50.00%> (ø)

Sources/sRouting/Models/RootRouter.swift
100.00% <100.00%> (ø)

...sts/sRoutingTests/Testcases/TestInitializers.swift
97.64% <100.00%> (+0.11%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d18a986...58e246c. Read the comment docs.


Feature Requests

First, many thanks for building this!  It's exactly what we need to increase adoption by lowering the technical knowledge barrier.  I've tried to spin up a node several times using different guides with no success.  I tried Docker, WSL, Rand's windows distro.  Nothing worked, and even the Discord group couldn't help me figure it out.  "Must be a hardware issue"......
I'm not an expert, but certainly a little more tech savvy than most users. No changes to my hardware and your distro works flawlessly.  Three nodes up and running in true one-click fashion.  Bravo!
On to the feature requests:

Ability to specify drive/directory for binary installation and working data files.
Option to run archive node.

Thanks, glad you enjoy it!
As for your feature requests, I'm not sure if they really fit into the one-click node. This really is meant for users who are not tech savvy. For them it doesn't really matter where the data directory or binaries are located. They also probably don't need to be running an archive node. If someone is wanting that level of control over their node, then I think they'd be better off running a node in the more traditional way.

Thanks, glad you enjoy it!
As for your feature requests, I'm not sure if they really fit into the one-click node. This really is meant for users who are not tech savvy. For them it doesn't really matter where the data directory or binaries are located. They also probably don't need to be running an archive node. If someone is wanting that level of control over their node, then I think they'd be better off running a node in the more traditional way.

This application is absolutely amazing!! How does the foundation not have a solution like this?
I would second the feature request for choosing a drive/directory. I can't run a node on the drive that has my OS, however I can run a node on my second drive.

Thanks, glad you enjoy it!
As for your feature requests, I'm not sure if they really fit into the one-click node. This really is meant for users who are not tech savvy. For them it doesn't really matter where the data directory or binaries are located. They also probably don't need to be running an archive node. If someone is wanting that level of control over their node, then I think they'd be better off running a node in the more traditional way.

This application is absolutely amazing!! How does the foundation not have a solution like this?
I would second the feature request for choosing a drive/directory. I can't run a node on the drive that has my OS (no space), however I can run a node on my second drive.
PS: I think there are tech savvy people that don't yet know, or want to know, the nitty gritty of running a node - they just want to participate.

I also agree with number 1 of the feature request... due to space issues, there are people like me who cannot run this on their OS drive and need to run it on a separate hard drive

Thanks, glad you enjoy it!
As for your feature requests, I'm not sure if they really fit into the one-click node. This really is meant for users who are not tech savvy. For them it doesn't really matter where the data directory or binaries are located. They also probably don't need to be running an archive node. If someone is wanting that level of control over their node, then I think they'd be better off running a node in the more traditional way.

Thanks for replying.  I see your point with running an archive node, however I humbly request that you revisit directory replacement.  As others have stated, I also have a system that can't support a node on the OS drive but do have capacity on a storage drive on that system.  Technical prowess shouldn't be a barrier to supporting the ecosystem.

Good points.
I'll work on adding a setting for it in the next release.

v1.4.0 lets you change the data directory now.


Production Profile with Docker not working

Describe the bug
Hey, I am using a Dockerfile like this but the production profile is not used from Serverpod. In the App Folder the production.yaml is visible with the values. If I take the Dockerfile locally from the app folder, I can also start it normally with dart bin/main.dart --apply-migrations --mode production and the config is loaded correctly!
# Stage 1: Build the server application
FROM --platform=$BUILDPLATFORM dart:3.5.0 AS build

# Resolve app dependencies.
WORKDIR /app
COPY server /app

RUN dart pub get
RUN dart compile exe bin/main.dart -o bin/main

# Stage 2: Create the final Docker image
FROM debian:buster-slim

# Set environment variables
ENV runmode=production
ENV serverid=default
ENV logging=verbose
ENV role=monolith

# Copy the built artifacts from the previous stage
COPY --from=build /app/bin/main /app/bin/main
COPY --from=build /app/config/ /app/config/
COPY --from=build /app/web/ /app/web/
COPY --from=build /app/migrations/ /app/migrations/

# Copy passwords file
COPY ./intolerapp_server/config/passwords.yaml /app/config/passwords.yaml

# Expose ports
EXPOSE 8083
EXPOSE 8084
EXPOSE 8085

# Define the command to run the server application
CMD /app/bin/main --mode $runmode --server-id $serverid --logging $logging --role $role --apply-migrations

My production yaml which is also visible in the Docker Container:
# Configuration for the main API server.
apiServer:
  port: 8083
  publicHost: api.xy.tools.xy.de
  publicPort: 443
  publicScheme: https

# Configuration for the Insights server.
insightsServer:
  port: 8084
  publicHost: insights.xy.tools.xy.de
  publicPort: 443
  publicScheme: https

# Configuration for the web server.
webServer:
  port: 8085
  publicHost: app.xy.tools.xy.de
  publicPort: 443
  publicScheme: https

2024-08-20T12:17:50.661668356Z mode: production, role: monolith, logging: verbose, serverId: default
2024-08-20T12:17:50.661670971Z api port: 8080
2024-08-20T12:17:50.661673445Z api public host: localhost
2024-08-20T12:17:50.661675810Z api public port: 8080
2024-08-20T12:17:50.661678435Z api public scheme: http

Expected behavior
The Server should start with wanted Ports 8083 instead of 8080 and the url instead of localhost
Serverpod versions

CLI Version: 2.0.2
Library version:   serverpod: 2.0.2

Platform information
On my VPS and Docker Desktop

Hey @FantaMagier!
Are you sure the production.yaml file is accessible in the container on the correct path?
If the config yaml can't be loaded, the instance will use the default values written mentioned:
https://github.com/serverpod/serverpod/blob/f1d65ebf578095ddcead0d352e402840bfb625ba/packages/serverpod_shared/lib/src/config.dart#L156-L170
If so, did you rebuild the Dockerfile once your project was done? If you inspect the production.yaml file in the container, does it match the one you have defined locally?

Hey @SandPod
Thanks for your reply. The production.yaml in the app folder is the same one I used locally. I tested the docker image offline and copied the app folder from the container to my local folder and launched the project manually and everything works as expected. Only using the dockerfile does not work.
I am using its GitHub actions code<EMAIL_ADDRESS>I save everything and run Serverpod generate und push the files to my git repo and the action builds the image.

@BenAuerDev Do recognize any issues with this?

@BenAuerDev Do recognize any issues with this?

I think the issue is related to the SSL/HTTPS configuration.
@FantaMagier Can you show me the part in your docker-compose.yaml that configures Traefik to handle SSL/HTTPS?
But please remove any sensitive information first
If you're using Traefik, it should look something like:
  flashlist-api:
    image: ghcr.io/benauerdev/flashlist:latest
    container_name: flashlist-api
    restart: always
    security_opt:
      - no-new-privileges:true
    networks:
      - web
    labels:
      - "traefik.enable=true"

      # For API
      - "traefik.http.routers.myapi.entrypoints=websecure"
      - "traefik.http.routers.myapi.rule=Host(`api.flashlistapp.com`)"
      - "traefik.http.routers.myapi.tls=true"
      - "traefik.http.routers.myapi.tls.certresolver=myresolver"
      - "traefik.http.services.myapi.loadbalancer.server.port=8080"

      # For Insights
      - "traefik.http.routers.myapi.entrypoints=websecure"
      - "traefik.http.routers.myapi.rule=Host(`insights.flashlistapp.com`)"
      - "traefik.http.routers.myapi.tls=true"
      - "traefik.http.routers.myapi.tls.certresolver=myresolver"
      - "traefik.http.services.myapi.loadbalancer.server.port=8081"

      # For Webserver
      - "traefik.http.routers.myapi.entrypoints=websecure"
      - "traefik.http.routers.myapi.rule=Host(`app.flashlistapp.com`)"
      - "traefik.http.routers.myapi.tls=true"
      - "traefik.http.routers.myapi.tls.certresolver=myresolver"
      - "traefik.http.services.myapi.loadbalancer.server.port=8082"

and
  traefik:
    image: "traefik:v2.5"
    container_name: "traefik"
    command:
      - "--log.level=DEBUG"
      - "--providers.docker=true"
      - "--providers.docker.exposedbydefault=true"
      - "--entrypoints.web.address=:80"
      - "--entrypoints.websecure.address=:443"
      - "--certificatesresolvers.myresolver.acme.httpchallenge=true"
      - "--certificatesresolvers.myresolver.acme.httpchallenge.entrypoint=web"
      # uncomment this line to test ssl generation first
      # - "--certificatesresolvers.myresolver.acme.caserver=https://acme-staging-v02.api.letsencrypt.org/directory"
      - "--certificatesresolvers.myresolver.acme.email=${TRAEFIK_EMAIL}"
      - "--certificatesresolvers.myresolver.acme.storage=/letsencrypt/acme.json"
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock:ro"
      - "./letsencrypt:/letsencrypt"
    networks:
      - web

Hey @BenAuerDev, thanks for your reply. I want to try Coolify for deployment because you can add Postgres and Redis easily.
My Traefik Conf for the Coolify Serverpod Container is:
traefik.enable=true
traefik.http.middlewares.gzip.compress=true
traefik.http.middlewares.redirect-to-https.redirectscheme.scheme=https
traefik.http.routers.http-0-coolifyContainerID.entryPoints=http
traefik.http.routers.http-0-coolifyContainerID.middlewares=redirect-to-https
traefik.http.routers.http-0-coolifyContainerID.rule=Host(`api.backend.url.com`) && PathPrefix(`/`)
traefik.http.routers.http-0-coolifyContainerID.service=http-0-coolifyContainerID
traefik.http.routers.http-1-coolifyContainerID.entryPoints=http
traefik.http.routers.http-1-coolifyContainerID.middlewares=redirect-to-https
traefik.http.routers.http-1-coolifyContainerID.rule=Host(`insights.backend.url.com`) && PathPrefix(`/`)
traefik.http.routers.http-1-coolifyContainerID.service=http-1-coolifyContainerID
traefik.http.routers.http-2-coolifyContainerID.entryPoints=http
traefik.http.routers.http-2-coolifyContainerID.middlewares=redirect-to-https
traefik.http.routers.http-2-coolifyContainerID.rule=Host(`app.backend.url.com`) && PathPrefix(`/`)
traefik.http.routers.http-2-coolifyContainerID.service=http-2-coolifyContainerID
traefik.http.routers.https-0-coolifyContainerID.entryPoints=https
traefik.http.routers.https-0-coolifyContainerID.middlewares=gzip
traefik.http.routers.https-0-coolifyContainerID.rule=Host(`api.backend.url.com`) && PathPrefix(`/`)
traefik.http.routers.https-0-coolifyContainerID.service=https-0-coolifyContainerID
traefik.http.routers.https-0-coolifyContainerID.tls.certresolver=letsencrypt
traefik.http.routers.https-0-coolifyContainerID.tls=true
traefik.http.routers.https-1-coolifyContainerID.entryPoints=https
traefik.http.routers.https-1-coolifyContainerID.middlewares=gzip
traefik.http.routers.https-1-coolifyContainerID.rule=Host(`insights.backend.url.com`) && PathPrefix(`/`)
traefik.http.routers.https-1-coolifyContainerID.service=https-1-coolifyContainerID
traefik.http.routers.https-1-coolifyContainerID.tls.certresolver=letsencrypt
traefik.http.routers.https-1-coolifyContainerID.tls=true
traefik.http.routers.https-2-coolifyContainerID.entryPoints=https
traefik.http.routers.https-2-coolifyContainerID.middlewares=gzip
traefik.http.routers.https-2-coolifyContainerID.rule=Host(`app.backend.url.com`) && PathPrefix(`/`)
traefik.http.routers.https-2-coolifyContainerID.service=https-2-coolifyContainerID
traefik.http.routers.https-2-coolifyContainerID.tls.certresolver=letsencrypt
traefik.http.routers.https-2-coolifyContainerID.tls=true
traefik.http.services.http-0-coolifyContainerID.loadbalancer.server.port=8083
traefik.http.services.http-1-coolifyContainerID.loadbalancer.server.port=8084
traefik.http.services.http-2-coolifyContainerID.loadbalancer.server.port=8085
traefik.http.services.https-0-coolifyContainerID.loadbalancer.server.port=8083
traefik.http.services.https-1-coolifyContainerID.loadbalancer.server.port=8084
traefik.http.services.https-2-coolifyContainerID.loadbalancer.server.port=8085
caddy_0.encode=zstd gzip
caddy_0.handle_path.0_reverse_proxy={{upstreams 8083}}
caddy_0.handle_path=/*
caddy_0.header=-Server
caddy_0.try_files={path} /index.html /index.php
caddy_0=https://api.backend.url.com
caddy_1.encode=zstd gzip
caddy_1.handle_path.1_reverse_proxy={{upstreams 8084}}
caddy_1.handle_path=/*
caddy_1.header=-Server
caddy_1.try_files={path} /index.html /index.php
caddy_1=https://insights.backend.url.com
caddy_2.encode=zstd gzip
caddy_2.handle_path.2_reverse_proxy={{upstreams 8085}}
caddy_2.handle_path=/*
caddy_2.header=-Server
caddy_2.try_files={path} /index.html /index.php
caddy_2=https://app.backend.url.com
caddy_ingress_network=coolify

I am using 8083,8084,8085 as port expose and for the domains https://api.backend.url.com:8083,https://insights.backend.url.com:8084,https://app.backend.url.com:8085
When I open a site I get a Bad Gateway Error but it is werid because running my Serverpod project locally in production the sever connects to the DB etc and uses the production conf but not with the Dockerimage / file.

@FantaMagier Unfortunately I have never used coolify. But I'll can take a look on the weekend 😊
Also just to be sure on top of your Dockerfile you have
COPY server /app
But when you copy the password you have
COPY ./intolerapp_server/config/passwords.yaml /app/config/passwords.yaml
Shouldn't it both be intolerapp_server?

@BenAuerDev Yes that is correct in my dockerfile, I changed it for the issue :D

@FantaMagier I had a closer look at your Dockerfile and yaml-file, apart from the above mentioned they look good.
I also checked the Traefik/Caddy config you provided. And I noticed that you  shared both a Traefik and a Caddy configuration. Where imho it would be better to choose one for handling SSL and routing to avoid conflicts.
Furthermore I think you're missing this part in your docker-compose.yaml:
If you use Traefik
  traefik:
    image: "traefik:v2.5"
    container_name: "traefik"
    command:
      - "--log.level=DEBUG"
      - "--providers.docker=true"
      - "--providers.docker.exposedbydefault=true"
      - "--entrypoints.web.address=:80"
      - "--entrypoints.websecure.address=:443"
      - "--certificatesresolvers.myresolver.acme.httpchallenge=true"
      - "--certificatesresolvers.myresolver.acme.httpchallenge.entrypoint=web"
      # uncomment this line to test ssl generation first
      # - "--certificatesresolvers.myresolver.acme.caserver=https://acme-staging-v02.api.letsencrypt.org/directory"
      - "--certificatesresolvers.myresolver.acme.email=${TRAEFIK_EMAIL}"
      - "--certificatesresolvers.myresolver.acme.storage=/letsencrypt/acme.json"
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock:ro"
      - "./letsencrypt:/letsencrypt"
    networks:
      - web

If you use Caddy (it would look something like this)
caddy:
  image: caddy:latest
  restart: unless-stopped
  ports:
    - "80:80"
    - "443:443"
  volumes:
    - ./Caddyfile:/etc/caddy/Caddyfile
    - caddy_data:/data
  networks:
    - web

volumes:
  caddy_data:

Without this setup, there is no container on your VPS running Traefik or Caddy. Which means the config you created is not being serviced, resulting in  bad gateway when trying to access the url.
Furthermore it's important to make sure there is a network that connects your Docker containers with each other, in my example I called it web.
networks:
  web:
    external: true

@BenAuerDev @SandPod
I sloved the issue by set the correct Permission in the Dockerfile :D
Thanks your help!
RUN chmod -R 755 /app

# Expose ports
EXPOSE 8083
EXPOSE 8084
EXPOSE 8085

# Define the entrypoint and command to run the server application
CMD /app/bin/main --mode $runmode --server-id $serverid --logging $logging --role $role --apply-migrations

Awesome! 🙌
Thanks for the help here @BenAuerDev 🚀


Clicking on channel name from topic view should go to channel feed

When you click on a channel in the left sidebar and you're already narrowed to some topic within the channel, the click should always take you to the channel feed. The "Channel links in the left sidebar go to" preference should be ignored in this situation.

CZO thread

@zulipbot claim

@zulipbot unclaim

@zulipbot claim

Seems im taking a little long to understand the code and commit. I'll free to other person, but if nobody assign i can come back l8.

@zulipbot unclaim

@zulipbot claim


add unix socket connector

add unix socket connector

thanks will make a release sometimes this week with it.


run comby-rust

run: https://github.com/huitseeker/comby-rust

It is better not to include it inside CI as it does the bruteforce search and replace and changed code needs to be manually approved, in this example out of 3 changed items only one was okay.


[FEA]: New archery game added

FIX #41

looks great! please resolve the conflicts and make PR again.

Is it ok now?


Unused parameter in function and  Adding DeepSource

Describe your changes
Unused parameter should be replaced by underscore into the function
Adding Deepsource to make things easier to analyse the code
What is DeepSource and How to use the DeepSource ?
  DeepSource is an automated code review tool that helps developers find and automatically fix bugs in their code.
  And most important it's save  the developer time

How to use ?
1.  Create an account on DeepSource.
2.  Pick a Python repository to analyse.
3.  Visit to DeepSource dashboard.
4.  Choose your project from the repository list.
5.  Add DeepSource configuration file.
6.  Run the analysis.
Note: It will shows the all kind of different  bugs and suitable fixes
Screenshots - If Any (Optional)

VISIT DEEPSOURCE DASHBOARD

SELECT LANGUAGE TO FIX BUGS

LANGUAGE AND FRAMEWORKS

THEN CODE ANALYSIS REPORT

Issue ticket number and link
696
Checklist before requesting a review

[✓ ] My code follows the style guidelines of this project
[✓] I have commented my code, particularly in hard-to-understand areas
[✓] I have performed a self-review of my code
[✓] If it is a core feature, I have added thorough tests.
[✓] New and existing unit tests pass locally with my changes

Please open the PR against the dev branch (Unless the PR contains only documentation changes)

Hello @dwertent  as you requested to make some changes and that i have done weeks before please review again

@Cvr421  Will the graph update automatically when we make changes? If not, what can we do so it will get updated?

@dwertent Did you mean that when we make any changes or fix bugs. That will show in Deepsource  analyzer and update the graph .  Is that i am getting you right

@dwertent
It will update the graph when we create the pull request directly from deep source account. when bugs are fixed

@dwertent Please review again

This should be broken down into two PRs, one for unused function, and the other for deepsource integration.
For now, I do not want to add deepsource integration since it is not clear to me how it will get updated without manually changing the badge.


Syntax highlightning incorrectly recognizes "|"

(defn | [n1 n2] (Long/parseLong (str n1 n2)))
(for [o [* + |]] (o 1 2))

Technically it’s not allowed but in practice Clojure doesn’t seem to mind. Added


Master

Add example for use addRoute function.

There are two different between this function and the original one of i18next-node:

ensureLngSupport
It was not included the ensureIsSupportedLanguage function

parameter: lng
this is converted with the value of this language to avoid that you can have the incorrect routes.
example:
if I have these routes:
/en/team
/fr/equipe
/it/squadra

if you leave the parameter: lng as a parameter would also agree to the following routes:
/it/team
/fr/team
/en/equipe
/it/equipe
/en/squadra
/fr/squadra
these routes are not correct and should not be accessible, for this reason I prefer put the language value into :lng parameter for localized routes

great...will be merged and published with next version....might get to this later the day...
thanks a lot for making this better.

in published v0.4.0


uncertainty for self-supervised learning

@yaringal
Thank you for your example, it helps a lot to understand the paper.  I am currently use the proposed formula (exp(-log_var)*loss+log_var)) in self-supervised learning with uncertainty estimation.
In my project,  the loss is L1 distance between input images pixels and warped images pixels, the loss works well along. But, when I take uncertainty into training together using the above formula, however, performance drops a lot.
I have totally no idea why.  Do you have any advice? By the way, before taking L1 distance, diff = warp_pixel - input_pixel  follows Gaussian distribution perfectly.

@griffintin hi, I think the problem maybe that in the uncertainty estimation, the loss item follows an L2 format(eg. Euclidean loss), instead of L1. So, maybe you can try out L2 loss, which makes the likelihood function reasonable.
Here I have another question @yaringal . In a case where the loss does not follow Euclidean loss for regression or cross entropy loss for classification, can we use this adaptive loss weights based on uncertainty? In my objection detection project, the loss for location is 1 - IoU, where the IoU is not regressed by the model directly, but derived by the regressed corner coordinates and ground truth. So, it seems that I can not simply replace the loss_{loc} with exp(-log_{\sigma})*loss_{loc}+log_{\sigma}. What do you think about this problem? Maybe I can forget the 1 - IoU and turn to use L2 loss of coordinates? Or there is some transformation to handle it?


getRecordAt is an O(n) operation

To move a record we have to get the record at a file position. This is done as a dumb scan of all the records in a map. We should put the records in a TreeMap keyed by file position. We can then find the first very fast to expand the index and can find adjacent records fast to update their free space.
Take extra care about updates to this map being in sync.

committed to 0.9.8-SNAPSHOT


Add exa

Hi @patsevanton
thank you for your contribution :tada:

cache.json is automatically generated by the CI, can you revert your change on this file ?
Can you also add the distribution in the Supported distributions section of README.md ?

thank you !

Hi @Cleming
Update PR

Perfect :pray:
But doesn't work here
❯ binenv install exa
2021-04-21T15:03:05+02:00 WRN version for "exa" not specified; using "0.10.1"
2021-04-21T15:03:06+02:00 ERR unable to install "exa" (0.10.1) error="unable to download binary at https://github.com/ogham/exa/releases/download/0.10.1/exa-Linux-amd64-0.10.1.zip: 404 Not Found"

I think you have to do something like :
  exa:
    description: A modern replacement for ‘ls’.
    map:
      amd64: x86_64
    list:
      type: github-releases
      url: https://api.github.com/repos/ogham/exa/releases
    fetch:
      url: https://github.com/ogham/exa/releases/download/v{{ .Version }}/exa-{{ .OS }}-{{ .Arch }}-v{{ .Version }}.zip
    install:
      type: zip
      binaries:
        - exa

You can test it by editing your ~/.config/binenv/distributions.yaml and then binenv update exa -f and binenv install exa

lgtm @patsevanton :tada:

Great! Thanks!

Thanks !


[LocaleSelector] Mobile issues for region selector view

Detailed description

Describe in detail the issue you're having.

Various loading issues in mobile for the region selector view. See below:

Close button is covered by the masthead
height issues with the tiles

What browser are you working in?

Browserstack: Pixel 3XL (Android) Chrome browser

What version of the IBM.com Library are you using?

v1.2.0-rc.1

Also on Iphone XS iOS 12:

I agree with Sev 1 priority


Multi-line block-level attributes require future line parsing

The syntax reference states:

The contribution a line makes to block-level structure never depends on a future line.

and

Block attributes have the same syntax as inline attributes, but they must fit on one line.

However, the implementation intentionally allows multi-line attributes if indentation is used:
{
 %
 %
 attr=val
 %
 %
 %
 %
 }
> a

a

This seems to contradict that a line's impact on a block level structure never depends on a future line. When looking at the first line we can't tell whether it is a block-level attribute or a paragraph without parsing future lines until the closing }.
I can see that it would be useful to be able to hard-wrap e.g. long values, though it does not seem to work currently: #162.

That's true, the docs need updating in this regard (or maybe we should change this feature).  The current implementation handles this as follows: we feed content line by line into an attribute parsing state machine, while also saving the raw lines, and when it fails, we convert the block from attributes to paragraph and feed the raw lines to the inline parser.
It feels slightly unprincipled compared to the rest, so maybe this should be reconsidered. My original vision was to require it to fit on one line.


Supply token as an environment variable

I'm not putting that in my codebase. Or is the intended usage to gitignore git-packages.json? The disadvantage to gitignoring would be having to regenerate the json file on each fresh clone of the repo.

Should be resolved by #9

Thanks, I missed that when looking through the issues. It might be a good idea to update the README and recommend this approach.

We are moving away from tarballs to using pure git. So this will be fixed in #19


docs: Update readme for Q3 2021

Hi Angular Material team,
Is it possible to update the readme with what's planned for Q3 2021?
Thank you!

I think that it's roughly the same, but we'll discuss and see if there are any changes we can make (possibly a minor addition). We do certainly need to update the mention of Q2 to Q3.

OK, we've got a list of updates and we'll update the readme soon.

Just updated this for Q4. Q3 was super hectic, so this slipped through the cracks.


s/Application Gateway/App Gateway/g

This is a no-op PR renaming all string literal and comment occurrences of Application Gateway to App Gateway
This is mainly for consistency - we tend to use these two interchangeably. In logs (which we have many of) would be great to consistently use the shorter form.

Should we make everything Application Gateway instead in full ?

shelving for later


Process qualities with HTML in name

A Quality name can contain HTML, most prominently <i> italic text. For example, in Khatun-class Electric Launch, the Khatun is italic. The addon would pass that as is to the wiki, leading to an error.
The fix strips HTML tags from the quality name.

@kkotenko I've assume you tested it? :)

@lensvol Yeah, but I really should have mentioned this, so here is some visual proof :)

Thank you! :)


Converting Torch modules that use the max function

Description of the bug:
Given a module that calculates the maximum of a tensor, the convert method failed to convert the model appropriately.
import ai_edge_torch
import torch

class Max(torch.nn.Module):
    def forward(self, x):
        return x.max()

sample_input = (torch.randn(8),)
edge_model = ai_edge_torch.convert(Max().eval(), sample_input)

Actual vs expected behavior:
Actual behavior:
ConverterError: Could not translate MLIR to FlatBuffer.<unknown>:0: error: loc(callsite(callsite(callsite("__main__.Max;" at fused["XlaCallModule:", "XlaCallModule@__inference_inner_2449"]) at fused["StatefulPartitionedCall:", "StatefulPartitionedCall@__inference_signature_wrapper_2455"]) at fused["StatefulPartitionedCall:", "StatefulPartitionedCall"])): 'vhlo.reduce_v1' op is not part of the vhlo support yet.
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
<unknown>:0: error: failed while converting: 'main':

Expected behavior:
The method successfully converts the model
Any other information you'd like to share?
No response

I was able to replicate, exactly as above on internal Debian.

Is there any update on support for vhlo.reduce_v1? I'm trying to convert a little variation on MobileNet v3 for audio EfficientAT. Such model can be obtained using models.mn.model.get_model() from referenced repository with defaults. This model successfully goes through torch.export for me, but fails on tflite conversion. I could export MobileNet v3 from TorchVision repository without any problem and this EfficientAT implementation is derived from here but is using models.mn.block_types.ConcurrentSEBlock with max aggregation.
ConverterError: Could not translate MLIR to FlatBuffer.<unknown>:0: error: loc(callsite(callsite(callsite("models.mn.model.MN/models.mn.block_types.InvertedResidual_4/torch.nn.modules.container.Sequential_block/models.mn.block_types.ConcurrentSEBlock_2;" at fused["XlaCallModule:", "XlaCallModule@__inference_inner_29013"]) at fused["StatefulPartitionedCall:", "StatefulPartitionedCall@__inference_signature_wrapper_29553"]) at fused["StatefulPartitionedCall:", "StatefulPartitionedCall"])): 'vhlo.reduce_v1' op is not part of the vhlo support yet.
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
<unknown>:0: note: loc(callsite(callsite(callsite("models.mn.model.MN/models.mn.block_types.InvertedResidual_4/torch.nn.modules.container.Sequential_block/models.mn.block_types.ConcurrentSEBlock_2;" at fused["XlaCallModule:", "XlaCallModule@__inference_inner_29013"]) at fused["StatefulPartitionedCall:", "StatefulPartitionedCall@__inference_signature_wrapper_29553"]) at fused["StatefulPartitionedCall:", "StatefulPartitionedCall"])): see current operation:
%128 = "vhlo.reduce_v1"(%127, %99) <{dimensions = #vhlo.tensor_v1<dense<0> : tensor<1xi64>>}> ({
^bb0(%arg15: tensor<f32>, %arg16: tensor<f32>):
  %280 = "tfl.maximum"(%arg15, %arg16) : (tensor<f32>, tensor<f32>) -> tensor<f32>
  "vhlo.return_v1"(%280) : (tensor<f32>) -> ()
}) : (tensor<1x1x72x8x8xf32>, tensor<f32>) -> tensor<1x72x8x8xf32>

Sorry for not concrete help on debugging this, but I've just started digging into your work and learning the innnards. I'm very excited about this set of tools, thank you :)

There is a current PR for this https://github.com/tensorflow/tensorflow/pull/73635.

I hit the same issue recently. I was able to export the model by using pinning tf-nightly==2.18.0.dev20240722 and using the latest ai-edge-torch-nightly.
See bottom of #246


Rule::when validation rule includes field in validated request even if no validations are ran against the field

Laravel Version: 9.39.0
PHP Version: 8.1.2
Database Driver & Version:

Description:
Using just the Rule::when($condition, $rules) on a field, includes the field on $request->validated() even if $condition fails and $rules are not actually checked. I would not expect the field to be included in $request->validated() if $condition does not pass, and no other validation rules are checked on the field.
Steps To Reproduce:
On a FormRequest:
public function rules(): array
    {
        return [
            'url_field' => [
                Rule::when(filter_var($this->url_field, FILTER_VALIDATE_URL), ['sometimes', 'required', 'url', 'bail'])
            ],
        ];
    }

-- Request passes validation (expected)
-- Field is present in validated attributes (not expected)
-- dd of $request->validated() on controller:

Current solution using array unpacking (requires PHP ^8.1):
public function rules(): array
    {
        return [
             ...(filter_var($this->url_field, FILTER_VALIDATE_URL)) ? [
                'url_field' => ['required', 'url', 'bail']
             ] : [],
        ];
    }

Please provide an example I can run with a simple Validator::make call.

Sure, hope this helps. Let me know if something else is needed.
$cases = [
    'non_url_field' => 'no file provided!', // fails when condition
    'empty_field' => '',                    // fails when condition
    'correct_field' => 'https://www.example.de/63344d8899032.jpg' // passes
];

$validator = Validator::make(
    data: $cases,
    rules: [
        'non_url_field' => Rule::when(
            $cases['non_url_field'] !== 'no file provided!',
            ['required', 'url']
        ),
        'empty_field' => Rule::when(
            !empty($cases['empty_field']),
            ['required', 'url']
        ),
        'correct_field' => Rule::when(
            filter_var($cases['correct_field'], FILTER_VALIDATE_URL),
            ['required', 'url']
        ),
    ]
);

return response()->json([
    'validated' => $validator->validated(), // all three fields
    'failed' => $validator->failed()        // empty array
]);

Response:
{
    "validated": {
        "non_url_field": "no file provided!",
        "empty_field": "",
        "correct_field": "https://www.example.de/63344d889903232532.jpg"
    },
    "failed": []
}

'non_url_field' and 'empty_field' are present in validated even tho clearly no rules are ran against them, (not including 'when' itself).

The validated method returns the fields that were validated, meaning all fields, not just the passing ones. Please see https://laravel.com/docs/9.x/validation#working-with-validated-input
The Rule::when adds rules based on if the boolean returns true. Since the first two return false, only the last defined rules are added.


Null result on GraphQL endpoint for collection names with more than one word

Similar behavior described in issue to https://github.com/DomDew/strapi-plugin-fuzzy-search/issues/37
While attempting to register a new model in fuzzy search with uid such as api::app-user.app-user and modelName app-user, always  returns null as a response, regardless if the query string is matched or not
Even if no results where to be found, the expected response would be data: []
Not working ❌
{
          uid: 'api::app-user.app-user',
          modelName: 'app-user',
          fuzzysortOptions: {
            characterLimit: 60,
            threshold: -100,
            limit: 20,
            keys: [
              {
                name: 'name',
                weight: 100,
              },
              {
                name: 'nickname',
                weight: 60,
              },
            ],
          },
        },

Response:
"data": {
    "search": {
      "appUsers": null
    }
  }

When querying a model that is named with a single word (no dashes autogenerated by strapi when creating the content type), the results are returned correctly (or with an empty array if no matches)
this works ✅
{
    uid: 'api::user.user',
    modelName: 'user',
    fuzzysortOptions: {
        characterLimit: 60,
        threshold: -100,
        limit: 20,
        keys: [{
                name: 'name',
                weight: 100,
            },

        ],
    },
},

Response:
{
  "data": {
    "search": {
      "users": {
        "data": [
          {
            "attributes": {
              "name": "myuser"
            }
          }
        ]
      }
    }
  }
}

This only happens when querying via GraphQL, querying through REST endpoint returns the array as expected:
http://localhost:1337/api/fuzzy-search/search?query=myuser
"app-users": [{
    "id": 1,
    "name: "myuser"
    "createdAt": "2023-01-26T14:57:48.151Z",
    "updatedAt": "2023-01-26T14:57:49.382Z",
    "publishedAt": "2023-01-26T14:57:49.375Z"
}],

Plugin versions
<EMAIL_ADDRESS><EMAIL_ADDRESS>

Hey @aadelgrossi ,
thanks for bringing this up!
Trying to fix this asap.

Wow, it was some serious finding of a needle in a haystack, but it should be fixed now in 1.10.3!
@all-contributors please add @aadelgrossi  for bug

@DomDew thank you so much for taking the time and patching this so quickly 😄🙏


Update for backwards compatibility; shifting timezones

Per #3 - added backwards compatibility by adding a field to the trigger comment.
For timestamptz column; explicitly provides UTC timezone in trigger and check constraints

Hey @richardsmd, finally got around to another review of this. Really well done! 💯  Will cut a new version in a bit.


Repo sync for protected CLA branch

The pull request is created from main637746050539666281 to main to fix git push error for protected CLA branch

@huypub : Thanks for your contribution! The author(s) have been notified to review your proposed change.


Merging #307 appears to have caused 'import caffe_pb2' errors on Ubuntu 14.04 AWS

Sorry, python is not really my thing, so I can't be much more helpful than that.
Here is the error:
libdc1394 error: Failed to initialize libdc1394
Traceback (most recent call last):
  File "/home/ubuntu/digits/digits-devserver", line 44, in <module>
    from digits.webapp import app, socketio, scheduler
  File "/home/ubuntu/digits/digits/webapp.py", line 8, in <module>
    import digits.scheduler
  File "/home/ubuntu/digits/digits/scheduler.py", line 18, in <module>
    from model import ModelJob
  File "/home/ubuntu/digits/digits/model/__init__.py", line 3, in <module>
    from job import ModelJob
  File "/home/ubuntu/digits/digits/model/job.py", line 4, in <module>
    from . import tasks
  File "/home/ubuntu/digits/digits/model/tasks/__init__.py", line 4, in <module>
    from caffe_train import CaffeTrainTask
  File "/home/ubuntu/digits/digits/model/tasks/caffe_train.py", line 13, in <module>
    import caffe_pb2
ImportError: No module named caffe_pb2

Configuration is:

Cuda 7.0
CuDNN v3
Caffe (nvidia fork) 0.13
Ubuntu 14.04
AWS g2.2xlarge
ATLAS

Images built with the same chef scripts a few days ago work, new ones do not. Reverting to
f83da34db60b93c5f2ec75ed701061fc2dd0675b fixes the issue, and digits starts normally.
Best guess is that it has something to do with these changes (removing try block, replacing with import caffe_pb2)
-                        try:
-                            import caffe_pb2
-                        except ImportError:
-                            # See issue #32
-                            from caffe.proto import caffe_pb2
                         import numpy as np
+                        import caffe_pb2

throughout the code. Again, python isn't my thing, but considering this error only started showing up yesterday, and picking a revision prior to #307 fixes it, it seems like a likely candidate.

If anyone else can confirm this, I'll revert the change immediately.
Thanks for the bug report @davidsmd. The changes in digits.config.caffe_option should have kept this from happening.
Do you mind adding these lines before this line in digits/model/tasks/caffe_train.py for me?
import pprint, sys
pprint.pprint(sys.path)

I want to double-check that the $CAFFE_HOME/python/caffe/proto path is getting added to sys.path.

Ah-ha. Got it. My bad I think. Didn't realize there was a change to digits/config/caffe_option.py ... I'm loading mine from a template to add:
import matplotlib
matplotlib.use("agg")

in order to avoid GTK errors when running headless...
Sorry about that.

Sorry about that.

No problem, glad you sorted it out.

I'm loading mine from a template to add:

So you need #239?

Yes, #239 would be helpful.

Hi @davidsmd, if you can reproduce the GTK error consistently, would you mind doing this on your machine:
$ python
Python 2.7.3 (default, Feb 27 2014, 19:58:35)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import matplotlib
>>> matplotlib.get_backend()
u'TkAgg'

I am curious to see what backend matplotlib is using by default on your machine. Thanks!

@gheinrich
Python 2.7.6 (default, Jun 22 2015, 17:58:13)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import matplotlib
>>> matplotlib.get_backend()
u'GTK3Agg'
>>>


Use react-markdown to map HTML elements to MUI components

Looking at 0.9.0 in embedded in Chronosphere, the markdown panel looks OK, but could look better...

Links shouldn't be green
Table border is too dark
Probably nicer to have border-radius on table (like other containers in the app)
Code background is too dark

At this point, just want to be able to take some of the raw HTML elements (e.g. <a> tag) and map them to Material UI components (e.g. <Link>) because the Material UI components know how to style themselves correctly.
So, with this PR, I'm changing the markdown-to-HTML libraries. Specifically, using react-markdown instead of marked so that we can use a feature of react-markdown that allows us to map HTML elements to MUI components.
This should make the panel look better in apps that are using Material UI.
Light mode

Dark mode

Signed-off-by: Christine Donovan<EMAIL_ADDRESS>

FYI this PR will stay open for a bit as I go focus on some cloud-ui things, but appreciate the feedback so far.
Will come back after to address PR feedback!

any chance to know why you closed this PR. These changes were interesting :)


chore(deps): update ci dependencies

This PR contains the following updates:

Package
Type
Update
Change

common-dev-assets

digest
cf15dc9 -> ede5101

github.com/gruntwork-io/terratest
require
patch
v0.46.13 -> v0.46.14

github.com/terraform-ibm-modules/ibmcloud-terratest-wrapper
require
minor
v1.30.8 -> v1.31.6

go (source)
toolchain
minor
1.21.3 -> 1.22.2

terraform-ibm-modules/common-pipeline-assets
action
minor
v1.21.9 -> v1.22.0

Release Notes

gruntwork-io/terratest (github.com/gruntwork-io/terratest)
v0.46.14
Compare Source
Modules affected

opa

Description

Updated go-getter to 1.7.4

Related links

https://github.com/gruntwork-io/terratest/pull/1404

terraform-ibm-modules/ibmcloud-terratest-wrapper (github.com/terraform-ibm-modules/ibmcloud-terratest-wrapper)
v1.31.6
Compare Source
Bug Fixes

deps: update gomod (#802) (efcfa2f)

v1.31.5
Compare Source
Bug Fixes

ensure all stack members are listed in the stack order (#800) (04e23c7)

v1.31.4
Compare Source
Bug Fixes

enhance error message (#799) (0409a07)

v1.31.3
Compare Source
Bug Fixes

update log messages (#798) (c1f41db)

v1.31.2
Compare Source
Bug Fixes

minor projects fixes (#797) (cfe0774)

v1.31.1
Compare Source
Bug Fixes

update stack input handling (#796) (99dbf4e)

v1.31.0
Compare Source
Features

alpha projects/stacks support (#793) (d0a6052)

terraform-ibm-modules/common-pipeline-assets (terraform-ibm-modules/common-pipeline-assets)
v1.22.0
Compare Source
Features

add pages release action (#681) (f06fe28)

Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
👻 Immortal: This PR will be recreated if closed unmerged. Get config help if that's undesired.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

/run pipeline

:tada: This PR is included in version 7.18.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


I have a great idea for the repository

Hi @JonathanLemke
Kindly reopen the issue with a proper description.
Regards,


transaction.Rollback() is not working in EF Core

Hi,
I am working EF core code, I tried as below, any exception occur in _saveDocumentToDb or SaveFile catch block is executing but Rollback not happen.
try
{
await _saveDocumentToDb(entityId, file, docRequest, guid, transaction);
await EntityDocumentsStorageHelper.SaveFile(file, guid);

                transaction.Commit();
            }
            catch (Exception e)
            {
                // TODO: BUG: This rollback does not rollback the database operation.
                transaction.Rollback();
                throw;
            }

If you are seeing an exception, include the full exceptions details (message and stack trace).
Exception message:
Stack trace:

Steps to reproduce
Include a complete code listing (or project/solution) that we can run to reproduce the issue.
Partial code listings, or multiple fragments of code, will slow down our response or cause us to push the issue back to you to provide code to reproduce the issue.
Console.WriteLine("Hello World!");

Further technical details
EF Core version: (found in project.csproj or packages.config)
Database Provider: (e.g. Microsoft.EntityFrameworkCore.SqlServer)
Operating system:
IDE: (e.g. Visual Studio 2017 15.4)

@divakarganta There isn't enough information here for us to be able to reproduce what you are seeing. Please post a small, runnable project/solution or complete code listing that demonstrates the behavior you are seeing.

EF Team Triage: Closing this issue as the requested additional details have not been provided and we have been unable to reproduce it.
BTW this is a canned response and may have info or details that do not directly apply to this particular issue. While we'd like to spend the time to uniquely address every incoming issue, we get a lot traffic on the EF projects and that is not practical. To ensure we maximize the time we have to work on fixing bugs, implementing new features, etc. we use canned responses for common triage decisions.


Add support for numbers as map keys

Codecov Report

Merging #84 into master will increase coverage by <.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master      #84      +/-   ##
==========================================
+ Coverage   98.33%   98.33%   +<.01%
==========================================
  Files          16       16
  Lines         959      962       +3
  Branches      194      195       +1
==========================================
+ Hits          943      946       +3
  Misses         16       16

Impacted Files
Coverage Δ

src/Decoder.ts
99.13% <100%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update efff31d...38e2c91. Read the comment docs.


atuin: fix build on darwin

xvfb-run is not available on darwin
Description of changes

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[x] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Hi there, thank you so much for picking up on this so quickly... however I'm still unable to build the latest atuin on Darwin, and I'm a little confused as to why that's the case. It seems xvfb-run.meta.available is still evaluating to true despite the fact that it's definitely not available on darwin.

I've hard-coded the value for now so this doesn't block me, but any insight here is very much appreciated. Thanks for all your hard work :)

@jrolfs  Can you try this change?
--- a/pkgs/tools/misc/atuin/default.nix
+++ b/pkgs/tools/misc/atuin/default.nix
@@ -6,7 +6,7 @@
 , libiconv
 , Security
 , SystemConfiguration
-, xvfb-run
+, withXvfb-run ? lib.meta.availableOn stdenv.hostPlatform xvfb-run, xvfb-run
 , nixosTests
 }:

@@ -34,11 +34,11 @@ rustPlatform.buildRustPackage rec {
       --zsh <($out/bin/atuin gen-completions -s zsh)
   '';

-  nativeCheckInputs = lib.optionals xvfb-run.meta.available [
+  nativeCheckInputs = lib.optionals withXvfb-run [
     xvfb-run
   ];

-  checkPhase = lib.optionalString xvfb-run.meta.available ''
+  checkPhase = lib.optionalString withXvfb-run ''
     runHook preCheck
     xvfb-run cargo test
     runHook postCheck

@0x4A6F works great. Thank you so much!


Update free-courses-en.md

adding A cloud security book link which is online for free

linter is picky about spaces

I could be wrong, but I'm not convinced this is an authorized distribution. :thinking:
This book is by Albert Anthony and published by Packt.
https://www.packtpub.com/product/mastering-aws-security/9781788293723
I don't see an affiliation between Albert Anthony and RMS Technology Consulting.

authorized yes, but free with subscription != free


Add FLU-Serve validation profiles to non-CDC subscription

We have received eICR structure definitions and data sample for FLU-Serv.
These will be added to FHIR Service and sample data tested for validation.

Resource profile (structure definition) was added for eICR document bundle. The eICR sample data was used to test validation via $validate operation, but it failed on bundle type (message vs document).

Currently working with Product Group on validation issue with nested eICR document bundles.


Cosmos Db serialization using JsonPropertyNames

Hello,
I am facing an issue with serialization during saving the data. One of the properties ("MyClass") in documents has pascal case naming (as below) whilst all the other properties have camel case naming. I want to retain the naming as it is so as not to break the downstream reporting. I have used " CosmosPropertyNamingPolicy.CamelCase" for my client instance as I am unable to save without it. Now the pascal case name of "MyClass" gets overwritten with "myClass". Please advise.
Public class MyModel{
[JsonPropertyName("MyClass")]
public MyClass MyClass{ get; set; }

[JsonPropertyName("transactionId")]
public string TransactionId { get; set; }

[JsonPropertyName("id")]
public string Id { get; set; }
}

I have the below code to replace existing records which I fetch from the db based on some criteria.
private static async Task ReplaceMultipleAsync(List<Record> records)
        {
            List<Task> concurrentTasks = new List<Task>();
            foreach (var record in records)
            {
                concurrentTasks.Add(_container.ReplaceItemAsync(record, record.Id, new PartitionKey(record.UserId)));
            }

            await Task.WhenAll(concurrentTasks);
        }

With this update, my existing documents gets overwritten and MyClass (pascal case) property becomes myclass (Camel case). I don't want this overwrite as we have a lot of records in our db and they are used by some reporting which would break if this is changed.
Creating cosmos client
private static void InitializeCosmosClientInstance(IConfigurationSection configurationSection)
        {
            IConfiguration configuration = new ConfigurationBuilder()
                                                .AddJsonFile("appsettings.json", true, true)
                                                .Build();
            //var playerSection = configuration.GetSection(nameof(Player));

            _databaseName = configurationSection["DatabaseName"];
            _containerName = configurationSection["ContainerName"];
            var account = configurationSection["Account"];
            var key = configurationSection["Key"];
            var options = new CosmosClientOptions()
            {
                AllowBulkExecution = true,
                SerializerOptions = new CosmosSerializationOptions()
                {
                    PropertyNamingPolicy = CosmosPropertyNamingPolicy.CamelCase,
                }
            };
            _cosmosClient = new CosmosClient(account, key, options);
        }

Getting container instance

 _database = _cosmosClient.GetDatabase(_databaseName);
 _container = _database.GetContainer(_containerName);

Environment summary
SDK Version: 3.20.1
OS Version: Windows
.Net Core: 3.1
Posted here too: https://stackoverflow.com/questions/68578008/force-use-jsonpropertyname-while-saving-into-cosmosdb

What is the error you get when you try to save it without CosmosPropertyNamingPolicy? Cosmos DB only requires the "id" (which must be lower case) and the partition key property. If you have both it should matter if what the other properties are.
Under the covers the SDK is just calling newtonsoft serializer.  https://github.com/Azure/azure-cosmos-dotnet-v3/blob/6a1e27a7ec3d64b2c52d3a95136ee6daca245b6f/Microsoft.Azure.Cosmos/src/Serializer/CosmosJsonDotNetSerializer.cs#L48
JsonSerializerSettings jsonSerializerSettings = new JsonSerializerSettings()
            {
                NullValueHandling = cosmosSerializerOptions.IgnoreNullValues ? NullValueHandling.Ignore : NullValueHandling.Include,
                Formatting = cosmosSerializerOptions.Indented ? Formatting.Indented : Formatting.None,
                ContractResolver = cosmosSerializerOptions.PropertyNamingPolicy == CosmosPropertyNamingPolicy.CamelCase
                    ? new CamelCasePropertyNamesContractResolver()
                    : null
            };

The error I am getting without specifying serializer options is
{"Response status code does not indicate success: BadRequest (400); Substatus: 0; ActivityId: ; Reason: ();"}
Can anyone please suggest a way to serialize as per JsonPropertyNames and save the records/documents into cosmos db collection?

@septst JsonPropertyName is coming from Newtonsoft.Json or System.Text.Json? If it's the latter, you are using the wrong serialization engine. V3 SDK works with Newtonsoft.Json.

@septst JsonPropertyName is coming from Newtonsoft.Json or System.Text.Json? If it's the latter, you are using the wrong serialization engine. V3 SDK works with Newtonsoft.Json. Could you also paste the entire exception details / stack trace?

Thanks for saving me for the second time. Changing the attributes from [JsonPropertyName("***")] that uses System.Text.Json to [JsonProperty("MyClass")] to Newtonsoft.Json did the work. Much appreciated.


Replaced code deprecated in iOS 9.0

CFURLCreateStringsByAddingPercentEscapes() function was deprecated in iOS 9.0, and Xcode was throwing up a warning each time I built my app. I updated the code in escape(string: String) -> String function to use NSString().stringByAddingPercentEncodingWithAllowedCharacters() instead. Autocomplete still works correctly and the warning has disappeared.
Massive thanks to watsonbox for publishing this code originally!

Well, that's weird. Why did the Travis-CI fail?


datepicker in Safari browser

Hi! I implemented a 'date' field and it works fine with Firefox (v65)  and also with (IE) but not with the browser Safari (v12). Any idea to fix this problem
{
"schema": {
"field": {
"type": "string",
"title": "A field"
}
},
"form": [
{
"key": "field",
"type": "date"
},
{
"type": "submit",
"title": "Submit"
}
]
}
example-A.html.zip

Hi - could you provide a working JSFiddle so I can help ?

Hi! Here it is.
https://jsfiddle.net/PeterGruber/8kwzf5p6/22/
When you use Firefox the datepicker pops up in the in the ‘Date’ field. Don’t with Safari browser.
Thanks for any help!

Safari does not support this input type unfortunately : https://caniuse.com/#search=input-date