HADOOP-19315. Bumped avro to 1.11.4 and implemented new setter/getter adjustments

Description of PR
Bumped avro version from 1.9.2 to 1.11.4 and adjusted some parts where still member variables instead of getter/setter methods where used (avro made member variables private).
How was this patch tested?
run mvn clean install javadoc:javadoc checkstyle:checkstyle
For code changes:

[x] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
[ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[ ] If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

I was confused looking at this as the classes were already using the setters; turns out I was on an internal version where somebody already did something similar. But their change actually modified more files than this. Is it a complete set?
Also: check that LICENSE-binary file.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
6m 43s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
xmllint
0m  0s

xmllint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
14m 18s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
18m 39s

trunk passed

-1 :x:
compile
0m 18s
/branch-compile-root-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt
root in trunk failed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.

+1 :green_heart:
compile
8m 12s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
checkstyle
2m  9s

trunk passed

+1 :green_heart:
mvnsite
2m 34s

trunk passed

+1 :green_heart:
javadoc
2m 16s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
1m 56s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+0 :ok:
spotbugs
0m 32s

branch/hadoop-project no spotbugs output file (spotbugsXml.xml)

+1 :green_heart:
shadedclient
20m 17s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 25s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
1m 22s

the patch passed

-1 :x:
compile
0m 13s
/patch-compile-root-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt
root in the patch failed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.

-1 :x:
javac
0m 13s
/patch-compile-root-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt
root in the patch failed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.

+1 :green_heart:
compile
8m  3s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
javac
8m  3s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
2m  5s
/results-checkstyle-root.txt
root: The patch generated 1 new + 89 unchanged - 0 fixed = 90 total (was 89)

+1 :green_heart:
mvnsite
2m 34s

the patch passed

+1 :green_heart:
javadoc
2m 11s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
2m  2s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+0 :ok:
spotbugs
0m 26s

hadoop-project has no data from spotbugs

+1 :green_heart:
shadedclient
21m 28s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
0m 18s

hadoop-project in the patch passed.

-1 :x:
unit
16m 30s
/patch-unit-hadoop-common-project_hadoop-common.txt
hadoop-common in the patch passed.

+1 :green_heart:
unit
6m 25s

hadoop-mapreduce-client-core in the patch passed.

+1 :green_heart:
unit
0m 26s

hadoop-rumen in the patch passed.

+1 :green_heart:
asflicense
0m 33s

The patch does not generate ASF License warnings.

152m 36s

Reason
Tests

Failed junit tests
hadoop.io.TestText

hadoop.fs.TestPath

Subsystem
Report/Notes

Docker
ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7128/2/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/7128

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint

uname
Linux 0d0abd0ff4b8 5.15.0-117-generic #127-Ubuntu SMP Fri Jul 5 20:13:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 55fd86ced77d3f28441be7ac2171871d4cac3c6f

Default Java
Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7128/2/testReport/

Max. process+thread count
1559 (vs. ulimit of 5500)

modules
C: hadoop-project hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-tools/hadoop-rumen U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7128/2/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 53s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
xmllint
0m  0s

xmllint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
14m 36s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
35m 52s

trunk passed

-1 :x:
compile
0m 26s
/branch-compile-root-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt
root in trunk failed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.

+1 :green_heart:
compile
18m  2s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
checkstyle
4m 50s

trunk passed

+1 :green_heart:
mvnsite
4m  5s

trunk passed

+1 :green_heart:
javadoc
3m 23s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
2m 49s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+0 :ok:
spotbugs
0m 42s

branch/hadoop-project no spotbugs output file (spotbugsXml.xml)

+1 :green_heart:
shadedclient
41m  3s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 35s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
2m 57s

the patch passed

-1 :x:
compile
0m 22s
/patch-compile-root-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt
root in the patch failed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.

-1 :x:
javac
0m 22s
/patch-compile-root-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt
root in the patch failed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.

+1 :green_heart:
compile
19m 43s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
javac
19m 43s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
4m 43s
/results-checkstyle-root.txt
root: The patch generated 1 new + 89 unchanged - 0 fixed = 90 total (was 89)

+1 :green_heart:
mvnsite
4m  7s

the patch passed

+1 :green_heart:
javadoc
3m 17s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
2m 50s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+0 :ok:
spotbugs
0m 36s

hadoop-project has no data from spotbugs

+1 :green_heart:
shadedclient
39m 54s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
0m 35s

hadoop-project in the patch passed.

-1 :x:
unit
19m 28s
/patch-unit-hadoop-common-project_hadoop-common.txt
hadoop-common in the patch passed.

+1 :green_heart:
unit
8m 27s

hadoop-mapreduce-client-core in the patch passed.

+1 :green_heart:
unit
0m 46s

hadoop-rumen in the patch passed.

+1 :green_heart:
asflicense
1m  3s

The patch does not generate ASF License warnings.

252m 16s

Reason
Tests

Failed junit tests
hadoop.fs.TestPath

hadoop.io.TestText

Subsystem
Report/Notes

Docker
ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7128/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/7128

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint

uname
Linux 506397af691e 5.15.0-119-generic #129-Ubuntu SMP Fri Aug 2 19:25:20 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / a0a429c66b2c3d742c7bfd79397edda2acc47f2d

Default Java
Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7128/1/testReport/

Max. process+thread count
1261 (vs. ulimit of 5500)

modules
C: hadoop-project hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-tools/hadoop-rumen U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7128/1/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

Avro test failures.
Forbidden class org.apache.hadoop.fs.Path! This class is not trusted to be included in Avro schema using java-class. Please set org.apache.avro.SERIALIZABLE_PACKAGES system property with the packages you trust.

looks like there's now a sysprop which needs to be set. There's some implications there for testing sysprop needs to be set in execution or test setup.
More importantly: mapreduce uses avro files in production. All uses of that code needs to be reviewed to make sure it isn't using standard object ser/deser as otherwise every runtime deployment of the services will need to set up a new sysprop -which is going to be an incompatible change

...looking at our internal changes; some files in rumen module needed fixing too. nothing significant

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 22s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
xmllint
0m  0s

xmllint was not available.

+0 :ok:
shelldocs
0m  0s

Shelldocs was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
14m 24s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
20m 11s

trunk passed

-1 :x:
compile
0m 15s
/branch-compile-root-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt
root in trunk failed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.

+1 :green_heart:
compile
9m 45s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
checkstyle
2m 37s

trunk passed

+1 :green_heart:
mvnsite
18m  6s

trunk passed

+1 :green_heart:
javadoc
5m 31s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
5m 15s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+0 :ok:
spotbugs
0m 15s

branch/hadoop-project no spotbugs output file (spotbugsXml.xml)

+1 :green_heart:
shadedclient
37m  3s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 25s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
18m 58s

the patch passed

-1 :x:
compile
0m 18s
/patch-compile-root-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt
root in the patch failed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.

-1 :x:
javac
0m 18s
/patch-compile-root-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt
root in the patch failed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.

+1 :green_heart:
compile
8m  4s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
javac
8m  4s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
2m  6s
/results-checkstyle-root.txt
root: The patch generated 1 new + 89 unchanged - 0 fixed = 90 total (was 89)

+1 :green_heart:
mvnsite
8m 43s

the patch passed

+1 :green_heart:
shellcheck
0m  0s

No new issues.

+1 :green_heart:
javadoc
5m 26s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
5m 12s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+0 :ok:
spotbugs
0m 14s

hadoop-project has no data from spotbugs

+1 :green_heart:
shadedclient
37m 17s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

-1 :x:
unit
665m 17s
/patch-unit-root.txt
root in the patch passed.

+1 :green_heart:
asflicense
1m  3s

The patch does not generate ASF License warnings.

876m  6s

Reason
Tests

Failed junit tests
hadoop.io.TestText

hadoop.fs.TestPath

Subsystem
Report/Notes

Docker
ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7128/3/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/7128

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint shellcheck shelldocs

uname
Linux 0ca594da4b8f 5.15.0-117-generic #127-Ubuntu SMP Fri Jul 5 20:13:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 3e859dd8037d749a8208c7c56c158feca4d80341

Default Java
Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7128/3/testReport/

Max. process+thread count
4346 (vs. ulimit of 5500)

modules
C: hadoop-project hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-tools/hadoop-rumen . U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7128/3/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2 shellcheck=0.7.0

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@steveloughran Can you please specify what the ToDo is here on my side? Seems like this sysprop needs to be set either way right? What do you mean by checking the standard serializer methods in mapreduce? I'm missing the full picture here.

@steveloughran I can see why Avro need this packages property but the implementation makes things difficult for Hadoop. One option would be for the Hadoop code to set the property itself. It could reset the property, taking the existing value and appending the necessary Hadoop packages.
If the property is empty, we would need to add the default setting too if we were to add some of our own. The default applied is:
https://github.com/apache/avro/blob/main/lang/java/avro/src/main/java/org/apache/avro/specific/SpecificDatumReader.java#L41

@dom93dd two issues

the test is clearly broken: properties need to be set
avro is used in bits of the code, and that stuff needs to work.

@pjfanning turns out the issue is AVRO-3985. It's not a recently discovered CVE as such, just that they've added the ability to lock down which packages are allowed to contain classes instantiated through reflection: this new property is the core aspect of the feature.

@steveloughran
For the test classes it is an easy fix. Just added a change.
Regarding the productive code, as @pjfanning already started a discussion about this property, we should have a common agreement where this property should be set.
This looks like a method where serialisation is being done. But I guess it wouldn't be an option to set a system property within this method or even the constructor.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 51s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
xmllint
0m  0s

xmllint was not available.

+0 :ok:
shelldocs
0m  0s

Shelldocs was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 3 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
14m 59s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
36m 43s

trunk passed

-1 :x:
compile
0m 27s
/branch-compile-root-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt
root in trunk failed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.

+1 :green_heart:
compile
18m  6s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
checkstyle
5m  0s

trunk passed

+1 :green_heart:
mvnsite
28m 42s

trunk passed

+1 :green_heart:
javadoc
10m 40s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
7m 53s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+0 :ok:
spotbugs
0m 19s

branch/hadoop-project no spotbugs output file (spotbugsXml.xml)

+1 :green_heart:
shadedclient
71m 24s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 35s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
36m 35s

the patch passed

-1 :x:
compile
0m 26s
/patch-compile-root-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt
root in the patch failed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.

-1 :x:
javac
0m 26s
/patch-compile-root-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt
root in the patch failed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.

+1 :green_heart:
compile
18m 18s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
javac
18m 18s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
4m 45s
/results-checkstyle-root.txt
root: The patch generated 1 new + 144 unchanged - 0 fixed = 145 total (was 144)

+1 :green_heart:
mvnsite
20m 13s

the patch passed

+1 :green_heart:
shellcheck
0m  0s

No new issues.

+1 :green_heart:
javadoc
10m 29s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
7m 50s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+0 :ok:
spotbugs
0m 19s

hadoop-project has no data from spotbugs

+1 :green_heart:
shadedclient
72m 58s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

-1 :x:
unit
812m 33s
/patch-unit-root.txt
root in the patch passed.

+1 :green_heart:
asflicense
1m 37s

The patch does not generate ASF License warnings.

1197m  6s

Reason
Tests

Failed junit tests
hadoop.fs.TestPath

hadoop.io.TestText

hadoop.yarn.server.timelineservice.security.TestTimelineAuthFilterForV2

Subsystem
Report/Notes

Docker
ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7128/4/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/7128

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint shellcheck shelldocs

uname
Linux 116d764a7d3a 5.15.0-119-generic #129-Ubuntu SMP Fri Aug 2 19:25:20 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / dccaddf030af3a29963208fb84dfb444536878d6

Default Java
Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7128/4/testReport/

Max. process+thread count
3087 (vs. ulimit of 5500)

modules
C: hadoop-project hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-tools/hadoop-rumen . U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7128/4/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2 shellcheck=0.7.0

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

Added new changes - the two serialization tests are successful locally.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 21s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
xmllint
0m  0s

xmllint was not available.

+0 :ok:
shelldocs
0m  0s

Shelldocs was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 4 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
14m 28s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
18m 50s

trunk passed

-1 :x:
compile
0m 17s
/branch-compile-root-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt
root in trunk failed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.

+1 :green_heart:
compile
8m  7s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
checkstyle
2m 12s

trunk passed

+1 :green_heart:
mvnsite
15m 58s

trunk passed

+1 :green_heart:
javadoc
5m 28s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
5m  8s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+0 :ok:
spotbugs
0m 14s

branch/hadoop-project no spotbugs output file (spotbugsXml.xml)

+1 :green_heart:
shadedclient
37m  4s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 25s

Maven dependency ordering for patch

-1 :x:
mvninstall
0m 31s
/patch-mvninstall-hadoop-common-project_hadoop-common.txt
hadoop-common in the patch failed.

-1 :x:
mvninstall
0m 41s
/patch-mvninstall-root.txt
root in the patch failed.

-1 :x:
compile
0m 12s
/patch-compile-root-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt
root in the patch failed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.

-1 :x:
javac
0m 12s
/patch-compile-root-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt
root in the patch failed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.

-1 :x:
compile
0m 41s
/patch-compile-root-jdkPrivateBuild-1.8.0_422-8u422-b05-1~20.04-b05.txt
root in the patch failed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05.

-1 :x:
javac
0m 41s
/patch-compile-root-jdkPrivateBuild-1.8.0_422-8u422-b05-1~20.04-b05.txt
root in the patch failed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05.

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
1m 49s
/results-checkstyle-root.txt
root: The patch generated 5 new + 144 unchanged - 0 fixed = 149 total (was 144)

-1 :x:
mvnsite
7m 50s
/patch-mvnsite-root.txt
root in the patch failed.

+1 :green_heart:
shellcheck
0m  0s

No new issues.

+1 :green_heart:
javadoc
5m 19s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
5m  8s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+0 :ok:
spotbugs
0m 15s

hadoop-project has no data from spotbugs

-1 :x:
spotbugs
0m 33s
/patch-spotbugs-hadoop-common-project_hadoop-common.txt
hadoop-common in the patch failed.

-1 :x:
spotbugs
1m  2s
/patch-spotbugs-root.txt
root in the patch failed.

-1 :x:
shadedclient
1m 55s

patch has errors when building and testing our client artifacts.

_ Other Tests _

-1 :x:
unit
4m 56s
/patch-unit-root.txt
root in the patch failed.

+1 :green_heart:
asflicense
0m 38s

The patch does not generate ASF License warnings.

147m 48s

Subsystem
Report/Notes

Docker
ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7128/5/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/7128

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint shellcheck shelldocs

uname
Linux 28d71436e062 5.15.0-117-generic #127-Ubuntu SMP Fri Jul 5 20:13:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 2927b6b7742a01427581875960db472207545559

Default Java
Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7128/5/testReport/

Max. process+thread count
552 (vs. ulimit of 5500)

modules
C: hadoop-project hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-tools/hadoop-rumen . U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7128/5/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2 shellcheck=0.7.0

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] /home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-7128/ubuntu-focal/src/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestPath.java:[409,24] cannot find symbol
  symbol:   variable Constants
  location: class org.apache.hadoop.fs.TestPath

[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] /home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-7128/ubuntu-focal/src/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestPath.java:[409,24] cannot find symbol
  symbol:   variable Constants
  location: class org.apache.hadoop.fs.TestPath

Typical friday - forgot to stage the changes for TestPath class.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
6m 41s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
xmllint
0m  0s

xmllint was not available.

+0 :ok:
shelldocs
0m  0s

Shelldocs was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 4 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
18m 19s

Maven dependency ordering for branch

-1 :x:
mvninstall
18m  5s
/branch-mvninstall-root.txt
root in trunk failed.

+1 :green_heart:
compile
9m  3s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
compile
8m  3s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
checkstyle
2m 10s

trunk passed

+1 :green_heart:
mvnsite
14m 42s

trunk passed

+1 :green_heart:
javadoc
5m 25s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
5m  2s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+0 :ok:
spotbugs
0m 14s

branch/hadoop-project no spotbugs output file (spotbugsXml.xml)

+1 :green_heart:
shadedclient
36m 56s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 23s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
18m 59s

the patch passed

+1 :green_heart:
compile
8m 39s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javac
8m 39s

the patch passed

+1 :green_heart:
compile
8m  8s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
javac
8m  8s

the patch passed

+1 :green_heart:
blanks
0m  1s

The patch has no blanks issues.

-0 :warning:
checkstyle
2m  5s
/results-checkstyle-root.txt
root: The patch generated 5 new + 144 unchanged - 0 fixed = 149 total (was 144)

+1 :green_heart:
mvnsite
8m 49s

the patch passed

+1 :green_heart:
shellcheck
0m  0s

No new issues.

+1 :green_heart:
javadoc
5m 25s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
5m  9s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+0 :ok:
spotbugs
0m 15s

hadoop-project has no data from spotbugs

+1 :green_heart:
shadedclient
37m 24s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

-1 :x:
unit
669m  2s
/patch-unit-root.txt
root in the patch passed.

+1 :green_heart:
asflicense
1m  4s

The patch does not generate ASF License warnings.

899m 18s

Reason
Tests

Failed junit tests
hadoop.hdfs.server.federation.router.TestRouterClientRejectOverload

Subsystem
Report/Notes

Docker
ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7128/6/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/7128

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint shellcheck shelldocs

uname
Linux 02e88c2056f0 5.15.0-124-generic #134-Ubuntu SMP Fri Sep 27 20:20:17 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 1173fe2de88c8927f92fee55a198bff1de19846a

Default Java
Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7128/6/testReport/

Max. process+thread count
4620 (vs. ulimit of 5500)

modules
C: hadoop-project hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-tools/hadoop-rumen . U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7128/6/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2 shellcheck=0.7.0

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

close. Four little checkstyles to fix, all minor
./hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/constants/ConfigConstants.java:21:public interface ConfigConstants {: interfaces should describe a type and hence have methods. [InterfaceIsType]
./hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/constants/ConfigConstants.java:27:   public static final String CONFIG_AVRO_SERIALIZABLE_PACKAGES = "org.apache.avro.SERIALIZABLE_PACKAGES";: 'member def modifier' has incorrect indentation level 3, expected level should be 2. [Indentation]
./hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/constants/ConfigConstants.java:27:   public static final String CONFIG_AVRO_SERIALIZABLE_PACKAGES = "org.apache.avro.SERIALIZABLE_PACKAGES";: Line is longer than 100 characters (found 106). [LineLength]
./hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/constants/ConfigConstants.java:27:   public static final String CONFIG_AVRO_SERIALIZABLE_PACKAGES = "org.apache.avro.SERIALIZABLE_PACKAGES";:4: Redundant 'public' modifier. [RedundantModifier]
./hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobQueueChangeEvent.java:23:import org.apache.avro.util.Utf8;:8: Unused import - org.apache.avro.util.Utf8. [UnusedImports]

+1 pending these being fixed

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
17m 48s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
detsecrets
0m  1s

detect-secrets was not available.

+0 :ok:
xmllint
0m  1s

xmllint was not available.

+0 :ok:
shelldocs
0m  1s

Shelldocs was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 4 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
14m 44s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
39m 54s

trunk passed

+1 :green_heart:
compile
20m 11s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
compile
18m 32s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
checkstyle
4m 50s

trunk passed

+1 :green_heart:
mvnsite
26m 49s

trunk passed

+1 :green_heart:
javadoc
10m 35s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
7m 57s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+0 :ok:
spotbugs
0m 19s

branch/hadoop-project no spotbugs output file (spotbugsXml.xml)

+1 :green_heart:
shadedclient
72m  2s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 35s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
37m  4s

the patch passed

+1 :green_heart:
compile
19m 20s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javac
19m 20s

the patch passed

+1 :green_heart:
compile
22m 49s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
javac
22m 49s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
5m 40s
/results-checkstyle-root.txt
root: The patch generated 3 new + 144 unchanged - 0 fixed = 147 total (was 144)

+1 :green_heart:
mvnsite
19m 59s

the patch passed

+1 :green_heart:
shellcheck
0m  1s

No new issues.

+1 :green_heart:
javadoc
10m 26s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
8m  1s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+0 :ok:
spotbugs
0m 20s

hadoop-project has no data from spotbugs

+1 :green_heart:
shadedclient
71m 46s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

-1 :x:
unit
821m 57s
/patch-unit-root.txt
root in the patch passed.

+1 :green_heart:
asflicense
1m 38s

The patch does not generate ASF License warnings.

1268m 45s

Reason
Tests

Failed junit tests
hadoop.hdfs.server.federation.router.TestRouterRpc

hadoop.hdfs.TestRollingUpgrade

Subsystem
Report/Notes

Docker
ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7128/7/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/7128

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint shellcheck shelldocs

uname
Linux e62e9f5102d4 5.15.0-119-generic #129-Ubuntu SMP Fri Aug 2 19:25:20 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / ab0417f8e5ff4cf108e2cb9679f8f672b74a0eea

Default Java
Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7128/7/testReport/

Max. process+thread count
3106 (vs. ulimit of 5500)

modules
C: hadoop-project hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-tools/hadoop-rumen . U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7128/7/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2 shellcheck=0.7.0

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

At least my local mvn checkstyle was clean.. Let's double check with jenkins.

Here's my draft Commit Message
We need to highlight it is not backwards compatibility, and include the CVEs to make log scanning find them.
Does it seem good?

HADOOP-19315. Upgrade Apache Avro to 1.11.4

All field access is now via setter/getter methods
To use Avro to marshal Serializable objects,
the packages they are in must be declared in the system property
"org.apache.avro.SERIALIZABLE_PACKAGES"

This is required to address

CVE-2024-47561
CVE-2023-39410

This change is not backwards compatible.
Contributed by Dominik Diedrich

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 50s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
xmllint
0m  0s

xmllint was not available.

+0 :ok:
shelldocs
0m  0s

Shelldocs was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 4 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
15m 17s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
36m 54s

trunk passed

+1 :green_heart:
compile
20m  1s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
compile
19m 25s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
checkstyle
4m 44s

trunk passed

+1 :green_heart:
mvnsite
26m 52s

trunk passed

+1 :green_heart:
javadoc
10m 31s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
8m  0s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+0 :ok:
spotbugs
0m 20s

branch/hadoop-project no spotbugs output file (spotbugsXml.xml)

+1 :green_heart:
shadedclient
71m 30s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 35s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
39m 56s

the patch passed

+1 :green_heart:
compile
19m 27s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javac
19m 27s

the patch passed

+1 :green_heart:
compile
18m 23s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
javac
18m 23s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
4m 47s

the patch passed

+1 :green_heart:
mvnsite
18m 37s

the patch passed

+1 :green_heart:
shellcheck
0m  0s

No new issues.

+1 :green_heart:
javadoc
10m 22s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
7m 59s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+0 :ok:
spotbugs
0m 20s

hadoop-project has no data from spotbugs

+1 :green_heart:
shadedclient
72m  0s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

-1 :x:
unit
837m 32s
/patch-unit-root.txt
root in the patch passed.

+1 :green_heart:
asflicense
1m 39s

The patch does not generate ASF License warnings.

1261m 34s

Reason
Tests

Failed junit tests
hadoop.hdfs.TestRollingUpgrade

hadoop.yarn.server.resourcemanager.reservation.TestCapacityOverTimePolicy

Subsystem
Report/Notes

Docker
ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7128/8/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/7128

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint shellcheck shelldocs

uname
Linux a62f36e2c1ba 5.15.0-119-generic #129-Ubuntu SMP Fri Aug 2 19:25:20 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 1d20cf146d563fa28a8d89d74dfb2d792c4d55a1

Default Java
Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7128/8/testReport/

Max. process+thread count
3089 (vs. ulimit of 5500)

modules
C: hadoop-project hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-tools/hadoop-rumen . U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7128/8/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2 shellcheck=0.7.0

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

Here's my draft Commit Message
We need to highlight it is not backwards compatibility, and include the CVEs to make log scanning find them.
Does it seem good?
HADOOP-19315. Upgrade Apache Avro to 1.11.4

All field access is now via setter/getter methods
To use Avro to marshal Serializable objects,
the packages they are in must be declared in the system property
"org.apache.avro.SERIALIZABLE_PACKAGES"

This is required to address

CVE-2024-47561
CVE-2023-39410

This change is not backwards compatible.
Contributed by Dominik Diedrich

Looks good to me. I presume that this can't be merged to Hadoop 3.4.2 due to the compatibility issues.
Should this PR update the shell scripts to set the org.apache.avro.SERIALIZABLE_PACKAGES system property?

@pjfanning

I presume that this can't be merged to Hadoop 3.4.2 due to the compatibility issues.

It's a bit nuanced w.r.t third party libraries, but if it blocks people upgrading then it keeps people on older releases, which is also a pain.
Something to raise on the common-dev list.
Maybe we should just go straight to a 3.5.0 release and make 3.4.x the maintenance branch.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 23s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
xmllint
0m  0s

xmllint was not available.

+0 :ok:
shelldocs
0m  0s

Shelldocs was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 5 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
13m 57s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
18m 54s

trunk passed

+1 :green_heart:
compile
8m 52s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
compile
8m  6s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
checkstyle
2m  6s

trunk passed

+1 :green_heart:
mvnsite
17m 26s

trunk passed

+1 :green_heart:
javadoc
5m 26s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
5m  6s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+0 :ok:
spotbugs
0m 15s

branch/hadoop-project no spotbugs output file (spotbugsXml.xml)

+1 :green_heart:
shadedclient
37m  2s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 25s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
19m 17s

the patch passed

+1 :green_heart:
compile
8m 40s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javac
8m 40s

the patch passed

+1 :green_heart:
compile
8m 16s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
javac
8m 16s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
2m  0s

the patch passed

+1 :green_heart:
mvnsite
8m 36s

the patch passed

+1 :green_heart:
shellcheck
0m  0s

No new issues.

+1 :green_heart:
javadoc
5m 22s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
5m 14s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+0 :ok:
spotbugs
0m 15s

hadoop-project has no data from spotbugs

+1 :green_heart:
shadedclient
37m 28s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

-1 :x:
unit
247m 36s
/patch-unit-root.txt
root in the patch failed.

+0 :ok:
asflicense
0m 31s

ASF License check generated no output?

470m 23s

Reason
Tests

Failed junit tests
hadoop.hdfs.tools.TestStoragePolicyCommands

hadoop.hdfs.TestDecommission

hadoop.hdfs.tools.TestViewFSStoragePolicyCommands

hadoop.hdfs.TestDFSStorageStateRecovery

hadoop.hdfs.TestDistributedFileSystemWithECFileWithRandomECPolicy

hadoop.hdfs.tools.TestStoragePolicySatisfyAdminCommands

hadoop.hdfs.tools.TestDFSAdminWithHA

hadoop.hdfs.tools.TestDebugAdmin

hadoop.hdfs.TestFileAppend4

hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer

hadoop.hdfs.tools.TestDFSAdmin

hadoop.hdfs.TestDFSPermission

hadoop.hdfs.TestErasureCodingPolicyWithSnapshotWithRandomECPolicy

hadoop.hdfs.tools.TestECAdmin

Subsystem
Report/Notes

Docker
ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7128/9/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/7128

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint shellcheck shelldocs

uname
Linux 98df2e46c6d2 5.15.0-124-generic #134-Ubuntu SMP Fri Sep 27 20:20:17 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 4c1fd826e7de318030ae8a0c7aecd44a0efbd0b1

Default Java
Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7128/9/testReport/

Max. process+thread count
4668 (vs. ulimit of 5500)

modules
C: hadoop-project hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-tools/hadoop-rumen . U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7128/9/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2 shellcheck=0.7.0

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

any plan to release this? we need this change to fix vulnerability on Avro which is coming via Hadoop


Service not Available

I input text but it returns: Service not available ("Service temporarily unavailable (lacking session IDs)")

Should be fixed now.

Yes, appreciate your help :)


Update Jaeger community-operators to v1.17.0

Thanks submitting your Operator. Please check below list before you create your Pull Request.

Flat operator directory structure is obsolete from 23-rd of October 2019, only nested directory structure will be accepted.

New Submissions

[x] Has you operator nested directory structure?
[x] Have you selected the Project Community Operator Submissions in your PR on the right-hand menu bar?
[x] Are you familiar with our contribution guidelines?
[x] Have you packaged and deployed your Operator for Operator Framework?

[x] Is your submission signed?

Updates to existing Operators

[x] Is your new CSV pointing to the previous version with the replaces property?
[x] Is your new CSV referenced in the appropriate channel defined in the package.yaml ?
[ ] Have you tested an update to your Operator when deployed via OLM?
[x] Is your submission signed?

Your submission should not

[x] Modify more than one operator
[x] Modify an Operator you don't own
[x] Rename an operator - please remove and add with a different name instead
[x] Submit operators to both upstream-community-operators and community-operators at once
[x] Modify any files outside the above mentioned folders
[x] Contain more than one commit. Please squash your commits.

Operator Description must contain (in order)

[x] Description about the managed Application and where to find more information
[x] Features and capabilities of your Operator and how to use it
[x] Any manual steps about potential pre-requisites for using your Operator

Operator Metadata should contain

[x] Human readable name and 1-liner description about your Operator
[x] Valid category name1
[x] One of the pre-defined capability levels2
[x] Links to the maintainer, source code and documentation
[x] Example templates for all Custom Resource Definitions intended to be used
[x] A quadratic logo

Remember that you can preview your CSV here.
--
1 If you feel your Operator does not fit any of the pre-defined categories, file a PR against this repo and explain your need
2 For more information see here

cc @objectiser

CVP/operator-olm-deployment

Is this check failure expected, or is there something wrong with our CSV?

/test CVP

Marking this as WIP, as I found a problem with this CSV, related to jaegertracing/jaeger-operator#931

Removing WIP.

/test CVP

@J0zi, can we get the logs for the check that failed? All I can see is this:

Failed to deploy the operator with OLM: java.lang.Exception: Error running Ansible playbook 'deploy-olm-operator.yml': hudson.AbortException: script returned exit code 2

/retest CVP

I will give it another chance, we will see. Waiting for a result.

/test CVP

investigating

@jpkrohling, it failed on the following:
TASK [deploy_olm_operator : Push the operator metadata to the private quay.io repository]

What does that mean? Is there something I have to fix in the CSV? During my manual tests, I was able to push this to my private quay.io repository and create an operator-source from it, and a subscription afterwards.

@jpkrohling @J0zi can you double check that your validation schema is correct? That crd is very large and it seems to contain a lot of duplicate fields. Can you confirm that this file is correct?

still investigating
ERROR:operatorcourier.push: 413 Request Entity Too Large

As part of jaegertracing/jaeger-operator#932 , we have recently removed the descriptions from the CRD, making it far smaller. I added another commit to this PR to use that CRD instead.
A similar PR to this one has been merged already, adding the Jaeger 1.17.0 CSV to the upstream community operators. To bring a similar change to that, I also created #1299, but this might be a sign that the tests between community operators and upstream community operators should be aligned.

Hi @jpkrohling, currently, catalog-initialization fails on following:
fatal - error loading manifests from directory: error checking provided apis in bundle : couldn't find jaegertracing.io/v1/Jaeger (jaegers) in bundle. found: map[]"

Looks like it's now passing!

@jpkrohling please squash your commits. And consider similar updates to #1299.

And consider similar updates to #1299.

It is already in sync with this PR here.

/test CVP


Fix id mapping when copying emotions and their translations

1. Why is this change necessary?
The fields start_row, start_col, end_row, end_col are always set to '1' in s_emotion_element. This leads to a non unique key that is used to map and copy the translation objects. All translation objects will be saved with the key 1111 in copyElementTranslations::$oldObjectKeys.
When an emotion is copied then all element translations except of one are getting lost.
2. What does this change do, exactly?
My change reads all the viewport positioning elements and concatinates the values into an identifier string. Each element gets its own unique key again.
The change is also backward compatible with emotions without an exact viewport positioning.
3. Describe each step to reproduce the issue or behaviour.

Create a new emotion with two different text elements.
Translate each element with different texts
Copy the emotion
Review the copied emotion. The translations are either completely lost or mapped incorrectly.

4. Please link to the relevant issues (if any).

5. Which documentation changes (if any) need to be made because of this PR?

6. Checklist

[ ] I have written tests and verified that they fail without my change
[✅] I have squashed any insignificant commits
[ ] This change has comments for package types, values, functions, and non-obvious lines of code
[✅] I have read the contribution requirements and fulfil them.

Hey @jeboehm,
can you please test if your changes still work with ZIPs with the old code?

Hi @PaddyS
I've successfully exported an emotion with the old code base and imported it on the new base.

Hello,
thank you for creating this pull request.
I have opened an issue on our Issue Tracker for you. See the issue link: https://issues.shopware.com/issues/SW-25225
Please use this issue to track the state of your pull request.

Hey @jeboehm,
can you please sign the CLA?

Done!

Anddd merged.
Thanks for your Contribution! :blue_heart:


Meenu/Stp 74243 translation for partner page

Changes:

...

Type of change

[ ] Bug fix
[ ] New feature
[ ] Update feature
[ ] Refactor code
[ ] Translation to code
[ ] Translation to crowdin
[ ] Script configuration
[ ] Improve performance
[ ] Style only
[ ] Dependency update
[ ] Documentation update
[ ] Release

Can you resolve the conflicts and fix the code smells?


init.sh dependencies function incorrect spelling

Running init.sh gives line 55: dependencies: command not found.
When looking at the file, the function is spelt with dependences and is being called with dependencies, 2 different spellings.

This should be resolved now, yes?


About critic-guide search in InternLM2.5-StepProver

Thank you for your work on InternLM2.5-StepProver. I have two questions:

The difference between critic-guide search and BFS lies in using the critic model's score to guide the search process. However, this model uses state information as input, so if the sampling number is 32, each step needs to interact with the Lean compiler up to 32 times, which is a significant time cost. In contrast, BFS only needs to interact with Lean once per step. Is this understanding correct?
For BFS, if an error occurs at a certain step, the search will terminate. But for critic-guide search, to obtain the critic model score, the prerequisite is that the compilation succeeds to get the state information. Therefore, the tactics that result in errors will be discarded. This setup seems somewhat unfair, doesn't it?

The sampling number is 32 for both Best-First and Critic-Guide. This represents the number of tactic candidates generated (they are "candidates" since they might not work), each of which is then passed to the Lean REPL to obtain the next Lean state. BF then ranks these states by averaging log probability scores, while CG passes these Lean states to the critique model for scoring. Thus, there is no difference in how they interact with Lean.

It depends. Since Lean has a C++ kernel, any exceptions thrown by C++ will instantly terminate the Lean process, regardless of CG/BF. Exceptions thrown by Lean itself indicate that certain tactic candidates cannot work. Such candidates will be discarded, of course. Again, the process is essentially the same for CG and BF.

Following the question, may I ask what is the template of the prompt for the critic model? I wrote a version according to what paper mentions, but not sure if it is proper: prompt = "Which state is closer to 'no goals'?"
answer1 = "no goals"

chat = [
    {"role": "user", "content": prompt},
    {"role": "assistant", "content": answer1},
]

"role": "user", "content": prompt},
{"role": "assistant"

Thank you for your reply. I have tried the two points mentioned in the paper:

Using your open-source critic model to score and guide BFS, which indeed achieved positive results on minif2f.
Adding the previous proof to the prompt and deleting some data that seemed problematic to me, but this actually led to a performance decline. Of course, it is possible that I mistakenly deleted some data, so I would like to ask about the following example of the rot_add_assoc theorem. It appears to be proved at step id=2, showing no_goals, so I discarded steps id=3-7. Is this the correct approach?

The sampling number is 32 for both Best-First and Critic-Guide. This represents the number of tactic candidates generated (they are "candidates" since they might not work), each of which is then passed to the Lean REPL to obtain the next Lean state. BF then ranks these states by averaging log probability scores, while CG passes these Lean states to the critique model for scoring. Thus, there is no difference in how they interact with Lean.
It depends. Since Lean has a C++ kernel, any exceptions thrown by C++ will instantly terminate the Lean process, regardless of CG/BF. Exceptions thrown by Lean itself indicate that certain tactic candidates cannot work. Such candidates will be discarded, of course. Again, the process is essentially the same for CG and BF.

Thank you for your reply. I have tried the two points mentioned in the paper:

Using your open-source critic model to score and guide BFS, which indeed achieved positive results on minif2f.
Adding the previous proof to the prompt and deleting some data that seemed problematic to me, but this actually led to a performance decline. Of course, it is possible that I mistakenly deleted some data, so I would like to ask about the following example of the rot_add_assoc theorem. It appears to be proved at step id=2, showing no_goals, so I discarded steps id=3-7. Is this the correct approach?

Following the question, may I ask what is the template of the prompt for the critic model? I wrote a version according to what the paper mentions, but not sure if it is proper:
prompt = "Which state is closer to 'no goals'?"
answer1 = "no goals"
chat = [
        {"role": "user", "content": prompt},
        {"role": "assistant", "content": answer1},
    ]
critic_model.get_score(tokenizer, chat)

Yes, I think it is right.

The sampling number is 32 for both Best-First and Critic-Guide. This represents the number of tactic candidates generated (they are "candidates" since they might not work), each of which is then passed to the Lean REPL to obtain the next Lean state. BF then ranks these states by averaging log probability scores, while CG passes these Lean states to the critique model for scoring. Thus, there is no difference in how they interact with Lean.
It depends. Since Lean has a C++ kernel, any exceptions thrown by C++ will instantly terminate the Lean process, regardless of CG/BF. Exceptions thrown by Lean itself indicate that certain tactic candidates cannot work. Such candidates will be discarded, of course. Again, the process is essentially the same for CG and BF.

Thank you for your reply. I have tried the two points mentioned in the paper:

Using your open-source critic model to score and guide BFS, which indeed achieved positive results on minif2f.
Adding the previous proof to the prompt and deleting some data that seemed problematic to me, but this actually led to a performance decline. Of course, it is possible that I mistakenly deleted some data, so I would like to ask about the following example of the rot_add_assoc theorem. It appears to be proved at step id=2, showing no_goals, so I discarded steps id=3-7. Is this the correct approach?

The search algorithm stops immediately when "no goals" is met. Regarding the previous proof issue, here are some related discussions: Lean Zulip Chat. It is designed to alleviate the "swinging back and forth" issue. However, we haven't conducted experiments on other base models, so whether this phenomenon is model/approach-agnostic is still unknown.

The sampling number is 32 for both Best-First and Critic-Guide. This represents the number of tactic candidates generated (they are "candidates" since they might not work), each of which is then passed to the Lean REPL to obtain the next Lean state. BF then ranks these states by averaging log probability scores, while CG passes these Lean states to the critique model for scoring. Thus, there is no difference in how they interact with Lean.
It depends. Since Lean has a C++ kernel, any exceptions thrown by C++ will instantly terminate the Lean process, regardless of CG/BF. Exceptions thrown by Lean itself indicate that certain tactic candidates cannot work. Such candidates will be discarded, of course. Again, the process is essentially the same for CG and BF.

Thank you for your reply. I have tried the two points mentioned in the paper:

Using your open-source critic model to score and guide BFS, which indeed achieved positive results on minif2f.
Adding the previous proof to the prompt and deleting some data that seemed problematic to me, but this actually led to a performance decline. Of course, it is possible that I mistakenly deleted some data, so I would like to ask about the following example of the rot_add_assoc theorem. It appears to be proved at step id=2, showing no_goals, so I discarded steps id=3-7. Is this the correct approach?

The search algorithm stops immediately when "no goals" is met. Regarding the previous proof issue, here are some related discussions: Lean Zulip Chat. It is designed to alleviate the "swinging back and forth" issue. However, we haven't conducted experiments on other base models, so whether this phenomenon is model/approach-agnostic is still unknown.

Thank you for your reply. I will conduct more experiments to verify this conclusion.


Aircon board fix alterative PR

Output IRED PWM on both old and new pins to catch all assembly methods

@matias-cphatomics This has been tested in a few loops, seems to work fine

Nice and simple way to be backwards compatible. We only need to move the AC related code to the correct branch after which I think it is ready for merging.


Adjustments to support multi-arch builds of OpenWhisk.

Changes to OpenWhisk to support Multi-Architecture builds:

Support of 'docker --build-args FOO=BAR' in gradle/docker.gradle
has been broken for who knows how long, but with modern docker
now support ARG before the FROM line, it's necessary that it work
for build customizition and configuration.
The AdoptOpenJDK project is working on multi-architecture alpine
builds, but they're not yet available on Docker Hub, so use that
ARG line to get them from the S390X builds.  Also bump the JDK
version (I could live with making that an argument).
Add some stuff to the 'tools/ow-util/Dockerfile' to support
architecture-aware downloads rather than naively downloading
everything for x86_64.

I think because of what I see in other programs, that OpenWhisk
is now using a more modern Docker to build images, but I'll let
Travis be the judge of that.

Description

Related issue and scope

[ ] I opened an issue to propose and discuss this change (#????)

My changes affect the following components

[ ] API
[X] Controller
[ ] Message Bus (e.g., Kafka)
[ ] Loadbalancer
[X] Invoker
[ ] Intrinsic actions (e.g., sequences, conductors)
[ ] Data stores (e.g., CouchDB)
[ ] Tests
[ ] Deployment
[ ] CLI
[X] General tooling
[ ] Documentation

Types of changes

[X] Bug fix (generally a non-breaking change which closes an issue).
[X] Enhancement or new feature (adds new functionality).
[ ] Breaking change (a bug fix or enhancement which changes existing behavior).

Checklist:

[X] I signed an Apache CLA.
[X] I reviewed the style guides and followed the recommendations (Travis CI will check :).
[ ] I added tests to cover my changes.
[ ] My changes require further changes to the documentation.
[ ] I updated the documentation where necessary.

Ugh.  I think I'm running into https://github.com/apache/incubator-openwhisk/issues/3804, and I don't have the energy to deal with it today.  Maybe tomorrow I'll try to work some ansible magic.

Do you have any updates? I also need this patch for multiarch support


(patch-my-pc) Fix rejected package

https://chocolatey.org/packages/patch-my-pc/<IP_ADDRESS>

Not sure about this one. What do you think @gep13 ?

I think it should be unlisted too. Scanned the site,

most of the stuff is available via chocolatey anyway.
very low number of choco users.

Simply doesn't deserve attention.


Remove Health Check as Fails with Halyard 1.45.0

Hi,
Here is a PR with a different way of handling the probe problem: https://github.com/OpsMx/spinnaker-helm/pull/27
I've tested successfully on our deploys.
Disabling the health check means that the install container might fail and restart several times while halyard is still not ready.


Issue 411 structure build and test pipeline

introduce makefile for eris-db with
make check : fails on gofmt
make fmt : applies gofmt
make lint : fails on golint
make vet : fails on go vet
make erase_vendor : deletes /vendor, as helper to ensure CI builds have clean vendor folder installed
make install_vendor : installs /vendor from glide.lock
make build : fail on check, build all targets
make build_race : fail on check, build all targets with check for race conditions
make test : fail on check, build and run go test (for all targets)
make test_race : as test, but with race conditions
make build_docker_db : build docker image for eris-db and tag with full version number
make test_docker_db : build docker image for eris-db and run go test inside the docker image

note: build also builds into /target
this set of instructions is not set in stone or incomplete, but it aims at providing a base-line to simplify the jenkins jobs needed to build, test, move artefacts, for multiple targets in a single repository as we consolidate the repositories.

@silasdavis I started using a feature branch to allow for more flexibility for discussion, rather than requesting a merge into develop straight away.

:smiley_cat:


SoftEther  Support Kernels

Discussed in https://github.com/SoftEtherVPN/SoftEtherVPN/discussions/1910

Originally posted by Sherry0808 September 27, 2023
Hi，
When will SoftEther support the 5.4 series of kernels?
After installing Softether, I added the tap_28 interface of device A to vrf-14 and also added the tap_28 interface of device B to vrf-14. Then, I tried to connect using Softclient and access the <IP_ADDRESS> address of device B's tap_28 interface, but I found that it was not accessible. Our current kernel version is 5.4, so I would like to know if Softether supports the 5.4 kernel. I noticed on the official website that it explicitly states support for 2.x, 3.x, and 4.x kernels, so I want to confirm.

duplicate of https://github.com/SoftEtherVPN/SoftEtherVPN/discussions/1910


Generate a keypair using embark

So I'm making a mobile app where each new user on the app would get assigned a key pair with the private key being stored locally on the phone and the public key on the blockchain. How is that possible with embark? Is there a specific request to generate a new account with that being associated with a keypair?

I don't think its possible with Embark atm. You could try https://github.com/ConsenSys/eth-lightwallet which seems to supporting thee features.

Ok thanks!! Is it possible to make these light wallets and embark work together?

Embark is not envisioned to be such a product. And yes it is possible to make it work like that. However you can add these 2 projects together and work on your logic right?

@hasanqusay you can use this with embark, just add one of these files https://github.com/ConsenSys/eth-lightwallet/tree/master/dist to app/js
then follow the instructions in the readme https://github.com/ConsenSys/eth-lightwallet


v0.10.0

See CHANGELOG
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Needs to get in the patch from #465. It might not need a separate entry in CHANGELOG though, as this is a fix over a fix that is already documented here and hadn't been released yet.


Depends.pm: unified interface for aur-depends and aur-graph

Related to:

https://github.com/AladW/aurutils/issues/1119
https://github.com/AladW/aurutils/issues/1086#issuecomment-1775010778
https://github.com/AladW/aurutils/issues/966
https://github.com/AladW/aurutils/issues/860

[x] Depends.pm module
[ ] Srcinfo.pm module
[x] aur-depends adaptation
[ ] aur-graph adaptation
[ ] aur-graph --table

Since this issue was posted, the only consumer of Depends.pm is aur-depends. Similarly for Repo.pm and aur-repo-parse. Since the AUR::Depends split complicates maintenance, I might revert it until there are some concrete plans for a "unified" aur-depends.


Support multiple server TlsCertificates per DownstreamTlsContext

In the v2 API, we allow multiple TlsCertificates to be specified per DownstreamTlsContext (https://github.com/lyft/envoy-api/blob/master/api/tls_context.proto#L107). In the current Envoy implementation, each TLS context only supports a single certificate/key (https://github.com/lyft/envoy/blob/master/source/common/ssl/context_impl.cc#L77). This issue will track the design and implementation of multiple cert support. @PiotrSikora

This is a dup of https://github.com/lyft/envoy/issues/95 right? We should implement SNI in the "v1" world also, as it is trivial to do so. @PiotrSikora if you are going to work on this can we maybe do the v1 work first since it will be used in v2 as well?

I was interested in understanding what it means to have multiple certs here. Piotr pointed out there are a number of different options, e.g. are we doing this with SNI, different cert types. If it's just SNI then it's a dupe.

Yeah that's true, this is beyond SNI, like having both EC and RSA certs potentially I guess? Still, I would like to see us implement SNI ASAP since I'm getting tired of getting asked about it. :)

This (repeated TlsCertificates in v2 API) is for having different types of certificates for the same context (e.g. RSA and ECDSA), which is different from full SNI support (#95). Let's keep this simple and move SNI discussion back to #95.

@ggreenway I discussed this on and off with @davidben. The reason why I didn't implement this in Envoy via "userland" certificate selection yet is that the plan is for BoringSSL to eventually re-add the support for multiple certificates of different types in the same context (exactly for the RSA & ECDSA use case).

Any idea what the timeline is for that change to BoringSSL?

@PiotrSikora is the "userland" approach to create a different SSL_CTX for each cert type, and set the CTX in one of the callbacks based on supported cipher types in the ClientHello?

Sorry about the delay on that feature. I'm kind of running around all over the place right now due to an upcoming move, and a couple of tasks popped up in front of it that took longer than expected. I do intend to finally chew on it this week, though if it's not done by IETF and my move, it might get delayed.

@ggreenway yes, pretty much. The "userland" approach isn't a complex change in Envoy, but the support for RSA+ECDSA isn't something frequently requested by the community, and since the proper support for it is going to be added to BoringSSL sooner rather than later, I'd prefer avoid adding it now, unless required for a production deployment.
Do you have any deadline for when you need this feature to be available?

I don't have a hard date right now, but I'd guess I'll probably need it within 3 months. I'll hold off as long as possible in hopes that BoringSSL adds the functionality we need. But if I need to do it in userland, and then we rip it out later, that's fine. I'll comment here when I have firmer dates, or when I'm going to start working on it.

I'm going to take a shot at this one, since we need this sooner than later.

FTR, I've had internal discussions with @PiotrSikora and @davidben. The PoR is to add in explicit client handshake and certificate type detection to Envoy for the initial implementation, as BoringSSL won't add this for some time. When this feature arrives in BoringSSL, we will replace our implementation with that.

Quick design question @mattklein123 @ggreenway @lizan @PiotrSikora. Do we want to do the client handshake capability detection in the TLS inspector or directly in the TLS transport socket?
A reason for TLS inspector include that we're doing ClientHello peaking already for SNI/ALPN. A reason against is that mixed certificate type doesn't necessarily imply SNI use, so we will have additional scenarios for requiring the inspector extension.
I'm thinking we'll be doing it in the inspector, since this is conceptually coherent with what's done for other handshake attributes, but just wanted to check your thoughts.

Oh right, we inject the TLS inspector automatically anyway, so the above is moot, I see.

In general doing it as part of the inspector sounds good to me, and then we can use that data to drive later decisions around which cert to use.

A couple more things to bounce off folks on this issue; the resolution order for certificates. What I'm thinking is:

We keep track of the first ECDSA and first RSA certs in a certificate chain.
For server contexts, we will use the client handshake ECDSA indicators from the group and cipher suite extensions to determine which to use.

Some design choices we can make:

We could skip any certificates that don't validate and only select the first matching one in (1) above. Or, we could maintain existing semantics and reject any invalid cert/key pairs as a config rejection. Which to do?
I'm wondering how best to handle client contexts. Do we just pick the first certificate and call it a day?

There's a tradeoff here between keeping it simple (i.e. the cert list has a cardinality <= 2) vs. being more flexible. Either way, we'll need to document this well, as it's probably going to be confusing to reason about. Ideally we maybe should just have a singleton ECDSA and a singleton RSA option in the config and let the rest of the pipeline sort it out.

IIUC per https://github.com/envoyproxy/envoy/blob/master/source/server/listener_manager_impl.cc#L243
TLS inspector is only injected when you have certain filter_chain_match. So if a listener is configured for pre-knowledge TLS, then it is valid that no TLS inspector is injected.
Also do we want to select filter chain based on ClientHello certificate type? If no perhaps adding in TLS transport is better and it is easier to replace with BoringSSL implementation? (Assuming BoringSSL implementation will be having multiple certificates in same SSL_CTX)

One thing to keep in mind is how you decide whether to use RSA or EC if both are supported. I think openssl decides based on the server cipher-suites config, which are already ordered by priority. I don't know how boringssl plans to implement that detail. But we'll want to sort this out so we don't introduce an unexpected behavior change.
OTOH, does anyone ever want to prefer RSA over EC? I've never seen anyone do that (on purpose). RSA is only there for backwards-compat with clients that can't do EC.

@ggreenway  Yeah, I was planning on picking EC in preference, but this is actually a small detail local to the code responsible for the picking (at that point we will have both knowledge of client ECDSA cipher suite capability and the list of server cipher suites).
@lizan Yeah, I hear you on this one. I'm inclined to go with TLS inspector based on my PoC work though, since if we do one certificate per SSL_CTX for now, it's easier to have TLS inspector provide the information and make the choice at the start of the connection, before the SslSocket starts to process bytes (but after the listener filter). What this means is that we will turn on TLS inspector auto-injection for multi-cert filter chains, and later when we replace with BoringSSL, we will turn it off.
@mattklein123 Agree on those simplified semantics.

After some discussion with @davidben and @PiotrSikora, I think it's clear that we should have the ECDSA detection happen in the TLS socket, rather than TLS inspector. The rationale here is that BoringSSL will eventually subsume this capability, so we should try and localize it rather than plumb deprecated ECDSA details (signature algorithms, cipher suites) across the listener socket objects.


Add support to extend Azure AD to query security settings.

https://github.com/turbot/steampipe-plugin-azure/issues/47.  duplicate


Extracting audio from a Python script

[x] I've verified and I assure that I'm running youtube-dl 2<IP_ADDRESS>
[x] At least skimmed through README and most notably FAQ and BUGS sections
[x] Searched the bugtracker for similar issues including closed ones
[x] Question

Hi! I'm trying to extract audio from downloading video while embedding youtube-dl into a Python script. How do I go about that? It doesn't seem to be described in options.py, I've tinkered with postprocessor_args etc. but nothing would help. Is it even possible while keeping youtube_dl embedded? If it's not, how do I go about fixing it? Any pointers?
Best regards,
Arsenijs.

Never mind, solved. Added this to "opts":
  "postprocessors":[
{"key":'FFmpegExtractAudio', "preferredcodec":"mp3", "preferredquality":"0"}]

Will soon send a pull request - there's a bug where you can't send an int as "preferredquality", and though it's certainly something intuitively possible, the error message is anything but helpful.


Image uploader doesn't allow SVG files

When adding a field like this:
$this->crud->addField([
    'label'  => "Icon",
    'name'   => "icon-image",
    'type'   => "image",
    'fake'   => true,
]);

... the JavaScript in src/resources/views/fields/image.blade.php will run a test against the file that will fail when it shouldn't, and the JS will then do alert("Please choose an image file.");. (Several times, in fact, but that's a different issue.) SVGs should just work, like the other image types do.
The test should be modified to allow SVG images to work.
See my original comment here for my suggested solution.

My only fear with that, is the possibility of injections with allowing the + by default but will see what @tabacitu thinks. For now my suggestion would be to publish the view resources and edit it within your application to allow it.

I'm late to the party but why do we have a regex that stops stuff?

That makes sense. Maybe a rule that is more specifically written to allow what it currently does, plus exactly image/svg+xml and nothing else. That'd be much more specific than just allowing any plus sign.
@lloy0076 I'm not certain of my guess, but I imagine this is only for the image uploader, so that a person doesn't try to upload a Word doc to an image field, or something nonsensical like that.

Yeah, I suppose but what happens when I invent an image with a mimetype of `application/acme-image-gazoo'? :(

https://laravel.com/docs/5.4/validation#rule-image - we could just lean on the Laravel validators...

In case it's helpful, there's a complete list of MIME types on the Mozilla Developer Network, and you can see that there are currently five different MIME types with a plus sign in them. I think any MIME type validator, if that's its purpose, should recognize all of the ones in that list as valid MIME types.
If we're purposefully trying to restrict to images, then I'd agree with @lloy0076's comment above.

I think I agree with you guys - I see no reason to use another validation on top of Laravel. We can just instruct people to use the proper validation in the field documentation.

I am facing this issue. Is there a hack to get it to work?

@packytagliaferro I proposed a hack in a comment here that allowed me to copy vendor/backpack/crud/src/resources/views/fields/image.blade.php to resources/views/vendor/backpack/crud/fields/image.blade.php and change if (/^image\/\w+$/.test(file.type)) to if (/^image\/[\w\+]+$/.test(file.type)). I don't know that that's the best solution, but it works for me quite well, so if you're looking for a "hack", there you go.

PRs are welcome - the hack would work or even removing the validation completely?
Closing for the moment.

Why close this if no fix has been made, and it's been acknowledged that this is a bug?

PRs are welcome.

Yup, I think we should implement the simple solution - reopening.

Thank you for the suggestion @joshlewis
Just went through this again. I'm afraid cropperJS doesn't work with SVGs. Applying your regex would indeed allow you to pick a SVG, but it wouldn't allow you to crop the SVG. If you try it, the resulting file will be broken. Which makes sense now that I think about it - cropping bitmaps and cropping vector files should indeed be very different.
It looks like we can't support cropping SVGs right now. For SVGs the only good Backpack field types are upload or browse.
Sorry to bring bad news.
Cheers!

@tabacitu Actually, I just tried this out, and cropping SVGs works for me when I only apply @joshlewis' change..? cropper.js will convert it to base64-encoded png even when not tightening the cropping however, which could be improved upon, but basically I don't see why this shouldn't work.

I went ahead and opened PR #2166 to fix this. I also disabled cropper.js when an SVG image is selected. Feedback would be appreciated!


Enhancement: Create landing pages for all the .NET Toolkit controls

There appears to be landing pages for some of the .NET Toolkit Controls but not all of them. These landing pages are useful to the .NET Runtime developers as they provide enhanced information for users of the .NET Toolkit controls (without having to really dig into the source code to figure out how they work).
The following .NET Controls have landing pages:

AR Toolkit: https://esri.github.io/arcgis-toolkit-dotnet/ar.html
BasemapGallery: https://esri.github.io/arcgis-toolkit-dotnet/basemap-gallery.html
BookmarksView: https://esri.github.io/arcgis-toolkit-dotnet/bookmarks-view.html
OverviewMap: https://esri.github.io/arcgis-toolkit-dotnet/overview-map.html
SearchView: https://esri.github.io/arcgis-toolkit-dotnet/search-view.html

The following .NET Toolkit Controls DO NOT have landing pages:

Compass
FeatureDataGrid
Legend
MeasureToolbar
PopupViewer
SclaeLine
SymbolDisplay
TimeSlider

@nCastle1 I am just looking at this Issue in terms of providing up-to-date information on the .NET Toolkit page. - https://developers.arcgis.com/net/toolkit/ I was wondering if any of these documentation pages have been written and are due to get pushed up for U14 for Compass, FeatureDataGrid etc. (as above). Thanks

Unfortunately no, the only exception being TimeSlider. I was planning on taking this on for the U14 release, but I think instead the Toolkit team will be taking this on for all Toolkits in the U15 timeframe.


Moved all argparse configs to .yaml files, added set_deterministic function, added class for easy model evaluation on one image

Moved all argparse configs to .yaml files. All configs are parsed from the .yaml file. If necessary, any parameter can be written to the terminal as before and these parameters will overwrite the .yaml file. All args are also stored in args_text variable as string.
Added set_deterministic function. This function can be useful for checking the representativeness of experiment.
Added class for easy model evaluation on one image.

I think it would be great if the config system can be like that of detectron2 or mmdetection, as they allow overriding config values through command line args.

I think it would be great if the config system can be like that of detectron2 or mmdetection, as they allow overriding config values through command line args.

This config system can receive command-line arguments as before:
python train.py --batch_size=128 ----weight_decay=0.001, etc
And then these config values will be override and used in pipeline.

Does it also allow inheritance from some basic configs?

Thanks for the contribution. I agree that config files can make more sense than the sprawl of args that the scrips here have developed. I had a different (eventual) plan in mind though. I intended to migrate to Hydra or ML Collections, a config scheme with hierarchical / typed struct support.
This approach sits in between those, it keeps the command line more aligned with what exists right now, but it lacks some of the features that'd motivate me to adopt this change. The existing code does support using yaml config files in a half-assed way. I use it all the time for resuming experiments or changing a few items in a sequence of experiments. If you start any train process, grab the args.yaml file in the output folder, you can tweak it to create new configs and just launch with --config my_config.yaml

Also, the defaults that were placed in the config files in this PR are different and break the standard ImageNet training hparam starting point. I'll keep this PR open as a reference for now and figure out when I'd like to tackle config...


tsukuba: Printer  found but not printing from Ubuntu 20.04

Similar to https://github.com/130s/hut_10sqft/issues/685 at an occasional home.
Issue

I do see a printer but cannot print.
On Firefox, it does seem to finish but nothing happens on the printer device.
On Google Chrome,

it does show "selected printer is not availbale or not instlalled correctly".
On an option to use system printing, it does completes but nothing happens on the printer device.

$ avahi-browse --all -t -r
+ wlp4s0 IPv4 Canon MF741C/743C                             Web Site             local
:
+ wlp4s0 IPv4 MR2200ac-ERTCDH                               Web Site             local
+ wlp4s0 IPv6 RT2600ac [00:11:32:c3:26:13]                  Workstation          local
+ wlp4s0 IPv6 MR2200ac-ERTCDH-262344 [00:11:32:bf:45:11]    Workstation          local
+ wlp4s0 IPv4 RT2600ac [00:11:32:c3:26:13]                  Workstation          local
+ wlp4s0 IPv4 MR2200ac-ERTCDH-262344 [00:11:32:bf:45:11]    Workstation          local
= wlp4s0 IPv4 Canon MF741C/743C                             Web Site             local
   hostname = [Canona6cb69.local]
   address = [<IP_ADDRESS>]
   port = [80]
   txt = ["path=/"]
:

Printer's job page doesn't seem to show any job even when I see completion on the client side.

I removed the printer and am re-adding. After selecting the driver (MF741C) I get this error:
There was an error during the CUPS operation: 'server-error-internal-error'.

Choosing ipp://Canon%20MF741C%2F743C._ipp._tcp.local/. Got the same error.
Choosing IPP network printer via DNS-SD. Got the same error.

Updating firmware immediately failed due to some sort of network issue. Assuming network wasn't working, I just rebooted the entire device. Now, firmware update seems to be working.

Not sure which one was a solution but I now got it working.

Reboot printer.
Updated firmware.


Example page not up to date?

It appears that the example page is not using the latest version of the library.
When I test the vertical scrolling in chrome it auto scrolls to the end, however when I run the same example locally with the latest version the behaviour is different.
I suspect somehow the demo is running on a previous version of the library but not entirely sure..
Would be great to update it.

I just updated it, sorry about that.

Awesome, hopefully that helps a few people out :)


renderjson doesn't work in installed app

In the installed version of the client, the renderjson.js library fails with this error:
Uncaught ReferenceError: renderjson is not defined at renderjson.js:58
That line is:
(_module || {}).exports = renderjson = function () {
When the client is run from the command line, it works normally.

What does this library even do?

@hoffmabc It's the plugin we use to make the contract be collapsable / expandable:

Ah ok. I think I saw contracts not showing up in the installer version and didn't think about this. Good catch.


Convert from CoffeeScript to JavaScript

because of https://github.com/atom/electron/pull/4065

We have no plans to convert wholesale from CoffeeScript to JavaScript at this time. Thanks for your passion and feedback!

darn. Death to coffeescript.

This would be a boon not only to the performance of atom (e.g. argument splats use [].slice.call() which is death for V8 performance), but also to how it can be contributed to.
It would be nice if the team would express whether or not it is open to this sort of migration, so that a contributor can create a JS conversion with confidence.


MIDI Support

Summary
This feature is about turning the swadge into a real MIDI synthesizer! This would mean, at a minimum, being able to use the swadge as a USB MIDI synth, and playing back MIDI files loaded into the firmware. The implementation would be targeted at implementing General MIDI to whatever extent is feasible.
Implementation Plan
The initial implementation can be very basic, as a proof of concept. TinyUSB comes with MIDI Streaming support, so the first step is to add the descriptors and pull that into a mode.
The next step would be creating a way to define an instrument to be synthesized. This is the area where we'd have the most freedom with the implementation, so we could start out with something very basic here too: pick a waveform and define an envelope. Once the rest of the MIDI player is fully implemented at a basic level, we can just go wild making the instruments fancier until we run out of time or CPU, or we end up with a full-blown programmable synthesizer. Some less-than-basic things to do here after the initial implementation would be adding low-frequency oscillators and filters without supporting totally arbitrary synth pipelines.
As for the instruments themselves, General MIDI specifies 128 specific instruments (apparently this is the culprit behind every keyboard ever having helicopter sounds, phone rings, and whatever a "Goblins" is), so coming up with definitions for all of these might not really be feasible, but we could probably get a decent number of them, especially if we make it easy for anyone to define an instrument voice with a simple config file or something like that. I assume you can probably export these from actual synth software? Also note that we can come up with our own instruments and define them in a separate "program bank" from the ones General MIDI supports, so we can fix the glaring omission of Cat Meow from the spec.
Along with the instruments, we would create the MIDI player which is responsible for actually "playing" the instruments with a bank of software synths, mixing them together, applying more filters, etc. This would be used directly by the USB MIDI mode, or controlled by an equivalent to the current Song Player when playing a MIDI file. The initial implementation of this could just control a single synth to play songs without any filters, mixing, or polyphony.
After that basic functionality is complete, the rest of the work is basically just improving and expanding the player to meet the General MIDI specs, which would mainly mean:

24-voice polyphony
Pitch bending
Note velocity
Sustain pedal signal

Again, all this is keeping in mind that implementing General MIDI seems like a decent goal to shoot for absent any technical limitations, but isn't a hard requirement and there's basically no downside to just partially implementing it.
For a stretch goal after synthesized instruments are working, it'd be nice to support at least one sample-based instrument. Extra cool points if we have a way to record with the swadge mic and then use that as an instrument.
Another thing that's technically possible is piping the audio back to the computer as a USB audio device. Which I guess would be cool if you want to compose swadge music on your computer and record it or play it back on your actual speakers. But you could also just plug the swadge into some headphones or the line in too...
MIDI Player - Technical Spec
The MIDI player itself will consist of two parts: the MIDI engine, which is responsible for accepting MIDI commands and synthesizing audio; and the MIDI file parser, which will either read MIDI commands from a MIDI file directly, or generate them from some equivalent structure, and send them to the MIDI engine for playback.
Resources

USB MIDI 1.0 Specification: https://www.usb.org/sites/default/files/midi10.pdf
A super basic page on the basic MIDI commands: https://computermusicresource.com/MIDI.Commands.html
Arduino MIDI synth project, which implements all the complicated synth stuff and is MIT licensed: https://github.com/DLehenbauer/arduino-midi-sound-module
Another Arduino MIDI Synth project, specifically for ESP32: https://github.com/esp32beans/ESP32USBMIDI

============
To-do List
Not necessarily planning to get to all of these but this is everything I can think of that wouldn't be totally crazy to support.
MIDI commands:

[x] Note on/off
[x] All notes off
[x] All sounds off (Panic Button)
[x] Program select
[x] Pitch bend
[x] Set tempo
[ ] Reset
[ ] Active Sense (stop playing if not received for >300ms)
[ ] Mono Operation
[ ] Poly Operation
[ ] Omni on/off (do/don't listen to all 16 channels)
[x] Control: Hold (sustain) pedal
[ ] Control: Channel volume
[ ] Control: Release
[ ] Control: Attack
[ ] Control: Registered Parameter: Pitch bend range
[ ] Control: General Purpose Sliders (maybe for LEDs?)
[ ] Universal SysEx: General MIDI Enable/Disable
[ ] Universal SysEx: Master Volume

MIDI engine:

[X] Wavetable instrument playback
[x] Percussion instrument playback (via callback)
[ ] Sampled instrument playback
[X] Polyphony per-channel
[X] Reserved percussion polyphony
[x] Channel-independent polyphony (e.g. 24 voices pooled across all channels)
[ ] Intelligent note stealing if all voices are in use
[ ] Handle note off when multiple of the same note are playing (stop either first or last played note)
[x] Handle special cases for percussion note on/off (hi-hats and whistles)
[ ] ADSR envelopes

MIDI file parser:

[x] Type 0 (single-track) file support
[x] Type 1 (multi-track, simultaneous -- most common) file support
[ ] Type 2 (multi-track, in sequence) file support
[x] Basic commands
[x] Timecode / time signature parsing & calculation
[x] Running status handling
[x] Text (text, track name, instrument name, lyrics, section, etc.)

Other resource: http://midi.teragonaudio.com/

A couple more resources:

http://www.philrees.co.uk/articles/midimode.htm
http://blitter.com/~russtopia/MIDI/~jglatt/tech/midispec.htm

A random thought: it should be possible for the swadge to appear as a USB mass storage device and let the user upload a MIDI file into a fake filesystem in SPIRAM. If it works well, we could even make it usable by other modes for something like uploading a custom level. Or we could use it to make restore/backup a thing. Obviously a MIDI file wouldn't realistically fit in NVS but I think it would still be cool to support user-supplied songs somehow even if it doesn't persist.

 We still have #299 for efficiency, but I don't think we're adding more core MIDI functionality. @dylwhich, feel free to reopen if I'm off the mark,


#6300 Avoid inconsistency occured when building Gallery menu items if the tenant is not the default one

This avoids to build the Gallery menu item in Modules,Themes and Settings if the tenant is not the default one since the permissions doesn't allow the interaction. Avoid inconsistency commented in issue  #6300

Hi @jardg, I'm your friendly neighborhood .NET Foundation Pull Request Bot (You can call me DNFBOT). Thanks for your contribution!

This seems like a small (but important) contribution, so no Contribution License Agreement is required at this point. Real humans will now evaluate your PR.

TTYL, DNFBOT;