HADOOP-18484. Upgrade hsqldb to v2.7.1 to mitigate CVE-2022-41853

Description of PR
Upgrade hsqldb to v2.7.1 to mitigate CVE-2022-41853
For code changes:

[X] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
[ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[ ] If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

@pjfanning - 2.7.1 doesnt seem to be there yet. Can you confirm once? Thanks.

@ashutoshcipher looks like you are right - Github seems to have made this issue public before hsqldb have managed to publish the release to Maven Central.
Maybe, if we wait for a few days and check https://repo1.maven.org/maven2/org/hsqldb/hsqldb/ again.

Yeah, I will keep the PR open and keep a check on mvn.

@ashutoshcipher @pjfanning  we may not be able to upgrade to 2.7.0, I tried to upgrade to 2.6.0 some time ago, but found that 2.6.0 and above require JDK 11, do we have a solution for this?

Thanks @slfan1989. I can here - http://hsqldb.org./ for Version 2.7.0  that
Individual jars for Java 6, 8 and 11 are available

It seems that you can download jdk8 friendly jars from

http://www.hsqldb.org/download/hsqldb_270/
https://repo1.maven.org/maven2/org/hsqldb/hsqldb/2.7.0/

I'm not a maven expert but I think you might be able to specify a 'classifier' - in this case, possibly 'jdk8' in order to have maven download hsqldb-2.7.0-jdk8.jar.
Presumably v2.7.1.jar will be similar (whenever it gets published).

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 57s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
detsecrets
0m  1s

detect-secrets was not available.

+0 :ok:
xmllint
0m  1s

xmllint was not available.

+0 :ok:
shelldocs
0m  1s

Shelldocs was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

-1 :x:
test4tests
0m  0s

The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ trunk Compile Tests _

+0 :ok:
mvndep
15m 24s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
25m 55s

trunk passed

+1 :green_heart:
compile
23m 18s

trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
compile
20m 55s

trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07

+1 :green_heart:
mvnsite
19m 17s

trunk passed

+1 :green_heart:
javadoc
8m 12s

trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
7m 26s

trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07

+1 :green_heart:
shadedclient
35m 23s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 24s

Maven dependency ordering for patch

-1 :x:
mvninstall
5m 17s
/patch-mvninstall-root.txt
root in the patch failed.

-1 :x:
compile
9m  5s
/patch-compile-root-jdkUbuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04.txt
root in the patch failed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04.

-1 :x:
javac
9m  5s
/patch-compile-root-jdkUbuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04.txt
root in the patch failed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04.

-1 :x:
compile
8m  8s
/patch-compile-root-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt
root in the patch failed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.

-1 :x:
javac
8m  8s
/patch-compile-root-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt
root in the patch failed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-1 :x:
mvnsite
2m 46s
/patch-mvnsite-root.txt
root in the patch failed.

+1 :green_heart:
shellcheck
0m  0s

No new issues.

-1 :x:
javadoc
3m 13s
/patch-javadoc-root-jdkUbuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04.txt
root in the patch failed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04.

-1 :x:
javadoc
3m  2s
/patch-javadoc-root-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt
root in the patch failed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.

-1 :x:
shadedclient
14m 44s

patch has errors when building and testing our client artifacts.

_ Other Tests _

-1 :x:
unit
602m 51s
/patch-unit-root.txt
root in the patch failed.

+1 :green_heart:
asflicense
1m 37s

The patch does not generate ASF License warnings.

789m  5s

Reason
Tests

Failed junit tests
hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes

hadoop.hdfs.server.diskbalancer.command.TestDiskBalancerCommand

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4991/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/4991

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint shellcheck shelldocs

uname
Linux deffae4b7a47 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 2b0d246283bb2d6ca778554c440cb4ec5632b794

Default Java
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4991/1/testReport/

Max. process+thread count
2909 (vs. ulimit of 5500)

modules
C: hadoop-project . U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4991/1/console

versions
git=2.25.1 maven=3.6.3 shellcheck=0.7.0

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

Looks like the hsqldb do not want to do a v2.7.1 release. They want users to set system properties.
https://sourceforge.net/p/hsqldb/discussion/73673/thread/0df9e4f45a/?limit=25#2489

@pjfanning Thank you very much for your feedback. From a personal point of view, we should still wait for v2.7.1. The way of setting properties does not seem to be very good. If we hard-code it, we may need to remove it later.

@pjfanning Thank you very much for your feedback. From a personal point of view, we should still wait for v2.7.1. The way of setting properties does not seem to be very good. If we hard-code it, we may need to remove it later.

Yes, it's quite messy all in all. I'm not sure why they won't do a release.

Thanks @pjfanning @slfan1989 . I think we can wait for a while for 2.7.1 release and take a call if we want to make changes in code or not

Yes, it's quite messy all in all. I'm not sure why they won't do a release.

let's wait more time.

@ashutoshcipher @pjfanning Version 2.7.1 is now available.  May I ask you to update the PR?

you might be able to specify a 'classifier' - in this case, possibly 'jdk8' in order to have maven download hsqldb-2.7.0-jdk8.jar.

Yes, <classifier>jdk8</classifier> is needed for hsqldb dependency in all affected POMs.

@ashutoshcipher @pjfanning Version 2.7.1 is now available. May I ask you to update the PR?

you might be able to specify a 'classifier' - in this case, possibly 'jdk8' in order to have maven download hsqldb-2.7.0-jdk8.jar.

Yes, <classifier>jdk8</classifier> is needed for hsqldb dependency in all affected POMs.

Thanks @adoroszlai. Triggering jenkins once again for cleaner build

Thanks @adoroszlai. I am resolving and updating exactly the same in my local :)

Thanks @ashutoshcipher for the patch.

Thank you so much @adoroszlai for reviewing and your comments.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
1m 47s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
detsecrets
0m  1s

detect-secrets was not available.

+0 :ok:
xmllint
0m  1s

xmllint was not available.

+0 :ok:
shelldocs
0m  1s

Shelldocs was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

-1 :x:
test4tests
0m  0s

The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ trunk Compile Tests _

+0 :ok:
mvndep
16m  3s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
26m 13s

trunk passed

+1 :green_heart:
compile
24m  8s

trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
compile
22m 11s

trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07

+1 :green_heart:
mvnsite
21m 38s

trunk passed

+1 :green_heart:
javadoc
9m 14s

trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
8m 26s

trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07

+1 :green_heart:
shadedclient
38m 37s

branch has no errors when building and testing our client artifacts.

-0 :warning:
patch
39m  2s

Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 31s

Maven dependency ordering for patch

-1 :x:
mvninstall
10m  0s
/patch-mvninstall-root.txt
root in the patch failed.

+1 :green_heart:
compile
24m 42s

the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javac
24m 42s

the patch passed

-1 :x:
compile
11m 56s
/patch-compile-root-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt
root in the patch failed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.

-1 :x:
javac
11m 56s
/patch-compile-root-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt
root in the patch failed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-1 :x:
mvnsite
14m 40s
/patch-mvnsite-root.txt
root in the patch failed.

+1 :green_heart:
shellcheck
0m  0s

No new issues.

+1 :green_heart:
javadoc
8m 31s

the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04

-1 :x:
javadoc
6m 18s
/patch-javadoc-root-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt
root in the patch failed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.

-1 :x:
shadedclient
27m 35s

patch has errors when building and testing our client artifacts.

_ Other Tests _

-1 :x:
unit
959m 42s
/patch-unit-root.txt
root in the patch failed.

+1 :green_heart:
asflicense
1m 48s

The patch does not generate ASF License warnings.

1205m  3s

Reason
Tests

Failed junit tests
hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes

hadoop.hdfs.server.blockmanagement.TestBlockTokenWithShortCircuitRead

hadoop.hdfs.server.diskbalancer.command.TestDiskBalancerCommand

hadoop.yarn.server.federation.store.sql.TestFederationSQLServerScriptAccuracy

hadoop.yarn.server.federation.store.impl.TestSQLFederationStateStore

hadoop.yarn.server.federation.store.sql.TestFederationMySQLScriptAccuracy

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4991/3/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/4991

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint shellcheck shelldocs

uname
Linux 0c4e20aa2853 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 7f8ec22c1ccff82c55cb9c9b7917a80b28fa17d2

Default Java
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4991/3/testReport/

Max. process+thread count
2774 (vs. ulimit of 5500)

modules
C: hadoop-project . U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4991/3/console

versions
git=2.25.1 maven=3.6.3 shellcheck=0.7.0

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 58s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
detsecrets
0m  1s

detect-secrets was not available.

+0 :ok:
xmllint
0m  1s

xmllint was not available.

+0 :ok:
shelldocs
0m  1s

Shelldocs was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

-1 :x:
test4tests
0m  0s

The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ trunk Compile Tests _

+0 :ok:
mvndep
15m 49s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
28m 35s

trunk passed

+1 :green_heart:
compile
25m 43s

trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
compile
22m 14s

trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07

+1 :green_heart:
mvnsite
21m 23s

trunk passed

+1 :green_heart:
javadoc
10m 12s

trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
8m 26s

trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07

+1 :green_heart:
shadedclient
41m 49s

branch has no errors when building and testing our client artifacts.

-0 :warning:
patch
42m 11s

Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 27s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
28m 39s

the patch passed

+1 :green_heart:
compile
24m 56s

the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javac
24m 56s

the patch passed

+1 :green_heart:
compile
22m 10s

the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07

+1 :green_heart:
javac
22m 10s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
mvnsite
20m 16s

the patch passed

+1 :green_heart:
shellcheck
0m  0s

No new issues.

+1 :green_heart:
javadoc
8m 20s

the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
7m 32s

the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07

+1 :green_heart:
shadedclient
40m 22s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

-1 :x:
unit
1033m 24s
/patch-unit-root.txt
root in the patch passed.

+1 :green_heart:
asflicense
1m 58s

The patch does not generate ASF License warnings.

1331m 30s

Reason
Tests

Failed junit tests
hadoop.hdfs.server.namenode.ha.TestObserverNode

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4991/5/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/4991

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint shellcheck shelldocs

uname
Linux 3cf6c144939e 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 4760a4dee3525d17f45f1398e91132af163be744

Default Java
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4991/5/testReport/

Max. process+thread count
3960 (vs. ulimit of 5500)

modules
C: hadoop-project hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient hadoop-mapreduce-project/hadoop-mapreduce-examples hadoop-mapreduce-project . U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4991/5/console

versions
git=2.25.1 maven=3.6.3 shellcheck=0.7.0

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop.hdfs.server.namenode.ha.TestObserverNode failure is unrelated. This UT is failing in many PRs as well.
JIRA for ref - https://issues.apache.org/jira/browse/HDFS-16801

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 56s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
detsecrets
0m  1s

detect-secrets was not available.

+0 :ok:
xmllint
0m  1s

xmllint was not available.

+0 :ok:
shelldocs
0m  1s

Shelldocs was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

-1 :x:
test4tests
0m  0s

The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ trunk Compile Tests _

+0 :ok:
mvndep
15m 41s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
29m 33s

trunk passed

+1 :green_heart:
compile
27m 17s

trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
compile
22m 12s

trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07

+1 :green_heart:
mvnsite
20m 23s

trunk passed

+1 :green_heart:
javadoc
8m 37s

trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
7m 32s

trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07

+1 :green_heart:
shadedclient
38m 51s

branch has no errors when building and testing our client artifacts.

-0 :warning:
patch
39m 13s

Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 29s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
28m 34s

the patch passed

+1 :green_heart:
compile
24m 57s

the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javac
24m 57s

the patch passed

+1 :green_heart:
compile
23m 34s

the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07

+1 :green_heart:
javac
23m 34s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
mvnsite
22m 22s

the patch passed

+1 :green_heart:
shellcheck
0m  0s

No new issues.

+1 :green_heart:
javadoc
8m 40s

the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
7m 30s

the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07

+1 :green_heart:
shadedclient
40m 16s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
976m 44s

root in the patch passed.

+1 :green_heart:
asflicense
2m  7s

The patch does not generate ASF License warnings.

1276m 49s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4991/6/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/4991

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint shellcheck shelldocs

uname
Linux ca37993ff5ae 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / ac3fb3151f49e4aa3a2acba072bdb310ec0bffa7

Default Java
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4991/6/testReport/

Max. process+thread count
3860 (vs. ulimit of 5500)

modules
C: hadoop-project hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient hadoop-mapreduce-project/hadoop-mapreduce-examples hadoop-mapreduce-project . U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4991/6/console

versions
git=2.25.1 maven=3.6.3 shellcheck=0.7.0

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

The only failing Yetus check is test4tests, as there are no new tests.  Existing tests have passed.  I have checked out the change and verified the dependency tree locally: it has the new org.hsqldb:hsqldb:jar:jdk8:2.7.1 everywhere, e.g.:
[INFO] org.apache.hadoop:hadoop-mapreduce:pom:3.4.0-SNAPSHOT
[INFO] \- org.hsqldb:hsqldb:jar:jdk8:2.7.1:compile

The only failing Yetus check is test4tests, as there are no new tests. Existing tests have passed. I have checked out the change and verified the dependency tree locally: it has the new org.hsqldb:hsqldb:jar:jdk8:2.7.1 everywhere, e.g.:
[INFO] org.apache.hadoop:hadoop-mapreduce:pom:3.4.0-SNAPSHOT
[INFO] \- org.hsqldb:hsqldb:jar:jdk8:2.7.1:compile

Thanks


Create master.yml

Added multiarch builds:
Essentially this will use buildx via qemu to build your image and then push it to dockerhub. You can duplicate this and change the branch in "branches" and in the bottom "run" command to have it work for any branch.
Add your dockerhub password and dockerhub username under DOCKERHUB_PASSWORD and DOCKERHUB_USERNAME in the repo's secrets to have it pushed.

I see that you're using drone.io to build this and they do not support buildx yet but it may be worth seeing if you can get similar results using github actions.

Docker images are now built using Github Actions.


Front camera flips photo

Photos took by front camera is flipped after saving.
Though this is the default iOS behavior, since there are lots of people who prefer their selfie not bing flipped, it'll be great if there is a property to flip the front camera photo when saving to photo library.

Same for me .. did you solve this ?

@FaridHaddad Created a new pull request resolving this issue.

Front camera flips video

I found movieOutput.connections always empty when movieOutput created, so func _setupVideoConnection call only when movieOutput is nil and setting mirror is not working, I change like this to work
fileprivate func _getMovieOutput() -> AVCaptureMovieFileOutput {
    if movieOutput == nil {
        _createMovieOutput()
    } else {
        _setupVideoConnection() // <--- add this
    }
    return movieOutput!
}


python buffer.append behaves strangely (compared to vimL append).

Steps to reproduce
CAVEAT:  I'm not certain this is a bug, but it seemed strange enough to report in case it's not on purpose.

Open test file with below text in it.

line1
line2
line3

Move to line 3 (with G).
:python3 vim.buffers[1].append(['hello', 'world'], 2)

Observation is that cursor ends up at start of hello on third line.
line1
line2
[h]ello
world
line3

Expected behaviour
I would have expected that the text is inserted, and cursor stays on the text line3.
line1
line2
hello
world
[l]ine3

This is what happens when calling vimL append(2, ['hello', 'world']), and also happens to be what neovim does with the python3 command above.
I thought it was what happened on earlier vim versions -- but have not checked that so could be misremembering.
Version of Vim
8.2.4919
Environment
Multiple environments.
Logs and stack traces
No response

Steps to reproduce
CAVEAT:  I'm not certain this is a bug, but it seemed strange enough to report in case it's not on purpose.

Open test file with below text in it.

line1
line2
line3

Move to line 3 (with G).
:python3 vim.buffers[1].append(['hello', 'world'], 2)

Observation is that cursor ends up at start of hello on third line.
line1
line2
[h]ello
world
line3

Expected behaviour
I would have expected that the text is inserted, and cursor stays on the text line3.
line1
line2
hello
world
[l]ine3

This is what happens when calling vimL append(2, ['hello', 'world']), and also happens to be what neovim does with the python3
command above.
I thought it was what happened on earlier vim versions -- but have
not checked that so could be misremembering.

I do not recall changes in this area, it must always have worked this
way.
Whether the cursor is moved when lines are inserted is a choice.
There is something to say for moving it or not moving it.  I can't say
that either is "right" or "wrong".  In cases like this it's better to
just leave it like it is, as changing it can only cause trouble.
Note that if marks are set at the lines, the marks do get moved.
--
hundred-and-one symptoms of being an internet addict:
67. Your hard drive crashes. You haven't logged in for two hours.  You start
to twitch. You pick up the phone and manually dial your ISP's access
number. You try to hum to communicate with the modem.  You succeed.
/// Bram Moolenaar -- @.*** -- http://www.Moolenaar.net   \
///                                                                      \
\\        sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ ///
\\            help me help AIDS victims -- http://ICCF-Holland.org    ///

so can we close it and live with the current behaviour?


Add basic LocalDateTime support WW-5175

Reviewed what I had done before, and here is another patch.
org.apache.struts2.components.Date.DATETAG_PROPERTY override, and localdate/localtime.

Coverage increased (+0.01%) to 50.634% when pulling 075ef7c4e16682cf3da3f7b2cedd7bcf13d0c0ad on gregh3269:More_localdatetime_support into 7e912742edd55c77cec37a9fbfbe077b683ebf83 on apache:master.

There are test errors with testLocalDateTime1Type on Java 9 and 11

jdk11 something to do with -Djava.locale.providers=COMPAT,CLDR.


More info about Selectors and SectionsToSkip

Thank you for this tool. I have it running in the meantime with 3 cron jobs (hourly, daily, weekly).
The readme is also quite comprehensive, but I'm still unsure which possibilities Selectors and SectionsToSkip do offer. The only example of Selectors is this:
  "Selectors" : { 	"http://www.fiio.me/forum.php?mod=viewthread&tid=39932" : { "Selector" : "td[id=postmessage_105396]", "SelectorsToSkip" : [ "ignore_js_op" ] }   },
Is there any in-depth documentation for Selectors and SectionsToSkip?
Thanks

Hello.
The implemented are really just regular CSS selectors (as available in other languages, javascript, and so on)..
Here this golang implementation is coming from the golang additional library "andybalholm/cascadia".
You can find additional examples / explanations in various regular areas like :

https://www.w3schools.com/cssref/css_selectors.asp
http://butlerccwebdev.net/support/css-selectors-cheatsheet.html
(most should work)

It's of course the same syntax here for "Selector" and "SelectorsToSkip". I had a few situation for which it was easier to filter (main "Selector" and then to exclude additional patterns (SelectorsToSkip) (which can of course be empty if not needed).

Great, thank you. Also for your quick answer.
Very helpful stuff, will dig into it.


converting LaneATT to TorchScript  Hi @dkhanna511,

Hi @dkhanna511,
Thanks for your response. Did you have any trouble when exporting the model in TorchScript format?! I got some errors when scripting the model. Could I ask you some questions about the process?!
Originally posted by @hthoai in https://github.com/lucastabelini/LaneATT/issues/30#issuecomment-801571909
Have you done converting LaneATT to TorchScript?

Please move this discussion to the correct issue (#77) and stop creating unnecessary new ones.


Create base VWSException

Hello,
First of all, thanks for the great package!
I'd like to propose a change regarding the custom exceptions.
Right now, all exceptions (exceptheh ConnectionErrorPossiblyImageTooLarge) extends directly from Exception. I think it's a good idea if you create a base exception, e.g. VWSException, and have all the other exceptions extend from it. This way, users can handle all exceptions that come from this package with a simple except VWSException (instead of the generic except Exception or specifying each exception).
If you agree, I can make a PR for this.
Thanks!

Hello @laymonage - do you have a use case for this, e.g. a case where you want to except VWSException? If so - go for it! Please make sure your changes are tested, too. I'm also happy to help out if needed.

Hi @adamtheturtle, one use case I have in mind is that I want to catch the exceptions that might arise from this package, but I don't want to handle them individually. Listing all the exceptions one by one is a bit handful since I don't really care what the exception is. I don't want to use Exception because that also handles exceptions that don't come from this package.
Other than that, I think it's just nice to have a base VWSException so that all the exceptions in (and only in) this package have the same parent.
I can try creating it in the next few days.

Sounds good @laymonage .

And to clarify:

Listing all the exceptions one by one is a bit handful since I don't really care what the exception is

This sounds like a good motivation - less code for your own codebase, and I'll assume that if it is useful for you it is useful for others.

Other than that, I think it's just nice to have a base VWSException so that all the exceptions in (and only in) this package have the same parent.

This one is not a useful motivation for me as I prefer to work on concrete use cases.


LuckPerms doesn't detect a user if it has - in nickname

I tried to add my friend to a group with /lp user Abu-Ha group add "group name" and i got "invalid nickname/uuid" error, i tried samething using console and i got same error again. I used the /lp editor to add my friend to group, my other friend and my nickname doesn't have a - symbol and i didn't got a error when i am adding myself and my other friend to a group.

The LP config has an option to allow/accept invalid usernames - try enabling that


Remove Nano Adblocker & Nano Defender from list.

With the news of the recent transfer of ownership by the developer, as seen here:
https://github.com/jspenguin2017/Snippets/issues/2
Along with the recent discovery by gorhill about how the new updates of the programs are now collecting user data:
https://github.com/NanoAdblocker/NanoCore/issues/362#issuecomment-709428210
I would like to request the removal of these programs from this list.

Nano Adblocker can obviously be replaced with uBlock Origin.
On FIREFOX, Nano Defender is still safe (due to having a different maintainer), but on Chrome or Edge what is the suggested replacement for Nano Defender?

Nano Adblocker can obviously be replaced with uBlock Origin.
On FIREFOX, Nano Defender is still safe (due to having a different maintainer), but on Chrome or Edge what is the suggested replacement for Nano Defender?

You can user uBlock Origin on Chrome, Brave, Vivaldi, etc..

Yes, but what about a replacement for Nano DEFENDER (which defeats many anti-adblocker issues on common sites)?

Yes, but what about a replacement for Nano DEFENDER (which defeats many anti-adblocker issues on common sites)?

Can I bring up filterlists.com for possible filter list suggestions regarding adblock defusion?

Yes, but what about a replacement for Nano DEFENDER (which defeats many anti-adblocker issues on common sites)?

uBlock Origin already takes care of anti-adblock.

Nano Adblocker can obviously be replaced with uBlock Origin.
On FIREFOX, Nano Defender is still safe (due to having a different maintainer), but on Chrome or Edge what is the suggested replacement for Nano Defender?

I did not know that Nano Defender has a different maintainer in Firefox.
In that case, just put a note to only use Nano Defender in Firefox and not in Chrome.

The (new) Nano Defender has officially been deemed malware by Google.

I trusted Nano Adblocker & Nano Defender and now they are infected by malware according to Google. So what are the recommended procedures now, basically change all our used passwords or they didn't compromise such data? What exactly were they spying? Just the sites we browsed, texts we may have typed or something more dangerous such as login credentials as well?
Planet Earth keeps being a difficult place to live, our trust gets broken regularly in so many things.

@givemeatip i will leave you with this link to this comment and follow the rest of comments there https://github.com/jspenguin2017/Snippets/issues/2#issuecomment-712469408

Removed them both. Thanks for flagging!

Yes, but what about a replacement for Nano DEFENDER (which defeats many anti-adblocker issues on common sites)?

In 99.99% of all possible cases, you don't need anything other than uBlock Origin.
So this question is totally moot.


Certain arrow functions are unsupported

As mentioned in #190, I noticed arrow functions like the following seems unsupported:
const arrow: (n: number) => number = (n: number) => 1;

Looked into this and it seems if (n: number) => number is omitted NaNoFuzz would ignore arrow since it doesn't have typing information, and when it is there NaNoFuzz doesn't support arrow types?

Does it work ok if the const is exported, e.g.,

Another random thought - might be handy if NaNoFuzz still shows the codelens for non-exported functions, and upon opening automatically exports the function using something like
export { arrow as arrowExportedForTesting }
which is the convention my previous employee uses, but not sure if that has any compatibility issues or is universally acceptable?

Tend to agree here. More generally would like the UI to provide feedback about why a function is not testable (e.g., not exported) and, as you say, suggest a quick fix if that might be appropriate.


List of potential improvements

I very briefly looked through the code. Here are a few of the things I noticed that are specifically related to the integration with Shopkeepers:

Try to reduce your dependency from the complete Shopkeepers-2.13.3.jar to only the API portion ShopkeepersAPI. This artifact can be found in this Maven repository: https://nexus.lichtspiele.org/repository/releases/. Even if you depend on the complete Shopkeepers jar, try to only use classes and interfaces from the api packages. All other classes are likely to change between plugin updates. Even the API classes are still rather unstable, but they should become more stable over time.
Commands class and InvUtils class: The SKShopkeepersPlugin import does not seem to be used for anything in those classes. And if it is used for something, try to avoid using it, since it is not part of the API portion of Shopkeepers.
You are using SKShopkeepersPlugin.getInstance() for all your Bukkit Scheduler tasks (e.g. in GuiListeners and in ShopGUI). Don't. Instead, use your own plugin instance for those tasks (i.e. your Main class instance).
ShopkeeperSorter: Again, don't use SKShopkeepersPlugin. Instead, you can retrieve the ShopkeeperRegistry via ShopkeepersAPI.getShopkeeperRegistry().
ShopkeeperSorter#getAllShopOwners owners, ShopkeeperSorter#shopkeeperItemStacks values, ShopkeeperSorter#sellingItemStacks: You are currently using ArrayLists for those. Probably because you require access by index inside ShopGUI. However, you are currently also doing many containment checks on these collections, for which a Hash-based data structure would be better suited. An idea could be to have both: One Hash-based data structure for efficient containment checks, and a consistently maintained ArrayList based structure for your GUI operations.
ShopkeeperSorter: All your update methods (updateVisualRepresentationOfShopkeepers, updateShopkeeperItemstacks, updateSellingItemStacks) are quite wasteful performance-wise. Especially since you invoke them repeatedly (see GuiListeners), whenever a shopkeeper changes, is removed, or a player joins. Try to optimize this to not have to recalculate all data freshly on every minor shopkeeper change. Instead, try to only update those portions that might actually be affected by a change. E.g. if a shopkeeper is removed or edited, try to only update the pieces of data that are affected by this particular shopkeeper. And ideally build your data structures in a way that allows you to update this information for a specific shopkeeper efficiently, without having to iterate over all shopkeepers or over all of your cached data to check if one element is affected, etc.
ShopkeeperSorter#getAllAdminShopkeepers: The result of this can probably also be cached instead of being recalculated every time. You only need to add or remove a shopkeeper from your cache whenever a new admin shopkeeper is added to the registry (ShopkeeperAddedEvent) or removed (ShopkeeperRemoveEvent).
ShopkeeperSorter: You are using Shopkeeper#getTradingRecipes(Player) to retrieve (and cache) information about the traded items. There are a few minor problems with this.

Minor (not an issue right now, but maybe in the future): In principle, this is allowed to return random or periodically changing results everytime you call it (e.g. some custom type of shopkeeper that periodically changes its trades; or a custom type of shopkeeper that offers each player a randomly picked list of trades). Right now I am not aware of such shopkeeper implementations, but this is a future possibility. Unfortunately, there is no general solution then to retrieve the trade offers of any kind of shopkeeper. Instead, you will probably have to account for each type of shopkeeper individually, and ignore any unknown shopkeeper types (e.g. custom ones like I mentioned above).
Minor (not an issue right now, but maybe in the future): You are using one specific player instance to retrieve the trading recipes for. Your cached information is then based on this specific player instance. Although this is not relevant right now, because none of the currently built-in types of shopkeepers actually use the player argument, in principle the information you get is not suited for all players on the server. E.g. a custom shopkeeper implementation might offer different players different trades. You would probably need to either ignore these (unknown) types of shopkeepers, or split your cache into 2 portions, one that is player-unspecific, and one that is specific to the player viewing the inventory menu. Though, I don't think that there is a way currently to check if a shopkeeper might return player-specific trade offers..
Related to that: Since there are currently no shop types with player-specific trading recipes yet, nor is your cache properly accounting for this future possibility anyway, you can probably just omit the player argument for now when you call #getTradingRecipes(player). I.e. simply call #getTradingRecipes(null). This will also simplify a few of your update methods, since they no longer require a player argument.

Thanks for taking your time and suggesting improvements. I will be definitely making the changes. Also one thing to note The cache update only happens when a shopkeeper is edited or when the first player joins the server after start and i wont be requiring this method as you mentioned i can pass null instead of having to pass a player.


view! failing when attribute name ends in -ref

Describe the bug
To integrate with Tailwind Elements, I need to create div (and other) elements with a data-te-dropdown-ref tag. I tried:
view! { cx,
  div(class="relative", data-te-dropdown-ref=true) {}
}

but this fails the perseus serve with:
error: expected identifier
 --> src/components/twelements/dropdown.rs:7:48
  |
7 |         div(class="relative", data-te-dropdown-ref=true) {}
  |                                                ^^^

Environment

Sycamore: 0.8.1 with Perseus 0.4.2
OS: Linux

This should be a pretty easy fix in the view! macro. Right now, we just parse attributes as a sequence of identifiers but ref is a keyword. Instead, we just need to use the Ident::parse_any function from syn instead of just Ident::parse.