HADOOP-17833. Improve Magic Committer performance

Speeding up the magic committer with key changes being

All writes under __magic trigger marker retention
(no DELETEs after file/dir creation)
create(path, overwrite) skips all overwrite checks, including
the LIST call intended to stop files being created over dirs
thread pool used for more parallelism in task commit.
open() of .pending/pendingset files will skip HEAD calls if going straight from dir listing; will save 1 HEAD/task in job commit

This is still WiP as it needs

cost tests to verify the optimisations are active
testing through spark

Lots of changes in the tests because the committer has added
a CommitContext class which manages the lifecycle of
the thread pool and a set of thread local JSON serializers;
this is now what is passed around in internal committer methods,
so breaking tests calling in to them.
It is a better design (one we should have done from the start);
manifest committer is even better as all its operations "stages"
are modular. Just means that a lot of tests stopped compiling.
And as usual, mock tests played up.
Removed the injection/handling of inconsistent S3
from the committer tests. Not needed, and simply complicating
the code needlessly.

Incremental listings of directories with processing as the pages of results come in
Maximised parallelism parsing and processing files.
per thread json serializers; no sharing or repeated creation.
when creating files under __magic paths, all safety checks (overwriting files, overwriting dirs) are skipped. This benefits parquet files even more than most as it seems to be saving files with overwrite=false
We also skip deleting marker paths even if the FS isn't configured to do this, knowing that in job commit all these markers will be purged. This behaviour is also possible through the createFile() API, which is used when saving the intermediate manifests and the _SUCCESS file.

The write optimisations should provide significant benefits when writing files: at least one LIST per creation, for parquet a HEAD too, and O(depth) delete calls which just generates write load, risk of throttling etc. There is still more IO taking place for each magic file than writing a small simple file (it is always multipart and we have to add a marker file too). Spark also has to add an extra getXAttrs call for each file when building its intermediate stats.
Also of note

The committer can write the summary _SUCCESS file to the path fs.s3a.committer.summary.report.directory, which can be in a different file system/bucket if desired, with the jobid as the filename. This can be used to collect all statistics of jobs even though switch over right at the same directory tree. This is the same as the Manifest Committer -that just collects more stats on its operations.
There's reuse of the hadoop common code and statistic names from the ManifestCommitter.
The hadoop-aws maven build blocks all import of mapreduce code in the production source except in the s3a.committer source tree, and even there we are selective and exclude all classes we know get referenced elsewhere.

TESTING IN PROGRESS

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
1m  3s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 18 new or modified test files.

_ trunk Compile Tests _

-1 :x:
mvninstall
1m 45s
/branch-mvninstall-root.txt
root in trunk failed.

+1 :green_heart:
compile
4m  4s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
compile
0m 32s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
checkstyle
0m 23s

trunk passed

+1 :green_heart:
mvnsite
0m 37s

trunk passed

+1 :green_heart:
javadoc
0m 20s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
0m 28s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
1m  6s

trunk passed

-1 :x:
shadedclient
7m 12s

branch has errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 35s

the patch passed

+1 :green_heart:
compile
0m 38s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javac
0m 39s

the patch passed

+1 :green_heart:
compile
0m 31s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
javac
0m 31s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
0m 21s
/results-checkstyle-hadoop-tools_hadoop-aws.txt
hadoop-tools/hadoop-aws: The patch generated 12 new + 25 unchanged - 1 fixed = 37 total (was 26)

+1 :green_heart:
mvnsite
0m 35s

the patch passed

+1 :green_heart:
javadoc
0m 17s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

-1 :x:
javadoc
0m 24s
/patch-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt
hadoop-aws in the patch failed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.

+1 :green_heart:
spotbugs
1m 10s

the patch passed

+1 :green_heart:
shadedclient
14m 35s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

-1 :x:
unit
0m 17s
/patch-unit-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch failed.

+1 :green_heart:
asflicense
0m 34s

The patch does not generate ASF License warnings.

38m 33s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3289

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell

uname
Linux bd273600a69a 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 208929ef08592d69309263588eb7fd01675c9229

Default Java
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/1/testReport/

Max. process+thread count
549 (vs. ulimit of 5500)

modules
C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/1/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 57s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 18 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
30m 51s

trunk passed

+1 :green_heart:
compile
0m 46s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
compile
0m 40s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
checkstyle
0m 31s

trunk passed

+1 :green_heart:
mvnsite
0m 47s

trunk passed

+1 :green_heart:
javadoc
0m 27s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
0m 35s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
1m  9s

trunk passed

+1 :green_heart:
shadedclient
14m 59s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 37s

the patch passed

+1 :green_heart:
compile
0m 38s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javac
0m 38s

the patch passed

+1 :green_heart:
compile
0m 31s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
javac
0m 31s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
0m 19s
/results-checkstyle-hadoop-tools_hadoop-aws.txt
hadoop-tools/hadoop-aws: The patch generated 13 new + 25 unchanged - 1 fixed = 38 total (was 26)

+1 :green_heart:
mvnsite
0m 36s

the patch passed

+1 :green_heart:
javadoc
0m 15s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
0m 25s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
1m 13s

the patch passed

+1 :green_heart:
shadedclient
14m  9s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

-1 :x:
unit
2m 48s
/patch-unit-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
0m 33s

The patch does not generate ASF License warnings.

74m 51s

Reason
Tests

Failed junit tests
hadoop.fs.s3a.commit.staging.TestDirectoryCommitterScale

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/2/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3289

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell

uname
Linux 11c1188c953b 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / a2166e1361641af251cc23a89a15adebf95f6fdf

Default Java
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/2/testReport/

Max. process+thread count
622 (vs. ulimit of 5500)

modules
C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/2/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

+plan to lift some of the statistic names from the manifest committer and do the same reporting as in manifest committer. will also include list costs in results. (side issue, thinking of whether the json deserializer could build stats on reading costs, which can then be collected too to measure cost of ser/deser and, by collecting stream read/write costs, those steps

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
1m  4s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 18 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
31m 19s

trunk passed

+1 :green_heart:
compile
0m 47s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
compile
0m 40s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
checkstyle
0m 32s

trunk passed

+1 :green_heart:
mvnsite
0m 47s

trunk passed

+1 :green_heart:
javadoc
0m 27s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
0m 36s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
1m 10s

trunk passed

+1 :green_heart:
shadedclient
14m 20s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 37s

the patch passed

+1 :green_heart:
compile
0m 38s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javac
0m 38s

the patch passed

+1 :green_heart:
compile
0m 31s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
javac
0m 31s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
0m 21s
/results-checkstyle-hadoop-tools_hadoop-aws.txt
hadoop-tools/hadoop-aws: The patch generated 13 new + 25 unchanged - 1 fixed = 38 total (was 26)

+1 :green_heart:
mvnsite
0m 35s

the patch passed

+1 :green_heart:
javadoc
0m 16s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
0m 26s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
1m 11s

the patch passed

+1 :green_heart:
shadedclient
14m 15s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

-1 :x:
unit
2m 43s
/patch-unit-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
0m 34s

The patch does not generate ASF License warnings.

75m  5s

Reason
Tests

Failed junit tests
hadoop.fs.s3a.commit.staging.TestDirectoryCommitterScale

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/3/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3289

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell

uname
Linux 5ca76b4e62d8 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / d6a0dcf41e78bf5b8155d0f49d0aec66224168fa

Default Java
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/3/testReport/

Max. process+thread count
698 (vs. ulimit of 5500)

modules
C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/3/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 56s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 18 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
33m 27s

trunk passed

+1 :green_heart:
compile
0m 44s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
compile
0m 36s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
checkstyle
0m 26s

trunk passed

+1 :green_heart:
mvnsite
0m 41s

trunk passed

+1 :green_heart:
javadoc
0m 22s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
0m 29s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
1m  9s

trunk passed

+1 :green_heart:
shadedclient
17m  0s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 34s

the patch passed

+1 :green_heart:
compile
0m 38s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javac
0m 38s

the patch passed

+1 :green_heart:
compile
0m 30s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
javac
0m 30s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
0m 19s
/results-checkstyle-hadoop-tools_hadoop-aws.txt
hadoop-tools/hadoop-aws: The patch generated 13 new + 25 unchanged - 1 fixed = 38 total (was 26)

+1 :green_heart:
mvnsite
0m 35s

the patch passed

+1 :green_heart:
javadoc
0m 15s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
0m 23s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
1m 12s

the patch passed

+1 :green_heart:
shadedclient
16m 38s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

-1 :x:
unit
2m 32s
/patch-unit-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
0m 29s

The patch does not generate ASF License warnings.

80m 48s

Reason
Tests

Failed junit tests
hadoop.fs.s3a.commit.staging.TestDirectoryCommitterScale

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/4/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3289

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell

uname
Linux<PHONE_NUMBER>4e 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / f85002192c2d940d5e911747f3ca3e84c7393ccb

Default Java
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/4/testReport/

Max. process+thread count
598 (vs. ulimit of 5500)

modules
C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/4/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
1m 12s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 20 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
38m 55s

trunk passed

+1 :green_heart:
compile
0m 58s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
compile
0m 44s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
checkstyle
0m 32s

trunk passed

+1 :green_heart:
mvnsite
0m 51s

trunk passed

+1 :green_heart:
javadoc
0m 25s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
0m 36s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
1m 21s

trunk passed

+1 :green_heart:
shadedclient
19m  5s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 34s

the patch passed

+1 :green_heart:
compile
0m 37s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javac
0m 37s

the patch passed

+1 :green_heart:
compile
0m 30s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
javac
0m 30s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
0m 19s
/results-checkstyle-hadoop-tools_hadoop-aws.txt
hadoop-tools/hadoop-aws: The patch generated 5 new + 25 unchanged - 1 fixed = 30 total (was 26)

+1 :green_heart:
mvnsite
0m 35s

the patch passed

+1 :green_heart:
javadoc
0m 14s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

-1 :x:
javadoc
0m 24s
/results-javadoc-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt
hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 generated 2 new + 63 unchanged - 0 fixed = 65 total (was 63)

+1 :green_heart:
spotbugs
1m 12s

the patch passed

+1 :green_heart:
shadedclient
16m 56s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 30s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
0m 30s

The patch does not generate ASF License warnings.

89m 47s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/5/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3289

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell

uname
Linux 0aac55373e8b 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / e3dc6e416309b07ae015f5ec7c07a5fb0784cc5d

Default Java
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/5/testReport/

Max. process+thread count
522 (vs. ulimit of 5500)

modules
C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/5/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 48s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 24 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
35m 32s

trunk passed

+1 :green_heart:
compile
0m 54s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
compile
0m 39s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
checkstyle
0m 33s

trunk passed

+1 :green_heart:
mvnsite
0m 53s

trunk passed

+1 :green_heart:
javadoc
0m 29s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
0m 38s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
1m 24s

trunk passed

+1 :green_heart:
shadedclient
19m 11s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 42s

the patch passed

+1 :green_heart:
compile
0m 44s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

-1 :x:
javac
0m 44s
/results-compile-javac-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.txt
hadoop-tools_hadoop-aws-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 generated 5 new + 19 unchanged - 2 fixed = 24 total (was 21)

+1 :green_heart:
compile
0m 36s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

-1 :x:
javac
0m 36s
/results-compile-javac-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt
hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 generated 5 new + 19 unchanged - 2 fixed = 24 total (was 21)

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
0m 23s
/results-checkstyle-hadoop-tools_hadoop-aws.txt
hadoop-tools/hadoop-aws: The patch generated 20 new + 27 unchanged - 1 fixed = 47 total (was 28)

+1 :green_heart:
mvnsite
0m 42s

the patch passed

+1 :green_heart:
javadoc
0m 17s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
0m 28s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
1m 28s

the patch passed

+1 :green_heart:
shadedclient
19m 20s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 43s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
0m 34s

The patch does not generate ASF License warnings.

89m 49s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/6/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3289

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell

uname
Linux 83c7e47ad7bf 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / e293343cc73b3d16dbb03ef8ca74737affcf3051

Default Java
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/6/testReport/

Max. process+thread count
705 (vs. ulimit of 5500)

modules
C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/6/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
1m  2s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 25 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
12m 41s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
24m 39s

trunk passed

+1 :green_heart:
compile
24m 35s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
compile
20m 57s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
checkstyle
3m 49s

trunk passed

+1 :green_heart:
mvnsite
2m 24s

trunk passed

+1 :green_heart:
javadoc
1m 30s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
2m 13s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
3m 37s

trunk passed

+1 :green_heart:
shadedclient
17m 18s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 21s

Maven dependency ordering for patch

-1 :x:
mvninstall
0m 20s
/patch-mvninstall-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch failed.

-1 :x:
compile
20m 42s
/patch-compile-root-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.txt
root in the patch failed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.

-1 :x:
javac
20m 42s
/patch-compile-root-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.txt
root in the patch failed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.

-1 :x:
compile
18m 22s
/patch-compile-root-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt
root in the patch failed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.

-1 :x:
javac
18m 22s
/patch-compile-root-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt
root in the patch failed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.

-1 :x:
blanks
0m  1s
/blanks-eol.txt
The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply

-0 :warning:
checkstyle
3m 50s
/results-checkstyle-root.txt
root: The patch generated 14 new + 28 unchanged - 1 fixed = 42 total (was 29)

-1 :x:
mvnsite
0m 36s
/patch-mvnsite-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch failed.

+1 :green_heart:
javadoc
1m 28s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
2m 13s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

-1 :x:
spotbugs
0m 34s
/patch-spotbugs-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch failed.

+1 :green_heart:
shadedclient
18m 32s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
16m 57s

hadoop-common in the patch passed.

-1 :x:
unit
0m 35s
/patch-unit-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch failed.

+1 :green_heart:
asflicense
0m 48s

The patch does not generate ASF License warnings.

206m 59s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/7/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3289

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell markdownlint

uname
Linux d315b1bd2163 4.15.0-153-generic #160-Ubuntu SMP Thu Jul 29 06:54:29 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / ade05b1fca5bcb7d5446fef87e061101e3eb4e93

Default Java
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/7/testReport/

Max. process+thread count
2894 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/7/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

This is at a point where it's ready for some review and any benchmarking people can do. I've cut out a lot of HTTP IO per file create and load
@mukund-thakur @mehakmeet @dongjoon-hyun @bogthe
The only other big thing to consider is could we route the parallel POST calls in job commit through a fork/join thread pool and does that deliver better throughput due to the fact there's no need to yield to the OS scheduler to pick up the next bit of work.
I am happy to do a live shared screen review of this PR next week, if people want do discuss things that way

Thank you, @steveloughran !

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
1m 28s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
markdownlint
0m  1s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 26 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
12m 46s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
27m 52s

trunk passed

+1 :green_heart:
compile
28m 26s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
compile
24m  6s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
checkstyle
4m 21s

trunk passed

+1 :green_heart:
mvnsite
2m 51s

trunk passed

+1 :green_heart:
javadoc
1m 48s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
2m 35s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
4m 28s

trunk passed

+1 :green_heart:
shadedclient
19m 34s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 25s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
1m 50s

the patch passed

+1 :green_heart:
compile
27m  5s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

-1 :x:
javac
27m  5s
/results-compile-javac-root-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.txt
root-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 generated 3 new + 1914 unchanged - 2 fixed = 1917 total (was 1916)

+1 :green_heart:
compile
23m 58s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

-1 :x:
javac
23m 58s
/results-compile-javac-root-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt
root-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 generated 3 new + 1791 unchanged - 2 fixed = 1794 total (was 1793)

-1 :x:
blanks
0m  0s
/blanks-eol.txt
The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply

-0 :warning:
checkstyle
4m 23s
/results-checkstyle-root.txt
root: The patch generated 2 new + 28 unchanged - 1 fixed = 30 total (was 29)

+1 :green_heart:
mvnsite
2m 36s

the patch passed

+1 :green_heart:
javadoc
1m 43s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

-1 :x:
javadoc
0m 42s
/patch-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt
hadoop-aws in the patch failed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.

+1 :green_heart:
spotbugs
5m 10s

the patch passed

+1 :green_heart:
shadedclient
19m 25s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
17m  8s

hadoop-common in the patch passed.

+1 :green_heart:
unit
2m 55s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
0m 50s

The patch does not generate ASF License warnings.

244m 26s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/9/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3289

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell markdownlint

uname
Linux 6e568f6fa2e1 4.15.0-153-generic #160-Ubuntu SMP Thu Jul 29 06:54:29 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 9d372118209e7d1e2c11cc0e881491d15c15d1b3

Default Java
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/9/testReport/

Max. process+thread count
1764 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/9/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/auth/delegation/ITestSessionDelegationTokens.java:207:6:[deprecation] <T>assertThat(String,T,Matcher<? super T>) in Assert has been deprecated
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/auth/delegation/ITestSessionDelegationInFileystem.java:260:44:[unchecked] unchecked cast
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3ATemporaryCredentials.java:204:6:[deprecation] <T>assertThat(String,T,Matcher<? super T>) in Assert has been deprecated

Data has 42 rows clustered true for 20000000
Generating table call_center in database to s3a://perf-team-west1-bucket/perf-team-data/tpcds/magic2/sf1000-parquet/useDecimal=true,useDate=true,filterNull=false/call_center with save mode Overwrite.
java.lang.NullPointerException
  at org.apache.hadoop.fs.s3a.commit.CommitContext.<init>(CommitContext.java:128)
  at org.apache.hadoop.fs.s3a.commit.CommitOperations.createCommitContext(CommitOperations.java:658)
  at org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter.initiateJobOperation(AbstractS3ACommitter.java:796)
  at org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter.abortJob(AbstractS3ACommitter.java:840)
  at org.apache.spark.internal.io.HadoopMapReduceCommitProtocol.abortJob(HadoopMapReduceCommitProtocol.scala:224)
  at org.apache.spark.internal.io.cloud.PathOutputCommitProtocol.abortJob(PathOutputCommitProtocol.scala:206)
  at org.apache.spark.sql.execution.datasources.FileFormatWriter$.write(FileFormatWriter.scala:202)
  at org.apache.spark.sql.execution.datasources.InsertIntoHadoopFsRelationCommand.run(InsertIntoHadoopFsRelationCommand.scala:169)
  at org.apache.spark.sql.execution.command.DataWritingCommandExec.sideEffectResult$lzycompute(commands.scala:104)
  at org.apache.spark.sql.execution.command.DataWritingCommandExec.sideEffectResult(commands.scala:102)
  at org.apache.spark.sql.execution.command.DataWritingCommandExec.doExecute(commands.scala:122)
  at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:141)
  at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:137)
  at org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:165)
  at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
13m 21s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 24 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
15m 53s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
24m 55s

trunk passed

+1 :green_heart:
compile
22m 53s

trunk passed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
compile
20m  6s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
checkstyle
3m 44s

trunk passed

+1 :green_heart:
mvnsite
2m 42s

trunk passed

+1 :green_heart:
javadoc
1m 57s

trunk passed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javadoc
2m 34s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
4m  5s

trunk passed

+1 :green_heart:
shadedclient
21m 28s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 27s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
1m 32s

the patch passed

+1 :green_heart:
compile
22m 14s

the patch passed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.20.04

-1 :x:
javac
22m 14s
/results-compile-javac-root-jdkUbuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.20.04.txt
root-jdkUbuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.20.04 with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.20.04 generated 1 new + 1817 unchanged - 0 fixed = 1818 total (was 1817)

+1 :green_heart:
compile
19m 54s

the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

-1 :x:
javac
19m 54s
/results-compile-javac-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt
root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 1 new + 1689 unchanged - 0 fixed = 1690 total (was 1689)

-1 :x:
blanks
0m  0s
/blanks-eol.txt
The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply

-0 :warning:
checkstyle
3m 38s
/results-checkstyle-root.txt
root: The patch generated 7 new + 18 unchanged - 2 fixed = 25 total (was 20)

+1 :green_heart:
mvnsite
2m 39s

the patch passed

+1 :green_heart:
javadoc
1m 52s

the patch passed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.20.04

-1 :x:
javadoc
0m 49s
/results-javadoc-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt
hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 3 new + 39 unchanged - 0 fixed = 42 total (was 39)

-1 :x:
spotbugs
1m 32s
/new-spotbugs-hadoop-tools_hadoop-aws.html
hadoop-tools/hadoop-aws generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)

+1 :green_heart:
shadedclient
21m 38s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
17m 49s

hadoop-common in the patch passed.

+1 :green_heart:
unit
2m 52s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
1m  2s

The patch does not generate ASF License warnings.

239m 50s

Reason
Tests

SpotBugs
module:hadoop-tools/hadoop-aws

Inconsistent synchronization of org.apache.hadoop.fs.s3a.commit.CommitContext.innerSubmitter; locked 50% of time  Unsynchronized access at CommitContext.java:50% of time  Unsynchronized access at CommitContext.java:[line 333]

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/10/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3289

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell markdownlint

uname
Linux 1112f003275e 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 30748c9e21124740c0c7b3ba99d0401368f82c54

Default Java
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/10/testReport/

Max. process+thread count
1267 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/10/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 38s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
markdownlint
0m  1s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 29 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
15m 59s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
25m 16s

trunk passed

+1 :green_heart:
compile
22m 54s

trunk passed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
compile
20m  7s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
checkstyle
3m 48s

trunk passed

+1 :green_heart:
mvnsite
3m 44s

trunk passed

+1 :green_heart:
javadoc
2m 42s

trunk passed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javadoc
3m 14s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
5m 44s

trunk passed

+1 :green_heart:
shadedclient
21m 29s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 27s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
2m  9s

the patch passed

+1 :green_heart:
compile
22m 20s

the patch passed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javac
22m 20s

the patch passed

+1 :green_heart:
compile
20m 11s

the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
javac
20m 11s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
3m 36s
/results-checkstyle-root.txt
root: The patch generated 9 new + 20 unchanged - 2 fixed = 29 total (was 22)

+1 :green_heart:
mvnsite
3m 39s

the patch passed

+1 :green_heart:
javadoc
2m 36s

the patch passed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.20.04

-1 :x:
javadoc
0m 48s
/results-javadoc-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt
hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 4 new + 39 unchanged - 0 fixed = 43 total (was 39)

-1 :x:
spotbugs
1m 32s
/new-spotbugs-hadoop-tools_hadoop-aws.html
hadoop-tools/hadoop-aws generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)

+1 :green_heart:
shadedclient
21m 40s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
17m 59s

hadoop-common in the patch passed.

+1 :green_heart:
unit
6m 17s

hadoop-mapreduce-client-core in the patch passed.

+1 :green_heart:
unit
2m 46s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
1m  3s

The patch does not generate ASF License warnings.

243m 43s

Reason
Tests

SpotBugs
module:hadoop-tools/hadoop-aws

Inconsistent synchronization of org.apache.hadoop.fs.s3a.commit.CommitContext.outerSubmitter; locked 60% of time  Unsynchronized access at CommitContext.java:60% of time  Unsynchronized access at CommitContext.java:[line 291]

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/11/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3289

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell markdownlint

uname
Linux 3cc258694db0 4.15.0-169-generic #177-Ubuntu SMP Thu Feb 3 10:50:38 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / dd5ef821cb82339894a8f823e8be713224a7330f

Default Java
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/11/testReport/

Max. process+thread count
1576 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/11/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 49s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  2s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 29 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
15m 54s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
26m  0s

trunk passed

+1 :green_heart:
compile
24m 27s

trunk passed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
compile
21m 41s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
checkstyle
4m 30s

trunk passed

+1 :green_heart:
mvnsite
4m 54s

trunk passed

-1 :x:
javadoc
1m 45s
/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.20.04.txt
hadoop-common in trunk failed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.20.04.

+1 :green_heart:
javadoc
4m 50s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
6m 59s

trunk passed

+1 :green_heart:
shadedclient
22m 59s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 29s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
2m 29s

the patch passed

+1 :green_heart:
compile
24m  8s

the patch passed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javac
24m  8s

the patch passed

+1 :green_heart:
compile
21m 38s

the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
javac
21m 38s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
4m 20s
/results-checkstyle-root.txt
root: The patch generated 10 new + 20 unchanged - 2 fixed = 30 total (was 22)

+1 :green_heart:
mvnsite
5m 17s

the patch passed

-1 :x:
javadoc
1m 33s
/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.20.04.txt
hadoop-common in the patch failed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.20.04.

-1 :x:
javadoc
1m  9s
/results-javadoc-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt
hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 5 new + 38 unchanged - 0 fixed = 43 total (was 38)

-1 :x:
spotbugs
2m  3s
/new-spotbugs-hadoop-tools_hadoop-aws.html
hadoop-tools/hadoop-aws generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)

+1 :green_heart:
shadedclient
23m 29s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
18m 47s

hadoop-common in the patch passed.

+1 :green_heart:
unit
6m 45s

hadoop-mapreduce-client-core in the patch passed.

+1 :green_heart:
unit
3m 11s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
1m 38s

The patch does not generate ASF License warnings.

270m  3s

Reason
Tests

SpotBugs
module:hadoop-tools/hadoop-aws

Inconsistent synchronization of org.apache.hadoop.fs.s3a.commit.CommitContext.outerSubmitter; locked 60% of time  Unsynchronized access at CommitContext.java:60% of time  Unsynchronized access at CommitContext.java:[line 291]

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/12/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3289

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell markdownlint

uname
Linux 435a1b7488cd 4.15.0-169-generic #177-Ubuntu SMP Thu Feb 3 10:50:38 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 87ae7e5515e4203826dd0a7eb89a9e2fc50bdade

Default Java
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/12/testReport/

Max. process+thread count
2828 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/12/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 44s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  2s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 29 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
16m 35s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
25m 19s

trunk passed

+1 :green_heart:
compile
23m 12s

trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
compile
20m 39s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
checkstyle
4m 25s

trunk passed

+1 :green_heart:
mvnsite
5m 20s

trunk passed

-1 :x:
javadoc
1m 52s
/branch-javadoc-hadoop-common-project_hadoop-common-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.txt
hadoop-common in trunk failed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.

+1 :green_heart:
javadoc
4m 49s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
7m 10s

trunk passed

+1 :green_heart:
shadedclient
22m 17s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 31s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
2m 30s

the patch passed

+1 :green_heart:
compile
22m 20s

the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

-1 :x:
javac
22m 20s
/results-compile-javac-root-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.txt
root-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS> with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS> generated 1 new + 1811 unchanged - 1 fixed = 1812 total (was 1812)

+1 :green_heart:
compile
20m 37s

the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

-1 :x:
javac
20m 37s
/results-compile-javac-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt
root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 1 new + 1688 unchanged - 1 fixed = 1689 total (was 1689)

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
4m 13s
/results-checkstyle-root.txt
root: The patch generated 10 new + 20 unchanged - 2 fixed = 30 total (was 22)

+1 :green_heart:
mvnsite
5m 16s

the patch passed

-1 :x:
javadoc
1m 43s
/patch-javadoc-hadoop-common-project_hadoop-common-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.txt
hadoop-common in the patch failed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.

-1 :x:
javadoc
1m 20s
/results-javadoc-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt
hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 5 new + 38 unchanged - 0 fixed = 43 total (was 38)

-1 :x:
spotbugs
2m  6s
/new-spotbugs-hadoop-tools_hadoop-aws.html
hadoop-tools/hadoop-aws generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)

+1 :green_heart:
shadedclient
22m 29s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
18m 27s

hadoop-common in the patch passed.

+1 :green_heart:
unit
6m 48s

hadoop-mapreduce-client-core in the patch passed.

+1 :green_heart:
unit
3m 10s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
1m 38s

The patch does not generate ASF License warnings.

265m 20s

Reason
Tests

SpotBugs
module:hadoop-tools/hadoop-aws

Inconsistent synchronization of org.apache.hadoop.fs.s3a.commit.CommitContext.outerSubmitter; locked 60% of time  Unsynchronized access at CommitContext.java:60% of time  Unsynchronized access at CommitContext.java:[line 291]

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/13/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3289

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell markdownlint

uname
Linux e482d1513a87 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 3b1331583dadfa2bce444477108490b53a324798

Default Java
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS> /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/13/testReport/

Max. process+thread count
1535 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/13/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 41s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  2s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 31 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
16m  6s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
25m 22s

trunk passed

+1 :green_heart:
compile
23m  5s

trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
compile
20m 36s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
checkstyle
4m 27s

trunk passed

+1 :green_heart:
mvnsite
4m 58s

trunk passed

-1 :x:
javadoc
1m 54s
/branch-javadoc-hadoop-common-project_hadoop-common-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.txt
hadoop-common in trunk failed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.

+1 :green_heart:
javadoc
4m 22s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
7m  4s

trunk passed

+1 :green_heart:
shadedclient
22m 50s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 31s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
2m 29s

the patch passed

+1 :green_heart:
compile
22m 16s

the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

-1 :x:
javac
22m 16s
/results-compile-javac-root-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.txt
root-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS> with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS> generated 1 new + 1811 unchanged - 1 fixed = 1812 total (was 1812)

+1 :green_heart:
compile
20m 35s

the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

-1 :x:
javac
20m 35s
/results-compile-javac-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt
root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 1 new + 1688 unchanged - 1 fixed = 1689 total (was 1689)

-1 :x:
blanks
0m  0s
/blanks-eol.txt
The patch has 4 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply

-0 :warning:
checkstyle
4m 17s
/results-checkstyle-root.txt
root: The patch generated 20 new + 20 unchanged - 2 fixed = 40 total (was 22)

+1 :green_heart:
mvnsite
5m 16s

the patch passed

-1 :x:
javadoc
1m 44s
/patch-javadoc-hadoop-common-project_hadoop-common-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.txt
hadoop-common in the patch failed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.

-1 :x:
javadoc
1m 20s
/results-javadoc-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt
hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 4 new + 38 unchanged - 0 fixed = 42 total (was 38)

-1 :x:
spotbugs
2m  5s
/new-spotbugs-hadoop-tools_hadoop-aws.html
hadoop-tools/hadoop-aws generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)

+1 :green_heart:
shadedclient
22m 41s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
18m 28s

hadoop-common in the patch passed.

+1 :green_heart:
unit
6m 48s

hadoop-mapreduce-client-core in the patch passed.

+1 :green_heart:
unit
3m 13s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
1m 38s

The patch does not generate ASF License warnings.

264m 18s

Reason
Tests

SpotBugs
module:hadoop-tools/hadoop-aws

Inconsistent synchronization of org.apache.hadoop.fs.s3a.commit.CommitContext.outerSubmitter; locked 60% of time  Unsynchronized access at CommitContext.java:60% of time  Unsynchronized access at CommitContext.java:[line 291]

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/14/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3289

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell markdownlint

uname
Linux e4d9e7cf1918 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 7dd315909bae5b88526d389272ab5ceb1c1badfb

Default Java
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS> /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/14/testReport/

Max. process+thread count
1612 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/14/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

This is ready for review as I've

stripped out the overheads from the mkdirs calls too.
added a createFile option "fs.s3a.create.performance" to offer the file creation speedup to any application, in any location. if you know the parent dir is ok and the file isn't overwriting anything, this saves

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 43s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  2s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 36 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
15m 57s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
27m 58s

trunk passed

+1 :green_heart:
compile
24m 13s

trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
compile
22m 26s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
checkstyle
4m 43s

trunk passed

+1 :green_heart:
mvnsite
5m 16s

trunk passed

-1 :x:
javadoc
1m 52s
/branch-javadoc-hadoop-common-project_hadoop-common-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.txt
hadoop-common in trunk failed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.

+1 :green_heart:
javadoc
4m 50s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
7m  9s

trunk passed

+1 :green_heart:
shadedclient
22m 22s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 29s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
2m 30s

the patch passed

+1 :green_heart:
compile
22m 21s

the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

-1 :x:
javac
22m 21s
/results-compile-javac-root-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.txt
root-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS> with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS> generated 2 new + 1817 unchanged - 2 fixed = 1819 total (was 1819)

+1 :green_heart:
compile
21m  6s

the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

-1 :x:
javac
21m  6s
/results-compile-javac-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt
root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 2 new + 1690 unchanged - 2 fixed = 1692 total (was 1692)

-1 :x:
blanks
0m  0s
/blanks-eol.txt
The patch has 4 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply

-0 :warning:
checkstyle
5m  8s
/results-checkstyle-root.txt
root: The patch generated 8 new + 30 unchanged - 6 fixed = 38 total (was 36)

+1 :green_heart:
mvnsite
4m 51s

the patch passed

-1 :x:
javadoc
1m 36s
/patch-javadoc-hadoop-common-project_hadoop-common-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.txt
hadoop-common in the patch failed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.

-1 :x:
javadoc
1m 15s
/results-javadoc-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt
hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 4 new + 38 unchanged - 0 fixed = 42 total (was 38)

-1 :x:
spotbugs
2m  5s
/new-spotbugs-hadoop-tools_hadoop-aws.html
hadoop-tools/hadoop-aws generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)

+1 :green_heart:
shadedclient
22m 38s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
18m 23s

hadoop-common in the patch passed.

+1 :green_heart:
unit
6m 46s

hadoop-mapreduce-client-core in the patch passed.

+1 :green_heart:
unit
3m  8s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
1m 38s

The patch does not generate ASF License warnings.

269m 31s

Reason
Tests

SpotBugs
module:hadoop-tools/hadoop-aws

Inconsistent synchronization of org.apache.hadoop.fs.s3a.commit.CommitContext.outerSubmitter; locked 60% of time  Unsynchronized access at CommitContext.java:60% of time  Unsynchronized access at CommitContext.java:[line 296]

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/16/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3289

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell markdownlint

uname
Linux 1de097c2249e 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 1050c5b57eb0e2cf12dccc00f0b19d71fc71ff3b

Default Java
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS> /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/16/testReport/

Max. process+thread count
1601 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/16/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 47s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  2s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 36 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
15m 51s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
28m 42s

trunk passed

+1 :green_heart:
compile
26m 30s

trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
compile
23m 16s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
checkstyle
4m 34s

trunk passed

+1 :green_heart:
mvnsite
5m  7s

trunk passed

-1 :x:
javadoc
1m 48s
/branch-javadoc-hadoop-common-project_hadoop-common-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.txt
hadoop-common in trunk failed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.

+1 :green_heart:
javadoc
4m 37s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
7m  5s

trunk passed

+1 :green_heart:
shadedclient
24m 29s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 28s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
2m 36s

the patch passed

+1 :green_heart:
compile
24m 30s

the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
javac
24m 30s

the patch passed

+1 :green_heart:
compile
23m 19s

the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
javac
23m 19s

the patch passed

-1 :x:
blanks
0m  0s
/blanks-eol.txt
The patch has 4 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply

-0 :warning:
checkstyle
4m 29s
/results-checkstyle-root.txt
root: The patch generated 9 new + 30 unchanged - 6 fixed = 39 total (was 36)

+1 :green_heart:
mvnsite
5m  3s

the patch passed

-1 :x:
javadoc
1m 41s
/patch-javadoc-hadoop-common-project_hadoop-common-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.txt
hadoop-common in the patch failed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.

-1 :x:
javadoc
1m  8s
/results-javadoc-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt
hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 4 new + 38 unchanged - 0 fixed = 42 total (was 38)

-1 :x:
spotbugs
2m  3s
/new-spotbugs-hadoop-tools_hadoop-aws.html
hadoop-tools/hadoop-aws generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)

+1 :green_heart:
shadedclient
23m 46s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
18m 41s

hadoop-common in the patch passed.

+1 :green_heart:
unit
6m 48s

hadoop-mapreduce-client-core in the patch passed.

+1 :green_heart:
unit
3m  9s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
1m 22s

The patch does not generate ASF License warnings.

280m 13s

Reason
Tests

SpotBugs
module:hadoop-tools/hadoop-aws

Inconsistent synchronization of org.apache.hadoop.fs.s3a.commit.CommitContext.outerSubmitter; locked 60% of time  Unsynchronized access at CommitContext.java:60% of time  Unsynchronized access at CommitContext.java:[line 296]

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/17/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3289

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell markdownlint

uname
Linux 0c35b3b6bf23 4.15.0-169-generic #177-Ubuntu SMP Thu Feb 3 10:50:38 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 4e628331fe90e3ed0ad38a7957bccebfa58f63da

Default Java
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS> /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/17/testReport/

Max. process+thread count
1353 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/17/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 40s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  2s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 36 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
16m 29s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
24m 56s

trunk passed

+1 :green_heart:
compile
23m  3s

trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
compile
20m 35s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
checkstyle
4m 23s

trunk passed

+1 :green_heart:
mvnsite
5m 13s

trunk passed

-1 :x:
javadoc
1m 46s
/branch-javadoc-hadoop-common-project_hadoop-common-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.txt
hadoop-common in trunk failed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.

+1 :green_heart:
javadoc
4m 49s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
7m  7s

trunk passed

+1 :green_heart:
shadedclient
22m  7s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 32s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
2m 30s

the patch passed

+1 :green_heart:
compile
22m 23s

the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
javac
22m 23s

the patch passed

+1 :green_heart:
compile
20m 40s

the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
javac
20m 40s

the patch passed

-1 :x:
blanks
0m  1s
/blanks-eol.txt
The patch has 4 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply

-0 :warning:
checkstyle
4m  8s
/results-checkstyle-root.txt
root: The patch generated 10 new + 30 unchanged - 6 fixed = 40 total (was 36)

+1 :green_heart:
mvnsite
5m  9s

the patch passed

-1 :x:
javadoc
1m 42s
/patch-javadoc-hadoop-common-project_hadoop-common-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.txt
hadoop-common in the patch failed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.

-1 :x:
javadoc
1m 19s
/results-javadoc-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt
hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 5 new + 38 unchanged - 0 fixed = 43 total (was 38)

-1 :x:
spotbugs
2m  4s
/new-spotbugs-hadoop-tools_hadoop-aws.html
hadoop-tools/hadoop-aws generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)

+1 :green_heart:
shadedclient
22m 34s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
18m 19s

hadoop-common in the patch passed.

+1 :green_heart:
unit
6m 47s

hadoop-mapreduce-client-core in the patch passed.

+1 :green_heart:
unit
3m 13s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
1m 37s

The patch does not generate ASF License warnings.

263m 35s

Reason
Tests

SpotBugs
module:hadoop-tools/hadoop-aws

Inconsistent synchronization of org.apache.hadoop.fs.s3a.commit.CommitContext.outerSubmitter; locked 60% of time  Unsynchronized access at CommitContext.java:60% of time  Unsynchronized access at CommitContext.java:[line 296]

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/18/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3289

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell markdownlint

uname
Linux 5c550aeeb1a0 4.15.0-169-generic #177-Ubuntu SMP Thu Feb 3 10:50:38 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 74ee6d211790c442b1a6f590fee081e2ad689b3a

Default Java
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS> /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/18/testReport/

Max. process+thread count
1564 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/18/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 40s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  2s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
detsecrets
0m  1s

detect-secrets was not available.

+0 :ok:
markdownlint
0m  1s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 36 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
14m 37s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
26m 33s

trunk passed

-1 :x:
compile
4m  6s
/branch-compile-root-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.txt
root in trunk failed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.

-1 :x:
compile
3m 36s
/branch-compile-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt
root in trunk failed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.

+1 :green_heart:
checkstyle
3m 56s

trunk passed

+1 :green_heart:
mvnsite
3m 49s

trunk passed

+1 :green_heart:
javadoc
2m 43s

trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
javadoc
2m 27s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
5m 39s

trunk passed

+1 :green_heart:
shadedclient
20m 27s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 32s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
2m 31s

the patch passed

-1 :x:
compile
3m 48s
/patch-compile-root-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.txt
root in the patch failed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.

-1 :x:
javac
3m 48s
/patch-compile-root-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.txt
root in the patch failed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.

-1 :x:
compile
3m 24s
/patch-compile-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt
root in the patch failed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.

-1 :x:
javac
3m 24s
/patch-compile-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt
root in the patch failed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.

-1 :x:
blanks
0m  0s
/blanks-eol.txt
The patch has 4 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply

-0 :warning:
checkstyle
3m 35s
/results-checkstyle-root.txt
root: The patch generated 10 new + 30 unchanged - 6 fixed = 40 total (was 36)

+1 :green_heart:
mvnsite
3m  9s

the patch passed

-1 :x:
javadoc
0m 57s
/results-javadoc-javadoc-hadoop-common-project_hadoop-common-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.txt
hadoop-common-project_hadoop-common-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS> with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS> generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)

-1 :x:
javadoc
0m 37s
/results-javadoc-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt
hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 5 new + 38 unchanged - 0 fixed = 43 total (was 38)

-1 :x:
spotbugs
1m 22s
/new-spotbugs-hadoop-tools_hadoop-aws.html
hadoop-tools/hadoop-aws generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)

+1 :green_heart:
shadedclient
20m 29s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
18m 11s

hadoop-common in the patch passed.

+1 :green_heart:
unit
6m 17s

hadoop-mapreduce-client-core in the patch passed.

+1 :green_heart:
unit
2m 34s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
0m 48s

The patch does not generate ASF License warnings.

166m 49s

Reason
Tests

SpotBugs
module:hadoop-tools/hadoop-aws

Inconsistent synchronization of org.apache.hadoop.fs.s3a.commit.CommitContext.outerSubmitter; locked 60% of time  Unsynchronized access at CommitContext.java:60% of time  Unsynchronized access at CommitContext.java:[line 296]

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/19/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3289

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint

uname
Linux 2f73b3f3b0d8 4.15.0-169-generic #177-Ubuntu SMP Thu Feb 3 10:50:38 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 1ac6a9afb79aedde02a1d9a4fa4cb6bf9c8332de

Default Java
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS> /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/19/testReport/

Max. process+thread count
3152 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/19/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 37s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  2s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
detsecrets
0m  1s

detect-secrets was not available.

+0 :ok:
markdownlint
0m  1s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 37 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
44m 24s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
24m 29s

trunk passed

-1 :x:
compile
4m  6s
/branch-compile-root-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.txt
root in trunk failed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.

-1 :x:
compile
3m 35s
/branch-compile-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt
root in trunk failed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.

+1 :green_heart:
checkstyle
3m 59s

trunk passed

+1 :green_heart:
mvnsite
3m 52s

trunk passed

+1 :green_heart:
javadoc
2m 47s

trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
javadoc
2m 29s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
5m 39s

trunk passed

+1 :green_heart:
shadedclient
20m 51s

branch has no errors when building and testing our client artifacts.

-0 :warning:
patch
21m 19s

Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 32s

Maven dependency ordering for patch

-1 :x:
mvninstall
0m 39s
/patch-mvninstall-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch failed.

-1 :x:
compile
3m 52s
/patch-compile-root-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.txt
root in the patch failed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.

-1 :x:
javac
3m 52s
/patch-compile-root-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.txt
root in the patch failed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.

-1 :x:
compile
3m 21s
/patch-compile-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt
root in the patch failed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.

-1 :x:
javac
3m 21s
/patch-compile-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt
root in the patch failed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
3m 43s
/results-checkstyle-root.txt
root: The patch generated 12 new + 30 unchanged - 6 fixed = 42 total (was 36)

-1 :x:
mvnsite
0m 45s
/patch-mvnsite-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch failed.

+1 :green_heart:
javadoc
1m 56s

the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

-1 :x:
javadoc
0m 36s
/results-javadoc-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt
hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 4 new + 38 unchanged - 0 fixed = 42 total (was 38)

-1 :x:
spotbugs
0m 43s
/patch-spotbugs-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch failed.

+1 :green_heart:
shadedclient
21m 42s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
17m 53s

hadoop-common in the patch passed.

+1 :green_heart:
unit
6m 15s

hadoop-mapreduce-client-core in the patch passed.

-1 :x:
unit
0m 49s
/patch-unit-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch failed.

+1 :green_heart:
asflicense
0m 50s

The patch does not generate ASF License warnings.

192m 59s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/20/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3289

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint

uname
Linux 6c3ee3531ab0 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / c585b629aec8c15cb7345b93432a695537eb218b

Default Java
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS> /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/20/testReport/

Max. process+thread count
2800 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/20/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 40s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  2s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
detsecrets
0m  1s

detect-secrets was not available.

+0 :ok:
markdownlint
0m  1s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 37 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
52m 22s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
28m 21s

trunk passed

-1 :x:
compile
4m 55s
/branch-compile-root-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.txt
root in trunk failed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.

-1 :x:
compile
3m 56s
/branch-compile-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt
root in trunk failed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.

+1 :green_heart:
checkstyle
4m 36s

trunk passed

+1 :green_heart:
mvnsite
4m  5s

trunk passed

+1 :green_heart:
javadoc
2m 57s

trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
javadoc
2m 20s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
5m 40s

trunk passed

+1 :green_heart:
shadedclient
21m 25s

branch has no errors when building and testing our client artifacts.

-0 :warning:
patch
21m 52s

Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 34s

Maven dependency ordering for patch

-1 :x:
mvninstall
0m 37s
/patch-mvninstall-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch failed.

-1 :x:
compile
4m 17s
/patch-compile-root-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.txt
root in the patch failed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.

-1 :x:
javac
4m 17s
/patch-compile-root-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.txt
root in the patch failed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.

-1 :x:
compile
3m 32s
/patch-compile-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt
root in the patch failed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.

-1 :x:
javac
3m 32s
/patch-compile-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt
root in the patch failed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
3m 47s
/results-checkstyle-root.txt
root: The patch generated 12 new + 30 unchanged - 6 fixed = 42 total (was 36)

-1 :x:
mvnsite
0m 47s
/patch-mvnsite-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch failed.

+1 :green_heart:
javadoc
1m 45s

the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

-1 :x:
javadoc
0m 35s
/results-javadoc-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt
hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 4 new + 38 unchanged - 0 fixed = 42 total (was 38)

-1 :x:
spotbugs
0m 39s
/patch-spotbugs-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch failed.

+1 :green_heart:
shadedclient
21m 48s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
17m 59s

hadoop-common in the patch passed.

+1 :green_heart:
unit
6m 11s

hadoop-mapreduce-client-core in the patch passed.

-1 :x:
unit
0m 44s
/patch-unit-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch failed.

+1 :green_heart:
asflicense
0m 47s

The patch does not generate ASF License warnings.

207m 30s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/21/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3289

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint

uname
Linux 2eb8f8e488a5 4.15.0-169-generic #177-Ubuntu SMP Thu Feb 3 10:50:38 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / c585b629aec8c15cb7345b93432a695537eb218b

Default Java
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS> /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/21/testReport/

Max. process+thread count
2645 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/21/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 38s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  2s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 37 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
14m 29s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
25m 45s

trunk passed

-1 :x:
compile
4m  6s
/branch-compile-root-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.txt
root in trunk failed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.

-1 :x:
compile
3m 48s
/branch-compile-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt
root in trunk failed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.

+1 :green_heart:
checkstyle
3m 56s

trunk passed

+1 :green_heart:
mvnsite
3m 48s

trunk passed

+1 :green_heart:
javadoc
2m 37s

trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
javadoc
2m 10s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
5m 38s

trunk passed

+1 :green_heart:
shadedclient
20m 29s

branch has no errors when building and testing our client artifacts.

-0 :warning:
patch
20m 55s

Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 30s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
2m 31s

the patch passed

-1 :x:
compile
4m 18s
/patch-compile-root-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.txt
root in the patch failed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.

-1 :x:
javac
4m 18s
/patch-compile-root-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.txt
root in the patch failed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.

-1 :x:
compile
3m 44s
/patch-compile-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt
root in the patch failed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.

-1 :x:
javac
3m 44s
/patch-compile-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt
root in the patch failed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
4m  9s
/results-checkstyle-root.txt
root: The patch generated 12 new + 30 unchanged - 6 fixed = 42 total (was 36)

+1 :green_heart:
mvnsite
3m 20s

the patch passed

+1 :green_heart:
javadoc
2m  0s

the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

-1 :x:
javadoc
0m 36s
/results-javadoc-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt
hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 4 new + 38 unchanged - 0 fixed = 42 total (was 38)

+1 :green_heart:
spotbugs
5m 44s

the patch passed

+1 :green_heart:
shadedclient
23m 27s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
18m 28s

hadoop-common in the patch passed.

+1 :green_heart:
unit
6m 19s

hadoop-mapreduce-client-core in the patch passed.

+1 :green_heart:
unit
2m 31s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
0m 52s

The patch does not generate ASF License warnings.

170m  9s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/22/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3289

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint

uname
Linux efa2f8171161 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / ff8eb5fed5ec506a17e86f0331d1d1aac295b340

Default Java
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS> /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/22/testReport/

Max. process+thread count
3152 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/22/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 38s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  2s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+0 :ok:
xmllint
0m  0s

xmllint was not available.

+1 :green_heart:
@author
0m  1s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 38 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
14m 36s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
25m  2s

trunk passed

+1 :green_heart:
compile
22m 57s

trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
compile
20m 33s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
checkstyle
4m 13s

trunk passed

+1 :green_heart:
mvnsite
5m 20s

trunk passed

+1 :green_heart:
javadoc
4m 16s

trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
javadoc
3m 54s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
7m  8s

trunk passed

+1 :green_heart:
shadedclient
25m 31s

branch has no errors when building and testing our client artifacts.

-0 :warning:
patch
26m  3s

Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 31s

Maven dependency ordering for patch

-1 :x:
mvninstall
0m 24s
/patch-mvninstall-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch failed.

-1 :x:
compile
20m 38s
/patch-compile-root-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.txt
root in the patch failed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.

-1 :x:
javac
20m 38s
/patch-compile-root-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.txt
root in the patch failed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.

-1 :x:
compile
19m 18s
/patch-compile-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt
root in the patch failed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.

-1 :x:
javac
19m 18s
/patch-compile-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt
root in the patch failed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
4m 20s
/results-checkstyle-root.txt
root: The patch generated 7 new + 30 unchanged - 6 fixed = 37 total (was 36)

-1 :x:
mvnsite
1m 13s
/patch-mvnsite-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch failed.

-1 :x:
javadoc
1m  9s
/patch-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.txt
hadoop-aws in the patch failed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>.

-1 :x:
javadoc
1m  8s
/patch-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt
hadoop-aws in the patch failed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.

-1 :x:
spotbugs
0m 58s
/patch-spotbugs-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch failed.

+1 :green_heart:
shadedclient
22m 28s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
18m 23s

hadoop-common in the patch passed.

+1 :green_heart:
unit
6m 49s

hadoop-mapreduce-client-core in the patch passed.

-1 :x:
unit
1m 19s
/patch-unit-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch failed.

+1 :green_heart:
asflicense
1m 35s

The patch does not generate ASF License warnings.

256m 20s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/24/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3289

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint xmllint

uname
Linux 265491951ae0 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 3532b151a972de2a045bb0ccc768346541dadfa8

Default Java
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS> /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/24/testReport/

Max. process+thread count
1560 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/24/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 43s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  2s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+0 :ok:
xmllint
0m  0s

xmllint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 37 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
14m 35s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
28m 14s

trunk passed

+1 :green_heart:
compile
26m 14s

trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
compile
21m 32s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
checkstyle
4m 51s

trunk passed

+1 :green_heart:
mvnsite
5m  5s

trunk passed

+1 :green_heart:
javadoc
3m 57s

trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
javadoc
3m 25s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
7m  2s

trunk passed

+1 :green_heart:
shadedclient
25m 31s

branch has no errors when building and testing our client artifacts.

-0 :warning:
patch
26m  1s

Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 33s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
2m 36s

the patch passed

+1 :green_heart:
compile
24m 30s

the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
javac
24m 30s

root-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS> with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS> generated 0 new + 2889 unchanged - 3 fixed = 2889 total (was 2892)

+1 :green_heart:
compile
22m 11s

the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
javac
22m 11s

root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 0 new + 2687 unchanged - 3 fixed = 2687 total (was 2690)

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
4m  6s
/results-checkstyle-root.txt
root: The patch generated 7 new + 30 unchanged - 6 fixed = 37 total (was 36)

+1 :green_heart:
mvnsite
4m 46s

the patch passed

+1 :green_heart:
javadoc
3m 40s

the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

-1 :x:
javadoc
0m 59s
/results-javadoc-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt
hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 4 new + 38 unchanged - 0 fixed = 42 total (was 38)

+1 :green_heart:
spotbugs
7m 20s

the patch passed

+1 :green_heart:
shadedclient
23m  5s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
18m 44s

hadoop-common in the patch passed.

+1 :green_heart:
unit
6m 55s

hadoop-mapreduce-client-core in the patch passed.

+1 :green_heart:
unit
3m  2s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
1m 32s

The patch does not generate ASF License warnings.

272m 44s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/26/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3289

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint xmllint

uname
Linux 6f9664a99a85 4.15.0-169-generic #177-Ubuntu SMP Thu Feb 3 10:50:38 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 385da6a8cfaa79326fe069997e6628a859acbd57

Default Java
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS> /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/26/testReport/

Max. process+thread count
1608 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/26/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 42s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  3s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+0 :ok:
xmllint
0m  0s

xmllint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 37 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
14m 36s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
24m 37s

trunk passed

+1 :green_heart:
compile
22m 50s

trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
compile
20m 26s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
checkstyle
4m 28s

trunk passed

+1 :green_heart:
mvnsite
5m 18s

trunk passed

+1 :green_heart:
javadoc
3m 58s

trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
javadoc
3m 55s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
7m  5s

trunk passed

+1 :green_heart:
shadedclient
22m 10s

branch has no errors when building and testing our client artifacts.

-0 :warning:
patch
22m 41s

Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 28s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
2m 29s

the patch passed

+1 :green_heart:
compile
22m  5s

the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
javac
22m  5s

root-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS> with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS> generated 0 new + 2889 unchanged - 3 fixed = 2889 total (was 2892)

+1 :green_heart:
compile
20m 17s

the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
javac
20m 17s

root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 0 new + 2682 unchanged - 3 fixed = 2682 total (was 2685)

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
4m 18s
/results-checkstyle-root.txt
root: The patch generated 7 new + 30 unchanged - 6 fixed = 37 total (was 36)

+1 :green_heart:
mvnsite
5m 16s

the patch passed

+1 :green_heart:
javadoc
4m  2s

the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

-1 :x:
javadoc
1m 16s
/results-javadoc-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt
hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 4 new + 38 unchanged - 0 fixed = 42 total (was 38)

+1 :green_heart:
spotbugs
7m 27s

the patch passed

+1 :green_heart:
shadedclient
22m 11s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
18m 22s

hadoop-common in the patch passed.

+1 :green_heart:
unit
6m 43s

hadoop-mapreduce-client-core in the patch passed.

+1 :green_heart:
unit
3m 12s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
1m 38s

The patch does not generate ASF License warnings.

258m 18s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/27/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3289

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint xmllint

uname
Linux 7cd3150095ff 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / f58b9e47260ad5fd451ab7bde478db6c8dca4860

Default Java
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS> /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/27/testReport/

Max. process+thread count
3152 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/27/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

Wrapping up createFile() with the ability to set headers on the file
builder.must("fs.s3a.create.header.my-header","my-value");
this will set a user metadata entry "my-header" which xattr will return
as "header.my-header".
This forced me to make PutOptions something passed around more rigorously and
used as the parameter class to innerCreateFile. This is ultimately good as it
lets us set new options later..encryption, storage class etc. if we so choose.
also: s3a path capabilities let you probe for this prefix, as well as the
performance one...think this should be the standard behaviour for custom options.
also: fixed the various style/javadoc warnings

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 42s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  3s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+0 :ok:
xmllint
0m  0s

xmllint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 39 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
17m 10s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
24m 51s

trunk passed

+1 :green_heart:
compile
22m 57s

trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
compile
20m 24s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
checkstyle
4m 25s

trunk passed

+1 :green_heart:
mvnsite
5m  4s

trunk passed

+1 :green_heart:
javadoc
4m 14s

trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
javadoc
3m 53s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
7m  6s

trunk passed

+1 :green_heart:
shadedclient
22m  9s

branch has no errors when building and testing our client artifacts.

-0 :warning:
patch
22m 40s

Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 29s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
2m 27s

the patch passed

+1 :green_heart:
compile
22m  6s

the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
javac
22m  6s

root-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS> with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS> generated 0 new + 2889 unchanged - 3 fixed = 2889 total (was 2892)

+1 :green_heart:
compile
20m 23s

the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
javac
20m 23s

root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 0 new + 2687 unchanged - 3 fixed = 2687 total (was 2690)

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
4m 18s
/results-checkstyle-root.txt
root: The patch generated 10 new + 30 unchanged - 6 fixed = 40 total (was 36)

+1 :green_heart:
mvnsite
5m  9s

the patch passed

+1 :green_heart:
javadoc
4m  5s

the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

-1 :x:
javadoc
1m 17s
/results-javadoc-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt
hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 1 new + 38 unchanged - 0 fixed = 39 total (was 38)

-1 :x:
spotbugs
2m  1s
/new-spotbugs-hadoop-tools_hadoop-aws.html
hadoop-tools/hadoop-aws generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)

+1 :green_heart:
shadedclient
22m 22s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
18m 29s

hadoop-common in the patch passed.

+1 :green_heart:
unit
6m 47s

hadoop-mapreduce-client-core in the patch passed.

+1 :green_heart:
unit
3m 14s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
1m 38s

The patch does not generate ASF License warnings.

261m 59s

Reason
Tests

SpotBugs
module:hadoop-tools/hadoop-aws

Dead store to recursive in org.apache.hadoop.fs.s3a.S3AFileSystem.innerCreateFile(Path, Progressable, AuditSpan, CreateFileBuilder$CreateFileOptions)  At S3AFileSystem.java:org.apache.hadoop.fs.s3a.S3AFileSystem.innerCreateFile(Path, Progressable, AuditSpan, CreateFileBuilder$CreateFileOptions)  At S3AFileSystem.java:[line 1648]

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/28/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3289

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint xmllint

uname
Linux 385c6d8ec3eb 4.15.0-169-generic #177-Ubuntu SMP Thu Feb 3 10:50:38 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 3af20704eff46d463a789cfdfa8d053593796a84

Default Java
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS> /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/28/testReport/

Max. process+thread count
1618 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/28/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@Sam-Kramer you ask a good question. know that you can create a file under a file today, but nobody has ever noticed. This is probably because mkdirs() does the check, and most code creates the destination directory before the files. They don't need to and if you try hard you can get in a mess.
the cloudstore listobjects command exists to get the untainted view of a store and is how we debug such things on the odd occasion when they surface.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
12m 59s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  2s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+0 :ok:
xmllint
0m  0s

xmllint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 39 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
14m 25s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
25m 33s

trunk passed

+1 :green_heart:
compile
24m 19s

trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
compile
21m 55s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
checkstyle
4m 12s

trunk passed

+1 :green_heart:
mvnsite
4m 20s

trunk passed

+1 :green_heart:
javadoc
3m 24s

trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
javadoc
2m 55s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
6m 29s

trunk passed

+1 :green_heart:
shadedclient
24m 24s

branch has no errors when building and testing our client artifacts.

-0 :warning:
patch
24m 56s