HADOOP-16202. Enhance openFile() for better read performance against object stores

This PR

declares a standard set of openFile() parameters for setting length, seek/read policy
and split start/end.
supports all of these in s3a
as well as the threshold to trigger an async drain of s3a stream in a seek,
fs.s3a.input.async.drain.threshold
documents all of this
fixes up all places in the code which reads a whole file (fs shell, distcp etc)
to set its read policy to be whole-file.

as a result of this, on the s3a client you can open files without needing
a head request, or even a filestatus, just the length.
and if, in a cluster, you set the default read policy to be random,
shell and distcp read performance doesn't collapse.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 51s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
markdownlint
0m  1s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:

0m  0s
test4tests
The patch appears to include 10 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
5m 15s

Maven dependency ordering for branch

-1 :x:
mvninstall
0m 23s
/branch-mvninstall-root.txt
root in trunk failed.

-1 :x:
compile
18m 15s
/branch-compile-root-jdkUbuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04.txt
root in trunk failed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04.

-1 :x:
compile
0m 28s
/branch-compile-root-jdkPrivateBuild-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01.txt
root in trunk failed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01.

+1 :green_heart:
checkstyle
7m  9s

trunk passed

-1 :x:
mvnsite
0m 36s
/branch-mvnsite-hadoop-common-project_hadoop-common.txt
hadoop-common in trunk failed.

-1 :x:
mvnsite
0m 21s
/branch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt
hadoop-yarn-common in trunk failed.

+1 :green_heart:
shadedclient
26m 40s

branch has no errors when building and testing our client artifacts.

+1 :green_heart:
javadoc
3m 36s

trunk passed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04

+1 :green_heart:
javadoc
4m 16s

trunk passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01

+0 :ok:
spotbugs
0m 44s

Used deprecated FindBugs config; considering switching to SpotBugs.

-1 :x:
findbugs
0m 20s
/branch-findbugs-hadoop-common-project_hadoop-common.txt
hadoop-common in trunk failed.

-1 :x:
findbugs
0m 19s
/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt
hadoop-yarn-common in trunk failed.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 51s

Maven dependency ordering for patch

-1 :x:
mvninstall
0m 12s
/patch-mvninstall-hadoop-common-project_hadoop-common.txt
hadoop-common in the patch failed.

-1 :x:
mvninstall
0m 12s
/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt
hadoop-yarn-common in the patch failed.

-1 :x:
mvninstall
0m 22s
/patch-mvninstall-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt
hadoop-mapreduce-client-core in the patch failed.

-1 :x:
mvninstall
0m 19s
/patch-mvninstall-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt
hadoop-mapreduce-client-app in the patch failed.

-1 :x:
mvninstall
0m 18s
/patch-mvninstall-hadoop-mapreduce-project_hadoop-mapreduce-examples.txt
hadoop-mapreduce-examples in the patch failed.

-1 :x:
mvninstall
0m 16s
/patch-mvninstall-hadoop-tools_hadoop-distcp.txt
hadoop-distcp in the patch failed.

-1 :x:
mvninstall
0m 20s
/patch-mvninstall-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch failed.

-1 :x:
compile
19m 11s
/patch-compile-root-jdkUbuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04.txt
root in the patch failed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04.

-1 :x:
javac
19m 11s
/patch-compile-root-jdkUbuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04.txt
root in the patch failed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04.

-1 :x:
compile
16m 20s
/patch-compile-root-jdkPrivateBuild-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01.txt
root in the patch failed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01.

-1 :x:
javac
16m 20s
/patch-compile-root-jdkPrivateBuild-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01.txt
root in the patch failed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01.

-0 :warning:
checkstyle
3m  1s
/diff-checkstyle-root.txt
root: The patch generated 4 new + 794 unchanged - 1 fixed = 798 total (was 795)

-1 :x:
mvnsite
0m 33s
/patch-mvnsite-hadoop-common-project_hadoop-common.txt
hadoop-common in the patch failed.

-1 :x:
mvnsite
0m 29s
/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt
hadoop-yarn-common in the patch failed.

-1 :x:
mvnsite
0m 40s
/patch-mvnsite-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt
hadoop-mapreduce-client-core in the patch failed.

-1 :x:
mvnsite
0m 33s
/patch-mvnsite-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt
hadoop-mapreduce-client-app in the patch failed.

-1 :x:
mvnsite
0m 32s
/patch-mvnsite-hadoop-mapreduce-project_hadoop-mapreduce-examples.txt
hadoop-mapreduce-examples in the patch failed.

-1 :x:
mvnsite
0m 33s
/patch-mvnsite-hadoop-tools_hadoop-distcp.txt
hadoop-distcp in the patch failed.

-1 :x:
mvnsite
0m 37s
/patch-mvnsite-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch failed.

-1 :x:
whitespace
0m  0s
/whitespace-eol.txt
The patch has 8 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply

+1 :green_heart:
shadedclient
15m 30s

patch has no errors when building and testing our client artifacts.

-1 :x:
javadoc
0m 39s
/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common-jdkUbuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04.txt
hadoop-yarn-common in the patch failed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04.

-1 :x:
javadoc
0m 35s
/patch-javadoc-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdkUbuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04.txt
hadoop-mapreduce-client-app in the patch failed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04.

-1 :x:
javadoc
0m 33s
/patch-javadoc-hadoop-mapreduce-project_hadoop-mapreduce-examples-jdkUbuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04.txt
hadoop-mapreduce-examples in the patch failed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04.

-1 :x:
javadoc
0m 33s
/patch-javadoc-hadoop-tools_hadoop-distcp-jdkUbuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04.txt
hadoop-distcp in the patch failed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04.

-1 :x:
javadoc
1m  7s
/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common-jdkPrivateBuild-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01.txt
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common-jdkPrivateBuild-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 generated 10 new + 4527 unchanged - 0 fixed = 4537 total (was 4527)

-1 :x:
javadoc
0m 38s
/diff-javadoc-javadoc-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdkPrivateBuild-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01.txt
hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdkPrivateBuild-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 generated 4 new + 38 unchanged - 0 fixed = 42 total (was 38)

-1 :x:
javadoc
0m 35s
/diff-javadoc-javadoc-hadoop-mapreduce-project_hadoop-mapreduce-examples-jdkPrivateBuild-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01.txt
hadoop-mapreduce-project_hadoop-mapreduce-examples-jdkPrivateBuild-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 generated 8 new + 143 unchanged - 0 fixed = 151 total (was 143)

-1 :x:
javadoc
0m 34s
/diff-javadoc-javadoc-hadoop-tools_hadoop-distcp-jdkPrivateBuild-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01.txt
hadoop-tools_hadoop-distcp-jdkPrivateBuild-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 generated 4 new + 41 unchanged - 0 fixed = 45 total (was 41)

-1 :x:
javadoc
0m 43s
/patch-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01.txt
hadoop-aws in the patch failed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01.

-1 :x:
findbugs
0m 32s
/patch-findbugs-hadoop-common-project_hadoop-common.txt
hadoop-common in the patch failed.

-1 :x:
findbugs
0m 28s
/patch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt
hadoop-yarn-common in the patch failed.

-1 :x:
findbugs
0m 41s
/patch-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt
hadoop-mapreduce-client-core in the patch failed.

-1 :x:
findbugs
0m 37s
/patch-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt
hadoop-mapreduce-client-app in the patch failed.

-1 :x:
findbugs
0m 36s
/patch-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-examples.txt
hadoop-mapreduce-examples in the patch failed.

-1 :x:
findbugs
0m 31s
/patch-findbugs-hadoop-tools_hadoop-distcp.txt
hadoop-distcp in the patch failed.

-1 :x:
findbugs
0m 38s
/patch-findbugs-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch failed.

_ Other Tests _

-1 :x:
unit
0m 30s
/patch-unit-hadoop-common-project_hadoop-common.txt
hadoop-common in the patch failed.

-1 :x:
unit
0m 29s
/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt
hadoop-yarn-common in the patch failed.

-1 :x:
unit
0m 43s
/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt
hadoop-mapreduce-client-core in the patch failed.

-1 :x:
unit
0m 36s
/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt
hadoop-mapreduce-client-app in the patch failed.

-1 :x:
unit
0m 32s
/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-examples.txt
hadoop-mapreduce-examples in the patch failed.

-1 :x:
unit
0m 33s
/patch-unit-hadoop-tools_hadoop-distcp.txt
hadoop-distcp in the patch failed.

-1 :x:
unit
0m 25s
/patch-unit-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch failed.

+1 :green_heart:
asflicense
0m 53s

The patch does not generate ASF License warnings.

150m 13s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2584/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/2584

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle markdownlint

uname
Linux 744cc5b526c8 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 2825d060cf9

Default Java
Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2584/1/testReport/

Max. process+thread count
538 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-examples hadoop-tools/hadoop-distcp hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2584/1/console

versions
git=2.17.1 maven=3.6.0 findbugs=4.0.6

Powered by
Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 51s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
markdownlint
0m  1s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:

0m  0s
test4tests
The patch appears to include 10 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
5m 15s

Maven dependency ordering for branch

-1 :x:
mvninstall
0m 23s
/branch-mvninstall-root.txt
root in trunk failed.

-1 :x:
compile
18m 15s
/branch-compile-root-jdkUbuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04.txt
root in trunk failed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04.

-1 :x:
compile
0m 28s
/branch-compile-root-jdkPrivateBuild-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01.txt
root in trunk failed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01.

+1 :green_heart:
checkstyle
7m  9s

trunk passed

-1 :x:
mvnsite
0m 36s
/branch-mvnsite-hadoop-common-project_hadoop-common.txt
hadoop-common in trunk failed.

-1 :x:
mvnsite
0m 21s
/branch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt
hadoop-yarn-common in trunk failed.

+1 :green_heart:
shadedclient
26m 40s

branch has no errors when building and testing our client artifacts.

+1 :green_heart:
javadoc
3m 36s

trunk passed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04

+1 :green_heart:
javadoc
4m 16s

trunk passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01

+0 :ok:
spotbugs
0m 44s

Used deprecated FindBugs config; considering switching to SpotBugs.

-1 :x:
findbugs
0m 20s
/branch-findbugs-hadoop-common-project_hadoop-common.txt
hadoop-common in trunk failed.

-1 :x:
findbugs
0m 19s
/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt
hadoop-yarn-common in trunk failed.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 51s

Maven dependency ordering for patch

-1 :x:
mvninstall
0m 12s
/patch-mvninstall-hadoop-common-project_hadoop-common.txt
hadoop-common in the patch failed.

-1 :x:
mvninstall
0m 12s
/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt
hadoop-yarn-common in the patch failed.

-1 :x:
mvninstall
0m 22s
/patch-mvninstall-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt
hadoop-mapreduce-client-core in the patch failed.

-1 :x:
mvninstall
0m 19s
/patch-mvninstall-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt
hadoop-mapreduce-client-app in the patch failed.

-1 :x:
mvninstall
0m 18s
/patch-mvninstall-hadoop-mapreduce-project_hadoop-mapreduce-examples.txt
hadoop-mapreduce-examples in the patch failed.

-1 :x:
mvninstall
0m 16s
/patch-mvninstall-hadoop-tools_hadoop-distcp.txt
hadoop-distcp in the patch failed.

-1 :x:
mvninstall
0m 20s
/patch-mvninstall-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch failed.

-1 :x:
compile
19m 11s
/patch-compile-root-jdkUbuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04.txt
root in the patch failed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04.

-1 :x:
javac
19m 11s
/patch-compile-root-jdkUbuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04.txt
root in the patch failed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04.

-1 :x:
compile
16m 20s
/patch-compile-root-jdkPrivateBuild-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01.txt
root in the patch failed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01.

-1 :x:
javac
16m 20s
/patch-compile-root-jdkPrivateBuild-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01.txt
root in the patch failed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01.

-0 :warning:
checkstyle
3m  1s
/diff-checkstyle-root.txt
root: The patch generated 4 new + 794 unchanged - 1 fixed = 798 total (was 795)

-1 :x:
mvnsite
0m 33s
/patch-mvnsite-hadoop-common-project_hadoop-common.txt
hadoop-common in the patch failed.

-1 :x:
mvnsite
0m 29s
/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt
hadoop-yarn-common in the patch failed.

-1 :x:
mvnsite
0m 40s
/patch-mvnsite-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt
hadoop-mapreduce-client-core in the patch failed.

-1 :x:
mvnsite
0m 33s
/patch-mvnsite-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt
hadoop-mapreduce-client-app in the patch failed.

-1 :x:
mvnsite
0m 32s
/patch-mvnsite-hadoop-mapreduce-project_hadoop-mapreduce-examples.txt
hadoop-mapreduce-examples in the patch failed.

-1 :x:
mvnsite
0m 33s
/patch-mvnsite-hadoop-tools_hadoop-distcp.txt
hadoop-distcp in the patch failed.

-1 :x:
mvnsite
0m 37s
/patch-mvnsite-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch failed.

-1 :x:
whitespace
0m  0s
/whitespace-eol.txt
The patch has 8 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply

+1 :green_heart:
shadedclient
15m 30s

patch has no errors when building and testing our client artifacts.

-1 :x:
javadoc
0m 39s
/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common-jdkUbuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04.txt
hadoop-yarn-common in the patch failed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04.

-1 :x:
javadoc
0m 35s
/patch-javadoc-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdkUbuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04.txt
hadoop-mapreduce-client-app in the patch failed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04.

-1 :x:
javadoc
0m 33s
/patch-javadoc-hadoop-mapreduce-project_hadoop-mapreduce-examples-jdkUbuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04.txt
hadoop-mapreduce-examples in the patch failed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04.

-1 :x:
javadoc
0m 33s
/patch-javadoc-hadoop-tools_hadoop-distcp-jdkUbuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04.txt
hadoop-distcp in the patch failed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04.

-1 :x:
javadoc
1m  7s
/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common-jdkPrivateBuild-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01.txt
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common-jdkPrivateBuild-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 generated 10 new + 4527 unchanged - 0 fixed = 4537 total (was 4527)

-1 :x:
javadoc
0m 38s
/diff-javadoc-javadoc-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdkPrivateBuild-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01.txt
hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdkPrivateBuild-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 generated 4 new + 38 unchanged - 0 fixed = 42 total (was 38)

-1 :x:
javadoc
0m 35s
/diff-javadoc-javadoc-hadoop-mapreduce-project_hadoop-mapreduce-examples-jdkPrivateBuild-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01.txt
hadoop-mapreduce-project_hadoop-mapreduce-examples-jdkPrivateBuild-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 generated 8 new + 143 unchanged - 0 fixed = 151 total (was 143)

-1 :x:
javadoc
0m 34s
/diff-javadoc-javadoc-hadoop-tools_hadoop-distcp-jdkPrivateBuild-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01.txt
hadoop-tools_hadoop-distcp-jdkPrivateBuild-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 generated 4 new + 41 unchanged - 0 fixed = 45 total (was 41)

-1 :x:
javadoc
0m 43s
/patch-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01.txt
hadoop-aws in the patch failed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01.

-1 :x:
findbugs
0m 32s
/patch-findbugs-hadoop-common-project_hadoop-common.txt
hadoop-common in the patch failed.

-1 :x:
findbugs
0m 28s
/patch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt
hadoop-yarn-common in the patch failed.

-1 :x:
findbugs
0m 41s
/patch-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt
hadoop-mapreduce-client-core in the patch failed.

-1 :x:
findbugs
0m 37s
/patch-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt
hadoop-mapreduce-client-app in the patch failed.

-1 :x:
findbugs
0m 36s
/patch-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-examples.txt
hadoop-mapreduce-examples in the patch failed.

-1 :x:
findbugs
0m 31s
/patch-findbugs-hadoop-tools_hadoop-distcp.txt
hadoop-distcp in the patch failed.

-1 :x:
findbugs
0m 38s
/patch-findbugs-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch failed.

_ Other Tests _

-1 :x:
unit
0m 30s
/patch-unit-hadoop-common-project_hadoop-common.txt
hadoop-common in the patch failed.

-1 :x:
unit
0m 29s
/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt
hadoop-yarn-common in the patch failed.

-1 :x:
unit
0m 43s
/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt
hadoop-mapreduce-client-core in the patch failed.

-1 :x:
unit
0m 36s
/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt
hadoop-mapreduce-client-app in the patch failed.

-1 :x:
unit
0m 32s
/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-examples.txt
hadoop-mapreduce-examples in the patch failed.

-1 :x:
unit
0m 33s
/patch-unit-hadoop-tools_hadoop-distcp.txt
hadoop-distcp in the patch failed.

-1 :x:
unit
0m 25s
/patch-unit-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch failed.

+1 :green_heart:
asflicense
0m 53s

The patch does not generate ASF License warnings.

150m 13s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2584/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/2584

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle markdownlint

uname
Linux 744cc5b526c8 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 2825d060cf9

Default Java
Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2584/1/testReport/

Max. process+thread count
538 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-examples hadoop-tools/hadoop-distcp hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2584/1/console

versions
git=2.17.1 maven=3.6.0 findbugs=4.0.6

Powered by
Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

MR client not compiling; not seeing useful information from yetus.

MR client not compiling; not seeing useful information from yetus.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 33s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:

0m  0s
test4tests
The patch appears to include 15 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
13m 49s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
27m 44s

trunk passed

+1 :green_heart:
compile
25m 51s

trunk passed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04

+1 :green_heart:
compile
22m 39s

trunk passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01

+1 :green_heart:
checkstyle
3m 33s

trunk passed

+1 :green_heart:
mvnsite
7m 39s

trunk passed

+1 :green_heart:
shadedclient
31m 37s

branch has no errors when building and testing our client artifacts.

+1 :green_heart:
javadoc
5m 52s

trunk passed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04

+1 :green_heart:
javadoc
6m 32s

trunk passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01

+0 :ok:
spotbugs
1m 31s

Used deprecated FindBugs config; considering switching to SpotBugs.

+1 :green_heart:
findbugs
12m 13s

trunk passed

-0 :warning:
patch
1m 56s

Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 29s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
5m 17s

the patch passed

+1 :green_heart:
compile
19m 33s

the patch passed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04

+1 :green_heart:
javac
19m 33s

the patch passed

+1 :green_heart:
compile
17m 23s

the patch passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01

+1 :green_heart:
javac
17m 23s

the patch passed

-0 :warning:
checkstyle
3m  1s
/diff-checkstyle-root.txt
root: The patch generated 6 new + 822 unchanged - 1 fixed = 828 total (was 823)

+1 :green_heart:
mvnsite
7m 25s

the patch passed

-1 :x:
whitespace
0m  0s
/whitespace-eol.txt
The patch has 14 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply

+1 :green_heart:
shadedclient
15m 28s

patch has no errors when building and testing our client artifacts.

-1 :x:
javadoc
1m  3s
/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04.txt
hadoop-common in the patch failed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04.

+1 :green_heart:
javadoc
6m 36s

the patch passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01

+1 :green_heart:
findbugs
12m  1s

the patch passed

_ Other Tests _

+1 :green_heart:
unit
9m 41s

hadoop-common in the patch passed.

+1 :green_heart:
unit
4m 22s

hadoop-yarn-common in the patch passed.

+1 :green_heart:
unit
7m  7s

hadoop-mapreduce-client-core in the patch passed.

+1 :green_heart:
unit
8m 30s

hadoop-mapreduce-client-app in the patch passed.

+1 :green_heart:
unit
12m 17s

hadoop-distcp in the patch passed.

+1 :green_heart:
unit
0m 59s

hadoop-mapreduce-examples in the patch passed.

+1 :green_heart:
unit
6m 38s

hadoop-streaming in the patch passed.

+1 :green_heart:
unit
1m 45s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
0m 56s

The patch does not generate ASF License warnings.

298m 38s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2584/2/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/2584

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle markdownlint

uname
Linux 79fae6f8feaf 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk /<PHONE_NUMBER>3

Default Java
Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2584/2/testReport/

Max. process+thread count
1665 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-tools/hadoop-distcp hadoop-mapreduce-project/hadoop-mapreduce-examples hadoop-tools/hadoop-streaming hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2584/2/console

versions
git=2.17.1 maven=3.6.0 findbugs=4.0.6

Powered by
Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 33s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:

0m  0s
test4tests
The patch appears to include 15 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
13m 49s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
27m 44s

trunk passed

+1 :green_heart:
compile
25m 51s

trunk passed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04

+1 :green_heart:
compile
22m 39s

trunk passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01

+1 :green_heart:
checkstyle
3m 33s

trunk passed

+1 :green_heart:
mvnsite
7m 39s

trunk passed

+1 :green_heart:
shadedclient
31m 37s

branch has no errors when building and testing our client artifacts.

+1 :green_heart:
javadoc
5m 52s

trunk passed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04

+1 :green_heart:
javadoc
6m 32s

trunk passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01

+0 :ok:
spotbugs
1m 31s

Used deprecated FindBugs config; considering switching to SpotBugs.

+1 :green_heart:
findbugs
12m 13s

trunk passed

-0 :warning:
patch
1m 56s

Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 29s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
5m 17s

the patch passed

+1 :green_heart:
compile
19m 33s

the patch passed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04

+1 :green_heart:
javac
19m 33s

the patch passed

+1 :green_heart:
compile
17m 23s

the patch passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01

+1 :green_heart:
javac
17m 23s

the patch passed

-0 :warning:
checkstyle
3m  1s
/diff-checkstyle-root.txt
root: The patch generated 6 new + 822 unchanged - 1 fixed = 828 total (was 823)

+1 :green_heart:
mvnsite
7m 25s

the patch passed

-1 :x:
whitespace
0m  0s
/whitespace-eol.txt
The patch has 14 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply

+1 :green_heart:
shadedclient
15m 28s

patch has no errors when building and testing our client artifacts.

-1 :x:
javadoc
1m  3s
/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04.txt
hadoop-common in the patch failed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04.

+1 :green_heart:
javadoc
6m 36s

the patch passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01

+1 :green_heart:
findbugs
12m  1s

the patch passed

_ Other Tests _

+1 :green_heart:
unit
9m 41s

hadoop-common in the patch passed.

+1 :green_heart:
unit
4m 22s

hadoop-yarn-common in the patch passed.

+1 :green_heart:
unit
7m  7s

hadoop-mapreduce-client-core in the patch passed.

+1 :green_heart:
unit
8m 30s

hadoop-mapreduce-client-app in the patch passed.

+1 :green_heart:
unit
12m 17s

hadoop-distcp in the patch passed.

+1 :green_heart:
unit
0m 59s

hadoop-mapreduce-examples in the patch passed.

+1 :green_heart:
unit
6m 38s

hadoop-streaming in the patch passed.

+1 :green_heart:
unit
1m 45s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
0m 56s

The patch does not generate ASF License warnings.

298m 38s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2584/2/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/2584

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle markdownlint

uname
Linux 79fae6f8feaf 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk /<PHONE_NUMBER>3

Default Java
Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2584/2/testReport/

Max. process+thread count
1665 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-tools/hadoop-distcp hadoop-mapreduce-project/hadoop-mapreduce-examples hadoop-tools/hadoop-streaming hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2584/2/console

versions
git=2.17.1 maven=3.6.0 findbugs=4.0.6

Powered by
Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

style
./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/functional/FutureIO.java:210:    FSBuilder<T, U> propagateOptions(: 'FSBuilder' has incorrect indentation level 4, expected level should be 6. [Indentation]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java:182:import static org.apache.hadoop.fs.impl.AbstractFSBuilderImpl.rejectUnknownMandatoryKeys;:15: Unused import - org.apache.hadoop.fs.impl.AbstractFSBuilderImpl.rejectUnknownMandatoryKeys. [UnusedImports]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/S3AOpenFileOperation.java:220:        fileStatus = createStatus(path, fileLength, blockSize);: 'if' child has incorrect indentation level 8, expected level should be 6. [Indentation]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/S3AOpenFileOperation.java:221:      }: 'if rcurly' has incorrect indentation level 6, expected level should be 4. [Indentation]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/S3AOpenFileOperation.java:323:    private OpenFileInformation(:13: More than 7 parameters (found 10). [ParameterNumber]
./hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/impl/TestS3AOpenFileOperation.java:119:    assertOpenFile(: 'assertOpenFile' has incorrect indentation level 4, expected level should be 6. [Indentation]

javadocs
[ERROR]                      ^
[ERROR] /home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-2584/src/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/impl/FutureIOSupport.java:126: error: bad use of '>'
[ERROR]    *   fs.example.s3a.option => s3a:option
[ERROR]                               ^
[ERROR] /home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-2584/src/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/impl/FutureIOSupport.java:127: error: bad use of '>'
[ERROR]    *   fs.example.fs.io.policy => s3a.io.policy
[ERROR]                                 ^
[ERROR] /home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-2584/src/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/impl/FutureIOSupport.java:128: error: bad use of '>'
[ERROR]    *   fs.example.something => something
[ERROR]                              ^
[ERROR] /home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-2584/src/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/impl/FutureIOSupport.java:154: error: bad use of '>'
[ERROR]    *   fs.example.s3a.option => s3a:option
[ERROR]                               ^
[ERROR] /home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-2584/src/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/impl/FutureIOSupport.java:155: error: bad use of '>'
[ERROR]    *   fs.example.fs.io.policy => s3a.io.policy
[ERROR]                                 ^
[ERROR] /home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-2584/src/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/impl/FutureIOSupport.java:156: error: bad use of '>'
[ERROR]    *   fs.example.something => something
[ERROR]                              ^
[ERROR] /home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-2584/src/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/functional/FutureIO.java:197: error: bad use of '>'
[ERROR]    *   fs.example.s3a.option => s3a:option
[ERROR]                               ^
[ERROR] /home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-2584/src/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/functional/FutureIO.java:198: error: bad use of '>'
[ERROR]    *   fs.example.fs.io.policy => s3a.io.policy
[ERROR]                                 ^
[ERROR] /home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-2584/src/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/functional/FutureIO.java:199: error: bad use of '>'
[ERROR]    *   fs.example.something => something
[ERROR]                              ^
[ERROR] /home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-2584/src/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/functional/FutureIO.java:227: error: bad use of '>'
[ERROR]    *   fs.example.s3a.option => s3a:option
[ERROR]                               ^
[ERROR] /home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-2584/src/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/functional/FutureIO.java:228: error: bad use of '>'
[ERROR]    *   fs.example.fs.io.policy => s3a.io.policy
[ERROR]                                 ^
[ERROR] /home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-2584/src/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/functional/FutureIO.java:229: error: bad use of '>'
[ERROR]    *   fs.example.something => something

style
./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/functional/FutureIO.java:210:    FSBuilder<T, U> propagateOptions(: 'FSBuilder' has incorrect indentation level 4, expected level should be 6. [Indentation]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java:182:import static org.apache.hadoop.fs.impl.AbstractFSBuilderImpl.rejectUnknownMandatoryKeys;:15: Unused import - org.apache.hadoop.fs.impl.AbstractFSBuilderImpl.rejectUnknownMandatoryKeys. [UnusedImports]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/S3AOpenFileOperation.java:220:        fileStatus = createStatus(path, fileLength, blockSize);: 'if' child has incorrect indentation level 8, expected level should be 6. [Indentation]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/S3AOpenFileOperation.java:221:      }: 'if rcurly' has incorrect indentation level 6, expected level should be 4. [Indentation]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/S3AOpenFileOperation.java:323:    private OpenFileInformation(:13: More than 7 parameters (found 10). [ParameterNumber]
./hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/impl/TestS3AOpenFileOperation.java:119:    assertOpenFile(: 'assertOpenFile' has incorrect indentation level 4, expected level should be 6. [Indentation]

javadocs
[ERROR]                      ^
[ERROR] /home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-2584/src/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/impl/FutureIOSupport.java:126: error: bad use of '>'
[ERROR]    *   fs.example.s3a.option => s3a:option
[ERROR]                               ^
[ERROR] /home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-2584/src/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/impl/FutureIOSupport.java:127: error: bad use of '>'
[ERROR]    *   fs.example.fs.io.policy => s3a.io.policy
[ERROR]                                 ^
[ERROR] /home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-2584/src/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/impl/FutureIOSupport.java:128: error: bad use of '>'
[ERROR]    *   fs.example.something => something
[ERROR]                              ^
[ERROR] /home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-2584/src/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/impl/FutureIOSupport.java:154: error: bad use of '>'
[ERROR]    *   fs.example.s3a.option => s3a:option
[ERROR]                               ^
[ERROR] /home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-2584/src/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/impl/FutureIOSupport.java:155: error: bad use of '>'
[ERROR]    *   fs.example.fs.io.policy => s3a.io.policy
[ERROR]                                 ^
[ERROR] /home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-2584/src/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/impl/FutureIOSupport.java:156: error: bad use of '>'
[ERROR]    *   fs.example.something => something
[ERROR]                              ^
[ERROR] /home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-2584/src/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/functional/FutureIO.java:197: error: bad use of '>'
[ERROR]    *   fs.example.s3a.option => s3a:option
[ERROR]                               ^
[ERROR] /home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-2584/src/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/functional/FutureIO.java:198: error: bad use of '>'
[ERROR]    *   fs.example.fs.io.policy => s3a.io.policy
[ERROR]                                 ^
[ERROR] /home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-2584/src/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/functional/FutureIO.java:199: error: bad use of '>'
[ERROR]    *   fs.example.something => something
[ERROR]                              ^
[ERROR] /home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-2584/src/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/functional/FutureIO.java:227: error: bad use of '>'
[ERROR]    *   fs.example.s3a.option => s3a:option
[ERROR]                               ^
[ERROR] /home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-2584/src/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/functional/FutureIO.java:228: error: bad use of '>'
[ERROR]    *   fs.example.fs.io.policy => s3a.io.policy
[ERROR]                                 ^
[ERROR] /home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-2584/src/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/functional/FutureIO.java:229: error: bad use of '>'
[ERROR]    *   fs.example.something => something

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 34s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  2s

No case conflicting files found.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:

0m  0s
test4tests
The patch appears to include 16 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
13m 56s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
21m  3s

trunk passed

+1 :green_heart:
compile
20m 12s

trunk passed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04

+1 :green_heart:
compile
17m 30s

trunk passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01

+1 :green_heart:
checkstyle
2m 58s

trunk passed

+1 :green_heart:
mvnsite
7m 45s

trunk passed

+1 :green_heart:
shadedclient
26m 36s

branch has no errors when building and testing our client artifacts.

+1 :green_heart:
javadoc
5m 17s

trunk passed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04

+1 :green_heart:
javadoc
6m 13s

trunk passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01

+0 :ok:
spotbugs
1m 16s

Used deprecated FindBugs config; considering switching to SpotBugs.

+1 :green_heart:
findbugs
10m 40s

trunk passed

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 25s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
4m 41s

the patch passed

+1 :green_heart:
compile
19m 28s

the patch passed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04

+1 :green_heart:
javac
19m 28s

the patch passed

+1 :green_heart:
compile
17m 23s

the patch passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01

+1 :green_heart:
javac
17m 23s

the patch passed

-0 :warning:
checkstyle
2m 59s
/diff-checkstyle-root.txt
root: The patch generated 3 new + 822 unchanged - 1 fixed = 825 total (was 823)

+1 :green_heart:
mvnsite
7m 25s

the patch passed

-1 :x:
whitespace
0m  0s
/whitespace-eol.txt
The patch has 16 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply

+1 :green_heart:
shadedclient
15m 19s

patch has no errors when building and testing our client artifacts.

+1 :green_heart:
javadoc
5m 55s

the patch passed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04

+1 :green_heart:
javadoc
6m 42s

the patch passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01

+1 :green_heart:
findbugs
11m 59s

the patch passed

_ Other Tests _

+1 :green_heart:
unit
9m 42s

hadoop-common in the patch passed.

+1 :green_heart:
unit
4m 21s

hadoop-yarn-common in the patch passed.

+1 :green_heart:
unit
7m  9s

hadoop-mapreduce-client-core in the patch passed.

+1 :green_heart:
unit
8m 32s

hadoop-mapreduce-client-app in the patch passed.

+1 :green_heart:
unit
12m 19s

hadoop-distcp in the patch passed.

+1 :green_heart:
unit
1m 11s

hadoop-mapreduce-examples in the patch passed.

+1 :green_heart:
unit
6m 41s

hadoop-streaming in the patch passed.

+1 :green_heart:
unit
1m 47s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
0m 59s

The patch does not generate ASF License warnings.

273m 24s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2584/4/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/2584

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle markdownlint

uname
Linux f4770a6b4bcd 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 630f8ddd2c7

Default Java
Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2584/4/testReport/

Max. process+thread count
1834 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-tools/hadoop-distcp hadoop-mapreduce-project/hadoop-mapreduce-examples hadoop-tools/hadoop-streaming hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2584/4/console

versions
git=2.17.1 maven=3.6.0 findbugs=4.0.6

Powered by
Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 34s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  2s

No case conflicting files found.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:

0m  0s
test4tests
The patch appears to include 16 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
13m 56s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
21m  3s

trunk passed

+1 :green_heart:
compile
20m 12s

trunk passed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04

+1 :green_heart:
compile
17m 30s

trunk passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01

+1 :green_heart:
checkstyle
2m 58s

trunk passed

+1 :green_heart:
mvnsite
7m 45s

trunk passed

+1 :green_heart:
shadedclient
26m 36s

branch has no errors when building and testing our client artifacts.

+1 :green_heart:
javadoc
5m 17s

trunk passed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04

+1 :green_heart:
javadoc
6m 13s

trunk passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01

+0 :ok:
spotbugs
1m 16s

Used deprecated FindBugs config; considering switching to SpotBugs.

+1 :green_heart:
findbugs
10m 40s

trunk passed

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 25s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
4m 41s

the patch passed

+1 :green_heart:
compile
19m 28s

the patch passed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04

+1 :green_heart:
javac
19m 28s

the patch passed

+1 :green_heart:
compile
17m 23s

the patch passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01

+1 :green_heart:
javac
17m 23s

the patch passed

-0 :warning:
checkstyle
2m 59s
/diff-checkstyle-root.txt
root: The patch generated 3 new + 822 unchanged - 1 fixed = 825 total (was 823)

+1 :green_heart:
mvnsite
7m 25s

the patch passed

-1 :x:
whitespace
0m  0s
/whitespace-eol.txt
The patch has 16 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply

+1 :green_heart:
shadedclient
15m 19s

patch has no errors when building and testing our client artifacts.

+1 :green_heart:
javadoc
5m 55s

the patch passed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04

+1 :green_heart:
javadoc
6m 42s

the patch passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01

+1 :green_heart:
findbugs
11m 59s

the patch passed

_ Other Tests _

+1 :green_heart:
unit
9m 42s

hadoop-common in the patch passed.

+1 :green_heart:
unit
4m 21s

hadoop-yarn-common in the patch passed.

+1 :green_heart:
unit
7m  9s

hadoop-mapreduce-client-core in the patch passed.

+1 :green_heart:
unit
8m 32s

hadoop-mapreduce-client-app in the patch passed.

+1 :green_heart:
unit
12m 19s

hadoop-distcp in the patch passed.

+1 :green_heart:
unit
1m 11s

hadoop-mapreduce-examples in the patch passed.

+1 :green_heart:
unit
6m 41s

hadoop-streaming in the patch passed.

+1 :green_heart:
unit
1m 47s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
0m 59s

The patch does not generate ASF License warnings.

273m 24s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2584/4/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/2584

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle markdownlint

uname
Linux f4770a6b4bcd 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 630f8ddd2c7

Default Java
Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2584/4/testReport/

Max. process+thread count
1834 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-tools/hadoop-distcp hadoop-mapreduce-project/hadoop-mapreduce-examples hadoop-tools/hadoop-streaming hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2584/4/console

versions
git=2.17.1 maven=3.6.0 findbugs=4.0.6

Powered by
Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

@ThomasMarquardt could you take a look @ this ?

I've updated the docs as suggested
proposed making the policy broader than just seek policy, so allowing stores to turn on whatever other tuning options they have, especially for file types they've profiled
The goal there is rather than set cluster wide options which work well for some datatypes but are suboptimal for others, the app provides more information down.

weakness there is that with multiple libraries working with Parquet data (spark, parquet.jar, iceberg, impala) it's not enough to declare the format. You'd really need to declare your app and version

@ThomasMarquardt could you take a look @ this ?

I've updated the docs as suggested
proposed making the policy broader than just seek policy, so allowing stores to turn on whatever other tuning options they have, especially for file types they've profiled
The goal there is rather than set cluster wide options which work well for some datatypes but are suboptimal for others, the app provides more information down.

weakness there is that with multiple libraries working with Parquet data (spark, parquet.jar, iceberg, impala) it's not enough to declare the format. You'd really need to declare your app and version

we've already shipped that
it's in createFile too

imagine in future you want to do something more than a hint. Case in point: the SQL API in AWS S3. there's no way an app could issue SQL commands which include transforming the response without knowing whether or not they are processed. hence must().
how about i clarify this in the docs and recommend use of opt() except in such special cases?

we've already shipped that
it's in createFile too

imagine in future you want to do something more than a hint. Case in point: the SQL API in AWS S3. there's no way an app could issue SQL commands which include transforming the response without knowing whether or not they are processed. hence must().
how about i clarify this in the docs and recommend use of opt() except in such special cases?

OK, so must() has already shipped, in that case sure, it should stay.  I still have a question.  If I pass an option via must(), can it be ignored by the implementation?  I read your documentation, specifically, "This means that it is not a requirement for the stores to actually read the the read policy or file length values and use them when opening files."  To me, this says that options passed via must() can be ignored.  If this is true, then what is the difference between must() and opt()?  Is it that must() can result in an unsupported exception, but opt() never results in an unsupported exception?

OK, so must() has already shipped, in that case sure, it should stay.  I still have a question.  If I pass an option via must(), can it be ignored by the implementation?  I read your documentation, specifically, "This means that it is not a requirement for the stores to actually read the the read policy or file length values and use them when opening files."  To me, this says that options passed via must() can be ignored.  If this is true, then what is the difference between must() and opt()?  Is it that must() can result in an unsupported exception, but opt() never results in an unsupported exception?

correct.
must() == raise an error if not recognised.
opt() entirely optional
note that createFile() has the same api/rules/ common builder codebase. i hope to add some options there for the stores to say "skip all probes for paths existing, being a dir, having a parent..." goal is max performance writes on paths where the writer knows it is building up a dir tree from scratch, or using unique names. in particular: flink/spark checkpoints

correct.
must() == raise an error if not recognised.
opt() entirely optional
note that createFile() has the same api/rules/ common builder codebase. i hope to add some options there for the stores to say "skip all probes for paths existing, being a dir, having a parent..." goal is max performance writes on paths where the writer knows it is building up a dir tree from scratch, or using unique names. in particular: flink/spark checkpoints

Thomas, to clarify a bit more
yes, an FS can choose to ignore an option, -but it must recognise the option and so make a conscious decision "this doesn't matter"

so with the standard length/read policy options, an FS can concude "I see that and don't support it", and just ignore it -a decision made on an option-by-option basis.
if something unrecognised comes in as .must() the FS must raise an exception "I don't recognise that".
if something it does recognise as a .must() which it knows it doesn't support and which it MUST be able to, then it MUST raise an exception.

Example: vectored IO will only work if the FS supports seek, which ftp doesn't. Assuming we add an option to say "I Want vectored IO" then all filesystems will get this out the box as a well known-and-supported feature, but ftp will need some explicit code to say "fail if vectored IO is requested"

Thomas, to clarify a bit more
yes, an FS can choose to ignore an option, -but it must recognise the option and so make a conscious decision "this doesn't matter"

so with the standard length/read policy options, an FS can concude "I see that and don't support it", and just ignore it -a decision made on an option-by-option basis.
if something unrecognised comes in as .must() the FS must raise an exception "I don't recognise that".
if something it does recognise as a .must() which it knows it doesn't support and which it MUST be able to, then it MUST raise an exception.

Example: vectored IO will only work if the FS supports seek, which ftp doesn't. Assuming we add an option to say "I Want vectored IO" then all filesystems will get this out the box as a well known-and-supported feature, but ftp will need some explicit code to say "fail if vectored IO is requested"

rebased to fix compile problems; the final patch is the one with changes since Thomas's last review

move all the text on options into the fsdatainputstreambuilder.md file
try to define the validation of must/opt in pseudo-python
make sure the doc is consistent in saying FNFE and permission issues MUST be delayed until future.get() and MAY be delayed until the first actual read.  This is to give object stores the maximum time for async probes. Example: you could initiate a HEAD request in  build() but not block for its completion until read()

S3A added some extra tuning of the input stream/openFile logic, so the stream no longer gets access to the S3 client, simply callbacks
Testing: in progress

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 30s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  2s

No case conflicting files found.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:

0m  0s
test4tests
The patch appears to include 16 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
14m  7s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
20m 12s

trunk passed

+1 :green_heart:
compile
20m 45s

trunk passed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
compile
18m  6s

trunk passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~20.04-b01

+1 :green_heart:
checkstyle
4m 11s

trunk passed

+1 :green_heart:
mvnsite
7m 20s

trunk passed

+1 :green_heart:
shadedclient
25m 33s

branch has no errors when building and testing our client artifacts.

+1 :green_heart:
javadoc
5m 54s

trunk passed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javadoc
6m 46s

trunk passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~20.04-b01

+0 :ok:
spotbugs
1m 16s

Used deprecated FindBugs config; considering switching to SpotBugs.

+1 :green_heart:
findbugs
10m 53s

trunk passed

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 25s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
4m 21s

the patch passed

+1 :green_heart:
compile
20m  8s

the patch passed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javac
20m  8s

the patch passed

+1 :green_heart:
compile
18m  3s

the patch passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~20.04-b01

+1 :green_heart:
javac
18m  3s

the patch passed

-0 :warning:
checkstyle
4m  0s
/diff-checkstyle-root.txt
root: The patch generated 5 new + 821 unchanged - 1 fixed = 826 total (was 822)

+1 :green_heart:
mvnsite
7m 22s

the patch passed

-1 :x:
whitespace
0m  0s
/whitespace-eol.txt
The patch has 10 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply

+1 :green_heart:
shadedclient
13m 12s

patch has no errors when building and testing our client artifacts.

+1 :green_heart:
javadoc
5m 48s

the patch passed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javadoc
6m 44s

the patch passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~20.04-b01

+1 :green_heart:
findbugs
11m 59s

the patch passed

_ Other Tests _

+1 :green_heart:
unit
17m 54s

hadoop-common in the patch passed.

+1 :green_heart:
unit
4m 43s

hadoop-yarn-common in the patch passed.

+1 :green_heart:
unit
7m 48s

hadoop-mapreduce-client-core in the patch passed.

+1 :green_heart:
unit
9m 49s

hadoop-mapreduce-client-app in the patch passed.

+1 :green_heart:
unit
15m 45s

hadoop-distcp in the patch passed.

+1 :green_heart:
unit
1m  9s

hadoop-mapreduce-examples in the patch passed.

-1 :x:
unit
6m 46s
/patch-unit-hadoop-tools_hadoop-streaming.txt
hadoop-streaming in the patch passed.

+1 :green_heart:
unit
2m 34s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
1m  1s

The patch does not generate ASF License warnings.

288m 49s

Reason
Tests

Failed junit tests
hadoop.streaming.TestSymLink

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2584/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/2584

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle markdownlint

uname
Linux c4889f76d37a 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 21a3fc3d2d7

Default Java
Private Build-1.8.0_275-8u275-b01-0ubuntu1~20.04-b01

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_275-8u275-b01-0ubuntu1~20.04-b01

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2584/1/testReport/

Max. process+thread count
3231 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-tools/hadoop-distcp hadoop-mapreduce-project/hadoop-mapreduce-examples hadoop-tools/hadoop-streaming hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2584/1/console

versions
git=2.25.1 maven=3.6.3 findbugs=4.0.6

Powered by
Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
14m 30s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
markdownlint
0m  1s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:

0m  0s
test4tests
The patch appears to include 16 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
14m  6s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
20m 21s

trunk passed

+1 :green_heart:
compile
21m 19s

trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
compile
19m  3s

trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
checkstyle
3m 59s

trunk passed

+1 :green_heart:
mvnsite
7m  6s

trunk passed

+1 :green_heart:
shadedclient
24m 48s

branch has no errors when building and testing our client artifacts.

+1 :green_heart:
javadoc
5m 25s

trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javadoc
6m 13s

trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+0 :ok:
spotbugs
1m 16s

Used deprecated FindBugs config; considering switching to SpotBugs.

+1 :green_heart:
findbugs
10m 52s

trunk passed

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 25s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
4m 26s

the patch passed

+1 :green_heart:
compile
20m  3s

the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javac
20m  2s

the patch passed

+1 :green_heart:
compile
18m  1s

the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
javac
18m  1s

the patch passed

-0 :warning:
checkstyle
4m  0s
/diff-checkstyle-root.txt
root: The patch generated 5 new + 821 unchanged - 1 fixed = 826 total (was 822)

+1 :green_heart:
mvnsite
7m 18s

the patch passed

-1 :x:
whitespace
0m  0s
/whitespace-eol.txt
The patch has 10 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply

+1 :green_heart:
shadedclient
13m  8s

patch has no errors when building and testing our client artifacts.

+1 :green_heart:
javadoc
5m 52s

the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javadoc
6m 46s

the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
findbugs
11m 55s

the patch passed

_ Other Tests _

+1 :green_heart:
unit
17m 49s

hadoop-common in the patch passed.

+1 :green_heart:
unit
4m 20s

hadoop-yarn-common in the patch passed.

+1 :green_heart:
unit
7m 17s

hadoop-mapreduce-client-core in the patch passed.

+1 :green_heart:
unit
8m 29s

hadoop-mapreduce-client-app in the patch passed.

+1 :green_heart:
unit
12m 16s

hadoop-distcp in the patch passed.

+1 :green_heart:
unit
0m 59s

hadoop-mapreduce-examples in the patch passed.

+1 :green_heart:
unit
6m 38s

hadoop-streaming in the patch passed.

+1 :green_heart:
unit
2m 10s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
0m 55s

The patch does not generate ASF License warnings.

295m 59s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2584/2/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/2584

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle markdownlint

uname
Linux 00a870d16cf0 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / c3134ab3a99

Default Java
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2584/2/testReport/

Max. process+thread count
1529 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-tools/hadoop-distcp hadoop-mapreduce-project/hadoop-mapreduce-examples hadoop-tools/hadoop-streaming hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2584/2/console

versions
git=2.25.1 maven=3.6.3 findbugs=4.0.6

Powered by
Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 39s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  2s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
markdownlint
0m  1s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 16 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
14m 19s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
20m  8s

trunk passed

+1 :green_heart:
compile
20m 58s

trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
compile
18m  2s

trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
checkstyle
3m 55s

trunk passed

+1 :green_heart:
mvnsite
7m 24s

trunk passed

-1 :x:
javadoc
1m  4s
/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04.txt
hadoop-common in trunk failed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04.

+1 :green_heart:
javadoc
6m 43s

trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
spotbugs
11m  6s

trunk passed

+1 :green_heart:
shadedclient
14m 27s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 25s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
4m 25s

the patch passed

+1 :green_heart:
compile
20m  1s

the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javac
20m  1s

the patch passed

+1 :green_heart:
compile
17m 59s

the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
javac
17m 59s

the patch passed

-1 :x:
blanks
0m  0s
/blanks-eol.txt
The patch has 11 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply

-0 :warning:
checkstyle
4m 26s
/results-checkstyle-root.txt
root: The patch generated 3 new + 821 unchanged - 2 fixed = 824 total (was 823)

+1 :green_heart:
mvnsite
7m 29s

the patch passed

-1 :x:
javadoc
1m  3s
/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04.txt
hadoop-common in the patch failed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04.

+1 :green_heart:
javadoc
6m 40s

the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
spotbugs
12m 38s

the patch passed

+1 :green_heart:
shadedclient
14m 39s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

-1 :x:
unit
18m 53s
/patch-unit-hadoop-common-project_hadoop-common.txt
hadoop-common in the patch passed.

+1 :green_heart:
unit
4m 55s

hadoop-yarn-common in the patch passed.

+1 :green_heart:
unit
7m 10s

hadoop-mapreduce-client-core in the patch passed.

+1 :green_heart:
unit
8m 36s

hadoop-mapreduce-client-app in the patch passed.

-1 :x:
unit
15m 46s
/patch-unit-hadoop-tools_hadoop-distcp.txt
hadoop-distcp in the patch passed.

+1 :green_heart:
unit
0m 57s

hadoop-mapreduce-examples in the patch passed.

+1 :green_heart:
unit
6m 44s

hadoop-streaming in the patch passed.

+1 :green_heart:
unit
2m  9s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
0m 59s

The patch does not generate ASF License warnings.

289m 18s

Reason
Tests

Failed junit tests
hadoop.ha.TestZKFailoverController

hadoop.tools.TestDistCpSystem

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2584/3/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/2584

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell markdownlint

uname
Linux 6d94139baff3 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / f5013f9993feaaacb03bfaec03114d64fb5914af

Default Java
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2584/3/testReport/

Max. process+thread count
3040 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-tools/hadoop-distcp hadoop-mapreduce-project/hadoop-mapreduce-examples hadoop-tools/hadoop-streaming hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2584/3/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 51s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 16 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
15m 14s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
23m  2s

trunk passed

+1 :green_heart:
compile
22m 12s

trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
compile
19m  0s

trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
checkstyle
4m  2s

trunk passed

+1 :green_heart:
mvnsite
6m 17s

trunk passed

+1 :green_heart:
javadoc
4m 46s

trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javadoc
5m 38s

trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
spotbugs
10m 19s

trunk passed

+1 :green_heart:
shadedclient
17m  4s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 21s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
4m 17s

the patch passed

+1 :green_heart:
compile
21m 39s

the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javac
21m 39s

the patch passed

+1 :green_heart:
compile
19m  0s

the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
javac
19m  0s

the patch passed

-1 :x:
blanks
0m  0s
/blanks-eol.txt
The patch has 11 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply

-0 :warning:
checkstyle
4m  6s
/results-checkstyle-root.txt
root: The patch generated 3 new + 820 unchanged - 1 fixed = 823 total (was 821)

+1 :green_heart:
mvnsite
6m 15s

the patch passed

+1 :green_heart:
javadoc
4m 45s

the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javadoc
5m 40s

the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
spotbugs
11m 53s

the patch passed

+1 :green_heart:
shadedclient
17m  8s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
17m 25s

hadoop-common in the patch passed.

+1 :green_heart:
unit
4m 42s

hadoop-yarn-common in the patch passed.

+1 :green_heart:
unit
7m  5s

hadoop-mapreduce-client-core in the patch passed.

+1 :green_heart:
unit
8m 22s

hadoop-mapreduce-client-app in the patch passed.

+1 :green_heart:
unit
37m 59s

hadoop-distcp in the patch passed.

+1 :green_heart:
unit
1m  2s

hadoop-mapreduce-examples in the patch passed.

+1 :green_heart:
unit
6m 49s

hadoop-streaming in the patch passed.

+1 :green_heart:
unit
2m 32s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
0m 50s

The patch does not generate ASF License warnings.

314m 16s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2584/5/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/2584

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell markdownlint

uname
Linux 3b389c521b83 4.15.0-126-generic #129-Ubuntu SMP Mon Nov 23 18:53:38 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 340be222f9f674f7100e23bf3844354c410ab421

Default Java
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2584/5/testReport/

Max. process+thread count
1638 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-tools/hadoop-distcp hadoop-mapreduce-project/hadoop-mapreduce-examples hadoop-tools/hadoop-streaming hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2584/5/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 53s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
markdownlint
0m  1s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 16 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
15m 47s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
22m 58s

trunk passed

+1 :green_heart:
compile
22m 19s

trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
compile
18m 59s

trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
checkstyle
4m  4s

trunk passed

+1 :green_heart:
mvnsite
6m 19s

trunk passed

+1 :green_heart:
javadoc
4m 45s

trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javadoc
5m 36s

trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
spotbugs
10m 22s

trunk passed

+1 :green_heart:
shadedclient
17m  4s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 20s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
4m 14s

the patch passed

+1 :green_heart:
compile
21m 37s

the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

-1 :x:
javac
21m 37s
/results-compile-javac-root-jdkUbuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04.txt
root-jdkUbuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 generated 1 new + 1933 unchanged - 0 fixed = 1934 total (was 1933)

+1 :green_heart:
compile
18m 55s

the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

-1 :x:
javac
18m 55s
/results-compile-javac-root-jdkPrivateBuild-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08.txt
root-jdkPrivateBuild-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 generated 1 new + 1829 unchanged - 0 fixed = 1830 total (was 1829)

-1 :x:
blanks
0m  0s
/blanks-eol.txt
The patch has 11 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply

-0 :warning:
checkstyle
4m  3s
/results-checkstyle-root.txt
root: The patch generated 3 new + 820 unchanged - 1 fixed = 823 total (was 821)

+1 :green_heart:
mvnsite
6m 16s

the patch passed

+1 :green_heart:
javadoc
4m 42s

the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javadoc
5m 36s

the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
spotbugs
13m 13s

the patch passed

+1 :green_heart:
shadedclient
17m 46s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
18m 38s

hadoop-common in the patch passed.

+1 :green_heart:
unit
5m 20s

hadoop-yarn-common in the patch passed.

+1 :green_heart:
unit
7m 32s

hadoop-mapreduce-client-core in the patch passed.

+1 :green_heart:
unit
8m 25s

hadoop-mapreduce-client-app in the patch passed.

+1 :green_heart:
unit
39m  6s

hadoop-distcp in the patch passed.

+1 :green_heart:
unit
0m 50s

hadoop-mapreduce-examples in the patch passed.

+1 :green_heart:
unit
6m 47s

hadoop-streaming in the patch passed.

+1 :green_heart:
unit
2m 31s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
0m 50s

The patch does not generate ASF License warnings.

319m 30s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2584/6/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/2584

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell markdownlint

uname
Linux 0d25d4d9dc4e 4.15.0-126-generic #129-Ubuntu SMP Mon Nov 23 18:53:38 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 700f99835f21571d9f29f2194327894776abe13f

Default Java
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2584/6/testReport/

Max. process+thread count
1275 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-tools/hadoop-distcp hadoop-mapreduce-project/hadoop-mapreduce-examples hadoop-tools/hadoop-streaming hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2584/6/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
1m 10s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
markdownlint
0m  1s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 17 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
12m 32s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
25m  6s

trunk passed

+1 :green_heart:
compile
28m 17s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
compile
23m 40s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
checkstyle
4m  1s

trunk passed

+1 :green_heart:
mvnsite
7m 14s

trunk passed

+1 :green_heart:
javadoc
5m 24s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
5m 38s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
11m  2s

trunk passed

+1 :green_heart:
shadedclient
18m 39s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 27s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
5m  9s

the patch passed

+1 :green_heart:
compile
26m  2s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

-1 :x:
javac
26m  2s
/results-compile-javac-root-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.txt
root-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 generated 1 new + 1924 unchanged - 0 fixed = 1925 total (was 1924)

+1 :green_heart:
compile
21m  5s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

-1 :x:
javac
21m  5s
/results-compile-javac-root-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt
root-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 generated 1 new + 1797 unchanged - 0 fixed = 1798 total (was 1797)

-1 :x:
blanks
0m  0s
/blanks-eol.txt
The patch has 13 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply

-0 :warning:
checkstyle
3m 47s
/results-checkstyle-root.txt
root: The patch generated 8 new + 853 unchanged - 1 fixed = 861 total (was 854)

+1 :green_heart:
mvnsite
6m 41s

the patch passed

+1 :green_heart:
javadoc
5m 18s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
5m 53s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

-1 :x:
spotbugs
1m 31s
/new-spotbugs-hadoop-tools_hadoop-aws.html
hadoop-tools/hadoop-aws generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)

+1 :green_heart:
shadedclient
17m 33s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
17m  8s

hadoop-common in the patch passed.

+1 :green_heart:
unit
4m 43s

hadoop-yarn-common in the patch passed.

+1 :green_heart:
unit
7m 11s

hadoop-mapreduce-client-core in the patch passed.

+1 :green_heart:
unit
8m 23s

hadoop-mapreduce-client-app in the patch passed.

+1 :green_heart:
unit
41m 59s

hadoop-distcp in the patch passed.

+1 :green_heart:
unit
0m 51s

hadoop-mapreduce-examples in the patch passed.

+1 :green_heart:
unit
6m 49s

hadoop-streaming in the patch passed.

+1 :green_heart:
unit
2m 37s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
0m 51s

The patch does not generate ASF License warnings.

341m 39s

Reason
Tests

SpotBugs
module:hadoop-tools/hadoop-aws

storeContext must be non-null but is marked as nullable  At AbstractStoreOperation.java:is marked as nullable  At AbstractStoreOperation.java:[lines 50-51]

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2584/9/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/2584

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell markdownlint

uname
Linux 2f0dd3878f2c 4.15.0-153-generic #160-Ubuntu SMP Thu Jul 29 06:54:29 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / c26e1339b8dac311917dbc72fa930e1e03673db5

Default Java
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2584/9/testReport/

Max. process+thread count
2299 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-tools/hadoop-distcp hadoop-mapreduce-project/hadoop-mapreduce-examples hadoop-tools/hadoop-streaming hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2584/9/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 49s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  2s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 17 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
12m 50s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
23m  5s

trunk passed

+1 :green_heart:
compile
24m 58s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
compile
21m 10s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
checkstyle
3m 56s

trunk passed

+1 :green_heart:
mvnsite
8m 30s

trunk passed

+1 :green_heart:
javadoc
6m 56s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
7m 25s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
12m 40s

trunk passed

+1 :green_heart:
shadedclient
22m  0s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 30s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
5m 12s

the patch passed

+1 :green_heart:
compile
23m 56s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

-1 :x:
javac
23m 56s
/results-compile-javac-root-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.txt
root-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 generated 1 new + 1889 unchanged - 0 fixed = 1890 total (was 1889)

+1 :green_heart:
compile
21m 11s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

-1 :x:
javac
21m 11s
/results-compile-javac-root-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt
root-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 generated 1 new + 1763 unchanged - 0 fixed = 1764 total (was 1763)

-1 :x:
blanks
0m  0s
/blanks-eol.txt
The patch has 12 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply

-0 :warning:
checkstyle
3m 56s
/results-checkstyle-root.txt
root: The patch generated 9 new + 835 unchanged - 1 fixed = 844 total (was 836)

+1 :green_heart:
mvnsite
8m 30s

the patch passed

+1 :green_heart:
javadoc
6m 56s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
7m 23s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

-1 :x:
spotbugs
1m 51s
/new-spotbugs-hadoop-tools_hadoop-aws.html
hadoop-tools/hadoop-aws generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)

+1 :green_heart:
shadedclient
22m  7s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
18m 20s

hadoop-common in the patch passed.

+1 :green_heart:
unit
5m 15s

hadoop-yarn-common in the patch passed.

+1 :green_heart:
unit
6m  3s

hadoop-mapreduce-client-core in the patch passed.

+1 :green_heart:
unit
9m  5s

hadoop-mapreduce-client-app in the patch passed.

+1 :green_heart:
unit
21m 36s

hadoop-distcp in the patch passed.

+1 :green_heart:
unit
1m  5s

hadoop-mapreduce-examples in the patch passed.

+1 :green_heart:
unit
7m  7s

hadoop-streaming in the patch passed.

+1 :green_heart:
unit
2m 47s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
1m  6s

The patch does not generate ASF License warnings.

335m 48s

Reason
Tests

SpotBugs
module:hadoop-tools/hadoop-aws

storeContext must be non-null but is marked as nullable  At AbstractStoreOperation.java:is marked as nullable  At AbstractStoreOperation.java:[lines 51-52]

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2584/10/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/2584

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell markdownlint

uname
Linux 886169ca5bb5 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / fe4fa5c697dcaf2433324f422842851eac2a3de3

Default Java
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2584/10/testReport/

Max. process+thread count
1549 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-tools/hadoop-distcp hadoop-mapreduce-project/hadoop-mapreduce-examples hadoop-tools/hadoop-streaming hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2584/10/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 51s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  2s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 16 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
11m 38s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
21m 44s

trunk passed

+1 :green_heart:
compile
22m 12s

trunk passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
compile
19m 29s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
checkstyle
3m 44s

trunk passed

+1 :green_heart:
mvnsite
7m 59s

trunk passed

+1 :green_heart:
javadoc
6m 35s

trunk passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javadoc
7m  5s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
11m 19s

trunk passed

+1 :green_heart:
shadedclient
20m 51s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 28s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
4m 43s

the patch passed

+1 :green_heart:
compile
21m 37s

the patch passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04

-1 :x:
javac
21m 37s
/results-compile-javac-root-jdkUbuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04.txt
root-jdkUbuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 generated 2 new + 1811 unchanged - 0 fixed = 1813 total (was 1811)

+1 :green_heart:
compile
19m 33s

the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

-1 :x:
javac
19m 33s
/results-compile-javac-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt
root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 2 new + 1685 unchanged - 0 fixed = 1687 total (was 1685)

-1 :x:
blanks
0m  0s
/blanks-eol.txt
The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply

-0 :warning:
checkstyle
3m 43s
/results-checkstyle-root.txt
root: The patch generated 1 new + 818 unchanged - 1 fixed = 819 total (was 819)

+1 :green_heart:
mvnsite
7m 56s

the patch passed

+1 :green_heart:
javadoc
6m 28s

the patch passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javadoc
7m  4s

the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

-1 :x:
spotbugs
1m 32s
/new-spotbugs-hadoop-tools_hadoop-aws.html
hadoop-tools/hadoop-aws generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)

+1 :green_heart:
shadedclient
21m 21s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
17m 30s

hadoop-common in the patch passed.

+1 :green_heart:
unit
5m  0s

hadoop-yarn-common in the patch passed.

+1 :green_heart:
unit
5m 44s

hadoop-mapreduce-client-core in the patch passed.

+1 :green_heart:
unit
8m 36s

hadoop-mapreduce-client-app in the patch passed.

+1 :green_heart:
unit
20m 31s

hadoop-distcp in the patch passed.

+1 :green_heart:
unit
0m 55s

hadoop-mapreduce-examples in the patch passed.

+1 :green_heart:
unit
6m 42s

hadoop-streaming in the patch passed.

+1 :green_heart:
unit
2m 20s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
0m 58s

The patch does not generate ASF License warnings.

312m 34s

Reason
Tests

SpotBugs
module:hadoop-tools/hadoop-aws

Null passed for non-null parameter of new AbstractStoreOperation(StoreContext, AuditSpan) in new org.apache.hadoop.fs.s3a.impl.AbstractStoreOperation(StoreContext)  Method invoked at AbstractStoreOperation.java:of new AbstractStoreOperation(StoreContext, AuditSpan) in new org.apache.hadoop.fs.s3a.impl.AbstractStoreOperation(StoreContext)  Method invoked at AbstractStoreOperation.java:[line 51]

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2584/12/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/2584

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell markdownlint

uname
Linux f494e5a13055 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 47b4335e1d34f9aacb1e270c745e5767ffbd9ddc

Default Java
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2584/12/testReport/

Max. process+thread count
1574 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-tools/hadoop-distcp hadoop-mapreduce-project/hadoop-mapreduce-examples hadoop-tools/hadoop-streaming hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2584/12/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
1m  9s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
markdownlint
0m  1s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 16 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
11m 32s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
24m 28s

trunk passed

+1 :green_heart:
compile
25m 11s

trunk passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
compile
20m 59s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
checkstyle
3m 58s

trunk passed

+1 :green_heart:
mvnsite
7m 52s

trunk passed

+1 :green_heart:
javadoc
6m 38s

trunk passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javadoc
6m 54s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
11m 48s

trunk passed

+1 :green_heart:
shadedclient
23m 15s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 30s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
5m  1s

the patch passed

+1 :green_heart:
compile
24m  4s

the patch passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04

-1 :x:
javac
24m  4s
/results-compile-javac-root-jdkUbuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04.txt
root-jdkUbuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 generated 2 new + 1811 unchanged - 0 fixed = 1813 total (was 1811)

+1 :green_heart:
compile
21m 10s

the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

-1 :x:
javac
21m 10s
/results-compile-javac-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt
root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 2 new + 1685 unchanged - 0 fixed = 1687 total (was 1685)

-1 :x:
blanks
0m  0s
/blanks-eol.txt
The patch has 2 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply

-0 :warning:
checkstyle
3m 55s
/results-checkstyle-root.txt
root: The patch generated 2 new + 836 unchanged - 1 fixed = 838 total (was 837)

+1 :green_heart:
mvnsite
8m 12s

the patch passed

+1 :green_heart:
javadoc
6m 25s

the patch passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javadoc
6m 52s

the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

-1 :x:
spotbugs
1m 38s
/new-spotbugs-hadoop-tools_hadoop-aws.html
hadoop-tools/hadoop-aws generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)

+1 :green_heart:
shadedclient
23m 27s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
18m 15s

hadoop-common in the patch passed.

+1 :green_heart:
unit
5m  7s

hadoop-yarn-common in the patch passed.

+1 :green_heart:
unit
6m 26s

hadoop-mapreduce-client-core in the patch passed.

+1 :green_heart:
unit
8m 54s

hadoop-mapreduce-client-app in the patch passed.

+1 :green_heart:
unit
56m 55s

hadoop-distcp in the patch passed.

+1 :green_heart:
unit
1m  0s

hadoop-mapreduce-examples in the patch passed.

+1 :green_heart:
unit
7m 53s

hadoop-streaming in the patch passed.

+1 :green_heart:
unit
3m 13s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
1m  4s

The patch does not generate ASF License warnings.

370m 32s

Reason
Tests

SpotBugs
module:hadoop-tools/hadoop-aws

Null passed for non-null parameter of new AbstractStoreOperation(StoreContext, AuditSpan) in new org.apache.hadoop.fs.s3a.impl.AbstractStoreOperation(StoreContext)  Method invoked at AbstractStoreOperation.java:of new AbstractStoreOperation(StoreContext, AuditSpan) in new org.apache.hadoop.fs.s3a.impl.AbstractStoreOperation(StoreContext)  Method invoked at AbstractStoreOperation.java:[line 51]

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2584/13/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/2584

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell markdownlint

uname
Linux 17b0eac715b9 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / c457c4c52bc2b4f006137c145feeda5749285b56

Default Java
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2584/13/testReport/

Max. process+thread count
3152 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-tools/hadoop-distcp hadoop-mapreduce-project/hadoop-mapreduce-examples hadoop-tools/hadoop-streaming hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2584/13/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 45s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  2s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
markdownlint
0m  1s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 16 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
11m 33s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
22m 44s

trunk passed

+1 :green_heart:
compile
27m  9s

trunk passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
compile
27m 46s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
checkstyle
5m 10s

trunk passed

+1 :green_heart:
mvnsite
7m 55s

trunk passed

+1 :green_heart:
javadoc
6m 45s

trunk passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javadoc
6m 46s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

-1 :x:
spotbugs
2m  1s
/branch-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt
hadoop-yarn-common in trunk failed.

-1 :x:
spotbugs
0m 59s
/branch-spotbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt
hadoop-mapreduce-client-app in trunk failed.

-1 :x:
spotbugs
0m 34s
/branch-spotbugs-hadoop-tools_hadoop-distcp.txt
hadoop-distcp in trunk failed.

-1 :x:
spotbugs
0m 25s
/branch-spotbugs-hadoop-mapreduce-project_hadoop-mapreduce-examples.txt
hadoop-mapreduce-examples in trunk failed.

-1 :x:
spotbugs
0m 42s
/branch-spotbugs-hadoop-tools_hadoop-streaming.txt
hadoop-streaming in trunk failed.

-1 :x:
spotbugs
0m 40s
/branch-spotbugs-hadoop-tools_hadoop-aws.txt
hadoop-aws in trunk failed.

+1 :green_heart:
shadedclient
2m 59s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 20s

Maven dependency ordering for patch

-1 :x:
mvninstall
0m 24s
/patch-mvninstall-hadoop-common-project_hadoop-common.txt
hadoop-common in the patch failed.

-1 :x:
mvninstall
0m 22s
/patch-mvninstall-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt
hadoop-mapreduce-client-app in the patch failed.

-1 :x:
mvninstall
0m 23s
/patch-mvninstall-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt
hadoop-mapreduce-client-core in the patch failed.

-1 :x:
mvninstall
0m 22s
/patch-mvninstall-hadoop-mapreduce-project_hadoop-mapreduce-examples.txt
hadoop-mapreduce-examples in the patch failed.

-1 :x:
mvninstall
0m 24s
/patch-mvninstall-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch failed.

-1 :x:
mvninstall
0m 23s
/patch-mvninstall-hadoop-tools_hadoop-distcp.txt
hadoop-distcp in the patch failed.

-1 :x:
mvninstall
0m 22s
/patch-mvninstall-hadoop-tools_hadoop-streaming.txt
hadoop-streaming in the patch failed.

-1 :x:
mvninstall
0m 23s
/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt
hadoop-yarn-common in the patch failed.

-1 :x:
compile
0m 23s
/patch-compile-root-jdkUbuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04.txt
root in the patch failed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04.

-1 :x:
javac
0m 23s
/patch-compile-root-jdkUbuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04.txt
root in the patch failed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04.

-1 :x:
compile
0m 23s
/patch-compile-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt
root in the patch failed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.

-1 :x:
javac
0m 23s
/patch-compile-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt
root in the patch failed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
0m 21s
/buildtool-patch-checkstyle-root.txt
The patch fails to run checkstyle in root

-1 :x:
mvnsite
0m 23s
/patch-mvnsite-hadoop-common-project_hadoop-common.txt
hadoop-common in the patch failed.

-1 :x:
mvnsite
0m 22s
/patch-mvnsite-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt
hadoop-mapreduce-client-app in the patch failed.

-1 :x:
mvnsite
0m 22s
/patch-mvnsite-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt
hadoop-mapreduce-client-core in the patch failed.

-1 :x:
mvnsite
0m 23s
/patch-mvnsite-hadoop-mapreduce-project_hadoop-mapreduce-examples.txt
hadoop-mapreduce-examples in the patch failed.

-1 :x:
mvnsite
0m 22s
/patch-mvnsite-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch failed.

-1 :x:
mvnsite
0m 23s
/patch-mvnsite-hadoop-tools_hadoop-distcp.txt
hadoop-distcp in the patch failed.

-1 :x:
mvnsite
0m 23s
/patch-mvnsite-hadoop-tools_hadoop-streaming.txt
hadoop-streaming in the patch failed.

-1 :x:
mvnsite
0m 22s
/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt
hadoop-yarn-common in the patch failed.

-1 :x:
javadoc
0m 22s
/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04.txt
hadoop-common in the patch failed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04.

-1 :x:
javadoc
0m 23s
/patch-javadoc-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdkUbuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04.txt
hadoop-mapreduce-client-app in the patch failed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04.

-1 :x:
javadoc
0m 23s
/patch-javadoc-hadoop-mapreduce-project_hadoop-mapreduce-examples-jdkUbuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04.txt
hadoop-mapreduce-examples in the patch failed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04.

-1 :x:
javadoc
0m 23s
/patch-javadoc-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04.txt
hadoop-aws in the patch failed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04.

-1 :x:
javadoc
0m 23s
/patch-javadoc-hadoop-tools_hadoop-distcp-jdkUbuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04.txt
hadoop-distcp in the patch failed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04.

-1 :x:
javadoc
0m 28s
/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common-jdkUbuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04.txt
hadoop-yarn-common in the patch failed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04.

-1 :x:
javadoc
0m 25s
/results-javadoc-javadoc-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt
hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 4 new + 38 unchanged - 0 fixed = 42 total (was 38)

-1 :x:
javadoc
0m 22s
/results-javadoc-javadoc-hadoop-mapreduce-project_hadoop-mapreduce-examples-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt
hadoop-mapreduce-project_hadoop-mapreduce-examples-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 8 new + 143 unchanged - 0 fixed = 151 total (was 143)

-1 :x:
javadoc
0m 28s
/patch-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt
hadoop-aws in the patch failed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.

-1 :x:
javadoc
0m 20s
/results-javadoc-javadoc-hadoop-tools_hadoop-distcp-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt
hadoop-tools_hadoop-distcp-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 4 new + 41 unchanged - 0 fixed = 45 total (was 41)

-1 :x:
javadoc
0m 44s
/results-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 10 new + 268 unchanged - 0 fixed = 278 total (was 268)

-1 :x:
spotbugs
1m  9s
/patch-spotbugs-hadoop-common-project_hadoop-common.txt
hadoop-common in the patch failed.

-1 :x:
spotbugs
0m 22s
/patch-spotbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt
hadoop-mapreduce-client-app in the patch failed.

-1 :x:
spotbugs
0m 29s
/patch-spotbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt
hadoop-mapreduce-client-core in the patch failed.

-1 :x:
spotbugs
0m 20s
/patch-spotbugs-hadoop-mapreduce-project_hadoop-mapreduce-examples.txt
hadoop-mapreduce-examples in the patch failed.

-1 :x:
spotbugs
0m 26s
/patch-spotbugs-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch failed.

-1 :x:
spotbugs
0m 19s
/patch-spotbugs-hadoop-tools_hadoop-distcp.txt
hadoop-distcp in the patch failed.

-1 :x:
spotbugs
0m 32s
/patch-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt
hadoop-yarn-common in the patch failed.

+1 :green_heart:
shadedclient
1m 41s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

-1 :x:
unit
18m 55s
/patch-unit-hadoop-common-project_hadoop-common.txt
hadoop-common in the patch passed.

-1 :x:
unit
0m 20s
/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt
hadoop-mapreduce-client-app in the patch failed.

-1 :x:
unit
0m 26s
/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt
hadoop-mapreduce-client-core in the patch failed.

-1 :x:
unit
0m 18s
/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-examples.txt
hadoop-mapreduce-examples in the patch failed.

-1 :x:
unit
0m 22s
/patch-unit-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch failed.

-1 :x:
unit
0m 20s
/patch-unit-hadoop-tools_hadoop-distcp.txt
hadoop-distcp in the patch failed.

+1 :green_heart:
unit
6m 59s

hadoop-streaming in the patch passed.

-1 :x:
unit
0m 30s
/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt
hadoop-yarn-common in the patch failed.

+1 :green_heart:
asflicense
0m 29s

The patch does not generate ASF License warnings.

181m 54s

Reason
Tests

Failed junit tests
hadoop.metrics2.source.TestJvmMetrics

hadoop.ipc.TestIPC

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2584/14/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/2584

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell markdownlint

uname
Linux ce4a7e0f25e4 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 7e7670c792de3df430ad14a1a8cfb1d369d0a5cb

Default Java
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2584/14/testReport/

Max. process+thread count
1295 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-examples hadoop-tools/hadoop-aws hadoop-tools/hadoop-distcp hadoop-tools/hadoop-streaming hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2584/14/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 44s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  2s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 16 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
13m 19s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
23m 15s

trunk passed

+1 :green_heart:
compile
22m 37s

trunk passed with JDK Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
compile
19m 57s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
checkstyle
3m 44s

trunk passed

+1 :green_heart:
mvnsite
7m 55s

trunk passed

+1 :green_heart:
javadoc
6m 32s

trunk passed with JDK Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
7m  1s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
11m 20s

trunk passed

+1 :green_heart:
shadedclient
21m 10s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 28s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
4m 29s

the patch passed

+1 :green_heart:
compile
22m  1s

the patch passed with JDK Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04

-1 :x:
javac
22m  1s
/results-compile-javac-root-jdkUbuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04.txt
root-jdkUbuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 with JDK Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 generated 2 new + 1811 unchanged - 0 fixed = 1813 total (was 1811)

+1 :green_heart:
compile
19m 47s

the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

-1 :x:
javac
19m 47s
/results-compile-javac-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt
root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 2 new + 1685 unchanged - 0 fixed = 1687 total (was 1685)

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
3m 44s
/results-checkstyle-root.txt
root: The patch generated 2 new + 836 unchanged - 1 fixed = 838 total (was 837)

+1 :green_heart:
mvnsite
7m 55s

the patch passed

+1 :green_heart:
javadoc
6m 30s

the patch passed with JDK Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
7m  4s

the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
12m 57s

the patch passed

+1 :green_heart:
shadedclient
21m  1s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
17m 52s

hadoop-common in the patch passed.

+1 :green_heart:
unit
5m  0s

hadoop-yarn-common in the patch passed.

+1 :green_heart:
unit
5m 45s

hadoop-mapreduce-client-core in the patch passed.

+1 :green_heart:
unit
8m 38s

hadoop-mapreduce-client-app in the patch passed.

+1 :green_heart:
unit
21m  0s

hadoop-distcp in the patch passed.

+1 :green_heart:
unit
1m  3s

hadoop-mapreduce-examples in the patch passed.

+1 :green_heart:
unit
6m 49s

hadoop-streaming in the patch passed.

+1 :green_heart:
unit
2m 35s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
1m  0s

The patch does not generate ASF License warnings.

318m 11s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2584/16/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/2584

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell markdownlint

uname
Linux ed08eac8853b 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 66d2bcb4759853cbe497b2b6edc786cc682e1af1

Default Java
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2584/16/testReport/

Max. process+thread count
2392 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-tools/hadoop-distcp hadoop-mapreduce-project/hadoop-mapreduce-examples hadoop-tools/hadoop-streaming hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2584/16/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

checkstyle
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java:120:import org.apache.hadoop.fs.s3a.select.InternalSelectConstants;:8: Unused import - org.apache.hadoop.fs.s3a.select.InternalSelectConstants. [UnusedImports]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java:194:import static org.apache.hadoop.fs.impl.AbstractFSBuilderImpl.rejectUnknownMandatoryKeys;:15: Unused import - org.apache.hadoop.fs.impl.AbstractFSBuilderImpl.rejectUnknownMandatoryKeys. [UnusedImports]'

javac warnings
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java:4099:14:[deprecation] getDefaultBlockSize() in FileSystem has been deprecated
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/CopyFromLocalOperation.java:235:16:[unchecked] unchecked method invocation: method sort in interface List is applied to given types

rebase against trunk
s3a input stream will drain the inner stream asynchronously in seek/unbuffer related calls if the # of bytes to drain is > a new config/openFile option fs.s3a.input.async.drain.threshold; default is 16000 bytes, which seems a good number in my long haul experiments. also draining is into a 16k byte buffer, which speeds it up.

This aims to reduce the cost of seeking where bytes do need to be discarded.
the time to drain is also measured. it's the max time which can be high, as it is time to read bytes remaining in the current read. abort is less expensive in the actual abort...its the negotiation of a new TLS channel later which costs.
(stream_read_remote_stream_aborted.mean=(samples=2, sum=2, mean=1.0000))
(stream_read_remote_stream_drain.mean=(samples=13, sum=29, mean=2.2308)));

(stream_read_remote_stream_aborted.max=1)
(stream_read_remote_stream_drain.max=25));