HADOOP-11452 make rename/3 public

PR with the previous patches.
This is a WiP but ready for some iniital review; lacks tests & spec.
Also, because the base FileSystem.rename/3 does its own src/dest checks, it's less efficient against object stores. They need their own high-perf subclass.
Change-Id: I1586ef2290d7a3d2d33b1a32e2f0999b07c26143

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

0
reexec
31
Docker mode activated.

_ Prechecks _

+1
@author
0
The patch does not contain any @author tags.

-1
test4tests
0
The patch doesn't appear to include any new or modified tests.  Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ trunk Compile Tests _

0
mvndep
101
Maven dependency ordering for branch

+1
mvninstall
1032
trunk passed

+1
compile
996
trunk passed

+1
checkstyle
132
trunk passed

+1
mvnsite
191
trunk passed

+1
shadedclient
1004
branch has no errors when building and testing our client artifacts.

+1
findbugs
334
trunk passed

+1
javadoc
153
trunk passed

_ Patch Compile Tests _

0
mvndep
22
Maven dependency ordering for patch

+1
mvninstall
141
the patch passed

+1
compile
949
the patch passed

+1
javac
949
root generated 0 new + 1491 unchanged - 5 fixed = 1491 total (was 1496)

-0
checkstyle
131
root: The patch generated 1 new + 173 unchanged - 4 fixed = 174 total (was 177)

+1
mvnsite
193
the patch passed

+1
whitespace
0
The patch has no whitespace issues.

+1
shadedclient
634
patch has no errors when building and testing our client artifacts.

+1
findbugs
360
the patch passed

-1
javadoc
59
hadoop-common-project_hadoop-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)

_ Other Tests _

-1
unit
475
hadoop-common in the patch failed.

+1
unit
115
hadoop-hdfs-client in the patch passed.

-1
unit
4733
hadoop-hdfs in the patch failed.

+1
asflicense
53
The patch does not generate ASF License warnings.

11852

Reason
Tests

Failed junit tests
hadoop.fs.viewfs.TestFSMainOperationsLocalFileSystem

hadoop.fs.TestFilterFileSystem

hadoop.fs.TestFSMainOperationsLocalFileSystem

hadoop.fs.TestSymlinkLocalFSFileSystem

hadoop.fs.TestHarFileSystem

hadoop.hdfs.TestFileAppend

hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure

hadoop.hdfs.web.TestWebHdfsTimeouts

Subsystem
Report/Notes

Docker
Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-743/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/743

Optional Tests
dupname  asflicense  compile  javac  javadoc  mvninstall  mvnsite  unit  shadedclient  findbugs  checkstyle

uname
Linux 7eefb4f84502 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
personality/hadoop.sh

git revision
trunk / 254efc9

maven
version: Apache Maven 3.3.9

Default Java
1.8.0_191

findbugs
v3.1.0-RC1

checkstyle
https://builds.apache.org/job/hadoop-multibranch/job/PR-743/1/artifact/out/diff-checkstyle-root.txt

javadoc
https://builds.apache.org/job/hadoop-multibranch/job/PR-743/1/artifact/out/diff-javadoc-javadoc-hadoop-common-project_hadoop-common.txt

unit
https://builds.apache.org/job/hadoop-multibranch/job/PR-743/1/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt

unit
https://builds.apache.org/job/hadoop-multibranch/job/PR-743/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt

Test Results
https://builds.apache.org/job/hadoop-multibranch/job/PR-743/1/testReport/

Max. process+thread count
4728 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: .

Console output
https://builds.apache.org/job/hadoop-multibranch/job/PR-743/1/console

Powered by
Apache Yetus 0.9.0 http://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

0
reexec
29
Docker mode activated.

_ Prechecks _

+1
@author
0
The patch does not contain any @author tags.

+1
test4tests
0
The patch appears to include 5 new or modified test files.

_ trunk Compile Tests _

0
mvndep
47
Maven dependency ordering for branch

+1
mvninstall
1346
trunk passed

+1
compile
1263
trunk passed

+1
checkstyle
154
trunk passed

+1
mvnsite
227
trunk passed

+1
shadedclient
1189
branch has no errors when building and testing our client artifacts.

+1
findbugs
415
trunk passed

+1
javadoc
174
trunk passed

_ Patch Compile Tests _

0
mvndep
68
Maven dependency ordering for patch

+1
mvninstall
190
the patch passed

+1
compile
1156
the patch passed

+1
javac
1156
root generated 0 new + 1491 unchanged - 5 fixed = 1491 total (was 1496)

-0
checkstyle
162
root: The patch generated 10 new + 230 unchanged - 6 fixed = 240 total (was 236)

+1
mvnsite
220
the patch passed

-1
whitespace
0
The patch has 3 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply

+1
shadedclient
722
patch has no errors when building and testing our client artifacts.

+1
findbugs
385
the patch passed

-1
javadoc
60
hadoop-common-project_hadoop-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)

_ Other Tests _

-1
unit
485
hadoop-common in the patch failed.

+1
unit
118
hadoop-hdfs-client in the patch passed.

-1
unit
5087
hadoop-hdfs in the patch failed.

+1
asflicense
72
The patch does not generate ASF License warnings.

13572

Reason
Tests

Failed junit tests
hadoop.fs.viewfs.TestFSMainOperationsLocalFileSystem

hadoop.fs.TestFilterFileSystem

hadoop.fs.TestSymlinkLocalFSFileSystem

hadoop.fs.TestFSMainOperationsLocalFileSystem

hadoop.fs.TestHarFileSystem

hadoop.hdfs.server.datanode.TestBPOfferService

hadoop.fs.contract.hdfs.TestHDFSContractRename

hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS

hadoop.hdfs.web.TestWebHdfsTimeouts

hadoop.fs.contract.hdfs.TestHDFSContractRenameEx

Subsystem
Report/Notes

Docker
Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-743/2/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/743

Optional Tests
dupname  asflicense  compile  javac  javadoc  mvninstall  mvnsite  unit  shadedclient  findbugs  checkstyle

uname
Linux 9c80918e87f7 4.4.0-143-generic #169~14.04.2-Ubuntu SMP Wed Feb 13 15:00:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
personality/hadoop.sh

git revision
trunk / 5583e1b

maven
version: Apache Maven 3.3.9

Default Java
1.8.0_191

findbugs
v3.1.0-RC1

checkstyle
https://builds.apache.org/job/hadoop-multibranch/job/PR-743/2/artifact/out/diff-checkstyle-root.txt

whitespace
https://builds.apache.org/job/hadoop-multibranch/job/PR-743/2/artifact/out/whitespace-eol.txt

javadoc
https://builds.apache.org/job/hadoop-multibranch/job/PR-743/2/artifact/out/diff-javadoc-javadoc-hadoop-common-project_hadoop-common.txt

unit
https://builds.apache.org/job/hadoop-multibranch/job/PR-743/2/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt

unit
https://builds.apache.org/job/hadoop-multibranch/job/PR-743/2/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt

Test Results
https://builds.apache.org/job/hadoop-multibranch/job/PR-743/2/testReport/

Max. process+thread count
3759 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: .

Console output
https://builds.apache.org/job/hadoop-multibranch/job/PR-743/2/console

Powered by
Apache Yetus 0.9.0 http://yetus.apache.org

This message was automatically generated.

Pushed up new version which does a lot more w.r.t testing, including updating FSMainOperationsBaseTest to java7/8 and implementing an S3A subclass (which fails, see below)
Also adds an override point to make it possible for FSs to implement their exception-raising rename after the common semantics checks.
I'm not comfortable with that design; I think it'd be better to pull the rename preconditions into some standalone class which takes a src & dest FS status and path strings, and do the checks there where they can validated in isolation. S3A would just precalculate those and then reuse them later on. Be easier for testing too.
but...that doesn't help abfs so much, does it?
[ERROR] testWorkingDirectory(org.apache.hadoop.fs.contract.s3a.ITestS3AFSMainOperations)  Time elapsed: 3.305 s  <<< FAILURE!
java.lang.AssertionError: expected:<s3a://hwdev-steve-new/ITestS3AFSMainOperations/test> but was:<>
        at org.junit.Assert.fail(Assert.java:88)
        at org.junit.Assert.failNotEquals(Assert.java:834)
        at org.junit.Assert.assertEquals(Assert.java:118)
        at org.junit.Assert.assertEquals(Assert.java:144)
        at org.apache.hadoop.fs.FSMainOperationsBaseTest.testWorkingDirectory(FSMainOperationsBaseTest.java:162)

[ERROR] testCopyToLocalWithUseRawLocalFileSystemOption(org.apache.hadoop.fs.contract.s3a.ITestS3AFSMainOperations)  Time elapsed: 1.291 s  <<< ERROR!
java.io.IOException: Mkdirs failed to create /ITestS3AFSMainOperations
        at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:317)
        at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:305)
        at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1111)
        at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1000)
        at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:988)
        at org.apache.hadoop.fs.FSMainOperationsBaseTest.writeFile(FSMainOperationsBaseTest.java:1042)
        at org.apache.hadoop.fs.FSMainOperationsBaseTest.testCopyToLocalWithUseRawLocalFileSystemOption(FSMainOperationsBaseTest.java:1033)

that new override point is what breaks the harfs test

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

0
reexec
21
Docker mode activated.

_ Prechecks _

+1
@author
0
The patch does not contain any @author tags.

+1
test4tests
0
The patch appears to include 9 new or modified test files.

_ trunk Compile Tests _

0
mvndep
31
Maven dependency ordering for branch

+1
mvninstall
1100
trunk passed

+1
compile
1051
trunk passed

+1
checkstyle
151
trunk passed

+1
mvnsite
294
trunk passed

+1
shadedclient
1193
branch has no errors when building and testing our client artifacts.

+1
findbugs
433
trunk passed

+1
javadoc
208
trunk passed

_ Patch Compile Tests _

0
mvndep
23
Maven dependency ordering for patch

+1
mvninstall
202
the patch passed

+1
compile
945
the patch passed

+1
javac
945
root generated 0 new + 1491 unchanged - 5 fixed = 1491 total (was 1496)

-0
checkstyle
144
root: The patch generated 44 new + 283 unchanged - 49 fixed = 327 total (was 332)

+1
mvnsite
319
the patch passed

-1
whitespace
0
The patch has 3 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply

+1
shadedclient
782
patch has no errors when building and testing our client artifacts.

+1
findbugs
565
the patch passed

-1
javadoc
75
hadoop-common-project_hadoop-common generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)

_ Other Tests _

-1
unit
603
hadoop-common in the patch failed.

+1
unit
135
hadoop-hdfs-client in the patch passed.

-1
unit
6103
hadoop-hdfs in the patch failed.

+1
unit
43
hadoop-openstack in the patch passed.

+1
unit
297
hadoop-aws in the patch passed.

+1
asflicense
70
The patch does not generate ASF License warnings.

14801

Reason
Tests

Failed junit tests
hadoop.fs.TestFilterFileSystem

hadoop.ha.TestZKFailoverController

hadoop.fs.TestHarFileSystem

hadoop.hdfs.TestReconstructStripedFile

hadoop.hdfs.server.datanode.TestBPOfferService

hadoop.fs.contract.hdfs.TestHDFSContractRename

hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerForContentSummary

hadoop.hdfs.web.TestWebHdfsTimeouts

hadoop.hdfs.TestDFSStripedOutputStreamWithFailureWithRandomECPolicy

hadoop.hdfs.tools.TestDFSAdmin

hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerWithStripedBlocks

hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics

Subsystem
Report/Notes

Docker
Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-743/3/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/743

Optional Tests
dupname  asflicense  compile  javac  javadoc  mvninstall  mvnsite  unit  shadedclient  findbugs  checkstyle

uname
Linux 58fb97ebe992 4.4.0-143-generic #169~14.04.2-Ubuntu SMP Wed Feb 13 15:00:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
personality/hadoop.sh

git revision
trunk / a5ceed2

maven
version: Apache Maven 3.3.9

Default Java
1.8.0_191

findbugs
v3.1.0-RC1

checkstyle
https://builds.apache.org/job/hadoop-multibranch/job/PR-743/3/artifact/out/diff-checkstyle-root.txt

whitespace
https://builds.apache.org/job/hadoop-multibranch/job/PR-743/3/artifact/out/whitespace-eol.txt

javadoc
https://builds.apache.org/job/hadoop-multibranch/job/PR-743/3/artifact/out/diff-javadoc-javadoc-hadoop-common-project_hadoop-common.txt

unit
https://builds.apache.org/job/hadoop-multibranch/job/PR-743/3/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt

unit
https://builds.apache.org/job/hadoop-multibranch/job/PR-743/3/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt

Test Results
https://builds.apache.org/job/hadoop-multibranch/job/PR-743/3/testReport/

Max. process+thread count
3148 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-tools/hadoop-openstack hadoop-tools/hadoop-aws U: .

Console output
https://builds.apache.org/job/hadoop-multibranch/job/PR-743/3/console

Powered by
Apache Yetus 0.9.0 http://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

0
reexec
24
Docker mode activated.

_ Prechecks _

+1
@author
0
The patch does not contain any @author tags.

+1
test4tests
1
The patch appears to include 9 new or modified test files.

_ trunk Compile Tests _

0
mvndep
74
Maven dependency ordering for branch

+1
mvninstall
1138
trunk passed

+1
compile
977
trunk passed

+1
checkstyle
153
trunk passed

+1
mvnsite
277
trunk passed

+1
shadedclient
1211
branch has no errors when building and testing our client artifacts.

+1
findbugs
435
trunk passed

+1
javadoc
212
trunk passed

_ Patch Compile Tests _

0
mvndep
20
Maven dependency ordering for patch

+1
mvninstall
198
the patch passed

+1
compile
972
the patch passed

+1
javac
972
root generated 0 new + 1476 unchanged - 5 fixed = 1476 total (was 1481)

-0
checkstyle
149
root: The patch generated 47 new + 283 unchanged - 49 fixed = 330 total (was 332)

+1
mvnsite
275
the patch passed

-1
whitespace
0
The patch has 3 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply

+1
shadedclient
727
patch has no errors when building and testing our client artifacts.

+1
findbugs
504
the patch passed

-1
javadoc
60
hadoop-common-project_hadoop-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)

_ Other Tests _

-1
unit
513
hadoop-common in the patch failed.

+1
unit
117
hadoop-hdfs-client in the patch passed.

-1
unit
4895
hadoop-hdfs in the patch failed.

+1
unit
33
hadoop-openstack in the patch passed.

+1
unit
299
hadoop-aws in the patch passed.

+1
asflicense
47
The patch does not generate ASF License warnings.

13333

Reason
Tests

Failed junit tests
hadoop.fs.TestFilterFileSystem

hadoop.fs.TestHarFileSystem

hadoop.hdfs.server.datanode.TestBPOfferService

hadoop.fs.contract.hdfs.TestHDFSContractRename

hadoop.hdfs.web.TestWebHdfsTimeouts

Subsystem
Report/Notes

Docker
Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-743/4/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/743

Optional Tests
dupname  asflicense  compile  javac  javadoc  mvninstall  mvnsite  unit  shadedclient  findbugs  checkstyle

uname
Linux 672fad23c84b 4.4.0-143-generic #169~14.04.2-Ubuntu SMP Wed Feb 13 15:00:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
personality/hadoop.sh

git revision
trunk / 4b4200f

maven
version: Apache Maven 3.3.9

Default Java
1.8.0_191

findbugs
v3.1.0-RC1

checkstyle
https://builds.apache.org/job/hadoop-multibranch/job/PR-743/4/artifact/out/diff-checkstyle-root.txt

whitespace
https://builds.apache.org/job/hadoop-multibranch/job/PR-743/4/artifact/out/whitespace-eol.txt

javadoc
https://builds.apache.org/job/hadoop-multibranch/job/PR-743/4/artifact/out/diff-javadoc-javadoc-hadoop-common-project_hadoop-common.txt

unit
https://builds.apache.org/job/hadoop-multibranch/job/PR-743/4/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt

unit
https://builds.apache.org/job/hadoop-multibranch/job/PR-743/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt

Test Results
https://builds.apache.org/job/hadoop-multibranch/job/PR-743/4/testReport/

Max. process+thread count
3878 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-tools/hadoop-openstack hadoop-tools/hadoop-aws U: .

Console output
https://builds.apache.org/job/hadoop-multibranch/job/PR-743/4/console

Powered by
Apache Yetus 0.9.0 http://yetus.apache.org

This message was automatically generated.

for commenters, yes, this needs a rebase against trunk and then a review of all your comments. I cherish your patience, especially as all the comments are good ones

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 31s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s
No case conflicting files found.

+0 :ok:
markdownlint
0m  0s
markdownlint was not available.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s
The patch appears to include 9 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
3m 17s
Maven dependency ordering for branch

+1 :green_heart:
mvninstall
25m 55s
trunk passed

+1 :green_heart:
compile
19m 31s
trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1

+1 :green_heart:
compile
16m 48s
trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01

+1 :green_heart:
checkstyle
2m 54s
trunk passed

+1 :green_heart:
mvnsite
5m 38s
trunk passed

+1 :green_heart:
shadedclient
23m 13s
branch has no errors when building and testing our client artifacts.

+1 :green_heart:
javadoc
3m 48s
trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1

+1 :green_heart:
javadoc
5m 19s
trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01

+0 :ok:
spotbugs
0m 54s
Used deprecated FindBugs config; considering switching to SpotBugs.

+1 :green_heart:
findbugs
10m  1s
trunk passed

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 25s
Maven dependency ordering for patch

+1 :green_heart:
mvninstall
3m 44s
the patch passed

+1 :green_heart:
compile
18m 50s
the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1

-1 :x:
javac
18m 50s
root-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 generated 1 new + 2050 unchanged - 5 fixed = 2051 total (was 2055)

+1 :green_heart:
compile
17m  1s
the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01

-1 :x:
javac
17m  1s
root-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 generated 1 new + 1944 unchanged - 5 fixed = 1945 total (was 1949)

-0 :warning:
checkstyle
2m 48s
root: The patch generated 63 new + 292 unchanged - 48 fixed = 355 total (was 340)

+1 :green_heart:
mvnsite
5m 38s
the patch passed

-1 :x:
whitespace
0m  0s
The patch has 3 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply

+1 :green_heart:
shadedclient
14m 22s
patch has no errors when building and testing our client artifacts.

+1 :green_heart:
javadoc
3m 45s
the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1

-1 :x:
javadoc
1m 34s
hadoop-common-project_hadoop-common-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1)

-1 :x:
findbugs
2m 22s
hadoop-common-project/hadoop-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)

_ Other Tests _

-1 :x:
unit
9m 29s
hadoop-common in the patch passed.

+1 :green_heart:
unit
2m 22s
hadoop-hdfs-client in the patch passed.

-1 :x:
unit
97m 40s
hadoop-hdfs in the patch passed.

+1 :green_heart:
unit
0m 46s
hadoop-openstack in the patch passed.

+1 :green_heart:
unit
1m 44s
hadoop-aws in the patch passed.

-1 :x:
asflicense
1m  8s
The patch generated 3 ASF License warnings.

310m  9s

Reason
Tests

FindBugs
module:hadoop-common-project/hadoop-common

Should org.apache.hadoop.fs.impl.RenameHelper$RenameValidationResult be a static inner class?  At RenameHelper.java:inner class?  At RenameHelper.java:[line 320]

Failed junit tests
hadoop.fs.TestHarFileSystem

hadoop.fs.TestFilterFileSystem

hadoop.hdfs.TestFileChecksumCompositeCrc

hadoop.hdfs.server.namenode.ha.TestHAAppend

hadoop.fs.contract.hdfs.TestHDFSContractRename

hadoop.hdfs.TestFileChecksum

hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped

Subsystem
Report/Notes

Docker
ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/2/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/743

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle markdownlint

uname
Linux 63395babc18e 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / a888d580d8a

Default Java
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01

javac
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/2/artifact/out/diff-compile-javac-root-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1.txt

javac
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/2/artifact/out/diff-compile-javac-root-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01.txt

checkstyle
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/2/artifact/out/diff-checkstyle-root.txt

whitespace
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/2/artifact/out/whitespace-eol.txt

javadoc
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/2/artifact/out/diff-javadoc-javadoc-hadoop-common-project_hadoop-common-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01.txt

findbugs
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/2/artifact/out/new-findbugs-hadoop-common-project_hadoop-common.html

unit
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/2/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt

unit
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/2/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/2/testReport/

asflicense
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/2/artifact/out/patch-asflicense-problems.txt

Max. process+thread count
3732 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-tools/hadoop-openstack hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/2/console

versions
git=2.17.1 maven=3.6.0 findbugs=4.0.6

Powered by
Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 29s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s
No case conflicting files found.

+0 :ok:
markdownlint
0m  0s
markdownlint was not available.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s
The patch appears to include 10 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
3m 18s
Maven dependency ordering for branch

+1 :green_heart:
mvninstall
25m 46s
trunk passed

+1 :green_heart:
compile
19m 27s
trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1

+1 :green_heart:
compile
16m 46s
trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01

+1 :green_heart:
checkstyle
2m 55s
trunk passed

+1 :green_heart:
mvnsite
5m 38s
trunk passed

+1 :green_heart:
shadedclient
23m 31s
branch has no errors when building and testing our client artifacts.

+1 :green_heart:
javadoc
3m 31s
trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1

+1 :green_heart:
javadoc
4m 55s
trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01

+0 :ok:
spotbugs
0m 49s
Used deprecated FindBugs config; considering switching to SpotBugs.

+1 :green_heart:
findbugs
9m 57s
trunk passed

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 28s
Maven dependency ordering for patch

+1 :green_heart:
mvninstall
3m 52s
the patch passed

+1 :green_heart:
compile
19m 47s
the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1

-1 :x:
javac
19m 47s
root-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 generated 1 new + 2050 unchanged - 5 fixed = 2051 total (was 2055)

+1 :green_heart:
compile
18m 21s
the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01

-1 :x:
javac
18m 21s
root-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 generated 1 new + 1944 unchanged - 5 fixed = 1945 total (was 1949)

-0 :warning:
checkstyle
2m 50s
root: The patch generated 65 new + 456 unchanged - 31 fixed = 521 total (was 487)

+1 :green_heart:
mvnsite
5m 27s
the patch passed

-1 :x:
whitespace
0m  0s
The patch has 4 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply

+1 :green_heart:
shadedclient
14m 30s
patch has no errors when building and testing our client artifacts.

+1 :green_heart:
javadoc
3m 28s
the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1

-1 :x:
javadoc
1m 31s
hadoop-common-project_hadoop-common-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1)

-1 :x:
findbugs
2m 28s
hadoop-common-project/hadoop-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)

_ Other Tests _

-1 :x:
unit
9m 27s
hadoop-common in the patch passed.

+1 :green_heart:
unit
2m 21s
hadoop-hdfs-client in the patch passed.

-1 :x:
unit
99m 38s
hadoop-hdfs in the patch passed.

+1 :green_heart:
unit
0m 48s
hadoop-openstack in the patch passed.

+1 :green_heart:
unit
1m 42s
hadoop-aws in the patch passed.

-1 :x:
asflicense
1m  8s
The patch generated 3 ASF License warnings.

313m 14s

Reason
Tests

FindBugs
module:hadoop-common-project/hadoop-common

Should org.apache.hadoop.fs.impl.RenameHelper$RenameValidationResult be a static inner class?  At RenameHelper.java:inner class?  At RenameHelper.java:[line 320]

Failed junit tests
hadoop.fs.TestChecksumFs

hadoop.fs.TestTrash

hadoop.fs.viewfs.TestFcCreateMkdirLocalFs

hadoop.fs.viewfs.TestChRootedFs

hadoop.fs.contract.localfs.TestLocalFSContractRenameEx

hadoop.fs.viewfs.TestViewFsTrash

hadoop.fs.TestSymlinkLocalFSFileContext

hadoop.fs.viewfs.TestViewFileSystemWithAuthorityLocalFileSystem

hadoop.fs.viewfs.TestViewFsWithAuthorityLocalFs

hadoop.fs.viewfs.TestFSMainOperationsLocalFileSystem

hadoop.fs.TestLocalFSFileContextCreateMkdir

hadoop.fs.contract.rawlocal.TestRawlocalContractRenameEx

hadoop.fs.viewfs.TestViewFileSystemLocalFileSystem

hadoop.fs.viewfs.TestViewFsLocalFs

hadoop.fs.viewfs.TestFcMainOperationsLocalFs

hadoop.fs.TestSymlinkLocalFSFileSystem

hadoop.fs.TestLocalFSFileContextMainOperations

hadoop.fs.TestFSMainOperationsLocalFileSystem

hadoop.hdfs.server.sps.TestExternalStoragePolicySatisfier

hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks

hadoop.hdfs.TestFileChecksumCompositeCrc

hadoop.hdfs.TestDecommissionWithStriped

hadoop.hdfs.server.balancer.TestBalancer

hadoop.fs.contract.hdfs.TestHDFSContractRename

hadoop.hdfs.TestHDFSTrash

hadoop.hdfs.TestFileChecksum

hadoop.hdfs.TestDFSInotifyEventInputStreamKerberized

hadoop.hdfs.server.namenode.TestCreateEditsLog

Subsystem
Report/Notes

Docker
ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/3/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/743

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle markdownlint

uname
Linux 107e1e01525d 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 56a5c360a10

Default Java
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01

javac
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/3/artifact/out/diff-compile-javac-root-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1.txt

javac
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/3/artifact/out/diff-compile-javac-root-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01.txt

checkstyle
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/3/artifact/out/diff-checkstyle-root.txt

whitespace
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/3/artifact/out/whitespace-eol.txt

javadoc
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/3/artifact/out/diff-javadoc-javadoc-hadoop-common-project_hadoop-common-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01.txt

findbugs
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/3/artifact/out/new-findbugs-hadoop-common-project_hadoop-common.html

unit
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/3/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt

unit
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/3/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/3/testReport/

asflicense
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/3/artifact/out/patch-asflicense-problems.txt

Max. process+thread count
3841 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-tools/hadoop-openstack hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/3/console

versions
git=2.17.1 maven=3.6.0 findbugs=4.0.6

Powered by
Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
1m 11s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+0 :ok:
markdownlint
0m  1s
markdownlint was not available.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s
The patch appears to include 10 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
3m 20s
Maven dependency ordering for branch

+1 :green_heart:
mvninstall
28m 46s
trunk passed

+1 :green_heart:
compile
20m 47s
trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1

+1 :green_heart:
compile
17m 39s
trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01

+1 :green_heart:
checkstyle
3m  1s
trunk passed

+1 :green_heart:
mvnsite
5m  5s
trunk passed

+1 :green_heart:
shadedclient
24m 24s
branch has no errors when building and testing our client artifacts.

+1 :green_heart:
javadoc
3m  6s
trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1

+1 :green_heart:
javadoc
4m 37s
trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01

+0 :ok:
spotbugs
0m 45s
Used deprecated FindBugs config; considering switching to SpotBugs.

+1 :green_heart:
findbugs
9m 54s
trunk passed

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 22s
Maven dependency ordering for patch

+1 :green_heart:
mvninstall
3m 39s
the patch passed

+1 :green_heart:
compile
20m 14s
the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1

-1 :x:
javac
20m 14s
root-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 generated 1 new + 2050 unchanged - 5 fixed = 2051 total (was 2055)

+1 :green_heart:
compile
17m 40s
the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01

-1 :x:
javac
17m 40s
root-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 generated 1 new + 1944 unchanged - 5 fixed = 1945 total (was 1949)

-0 :warning:
checkstyle
3m  3s
root: The patch generated 103 new + 484 unchanged - 30 fixed = 587 total (was 514)

+1 :green_heart:
mvnsite
5m  0s
the patch passed

-1 :x:
whitespace
0m  0s
The patch has 8 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply

+1 :green_heart:
shadedclient
15m 43s
patch has no errors when building and testing our client artifacts.

+1 :green_heart:
javadoc
3m  7s
the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1

-1 :x:
javadoc
1m 30s
hadoop-common-project_hadoop-common-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 generated 3 new + 1 unchanged - 0 fixed = 4 total (was 1)

-1 :x:
findbugs
2m 22s
hadoop-common-project/hadoop-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)

_ Other Tests _

-1 :x:
unit
9m 48s
hadoop-common in the patch passed.

+1 :green_heart:
unit
2m  7s
hadoop-hdfs-client in the patch passed.

-1 :x:
unit
126m 42s
hadoop-hdfs in the patch passed.

+1 :green_heart:
unit
0m 38s
hadoop-openstack in the patch passed.

+1 :green_heart:
unit
1m 58s
hadoop-aws in the patch passed.

-1 :x:
asflicense
0m 59s
The patch generated 3 ASF License warnings.

345m  9s

Reason
Tests

FindBugs
module:hadoop-common-project/hadoop-common

Should org.apache.hadoop.fs.impl.RenameHelper$RenameValidationResult be a static inner class?  At RenameHelper.java:inner class?  At RenameHelper.java:[line 320]

Failed junit tests
hadoop.fs.contract.rawlocal.TestRawlocalContractRenameEx

hadoop.fs.TestFSMainOperationsLocalFileSystem

hadoop.fs.viewfs.TestViewFsWithAuthorityLocalFs

hadoop.fs.TestSymlinkLocalFSFileContext

hadoop.fs.TestChecksumFs

hadoop.fs.viewfs.TestFSMainOperationsLocalFileSystem

hadoop.fs.viewfs.TestFcMainOperationsLocalFs

hadoop.fs.TestLocalFSFileContextMainOperations

hadoop.fs.TestLocalFSFileContextCreateMkdir

hadoop.fs.viewfs.TestFcCreateMkdirLocalFs

hadoop.fs.viewfs.TestViewFileSystemWithAuthorityLocalFileSystem

hadoop.fs.viewfs.TestViewFsLocalFs

hadoop.fs.viewfs.TestViewFileSystemLocalFileSystem

hadoop.fs.viewfs.TestViewFsTrash

hadoop.fs.TestSymlinkLocalFSFileSystem

hadoop.fs.TestTrash

hadoop.fs.viewfs.TestChRootedFs

hadoop.fs.contract.localfs.TestLocalFSContractRenameEx

hadoop.hdfs.TestViewDistributedFileSystem

hadoop.hdfs.server.balancer.TestBalancerWithHANameNodes

hadoop.fs.contract.hdfs.TestHDFSContractRename

hadoop.hdfs.server.sps.TestExternalStoragePolicySatisfier

hadoop.hdfs.TestFileChecksum

hadoop.hdfs.TestFileChecksumCompositeCrc

hadoop.hdfs.server.datanode.TestDataNodeUUID

hadoop.hdfs.TestHDFSTrash

hadoop.hdfs.server.namenode.TestCreateEditsLog

Subsystem
Report/Notes

Docker
ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/4/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/743

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle markdownlint

uname
Linux ec536684cf51 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 7e671f3b657

Default Java
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01

javac
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/4/artifact/out/diff-compile-javac-root-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1.txt

javac
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/4/artifact/out/diff-compile-javac-root-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01.txt

checkstyle
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/4/artifact/out/diff-checkstyle-root.txt

whitespace
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/4/artifact/out/whitespace-eol.txt

javadoc
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/4/artifact/out/diff-javadoc-javadoc-hadoop-common-project_hadoop-common-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01.txt

findbugs
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/4/artifact/out/new-findbugs-hadoop-common-project_hadoop-common.html

unit
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/4/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt

unit
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/4/testReport/

asflicense
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/4/artifact/out/patch-asflicense-problems.txt

Max. process+thread count
3144 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-tools/hadoop-openstack hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/4/console

versions
git=2.17.1 maven=3.6.0 findbugs=4.0.6

Powered by
Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 36s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+0 :ok:
markdownlint
0m  0s
markdownlint was not available.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s
The patch appears to include 10 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
3m 26s
Maven dependency ordering for branch

+1 :green_heart:
mvninstall
30m 11s
trunk passed

+1 :green_heart:
compile
23m 30s
trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1

-1 :x:
compile
0m 35s
root in trunk failed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01.

-0 :warning:
checkstyle
0m 22s
The patch fails to run checkstyle in root

-1 :x:
mvnsite
0m 29s
hadoop-common in trunk failed.

-1 :x:
mvnsite
0m 34s
hadoop-hdfs in trunk failed.

-1 :x:
mvnsite
0m 29s
hadoop-hdfs-client in trunk failed.

-1 :x:
mvnsite
0m 35s
hadoop-aws in trunk failed.

-1 :x:
shadedclient
12m 52s
branch has errors when building and testing our client artifacts.

+1 :green_heart:
javadoc
4m  6s
trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1

+1 :green_heart:
javadoc
5m 41s
trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01

+0 :ok:
spotbugs
0m 53s
Used deprecated FindBugs config; considering switching to SpotBugs.

+1 :green_heart:
findbugs
12m  7s
trunk passed

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 31s
Maven dependency ordering for patch

+1 :green_heart:
mvninstall
4m 23s
the patch passed

-1 :x:
compile
15m 11s
root in the patch failed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1.

-1 :x:
javac
15m 11s
root in the patch failed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1.

-1 :x:
compile
0m 27s
root in the patch failed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01.

-1 :x:
javac
0m 27s
root in the patch failed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01.

-0 :warning:
checkstyle
3m  8s
root: The patch generated 587 new + 0 unchanged - 0 fixed = 587 total (was 0)

+1 :green_heart:
mvnsite
4m 43s
the patch passed

-1 :x:
whitespace
0m  0s
The patch has 8 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply

+1 :green_heart:
shadedclient
14m 15s
patch has no errors when building and testing our client artifacts.

+1 :green_heart:
javadoc
2m 31s
the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1

-1 :x:
javadoc
1m 20s
hadoop-common-project_hadoop-common-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 generated 3 new + 1 unchanged - 0 fixed = 4 total (was 1)

-1 :x:
findbugs
2m 15s
hadoop-common-project/hadoop-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)

_ Other Tests _

-1 :x:
unit
9m 24s
hadoop-common in the patch passed.

+1 :green_heart:
unit
2m  3s
hadoop-hdfs-client in the patch passed.

-1 :x:
unit
96m 17s
hadoop-hdfs in the patch passed.

+1 :green_heart:
unit
0m 33s
hadoop-openstack in the patch passed.

+1 :green_heart:
unit
1m 24s
hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
0m 48s
The patch does not generate ASF License warnings.

267m 36s

Reason
Tests

FindBugs
module:hadoop-common-project/hadoop-common

Should org.apache.hadoop.fs.impl.RenameHelper$RenameValidationResult be a static inner class?  At RenameHelper.java:inner class?  At RenameHelper.java:[line 320]

Failed junit tests
hadoop.fs.contract.rawlocal.TestRawlocalContractRenameEx

hadoop.fs.viewfs.TestFcMainOperationsLocalFs

hadoop.fs.TestSymlinkLocalFSFileSystem

hadoop.fs.TestTrash

hadoop.fs.TestFSMainOperationsLocalFileSystem

hadoop.fs.viewfs.TestViewFsWithAuthorityLocalFs

hadoop.fs.viewfs.TestViewFsLocalFs

hadoop.fs.viewfs.TestViewFsTrash

hadoop.fs.viewfs.TestChRootedFs

hadoop.fs.contract.localfs.TestLocalFSContractRenameEx

hadoop.fs.viewfs.TestViewFileSystemWithAuthorityLocalFileSystem

hadoop.fs.TestChecksumFs

hadoop.fs.TestLocalFSFileContextMainOperations

hadoop.fs.TestLocalFSFileContextCreateMkdir

hadoop.fs.viewfs.TestFcCreateMkdirLocalFs

hadoop.fs.viewfs.TestViewFileSystemLocalFileSystem

hadoop.fs.TestSymlinkLocalFSFileContext

hadoop.fs.viewfs.TestFSMainOperationsLocalFileSystem

hadoop.hdfs.server.namenode.TestCreateEditsLog

hadoop.hdfs.TestFileChecksum

hadoop.fs.contract.hdfs.TestHDFSContractRename

hadoop.hdfs.TestStripedFileAppend

hadoop.hdfs.TestFileChecksumCompositeCrc

hadoop.hdfs.server.datanode.TestBPOfferService

hadoop.hdfs.server.sps.TestExternalStoragePolicySatisfier

hadoop.hdfs.TestHDFSTrash

hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped

hadoop.hdfs.server.namenode.TestNameNodeRetryCacheMetrics

hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks

Subsystem
Report/Notes

Docker
ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/5/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/743

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle markdownlint

uname
Linux a3de12066332 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / e5fe3262702

Default Java
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01

compile
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/5/artifact/out/branch-compile-root-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01.txt

checkstyle
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/5/artifact/out/buildtool-branch-checkstyle-root.txt

mvnsite
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/5/artifact/out/branch-mvnsite-hadoop-common-project_hadoop-common.txt

mvnsite
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/5/artifact/out/branch-mvnsite-hadoop-hdfs-project_hadoop-hdfs.txt

mvnsite
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/5/artifact/out/branch-mvnsite-hadoop-hdfs-project_hadoop-hdfs-client.txt

mvnsite
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/5/artifact/out/branch-mvnsite-hadoop-tools_hadoop-aws.txt

compile
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/5/artifact/out/patch-compile-root-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1.txt

javac
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/5/artifact/out/patch-compile-root-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1.txt

compile
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/5/artifact/out/patch-compile-root-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01.txt

javac
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/5/artifact/out/patch-compile-root-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01.txt

checkstyle
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/5/artifact/out/diff-checkstyle-root.txt

whitespace
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/5/artifact/out/whitespace-eol.txt

javadoc
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/5/artifact/out/diff-javadoc-javadoc-hadoop-common-project_hadoop-common-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01.txt

findbugs
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/5/artifact/out/new-findbugs-hadoop-common-project_hadoop-common.html

unit
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/5/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt

unit
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/5/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/5/testReport/

Max. process+thread count
4168 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-tools/hadoop-openstack hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-743/5/console

versions
git=2.17.1 maven=3.6.0 findbugs=4.0.6

Powered by
Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

as per Daryn's comments in : https://issues.apache.org/jira/browse/HADOOP-16281?focusedCommentId=16832628&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16832628
No changes to HDFS error messages. Review this

closing and creating a new rebased/squashed PR. Still need to look at the comments related to permissions


Add support for batch read to DynamoDocumentStoreTemplate

Supplying a KeySetQuery to a fetch on a document store template will now support batch read. Pattern for retry unprocessed keys taken from AWS best practices.
Signed-off-by: Robin Smith<EMAIL_ADDRESS>

@ClickTravel/cheddarchapter

I'm not sure if it's going to do the right thing. An integration test will disprove this.

I think the algorithm to split the request to batches of 100 ids is not optimal. It can result in more requests than necessary if not all 100 items are processed by each request. I suggest something like this instead (pseudocode)
Set idsRemaining = query.itemIds();
while (idsRemaining.notEmpty()) {
    Set idsToGet = first 100 keys of idsRemaining
    outcome = batchGetItems(idsToGet)
    for each item in outcome {
       idsRemaining.remove(item.key)
    }
}

@clicktravel-steffan this assumes that the internals of the special sdk method
dynamoDBClient.batchGetItemUnprocessed(...)
is not already optimised to work differently to the normal batch get item. I'll have to check this.

@clicktravel-robin I just wanted to ensure we avoid situations like this:

Get 97 items
Get 3 items
Get 97 items
Get 3 items

...which instead would be more optimally dealt with like this:

Get 97 items
Get 97 items
Get 6 items

@clicktravel-basharat i have changed to instead use a recursive call to process the results rather than the looping method suggested by AWS.
Does this look better, integration tests to proce this are going to be a problem because i cant force the dynamo service to return unprocessed keys

@clicktravel-basharat added a integration test that proves it works


Provide an option to fail fast when LLM calls timeout or are exhausted

Pipeline yields "corrupted" dataframes when Endpoint TPM thresholds have been exceeded to many times.
We should provide options to determine the failure behavior on LLM-based operations, so we can fail fast and emit the issue instead of continuing and thus generating incorrect dataframes.

This issue is still relevant. In situations of having low TPM or too many requests being made to the LLM, graphrag users are starting to open issues where this is the root cause (i.e. silent failure due to not enough data validation happening and thus leading to corrupt data frames).

Related to #811


Task Graph / Job System

There should be some kind of parallel task graph/job system for handling large parallel for type tasks. This will be useful for large calculations for things like physics. We also need this so that we can multithread our command buffers

Again, out of scope for this semester. Closing and backlogging


Plugin crashes after update

Hi chris,
I built my ionic app  today after some other changes ( not related to this plugin).. and i saw it updated the cordova scan plugin.. and since then this plugin starts to crash. it does not open camera or gallery options :( ..
and in the phone log i can see this exception
java.lang.UnsatisfiedLinkError: dalvik.system.PathClassLoader[DexPathList[[zip file "/data/app/io.ionic.starter-eC91gWckgNN3MK_UB9WYeA==/base.apk"],nativeLibraryDirectories=[/data/app/io.ionic.starter-eC91gWckgNN3MK_UB9WYeA==/lib/arm64, /system/lib64, /system/vendor/lib64]]] couldn't find "libopencv_java3.so"
I was able to solve this by following the solution in this thread https://stackoverflow.com/questions/33030900/android-unsatisfiedlinkerror-with-tesseract-and-opencv
any ideas about a proper fix?

Hey @ChrisTomAlx ,
I have tested now without the workaround and its works fine. Thanks for the help mate 👍
Regards,
Waqas

@waqasmq
That is great news. Will let the people on the other issue know this has been fixed as well.
Cheers and thanks a ton for your help :)
Chris
Neutrinos

i got the same crash issue on Android before...
but i "git clone" my ionic project
and then "npm i", "ionic cordova platform add android"
right now it works perfectly on the phone !!
sooo thank you for fix this issue

@openopen114
I am glad that it worked for you ;)
Cheers,
Chris
Neutrinos


Offline version of cartopy

@willirath commented on Sun Feb 05 2017
(I'd be happy to provide a PR myself.)
Problem
Cartopy provides a very limited set of feature data (shapefiles et al.) and downloads features to the user cache whenever they are needed.  This is usually a good practice, but interferes with people working without permanent internet access (seagoing oceanographers, people in large data-centers who are trapped behind firewalls, someone working on a plane, etc.).
Proposed solution
Create a feedstock which downloads all features upon installation.
To download all features, use (on python2.7)
feature_download.py \
    --output ${CONDA_PREFIX}/lib/python2.7/site-packages/cartopy/data physical \
    gshhs cultural cultural-extra

with feature_download.py from the cartopy repo.
Caveats
Make sure to make use of conda's hard linking across virtual environments?

@jakirkham commented on Sun Feb 05 2017
For future reference, discussion of changes to an individual package should be conducted on the issue tracker of that feedstock. In this case, the cartopy feedstock would be the right place. Am moving this issue over to that feedstock. Please discuss more over there.

@willirath, this is a great idea.  I have also run into this problem several times.
I presume we would want to create a second, optional conda package that contains the data, say cartopy-data?
Also, there doesn't seem to be a way to set an environment variable that cartopy recognizes to point to the path for the downloaded data.  Given that the default location is outside of a conda environment, this could be a problem.  I need to look into this a bit more so I can provide more details.

I've solved this problem for me using https://git.geomar.de/open-source/cartopy_offlinedata
It has most of the logic needed for building a conda-forge package containing the data.
And there is a mechanism for overriding standard configs in cartopy: https://github.com/SciTools/cartopy/blob/9114626b429627a90c6ed65684e72f4d4384fce7/lib/cartopy/__init__.py#L40
So from my PoV, creating a cartopy-data package boils down to:

[ ] Check if it's allowed to redistributed the  https://naciscdn.org data.
[ ] Use https://github.com/SciTools/cartopy/blob/master/tools/feature_download.py to download a full copy of the data when the package is built. (See https://git.geomar.de/open-source/cartopy_offlinedata/blob/23ea474df8b70b65b68032a0074cab2c5289f922/bin/download_cartopy_data.sh for a bash script already doing this.)
[ ] Ensure that cartopy_userconfig.update_config is importable and sets the path to the data. (Preferable to do this via an env var?)

I don't really have time to do this myself. But if you want to give it a shot, I'd chime in whenever necessary.

Thanks so very much for the instructions.  I hadn't realized there was this this mechanism for creating a cartopy_userconfig (documentation remains sparse), and that is just the ticket!

Check if it's allowed to redistributed the https://naciscdn.org data.

This link makes is quite clear that redistribution won't be a problem: http://www.naturalearthdata.com/about/terms-of-use/
I have a little time so I'll see if I can get this going today.

@willirath, I've made some progress.  Before I get too far, I wondered if:

you are willing to accept a pull request to the https://git.geomar.de/open-source/cartopy_offlinedata repo (and if this is possible)?
whether the changes on the following branch (which I'm happy to explain if needed) would be acceptable to you: https://github.com/xylar/cartopy_offline_data/tree/add_license
whether you would be willing to make a 0.2.0 release once these changes are merged?

Unfortunately, I can't make a conda-forge package from your repo unless there is a license file and an official release.

Here is the draft recipe on staged-recipes, awaiting a release of cartopy_offlinedata from @willirath:
https://github.com/xylar/staged-recipes/tree/cartopy_offlinedata/recipes/cartopy_offlinedata

@xylar Thanks a lot! I've moved this to Github and invited you as a collaborator.

:tada: https://github.com/conda-forge/cartopy_offlinedata-feedstock


Return shortlink referring to a given longlink

Currently, we're only searching for/ returning the long link for a given resource. However, the archives also contain information about the shortlink within the file (the format of the file is in the Beacon Link Dump format:
b9YiMs|https://www.google.com

It'd be helpful to also return the shortlink which refers to the longlink. This could be implemented by scanning the archive to find the entire line that contains a resource instead of just the specific string that was matched.

@utkusen I dug through the code a bit, and it looked like the current behavior has been hardcoded in: https://github.com/utkusen/urlhunter/blob/020010daa28a71c2b71a1b40d3f9c68fe327d59f/main.go#L224
Would you be OK with also logging the first part?


failed to resolve compile 'com.github.ybq:Android-SpinKit:1.0.4'

failed to resolve compile 'com.github.ybq:Android-SpinKit:1.0.4'

I'm getting this error.

Doesnt work either @mesutpiskin

Try updating to 1.2.0 and try
'com.github.ybq:Android-SpinKit:1.2.0'
@Nikhil-z @Seffah


Updates for ROCM 5.6

retest gpu-pycpp please

retest gpu-pycpp please

retest gpu-non-pip-multi please

retest cpu-pycpp please

retest code-check please

retest Ubuntu-CPU please

retest cpu-pycpp please

retest Ubuntu-GPU-single please


Standerize the use of elements using ele

Is there a reason we are sub-classing Element from openMMin some avenues in the code base as well as using parmed's periodic table for inferring elements in some other avenues in the code base? It seems to me that a lot of this can be standardized using ele.

the parmed and openmm element are from the old implementation since gmso and ele does not exist. and I think we should replace them soon to eventually move away from parmed an openmm


FIX: orientation on iOS 7 and earlier after rotation, due to adding s…

Hi, I am using your cool AlertView library in my app, it's very cool and highly customizable. However, two issues occurs when I integrate then into my app.
First one is rotation in iOS 7 and earlier systems. My app runs in iOS 7 (iPad landscape), I found that the alert view is still portrait (never rotates according to the current device orientation). I checked the source code and found that the alert view was added as a subview of UIWindow when showing  alert view in window. That's the problem, because before iOS 8, UIWindow is always rotating with the device (or UIScreen), and internal rotation is done by setting the transform of the Window's rootViewController.
Second one is that the blackOpaqueView used for dimming background like the visual effect of UIAlertView. This view is added as a subview of the applications key window, thus never will this blackOpaqueView dim the status bar.
I have tried to solve these two problems and created this PR. Wish you a good day.

Many thanks for fixing those issues for my library, specially the first issue about which I received some comments of developers.
Your fixes are really good.
I have merged your pull request.
:] :]

My pleasure :]


readme clean up

Change distance with button and + - 5mph is also working only for Toyota. Can we added that somewhere and take it fr to-do :)

[ ] Dynamic distance profiles Can be taken off

Sweet! Thank you kind sir. Will merge it once the Subaru pr is merge :)


Fill out all required Documentation

Is your documentation request related to a problem? Please describe.
Is your documentation request related to a problem? Please describe.
I need to complete all the missing documentation in the project, including but not limited to README files, installation guides, API documentation, and any other relevant sections that are yet to be implemented. The current documentation is either incomplete or missing entirely, making it challenging for new users or contributors to understand and work with the project effectively.

Approved

Approved


Expand the auditlog entry with more data

Depends on https://github.com/SURFnet/Stepup-Middleware/pull/76

Redundant PR, must have been sleeping...

superseded by #27


refactor: removal of the old version of ProductPage; OFF language and country

Deleted files:

attribute_card.dart
attribute_list_expandable.dart
category_card.dart
category_chip.dart
category_product_query.dart
choose_page.dart
group_product_query.dart
product_page.dart
smooth_expandable_card.dart
subcategory_card.dart
text_search_widget.dart

Impacted files:

barcode_product_query.dart: now using OFF classes for language and country instead of Strings
continuous_scan_model.dart: now using OFF classes for language and country instead of Strings
keywords_product_query.dart: now using OFF classes for language and country instead of Strings
knowlegde_panel_query.dart: now using OFF classes for country instead of String
product_cards_helper.dart: moved here method from now defunct AttributeListExpandable
product_dialog_helper.dart: now using OFF classes for language and country instead of Strings
product_query.dart: now computing values for OFF classes for language and country instead of Strings
pubspec.yaml: removed now useless libraries flutter_staggered_animations, flutter_staggered_grid_view, flutter_typeahead and share_plus, and upgraded some version numbers
scan_page.dart: now using OFF classes for language and country instead of Strings
search_page.dart: moved methods from now defunct ChoosePage
smooth_product_card_found.dart: removed a never used "old style" version; use a method moved to product_dialog_helper.dart
smooth_ui_library.dart: removed the entry about now defunct smooth_expandable_card.dart
summary_card.dart: use a method moved to product_dialog_helper.dart

Thank you @jasmeet0817!


LobbyItems

Hi.
Excuse me, can you update TntTag plugin to pm5?

I Didn't Create a TntTag Plugin

I Didn't Create a TntTag Plugin
Are you the wrong user? I confirm that I did not create a TntTag plugin in PmmP

I know, but you have updated plugins that were already made by other people.


When a spawned process terminates, goon may keep its input pipe open

Investigate if that is the case. Originally reported in https://github.com/alco/porcelain/issues/15.

I can reproduce this using the following steps:

Listen on a UNIX socket: $ echo 'Hello, world!' | socat UNIX-LISTEN:/tmp/test.sock -
In another terminal: $ goon -proto 1.0 -in=true -out=true socat UNIX-CONNECT:/tmp/test.sock -
Expected: goon should print the message and terminate
Actual: goon prints out "Hello, world!", but does not terminate. $ ps -A | grep socat shows socat <defunct>, so the child process has terminated. When I type some characters into stdin and press Enter,  goon terminates with panic: write |1: broken pipe.

In my Elixir program, I currently work around this by exchanging inexpensive messages in 10-second intervals with the spawned process. Falling back to the basic driver also works.


Sending a node to sleep does not require a response

Bug happens on the fsm/nodeStatus branch
16:44:43.384 DRIVER » [Node 020] [REQ] [SendData]
                      │ transmitOptions: 0x25
                      │ callbackId:      173
                      └─[SecurityCCCommandEncapsulation]
                        │ payload: 0x9b015f1f1e18f5f903b0bc6c53465ba0a3a7f59b
                        └─[WakeUpCCNoMoreInformation]
16:44:43.394 SERIAL « [ACK]                                                                   (0x06)
16:44:43.401 SERIAL « 0x0104011301e8                                                       (6 bytes)
16:44:43.407 SERIAL » [ACK]                                                                   (0x06)
16:44:43.410 DRIVER « [RES] [SendData]
                        wasSent: true
16:44:43.422 SERIAL « 0x01180013ad000003007e7f7f7f7f000003000000000301000025              (26 bytes)
16:44:43.424 SERIAL » [ACK]                                                                   (0x06)
16:44:43.427 DRIVER « [REQ] [SendData]
                        callbackId:     173
                        transmitStatus: OK
16:44:45.043 CNTRLR   [Node 020] did not respond after 1/3 attempts. Scheduling next try in 500 ms.

solved in fsm/all


[tool_crash] ProcessException: The command failed Command: hvigorw, OS error code: 255

Command
flutter build hap --debug

Steps to Reproduce

...
...
...

Logs
ProcessException: The command failed Command: hvigorw, OS error code: 255
#0      RunResult.throwException (package:flutter_tools/src/base/process.dart:126:5)
#1      _DefaultProcessUtils.runSync (package:flutter_tools/src/base/process.dart:414:17)
#2      hvigorwTask (package:flutter_tools/src/ohos/hvigor.dart:141:41)
#3      assembleHar (package:flutter_tools/src/ohos/hvigor.dart:206:10)
#4      OhosHvigorBuilder.assembleHars (package:flutter_tools/src/ohos/hvigor.dart:704:33)
#5      OhosHvigorBuilder.buildHap (package:flutter_tools/src/ohos/hvigor.dart:461:11)
<asynchronous suspension>
#6      BuildHapCommand

This should be reported on gitee.com/openharmony-sig/flutter_flutter.
Make sure you have all the required tools installed.
Closing from here.
Thank you


create langgraph introductory notebook

Create a new notebook before notebook 4 with a langgraph introduction.
mention checkpoints (user interrupt)

is anyone working on this issue?

Gelesen & Getestet :heavy_check_mark:


fix: trim off newline security vuln

:tada: This PR is included in version 1.14.2 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


chore: remove split out code

Remove code that has been split out to its own repo.
This includes the following projects:

compute
logging
dialogflow
dlp
dns
errorreporting
kms
language
monitoring
resourcemanager
talent
video_intelligence
vision

Fixes #<issue_number_goes_here> (it's a good idea to open an issue first for context and/or discussion)

Speech has already been removed, which is why the test is failing. We still need to clean up the kokoro job


[BUG] %player% token no longer being resolved

What happened?
-Since updating to DCIntegration 1.1.8-1.14, messages relayed from Discord to Minecraft now appear as "%player% " rather than the usual player name.
What should happen?
-Presumably the token should be replace with the player's name.
Steps to reproduce

Unable to give indication of reproduction steps.

Version

[ ] 1.12
[x] 1.14

Mod version 1.1.8
Crash report if available (use https://paste.dimdev.org)
Additional info

Same problem

Use %user% here.
https://github.com/ErdbeerbaerLP/Discord-Chat-Integration/issues/25

Thanks for the clarification!
The default config still uses %player% which is why I wasn't sure, as I deleted the config to force it to regenerate a new one and then copied my Discord information back over.

For some reason mod uses msgChatMessage (which is currently = "<%player%> %msg%") instead of ingameDiscordMsg.

oh, that could have happened. Let me look at it

Yes, update gets pushed to curseforge (might take some time)


NIFI-11885 Update asn1bean-compiler to 1.14.0

Summary
NIFI-11885
Tracking
Please complete the following tracking steps prior to pull request creation.
Issue Tracking

[X] Apache NiFi Jira issue created

Pull Request Tracking

[X] Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
[X] Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000

Pull Request Formatting

[X] Pull Request based on current revision of the main branch
[X] Pull Request refers to a feature branch with one commit containing changes

Verification
Please indicate the verification steps performed prior to pull request creation.
Build

[ ] Build completed using mvn clean install -P contrib-check

[ ] JDK 17

Licensing

[ ] New dependencies are compatible with the Apache License 2.0 according to the License Policy
[ ] New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

[ ] Documentation formatting appears as expected in rendered files

If committed, will immediately do the asn1bean update to this same version

If committed, will immediately do the asn1bean update to this same version

@mr1716 instead of handling that in a separate PR, I recommend updating this PR to include that change since the versions should be aligned.


Reloading the page for voting app increases counter

Refer to the attached gif for proof.

Thank you for the feedback @dewan-ahmed
This is not an issue we're planning to address for this demo application.


First example: 'utils' not found

The first example in your documentation (DensityBaseline) does not run right on my machine: it throws a 'module not found' exception at the call to 'utils'.
This might be a python version problem (I am using 3.6), so perhaps it's not an error in the normal sense - though I don't see any specification that the package required a particular python version. Apologies if I missed it: in any case, I fixed it by importing matplotlib instead: i.e.
import matplotlib.pyplot as plt
plt.scatter(y_test, y_pred)
instead of:
import utils
utils.plot_performance(y_test, y_pred)

Ah - it seems this does run fine when using the 'bleeding edge' implementation. Just didn't have the relevant 'utils.py' in the stable release.

Hi, thanks for raising this. The utils module is not part of the actual skpro package but just a python module in the examples folder here. To make the import work you have to make sure that the utils.py file is present at the same directory of your python script. Alternatively, you can copy the plot_performance function directly from the utils.py.
Hope that helps.


mediapipe hangs if tensorflow imported

Hello,
I am using a mac with macOS Catalina (version 10.15.7) - intel core i7. If I import tensorflow first, then mediapipe hangs and waits. If I don't mediapipe, then it works. I write the following code to test importing and not importing tensorflow cases.
BTW, my python version is 3.8.12 and mediapipe version is <IP_ADDRESS>.
if False:
    import tensorflow as tf
    print(tf.__version__) #mine version is 2.7.0

import mediapipe as mp
import cv2

img = cv2.imread("deepface/tests/dataset/img1.jpg")

mp_face_detection = mp.solutions.face_detection

#this line prints "INFO: Created TensorFlow Lite XNNPACK delegate for CPU." if tensorflow was not imported yet
face_detector = mp_face_detection.FaceDetection( min_detection_confidence=0.6)

#this line hangs and waiting if tensorflow was imported
results = face_detector.process(img)

print(results.detections)

I found a similar issue here, but it is closed because of inactivity.

Duplicate https://github.com/google/mediapipe/issues/1461#issuecomment-754868931

But that is not a solution, right?

Yes its known issue in MediaPipe and We don't have any resolution at present. We will keep you inform on the main thread once we found any update. Thanks!

Hello @serengil,
We are upgrading the MediaPipe Legacy Solutions to new MediaPipe solutions However, the libraries, documentation, and source code for all the MediaPipe Legacy Solutions will continue to be available in our GitHub repository and through library distribution services, such as Maven and NPM.
You can continue to use those legacy solutions in your applications if you choose. Though, we would request you to check new MediaPipe solutions which can help you more easily build and customize ML solutions for your applications. These new solutions will provide a superset of capabilities available in the legacy solutions.


CODE RUB: CompletionDependencyValidationException Class Update v2.10.1

Updated CompletionDependencyValidationException class by removing constructors in which the error message was hard coded in. Updated all relevant service files to include message.

Closed Pull Request. Resolved in PR #557.


DOC Prepend version with 'v' in switcher

Don't think this will fix the problem with the version switch drop down but standardises naming to add 'v' (which most other projects use)

Closing as I got it mixed up!


Test testMetrics fails intermittently

Description:
Below is the error log. More details can be found in scans.gradle (not sure for long it will be retained though)
[2020-06-03 10:48:51,786] INFO {org.ballerinalang.test.util.websocket.client.WebSocketTestClient} - WebSocket Client connected!
[2020-06-03 10:48:51,977] INFO {org.ballerinalang.test.util.websocket.client.WebSocketTestClient} - WebSocket Client disconnected!
[2020-06-03 10:48:52,488] INFO {org.ballerinalang.test.observability.metrics.WebSocketMetricsTestCase} - ws_messages_sent_value{client_or_server="server",service="/basic/ws",type="text",} -- 5.0
[2020-06-03 10:48:52,489] INFO {org.ballerinalang.test.observability.metrics.WebSocketMetricsTestCase} - ws_errors_value{client_or_server="server",service="/basic/ws",type="close",} -- 1.0
[2020-06-03 10:48:52,490] SEVERE {org.ballerinalang.test.observability.metrics.WebSocketMetricsTestCase} - testMetrics: FAILED -> Unexpected value found for metric ws_errors_value{client_or_server="server",service="/basic/ws",type="close",}. expected [2.0] but found [1.0]

I have attached the complete log file as well.
log-websocket.txt

These test cases had been removed when the websocket module was moved out of the lang repo.
Hence closing the issue.


Ye

I'd like to have my profile on ghuser.io, thanks!

here you go :) https://ghuser.io/OdarGM
you'll probably want to read https://github.com/AurelienLourot/ghuser.io/#some-of-my-repos-are-not-showing-up-on-my-profile-why

Thx


settings panel does not show

Not sure why/when this broke. When clicking the icon, the translucent overlay appears, but no dialog.
Issue #3 might be related.

Fixed with https://github.com/stapelberg/scan2drive/commit/ae154125e0f38dcbe633bf3ba2a3aeb8757d6eb6


[Feature][dolphinscheduler-api] ProcessDefinitionController save API returns ProcessDefinitionId

current ProcessDefinitionController   save API didn't returns a new ProcessDefinition's ID,if use API to create ProcessDefinition ,the default release_state is  0(off_line) and no   ProcessDefinitionId returns, so if i want this ProcessDefinition online, i must call query Api to get the ProcessDefinition,then  call  release API to make the ProcessDefinition  online

i have fork the git and fix the bug,lets contact in WeChat

i have fork the git and fix the bug,lets contact in WeChat

I found that the dev branch has fixed your requirement,It returns processdefinitionid after the workflow definition is saved successfully.

I found that the dev branch has fixed your requirement,It returns processdefinitionid after the workflow definition is saved successfully.

I found that the dev branch has fixed your requirement,It returns processdefinitionid after the workflow definition is saved successfully.

i see,

but is the implementation suitable? why not only return processDefineId? and  if so,why should query DB?and  if return the processDefine Entity,will Impact on Performance？

I found that the dev branch has fixed your requirement,It returns processdefinitionid after the workflow definition is saved successfully.

i see,

but is the implementation suitable? why not only return processDefineId? and  if so,why should query DB?and  if return the processDefine Entity,will Impact on Performance？


DCA-148 Revert Capitalisation

Fix general content issues in preparation for our repo to consume and publish this in the devportal
Description
This is same PR as DCA-148-Content-Fix, except all capitalisation for accounting-api endpoints have been reverted
Fix general content issues (spelling errors, capitalisation consistency, invalid guids, summary content etc.)
No changes in terms of actual structure, just content :)
Release Notes
As we are going to publish our public api reference using this repo in the near future, we need to fix any content related issues that may appear in the page (spellings, consistent text etc.)
Types of Changes

[/ ] Bug fix (non-breaking change that fixes an issue)
[ ] New feature (non-breaking change that adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

@SidneyAllen, ahh my bad. I thought the issue was specific to the accounting yaml file.
I've updated all instances of endpoints with the term "ID" back to "Id"


Add 20-inch Gray Performance Wheels for Model 3

As titled. Option code is taken from my vehicle invoice.

Thanks!


Notification Volume

Hi!
The notification sound is really, really loud, it scares me everytime haha
I work on mac and can't adjust the volume of google chrome or any application individually
hope a kind person can help solving this problem
thank you so much

Hi,
I'll try to add volume control for the next update.
For now maybe this can help :)
https://github.com/kyleneideck/BackgroundMusic

I agree because my autism was extremely painful and might be suffering for some time I had my headphones in and it really really really hurt please do update soon for now i can turn the sound off and await your brilliance in making it autism-friendly, thanks for what you have done so far awesome stuff

Thank you so much for working on this feature! I tried background music and other things that didn't work, unfortunately

@meepetpet  @jamieenby  Volume and new sounds are added in the next update (5.75) !


[bug] Support integers as column names

Hi, I recently stumbled about the following problem:
In[98]: adata
Out[98]:
AnnData object with n_obs × n_vars = 20728 × 32738
    obs: 0, 'batch', 'condition', 'source'
    var: 0, 1
In[99]: adata.write(file)
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2963, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-99-c21b8d69b5d6>", line 1, in <module>
    adata.write(file)
  File "/usr/lib/python3.6/site-packages/anndata/base.py", line 1779, in write
    _write_h5ad(filename, self, compression=compression, compression_opts=compression_opts)
  File "/usr/lib/python3.6/site-packages/anndata/readwrite/write.py", line 94, in _write_h5ad
    d = adata._to_dict_fixed_width_arrays()
  File "/usr/lib/python3.6/site-packages/anndata/base.py", line 1926, in _to_dict_fixed_width_arrays
    obs_rec, uns_obs = df_to_records_fixed_width(self._obs)
  File "/usr/lib/python3.6/site-packages/anndata/base.py", line 176, in df_to_records_fixed_width
    uns[k + '_categories'] = c.cat.categories.values
TypeError: unsupported operand type(s) for +: 'int' and 'str'

As it seems, df_to_records_fixed_width has problems when some column names are actually integers.
The following solves this problem:
adata.var.columns = adata.var.columns.astype(str)
adata.obs.columns = adata.obs.columns.astype(str)

Related:
When all columns are integers or strings of integers, anndata.read_h5ad() fails to read a written dataset:
# dataset = anndata.read_h5ad(file)
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/pandas/core/arrays/categorical.py", line 581, in from_codes
    codes = coerce_indexer_dtype(np.asarray(codes), categories)
  File "/usr/lib/python3.6/site-packages/pandas/core/dtypes/cast.py", line 606, in coerce_indexer_dtype
    return _ensure_int16(indexer)
  File "pandas/_libs/algos_common_helper.pxi", line 3209, in pandas._libs.algos.ensure_int16
  File "pandas/_libs/algos_common_helper.pxi", line 3214, in pandas._libs.algos.ensure_int16
ValueError: invalid literal for int() with base 10: 'ENSG00000243485'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2963, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-9-4e533aad0e71>", line 1, in <module>
    dataset = anndata.read_h5ad(file)
  File "/usr/lib/python3.6/site-packages/anndata/readwrite/read.py", line 345, in read_h5ad
    return AnnData(d)
  File "/usr/lib/python3.6/site-packages/anndata/base.py", line 636, in __init__
    filename=filename, filemode=filemode)
  File "/usr/lib/python3.6/site-packages/anndata/base.py", line 755, in _init_as_actual
    X, obs, var, uns, obsm, varm, raw = self._from_dict(X)
  File "/usr/lib/python3.6/site-packages/anndata/base.py", line 1878, in _from_dict
    categories=v)
  File "/usr/lib/python3.6/site-packages/pandas/core/arrays/categorical.py", line 584, in from_codes
    "codes need to be convertible to an arrays of integers")
ValueError: codes need to be convertible to an arrays of integers

Yes, currently only string column names are allowed. It would take a while to adapt anndata to integer names. Why do you need it?
In any case: there should more meaningful error messages. Sorry for not having thought of this.

I stumbled across this when I read some barcodes.tsv with Pandas and headers=False.
Headers=False causes Pandas to create column names by simply numbering them.
This is usually no problem since the column names are usually set afterwards (e.g. adata.obs_names=...).
However, I tried to join multiple AnnData objects and did not want to name any columns before merging them.
I do not need integer column names, I just thought this would be a bug :)

It is a bug! 🙂 Thanks for pointing it out!

oh yes good to know... I was just writing an issue on this. This could easily be fixed by just converting integer arguments to strings.

that would be too surprising. if people then try to index with integers and it doesn’t work as expected, they’ll get confused

Because after loading the columns could not be indexed by integers anymore? Damn...
Would it be possible to write an error message until this works?

Certainly.
I think the problem with the whole thing is that I was wrong. there is a non-technical reason why this is not supported:
If we support integers as column names, indexing becomes ambiguous: does adata[0,:] mean the first row or the row with the column name 0?
pandas got around it via .loc and .iloc where one operates on names and one on indices. What should we do?

So the problem arises with the annotations, which are pandas dataframes.
In general, it seems reasonable not to change the functionality and behaviour of third party packages, so since pandas supports indexing by integers, you may also support it with anndata, with all its caveats.
When you convert a data frame to a record array and back, all numerical string values, will be converted to integers.
>>> df = pd.DataFrame({0: [1, 2, 3], '1': [4, 5, 6]})
>>> pd.DataFrame.from_records(df.to_records())
   index  0  1
0      0  1  4
1      1  2  5
2      2  3  6

I think it is more important that the data is written and can be read properly. You can easily warn the user about problems when mixing up int and str indexes, within the documentation.

pandas got around it via .loc and .iloc where one operates on names and one on indices. What should we do?

Yes, I'm well aware of that and I've thought several times about whether we should do this, too. but as of today, I vote against it. I cannot think of a single good practical reason where integer indices are meaningful. The only practical reason that I can think of concerns memory usage. The only convincing - but not practical - reason would be full consistency with the full flexibility of pandas. But I like the current behavior of []; and the docs explain it well, I think:

Indexing into an AnnData object with a numeric is supposed to be positional, like pandas .iloc method, while indexing with a string/ categorical is supposed to behave like .loc.

In general, it seems reasonable not to change the functionality and behaviour of third party packages,

We don't change functionality of pandas. But you're right, we make more assumptions than pandas: only string indices, only string categories (like R's factors). That's not a good thing and it should usually be solved by generating a subclass that restricts to that functionality. But that is pretty tricky with pandas dataframes and we don't want to get into that. Writing warnings is a good thing. I don't know why I still haven't added these warnings. I'd actually even throw an error. But in a way the solution will always be half-cooked, because I don't want to rewrite the actual index object of pandas via subclassing, but just the setters in AnnData for .obs, .var, .var_names, .obs_names, which are also called in the constructor. So, people will still be able to call .obs[whatever_you_like] and put there whatever they like, which is bad... Hm.
Happy to discuss more.

We were mixing up things, here. Integer column names are no technical problem. Integer indices are a technical problem. The latter is addressed through the warnings, here: https://github.com/theislab/anndata/commit/51be5d85e62b1aa19d6f55ffc21aedbe9628c4c5.
The former should be addressed in a different way.

I had this issue as well–a very simple use case where this comes up is if I try to use lists for gene and cell names:
adata = anndata.AnnData(X, obs=cell_names, var=gene_names)

The resulting object has integer column names for obs and var. From a usability perspective, I think the above code (which is simple and intuitive) should Just Work™. The solution is to create an empty pandas DataFrame with the specified index (but not an Index or a Series!), but that was not at all obvious to me...

Hm, I disagree: the above shouldn't work, as the docs clearly state that you need to call obs=pd.DataFrame(index=cell_names). Alternatively, you can also create the object and then do adata.obs.index = cell_names.
It's a bug that we don't throw an error if something other than a dataframe or dict is passed.
@flying-sheep: I'll move forward and add the type check, you agree?

Sure, it should either work correctly or raise an error immediately–silently working until you try to write a file is the worst of both worlds.
For what it's worth I disagree that "the docs clearly state" the required usage–the DataFrame(index=cell_names) syntax is never actually mentioned.

Thanks for pointing that out! :)
I meant the docs "clearly state that one should pass a dataframe or a dict". But right, we should add some more explanation. I'll add that together with the type check. I agree that silently working is the worst that can happen. It has just never been pointed out by anyone that you could pass a simple iterable to obs and the constructor wouldn't complain. 🙈

Update on this:
Previously, conversion to a rec array converted any columns names of obs to strings. Now we don't use a recarray intermediate, but instead write each column as it's own hdf5 dataset to the h5py file. hdf5 datasets can't have integers for names, so that fails.

#777 is a more complete version of this issue, so closing this to retain only one duplicate


CFFL

CFFL

Linter check found the following problems:
The following problems have been found:
ERROR: /tmp/abs_6121dwokoa/clone/recipe/meta.yaml:15: pip_install_args: pip install should be run with --no-deps.
ERROR: /tmp/abs_6121dwokoa/clone/recipe/meta.yaml:20: missing_wheel: For pypi packages, wheel should be present in the host section
ERROR: /tmp/abs_6121dwokoa/clone/recipe/meta.yaml:27: missing_pip_check: For pypi packages, pip check should be present in the test commands
Errors were found


Fix composer requirement.

Because the last commit requires adamwathan/form 0.7.1 or greater.

Travis, I forgot to write a test for this change.