HDFS-17381. Distcp of EC files should not be limited to DFS.

Description of PR
Change to support distcp of EC files to and from Ozone FS
How was this patch tested?
Will add tests

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
8m  0s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  1s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

-1 :x:
test4tests
0m  0s

The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ trunk Compile Tests _

-1 :x:
mvninstall
0m 17s
/branch-mvninstall-root.txt
root in trunk failed.

-1 :x:
compile
4m  0s
/branch-compile-hadoop-tools_hadoop-distcp-jdkUbuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04.txt
hadoop-distcp in trunk failed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04.

-1 :x:
compile
0m 22s
/branch-compile-hadoop-tools_hadoop-distcp-jdkPrivateBuild-1.8.0_392-8u392-ga-1~20.04-b08.txt
hadoop-distcp in trunk failed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08.

-0 :warning:
checkstyle
0m 20s
/buildtool-branch-checkstyle-hadoop-tools_hadoop-distcp.txt
The patch fails to run checkstyle in hadoop-distcp

-1 :x:
mvnsite
0m 21s
/branch-mvnsite-hadoop-tools_hadoop-distcp.txt
hadoop-distcp in trunk failed.

-1 :x:
javadoc
0m 22s
/branch-javadoc-hadoop-tools_hadoop-distcp-jdkUbuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04.txt
hadoop-distcp in trunk failed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04.

-1 :x:
javadoc
0m 23s
/branch-javadoc-hadoop-tools_hadoop-distcp-jdkPrivateBuild-1.8.0_392-8u392-ga-1~20.04-b08.txt
hadoop-distcp in trunk failed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08.

-1 :x:
spotbugs
0m 24s
/branch-spotbugs-hadoop-tools_hadoop-distcp.txt
hadoop-distcp in trunk failed.

-1 :x:
shadedclient
2m 21s

branch has errors when building and testing our client artifacts.

_ Patch Compile Tests _

-1 :x:
mvninstall
0m 21s
/patch-mvninstall-hadoop-tools_hadoop-distcp.txt
hadoop-distcp in the patch failed.

-1 :x:
compile
0m 22s
/patch-compile-hadoop-tools_hadoop-distcp-jdkUbuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04.txt
hadoop-distcp in the patch failed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04.

-1 :x:
javac
0m 22s
/patch-compile-hadoop-tools_hadoop-distcp-jdkUbuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04.txt
hadoop-distcp in the patch failed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04.

-1 :x:
compile
0m 21s
/patch-compile-hadoop-tools_hadoop-distcp-jdkPrivateBuild-1.8.0_392-8u392-ga-1~20.04-b08.txt
hadoop-distcp in the patch failed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08.

-1 :x:
javac
0m 21s
/patch-compile-hadoop-tools_hadoop-distcp-jdkPrivateBuild-1.8.0_392-8u392-ga-1~20.04-b08.txt
hadoop-distcp in the patch failed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08.

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
0m 19s
/buildtool-patch-checkstyle-hadoop-tools_hadoop-distcp.txt
The patch fails to run checkstyle in hadoop-distcp

-1 :x:
mvnsite
0m 22s
/patch-mvnsite-hadoop-tools_hadoop-distcp.txt
hadoop-distcp in the patch failed.

-1 :x:
javadoc
0m 22s
/patch-javadoc-hadoop-tools_hadoop-distcp-jdkUbuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04.txt
hadoop-distcp in the patch failed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04.

-1 :x:
javadoc
0m 22s
/patch-javadoc-hadoop-tools_hadoop-distcp-jdkPrivateBuild-1.8.0_392-8u392-ga-1~20.04-b08.txt
hadoop-distcp in the patch failed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08.

-1 :x:
spotbugs
0m 22s
/patch-spotbugs-hadoop-tools_hadoop-distcp.txt
hadoop-distcp in the patch failed.

+1 :green_heart:
shadedclient
4m  8s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

-1 :x:
unit
0m 22s
/patch-unit-hadoop-tools_hadoop-distcp.txt
hadoop-distcp in the patch failed.

+0 :ok:
asflicense
0m 22s

ASF License check generated no output?

24m 43s

Subsystem
Report/Notes

Docker
ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6551/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6551

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux 77c2b8d5c370 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 616210b04ccc23eb56b3f53aeeb3f36ddee988fd

Default Java
Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6551/1/testReport/

Max. process+thread count
51 (vs. ulimit of 5500)

modules
C: hadoop-tools/hadoop-distcp U: hadoop-tools/hadoop-distcp

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6551/1/console

versions
git=2.25.1 maven=3.6.3

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 36s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

-1 :x:
test4tests
0m  0s

The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ trunk Compile Tests _

+0 :ok:
mvndep
14m 41s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
35m 47s

trunk passed

+1 :green_heart:
compile
17m 44s

trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
compile
16m 26s

trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
checkstyle
4m 24s

trunk passed

+1 :green_heart:
mvnsite
3m 47s

trunk passed

+1 :green_heart:
javadoc
3m  2s

trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
2m 37s

trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

-1 :x:
spotbugs
2m 38s
/branch-spotbugs-hadoop-common-project_hadoop-common-warnings.html
hadoop-common-project/hadoop-common in trunk has 1 extant spotbugs warnings.

-1 :x:
spotbugs
2m 48s
/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client-warnings.html
hadoop-hdfs-project/hadoop-hdfs-client in trunk has 1 extant spotbugs warnings.

+1 :green_heart:
shadedclient
33m 24s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 32s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
2m  7s

the patch passed

+1 :green_heart:
compile
17m 27s

the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javac
17m 27s

the patch passed

+1 :green_heart:
compile
16m 35s

the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
javac
16m 35s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
4m 17s
/results-checkstyle-root.txt
root: The patch generated 7 new + 28 unchanged - 0 fixed = 35 total (was 28)

+1 :green_heart:
mvnsite
3m 37s

the patch passed

-1 :x:
javadoc
1m 11s
/results-javadoc-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04.txt
hadoop-common-project_hadoop-common-jdkUbuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 generated 11 new + 0 unchanged - 0 fixed = 11 total (was 0)

+1 :green_heart:
javadoc
2m 35s

the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
spotbugs
6m 56s

the patch passed

+1 :green_heart:
shadedclient
33m 31s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
19m 28s

hadoop-common in the patch passed.

+1 :green_heart:
unit
2m 44s

hadoop-hdfs-client in the patch passed.

+1 :green_heart:
unit
15m 22s

hadoop-distcp in the patch passed.

+1 :green_heart:
asflicense
1m  4s

The patch does not generate ASF License warnings.

275m 59s

Subsystem
Report/Notes

Docker
ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6551/3/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6551

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux 923a0fccfdf0 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / b0f92fb05bd02bf03620e8f0ce6a0f5892465ca0

Default Java
Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6551/3/testReport/

Max. process+thread count
3148 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-tools/hadoop-distcp U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6551/3/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
19m  9s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
detsecrets
0m  1s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

-1 :x:
test4tests
0m  0s

The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ trunk Compile Tests _

+0 :ok:
mvndep
14m 18s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
36m 48s

trunk passed

+1 :green_heart:
compile
19m 48s

trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
compile
18m 51s

trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
checkstyle
4m 46s

trunk passed

+1 :green_heart:
mvnsite
3m 40s

trunk passed

+1 :green_heart:
javadoc
2m 57s

trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
2m 26s

trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

-1 :x:
spotbugs
2m 36s
/branch-spotbugs-hadoop-common-project_hadoop-common-warnings.html
hadoop-common-project/hadoop-common in trunk has 1 extant spotbugs warnings.

-1 :x:
spotbugs
2m 49s
/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client-warnings.html
hadoop-hdfs-project/hadoop-hdfs-client in trunk has 1 extant spotbugs warnings.

+1 :green_heart:
shadedclient
38m 56s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 31s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
2m  8s

the patch passed

+1 :green_heart:
compile
18m 55s

the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javac
18m 55s

the patch passed

+1 :green_heart:
compile
17m 52s

the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
javac
17m 52s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
4m 42s
/results-checkstyle-root.txt
root: The patch generated 7 new + 29 unchanged - 0 fixed = 36 total (was 29)

+1 :green_heart:
mvnsite
3m 35s

the patch passed

-1 :x:
javadoc
1m 11s
/results-javadoc-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04.txt
hadoop-common-project_hadoop-common-jdkUbuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 generated 11 new + 0 unchanged - 0 fixed = 11 total (was 0)

+1 :green_heart:
javadoc
2m 29s

the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
spotbugs
6m 55s

the patch passed

+1 :green_heart:
shadedclient
38m 48s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
19m 23s

hadoop-common in the patch passed.

+1 :green_heart:
unit
2m 42s

hadoop-hdfs-client in the patch passed.

+1 :green_heart:
unit
16m 18s

hadoop-distcp in the patch passed.

+1 :green_heart:
asflicense
1m  4s

The patch does not generate ASF License warnings.

314m 24s

Subsystem
Report/Notes

Docker
ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6551/2/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6551

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux 64057d38dc66 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / b0f92fb05bd02bf03620e8f0ce6a0f5892465ca0

Default Java
Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6551/2/testReport/

Max. process+thread count
2765 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-tools/hadoop-distcp U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6551/2/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
7m 19s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

-1 :x:
test4tests
0m  0s

The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ trunk Compile Tests _

+0 :ok:
mvndep
14m 46s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
21m 44s

trunk passed

+1 :green_heart:
compile
10m 30s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
compile
9m 55s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
checkstyle
2m 11s

trunk passed

+1 :green_heart:
mvnsite
2m  7s

trunk passed

+1 :green_heart:
javadoc
1m 29s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
1m 19s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
spotbugs
3m 37s

trunk passed

+1 :green_heart:
shadedclient
21m 53s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 24s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
1m 19s

the patch passed

+1 :green_heart:
compile
9m 55s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javac
9m 55s

the patch passed

+1 :green_heart:
compile
9m 43s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
javac
9m 43s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
2m 12s
/results-checkstyle-root.txt
root: The patch generated 6 new + 45 unchanged - 0 fixed = 51 total (was 45)

+1 :green_heart:
mvnsite
1m 54s

the patch passed

-1 :x:
javadoc
0m 36s
/results-javadoc-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt
hadoop-common-project_hadoop-common-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)

+1 :green_heart:
javadoc
1m 22s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
spotbugs
4m  5s

the patch passed

+1 :green_heart:
shadedclient
22m  1s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
16m 14s

hadoop-common in the patch passed.

+1 :green_heart:
unit
1m 56s

hadoop-hdfs-client in the patch passed.

+1 :green_heart:
unit
24m 19s

hadoop-distcp in the patch passed.

+1 :green_heart:
asflicense
0m 36s

The patch does not generate ASF License warnings.

196m 56s

Subsystem
Report/Notes

Docker
ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6551/5/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6551

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux e965f57831f3 5.15.0-117-generic #127-Ubuntu SMP Fri Jul 5 20:13:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / db67ccf97b29f9aa89883f29449aa0ac70521187

Default Java
Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6551/5/testReport/

Max. process+thread count
1276 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-tools/hadoop-distcp U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6551/5/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
5m 10s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

-1 :x:
test4tests
0m  0s

The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ trunk Compile Tests _

+0 :ok:
mvndep
14m 46s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
21m 37s

trunk passed

+1 :green_heart:
compile
10m 23s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
compile
9m 43s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
checkstyle
2m 20s

trunk passed

+1 :green_heart:
mvnsite
2m  2s

trunk passed

+1 :green_heart:
javadoc
1m 36s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
1m 17s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
spotbugs
3m 45s

trunk passed

+1 :green_heart:
shadedclient
21m 47s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 20s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
1m 18s

the patch passed

+1 :green_heart:
compile
9m 56s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javac
9m 56s

the patch passed

+1 :green_heart:
compile
9m 54s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
javac
9m 54s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
2m 12s
/results-checkstyle-root.txt
root: The patch generated 6 new + 45 unchanged - 0 fixed = 51 total (was 45)

+1 :green_heart:
mvnsite
1m 59s

the patch passed

-1 :x:
javadoc
0m 36s
/results-javadoc-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt
hadoop-common-project_hadoop-common-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)

+1 :green_heart:
javadoc
1m 23s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
spotbugs
4m  7s

the patch passed

+1 :green_heart:
shadedclient
22m  4s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
16m 16s

hadoop-common in the patch passed.

+1 :green_heart:
unit
2m  0s

hadoop-hdfs-client in the patch passed.

+1 :green_heart:
unit
24m 11s

hadoop-distcp in the patch passed.

+1 :green_heart:
asflicense
0m 37s

The patch does not generate ASF License warnings.

194m 54s

Subsystem
Report/Notes

Docker
ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6551/6/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6551

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux 00ace163478c 5.15.0-117-generic #127-Ubuntu SMP Fri Jul 5 20:13:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / db67ccf97b29f9aa89883f29449aa0ac70521187

Default Java
Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6551/6/testReport/

Max. process+thread count
3152 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-tools/hadoop-distcp U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6551/6/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 18s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
14m  7s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
20m 48s

trunk passed

+1 :green_heart:
compile
9m  1s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
compile
8m 27s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
checkstyle
2m  9s

trunk passed

+1 :green_heart:
mvnsite
2m 14s

trunk passed

+1 :green_heart:
javadoc
1m 54s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
1m 46s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
spotbugs
3m 46s

trunk passed

+1 :green_heart:
shadedclient
21m  6s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 21s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
1m 12s

the patch passed

+1 :green_heart:
compile
8m 46s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javac
8m 46s

the patch passed

+1 :green_heart:
compile
8m 28s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
javac
8m 28s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
2m  0s
/results-checkstyle-root.txt
root: The patch generated 7 new + 88 unchanged - 0 fixed = 95 total (was 88)

+1 :green_heart:
mvnsite
2m 10s

the patch passed

-1 :x:
javadoc
0m 44s
/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt
hadoop-common in the patch failed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.

+1 :green_heart:
javadoc
1m 42s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
spotbugs
4m  7s

the patch passed

+1 :green_heart:
shadedclient
21m 14s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
16m 13s

hadoop-common in the patch passed.

-1 :x:
unit
2m  3s
/patch-unit-hadoop-hdfs-project_hadoop-hdfs-client.txt
hadoop-hdfs-client in the patch passed.

+1 :green_heart:
unit
24m 15s

hadoop-distcp in the patch passed.

+1 :green_heart:
asflicense
0m 44s

The patch does not generate ASF License warnings.

183m 45s

Reason
Tests

Failed junit tests
hadoop.hdfs.protocol.TestHdfsFileStatusMethods

Subsystem
Report/Notes

Docker
ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6551/7/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6551

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux 880b3c77f96a 5.15.0-117-generic #127-Ubuntu SMP Fri Jul 5 20:13:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 292e110f05630c6df1e68912911a44e85f8300ef

Default Java
Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6551/7/testReport/

Max. process+thread count
2503 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-tools/hadoop-distcp U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6551/7/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 19s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
detsecrets
0m  1s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
14m  8s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
20m 28s

trunk passed

+1 :green_heart:
compile
9m  2s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
compile
8m 26s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
checkstyle
2m 11s

trunk passed

+1 :green_heart:
mvnsite
2m 12s

trunk passed

+1 :green_heart:
javadoc
1m 55s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
1m 42s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
spotbugs
3m 38s

trunk passed

+1 :green_heart:
shadedclient
20m 52s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 23s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
1m 17s

the patch passed

+1 :green_heart:
compile
8m 36s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javac
8m 36s

the patch passed

+1 :green_heart:
compile
8m 30s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
javac
8m 30s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
2m  6s
/results-checkstyle-root.txt
root: The patch generated 7 new + 106 unchanged - 0 fixed = 113 total (was 106)

+1 :green_heart:
mvnsite
2m 10s

the patch passed

+1 :green_heart:
javadoc
1m 48s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
1m 44s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
spotbugs
4m  1s

the patch passed

+1 :green_heart:
shadedclient
20m 58s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
16m 11s

hadoop-common in the patch passed.

+1 :green_heart:
unit
2m  1s

hadoop-hdfs-client in the patch passed.

+1 :green_heart:
unit
24m 44s

hadoop-distcp in the patch passed.

+1 :green_heart:
asflicense
0m 43s

The patch does not generate ASF License warnings.

183m  9s

Subsystem
Report/Notes

Docker
ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6551/8/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6551

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux 892917e01b88 5.15.0-117-generic #127-Ubuntu SMP Fri Jul 5 20:13:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / f22360af04c4df7182297f472be768e5c6ae3be5

Default Java
Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6551/8/testReport/

Max. process+thread count
3153 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-tools/hadoop-distcp U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6551/8/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

+can i encourage you to use git merge to resync with with trunk once people have started reviewing? its a lot easier to review with the "changes since your last review" option. thanks

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 29s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
14m 45s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
32m 54s

trunk passed

+1 :green_heart:
compile
17m 26s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
compile
16m 43s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
checkstyle
4m 31s

trunk passed

+1 :green_heart:
mvnsite
3m 47s

trunk passed

+1 :green_heart:
javadoc
3m  1s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
2m 39s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
spotbugs
6m 28s

trunk passed

+1 :green_heart:
shadedclient
35m 14s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 33s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
2m 13s

the patch passed

+1 :green_heart:
compile
19m 44s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javac
19m 44s

the patch passed

+1 :green_heart:
compile
17m 20s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
javac
17m 20s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
4m 26s
/results-checkstyle-root.txt
root: The patch generated 8 new + 142 unchanged - 0 fixed = 150 total (was 142)

+1 :green_heart:
mvnsite
3m 45s

the patch passed

-1 :x:
javadoc
1m 12s
/results-javadoc-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt
hadoop-common-project_hadoop-common-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 generated 4 new + 0 unchanged - 0 fixed = 4 total (was 0)

+1 :green_heart:
javadoc
2m 39s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
spotbugs
6m 58s

the patch passed

+1 :green_heart:
shadedclient
35m 39s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
19m 51s

hadoop-common in the patch passed.

+1 :green_heart:
unit
2m 45s

hadoop-hdfs-client in the patch passed.

+1 :green_heart:
unit
24m 51s

hadoop-distcp in the patch passed.

+1 :green_heart:
asflicense
1m  6s

The patch does not generate ASF License warnings.

287m 46s

Subsystem
Report/Notes

Docker
ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6551/11/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6551

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux f3b848cbecdd 5.15.0-117-generic #127-Ubuntu SMP Fri Jul 5 20:13:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 44ee8e1d18df1bc1735397601cbd8c4453ff54a0

Default Java
Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6551/11/testReport/

Max. process+thread count
1251 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-tools/hadoop-distcp U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6551/11/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 19s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
detsecrets
0m  1s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
14m 26s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
20m 20s

trunk passed

+1 :green_heart:
compile
9m  1s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
compile
8m 34s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
checkstyle
2m  5s

trunk passed

+1 :green_heart:
mvnsite
2m 13s

trunk passed

+1 :green_heart:
javadoc
1m 56s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
1m 41s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
spotbugs
3m 43s

trunk passed

+1 :green_heart:
shadedclient
20m 42s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 22s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
1m 17s

the patch passed

+1 :green_heart:
compile
8m 33s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javac
8m 33s

the patch passed

+1 :green_heart:
compile
8m 25s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
javac
8m 25s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
2m  5s
/results-checkstyle-root.txt
root: The patch generated 8 new + 142 unchanged - 0 fixed = 150 total (was 142)

+1 :green_heart:
mvnsite
2m 16s

the patch passed

+1 :green_heart:
javadoc
1m 51s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
1m 43s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
spotbugs
4m  7s

the patch passed

+1 :green_heart:
shadedclient
21m  5s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
16m 13s

hadoop-common in the patch passed.

+1 :green_heart:
unit
2m  3s

hadoop-hdfs-client in the patch passed.

+1 :green_heart:
unit
24m 24s

hadoop-distcp in the patch passed.

+1 :green_heart:
asflicense
0m 41s

The patch does not generate ASF License warnings.

183m 39s

Subsystem
Report/Notes

Docker
ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6551/12/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6551

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux ad0713a08e22 5.15.0-117-generic #127-Ubuntu SMP Fri Jul 5 20:13:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 3c0b1fb6757a53640167af52c3826258bd7cf21b

Default Java
Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6551/12/testReport/

Max. process+thread count
3152 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-tools/hadoop-distcp U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6551/12/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

Thanks @steveloughran for the reviews, updated the patch.

@sadanand48 merged to trunk; if you can do a PR on hadoop branch-3.4 then once Yetus is happy we can merge it there too

Thanks @steveloughran , HDFS-17376 is a dependency not present in the 3.4 branch,  I have raised https://github.com/apache/hadoop/pull/7073 for that cherry-pick. Once that goes in, I will create a backport for this PR too or  can we club them both in one ?

separately please, easier to track, revert etc.


Make help window content configurable

The content of the HelpWin can currently only be set / changed by changing the source code / overwriting the component.
It would be very useful to have a configurable way of setting the content (preferably HTML) like we can do for other texts like in the footer.

see #173


Fix the repo CI

That red Ci failing badge was bothering me 😄 ... also hoping this makes it easier for you to cut a 0.12 release with the skip_none fix in it!

Not sure why CI didn’t run on this PR, but I manually verified that all tests pass with this PR. Let’s hope I’m doing this right 🤞


Mouse controls should be supported

You should be able to interact with the Dialog component using pixel components. Possible features that come to mind:

Signalling to the component that something is hovered by the mouse, and have that reflected in its graphics
Signalling to the component that a mouse click occurred at position (x, y) and have the component react accordingly to that

Fixed by https://github.com/JonathanMurray/dialog-tree-py/pull/8


Fix Networks, implement NetworkSystem

Network is no longer working after the updates to ConductanceSystem and CompartmentSystem. It should be fixed by writing a NetworkSystem that similar implements the ModelingToolkit system interface.
The default constructor for NetworkSystem should support taking a Metagraph or similar datastrcuture capable of describing network topology, neuron types, synapse types and weights.
For now we will create a directly coupled system of equations. Future features may include spike-triggered discrete synaptic events.

NeuronalNetworkSystem now implemented by #18
Support for graph objects as input punted to #24


Add SpaceDustUnbound V 1.0.2 (More info, recommendations fix)

spec_version: 1
identifier: SpaceDustUnbound
name: Space Dust Unbound
abstract: >-
  Adds Space Dust bands for popular Community Resource Pack resources to
  celestial bodies from the stock game, Galaxies Unbound systems and the Outer
  Planets Mod
author:
  - Vexxel
license: MIT
download: >-
  https://github.com/Vexxel/SpaceDustUnbound/releases/download/v1.0.2/SpaceDustUnbound.v1.0.2.zip
version: 1.0.2
$kref: '#/ckan/github/Vexxel/SpaceDustUnbound'
release_status: stable
resources:
  bugtracker: https://github.com/Vexxel/SpaceDustUnbound/issues
  license: https://github.com/Vexxel/SpaceDustUnbound/blob/main/LICENSE
  spacedock: https://spacedock.info/mod/2970/SpaceDustUnbound
  homepage: https://spacedock.info/mod/2970/SpaceDustUnbound
  repository: https://github.com/Vexxel/SpaceDustUnbound
  x_screenshot: >-
    https://spacedock.info/content/Vexxel_76203/SpaceDustUnbound/SpaceDustUnbound-1643969459.png
ksp_version_min: '1.11'
depends:
  - name: CommunityResourcePack
  - name: SpaceDust
  - name: ModuleManager
recommends:
  - name: OuterPlanetsMod
  - name: GalaxiesUnbound-Core
suggests:
  - name: Blueshift
x_via: Generated by Metadata Webtool

Hi @Vexxel,
You can control the name and abstract on SpaceDock:
https://spacedock.info/mod/2970/SpaceDustUnbound/edit
And the game version compatibility is based on the SpaceDustUnbound.version file in the download, documentation here:
https://github.com/CYBUTEK/KSPAddonVersionChecker/blob/master/Documents/MiniAVC/README.md
I'll start a pull request in a moment for the recommends/suggests change...

Since the .version file has the URL property set properly, you don't even need to change the download, you can update the compatibility in the online copy post-release:
https://github.com/Vexxel/SpaceDustUnbound/blob/main/Version/SpaceDustUnbound.version


[ez] Redirect users to our issues creation page when submitting feedback

[ez] Redirect users to our issues creation page when submitting feedback
Super simple for now. Tried at first to connect to Jonathan's rage functiona but that was defined in the editor package: https://github.com/lastmile-ai/aiconfig/pull/870
Test Plan
https://github.com/lastmile-ai/aiconfig/assets/151060367/1ab905d0-dec8-4168-9a4c-995b7af0cd28

Stack created with Sapling. Best reviewed with ReviewStack.

-> #1362
#1360

I think this is a good starting point but +1 to the TODO for adding the additional info like log

I agree, just wanted something quick and dirty for now! Can fix forward later!


⚠ Tyre Pharmacy has degraded performance

In a3570d7, Tyre Pharmacy (https://tyrepharm.ru/) experienced degraded performance:

HTTP code: 200
Response time: 8836 ms

Resolved: Tyre Pharmacy performance has improved in 500ee0a after 57 minutes.


🎮 Game Request
The objective of the game "StackUp" is to stack the blocks on top of each other in such a way that you reach to the pinnacle by getting one on the top of another and secure a win.
Point down the features

The blocks will be rolling down freely in horizontal manner you have to be specific with the time and put them up on each other.
No time limit.
If not be able to put the block on the lower one then "GAME OVER".

Select program in which you are contributing
GSSoC24
Code of Conduct

[X] I follow CONTRIBUTING GUIDELINE of this project.

Hey @Github-RiyaGupta !
Thank you for raising an issue 💗
You can self assign the issue by commenting /assign in comment 😀
Make sure you follow CODE OF CONDUCT and CONTRIBUTING GUIDELINES 🚀
Don’t Forget to ⭐ our GameZone🎮
Make sure you join our Discord🕹

/assign

please add GSSOC label too.

Hello @Github-RiyaGupta, Time's up!⏰
Sorry for closing your issue!
But it's more than a week since we haven't received anything from your side 😢 .
Come up with new ideas, create a new issue and make sure you finish it within a week! 🔥
All the best! 🚀
Happy Hacking! 💗

Hey @Github-RiyaGupta ! Thank you so much for your raising the issue💗
It’s all yours, you can come anytime again and make some contributions! 🚀
Alone, we can do little, but together we can do so much! 😇


Automated provisioning of Actions runners

Feature Description
Currently Actions runner tokens are retrieved from the web UI and are one-time use. This makes it difficult to provision runners in an automated fashion for CI and such.
Woodpecker and Drone accept a symmetic secret via environment variables for both the application and the runners, making automated provisioning zero-touch. Could such a flow be considered for Gitea please?
P.S. Since it's Gitea issuing and validating the tokens it seemed more appropriate to open this issue here, but I'm happy to move or open a companion issue on act_runner instead :+1:
Screenshots
No response

Isn't that already implemented by #23762?
An alternative to it would be offering an API route that would need to ensure that the caller is an admin for the respective scope.

https://github.com/go-gitea/gitea/pull/23762 goes some way to making automation easier, but would still require executing the CLI in an automated fashion, extracting the token from the output, and applying it to the runner config, repeated for each runner. This would be particularly messy in an integration test.
Since runner tokens are just fixed-length random strings, would it be acceptable to have admin-supplied tokens/scopes retrieved from the ini with the ability to be reused?

perhaps me completing this PR https://github.com/go-gitea/gitea/pull/23761 would be helpful

Duplicate of #23703

So it is, bugger it. Thanks @benyanke :blue_heart:


Locking Middleware is not working

Locking Middleware is not preventing multiple non-get requests from being processed at the same time. As I had added the middleware to group routes and when I send two post requests to two different routes in that group then these two requests are being processed simultaneously as there is no delay between their processing.

Locking is per session by default. If you need something different, then you need to implement your own middleware.


New Makerspace FAQ page - Discussion

The Asks:

The Makerspace FAQ section should be first when someone navigates to /makers/faq
Several Q&A on the base FAQ need to be omitted or not-displayed

Do we make a new dedicated maker-faq.handlebar?
Or we automate this ordering upon /makers referral which also triggers certain questions to be made invisible?

There is new/more thorough Maker FAQ that should be listed first on /makeres/faq https://docs.google.com/document/d/1b9lj2j3vqJrZgqfozbZ2reSO6xnC_a00VtKO-xxUB7g/edit#heading=h.v4fuab4vcc6s


Need to assess whether building a bvh from a rank-2 view impacts performance

c.f. https://github.com/ORNL-CEES/DataTransferKit/pull/513#discussion_r243712391

Per https://github.com/ORNL-CEES/DataTransferKit/pull/513#discussion_r243726246 and following posts on the discussion, there is a cost but it vanishes as the size of the problem grows.
#513 was merged as is with Traits::Access<Rank2View>::get(i) returning a point by value hence making a copy.  We intend to revisit this in the future.


dart2js does not accept values of type Symbol in switch and const map keys

This issue was originally filed by @mhausner

The spec has been updated to allow Symbol literals in switch statements and as key values in const maps.
See also issue #15445.
tests/language/symbol_literal_test.dart

dart2js accepts Symbols now.


Impl HttpTryFrom for Uri

This allows Parts to be used with the builder fns.

There currently is an Into implementation, so if there is it would be incorrect...
That said, it might be possible, so that should be audited. I will open an issue to check.

I created #96 to revisit the Into implementation. If there are error cases, they should be reflected in the HttpTryInto impl as part of resolving #96.


display of stanza numbers, of @met and @real

@ajaniak — could we add the basic of layout of verse texts to our display code?
For code like this
<div type="canto" n="1" met="jagaddhita" real="+++--+-+---+------+-+-+">
I expect a kind of header looking like this
Canto 1.
Jagaddhita: – – – ⏑ ⏑ – ⏑ – ⏑ ⏑ ⏑ – ⏑ ⏑ ⏑ ⏑ ⏑ ⏑ – ⏑ – ⏑ –
If you approve of this use of @real, then we need to add it to EGD. If not, we need another solution to display the metrical pattern.
Finally, every stanza should show a double number, one for the canto and one for the stanza numbe, like "1.1", etc., in the left margin.

A first display has been made

Thanks. Got it now.
I like it. But can the code display + and - as  the metrical symbols – and ⏑ and insert a space between every symbol?

This has been done.


Add pmem media class support

Add support for storing HSE's storage resident structures on a pmem media class configured in fsdax mode.

RFC accepted and merged.


Build from source

Add make ezs so that we can build .ez files for all deps & plugin with a single command.
The resulting files will be placed in ./tmp:
ls -l tmp/
total 992
-rw-r--r--  1 gerhard  staff   12548 20 Feb 18:02 accept-0.3.4.ez
-rw-r--r--  1 gerhard  staff  211232 20 Feb 18:02 prometheus-4.2.2.ez
-rw-r--r--  1 gerhard  staff   14053 20 Feb 18:02 prometheus_cowboy-0.1.7.ez
-rw-r--r--  1 gerhard  staff   21638 20 Feb 18:02 prometheus_httpd-2.1.10.ez
-rw-r--r--  1 gerhard  staff   10708 20 Feb 18:02 prometheus_process_collector-1.4.0.ez
-rw-r--r--  1 gerhard  staff  223943 20 Feb 18:02 prometheus_rabbitmq_exporter-3.7.20190220.ez

I've changed the version of this plugin to use the date for the patch part, feel free to change back if you disagree.
Notice that I've deleted the prometheus_rabbitmq_exporter.app.src, this is now generated by erlang.mk and dropped in ebin/prometheus_rabbitmq_exporter.app - does it look correct @deadtrickster?
Want to integrate with Docker & testing before merging.
This started as deadtrickster/prometheus_rabbitmq_exporter#63, see the discussion for more context.

cc @dcorbacho

We prefer it if rebar3 is not committed to repos.

I'm ok with the date, wondering why, though.
Does patched rebar3 archive work for you?

I'm ok with the date, wondering why, though.

The last release, prometheus_rabbitmq_exporter v<IP_ADDRESS>, indicates to me that it requires RabbitMQ v3.7.2. But does it work with RabbitMQ v3.7.12? What about RabbitMQ v3.7.1? By using a date, the indication is that it works with any RabbitMQ version published on or after this date. Is that a fair assumption to make? Can you think of a better approach?
Maybe @michaelklishin can suggest a better versioning schema. @dumbbell might also be able to help.

Does patched rebar3 archive work for you?

No, it doesn't. 72ad2f2b442c1b7c58242d0b6f23af149213e52a uses rebar3_archive_plugin v0.0.2, as captured in rebar.config.

But does it work with RabbitMQ v3.7.12? What about RabbitMQ v3.7.1?

Hmm, I explained that in the README and issues, guess not clear, but - it works up to most recent. Really simple.
I just realized you tried to use rebar to build the plugin. Never tried that yet.

@deadtrickster this is now ready for you. I've tested both deadtrickster/rabbitmq_prometheus:3.7 & deadtrickster/rabbitmq_prometheus:3.7-alpine Docker image, prometheus_rabbitmq_exporter works as expected:
gmake docker_run

curl -svL -o /dev/null localhost:15672/api/metrics
*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 15672 (#0)
> GET /api/metrics HTTP/1.1
> Host: localhost:15672
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 200 OK
< content-encoding: identity
< content-length: 144853
< content-type: text/plain; version=0.0.4
< date: Mon, 25 Feb 2019 17:05:48 GMT
< server: Cowboy
<
{ [16219 bytes data]
* Connection #0 to host localhost left intact

Thanks, that's amazing. do you think
gmake docker_run

curl -svL -o /dev/null localhost:15672/api/metrics

is a good candidate for Travis/Circle?

hmm, ln -h is bsd specific?

gmake docker_run

curl -svL -o /dev/null localhost:15672/api/metrics

is a good candidate for Travis/Circle?

I almost added a make target that runs an acceptance test on both containers, but then didn't want to extend the PR too far. Yes, adding it to your CI of choice is a great idea. I would use curl's --fail --fail-early flags.

hmm, ln -h is bsd specific?

Yes, it is. -n has the same effect & is cross-platform compatible:
LN(1)                     BSD General Commands Manual                    LN(1)

NAME
     link, ln -- make links

SYNOPSIS
     ln [-Ffhinsv] source_file [target_file]
     ln [-Ffhinsv] source_file ... target_dir

...
     -h    If the target_file or target_dir is a symbolic link, do not follow it.  This is most useful with the -f option, to replace a symlink which may point to a directory.

...

     -n    Same as -h, for compatibility with other ln implementations.

so the sequence is make ezs && make docker_build && make docker_push/run. Or I'm missing the shortcut?

Single command, multiple targets: make distclean up app ezs docker_build docker_run

Will try on the Linux workstation later on today and add a few improvements. I've only made it work. Will make it right next & eventually make it fast 😉

Picking this now on clean Linux: docker run -it ubuntu

I added make test. It work, but I'm not sure I manager test container in the best possible way. Also, despite @ it still produces some noise :-)


Changelog for v1.9.4 (#8422)

I think this is needed in master as well?

Ah yes. Thanks for "frontporting" :)

Codecov Report

Merging #8433 into master will decrease coverage by <.01%.
The diff coverage is n/a.

@@            Coverage Diff            @@
##           master   #8433      +/-   ##
=========================================
- Coverage   41.81%   41.8%   -0.01%
=========================================
  Files         497     497
  Lines       65631   65631
=========================================
- Hits        27441   27437       -4
- Misses      34674   34678       +4
  Partials     3516    3516

Impacted Files
Coverage Δ

modules/log/event.go
64.61% <0%> (-1.03%)
:arrow_down:

models/repo_list.go
73.17% <0%> (-0.98%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7408942...66a913a. Read the comment docs.


No notifications sounding through Android

The app recently stopped notifying of new messages on at least Android phones. I'm running app version 1.5 with latest android 14 updates.

It appears google has deprecated the API I was using for delivering push notifications to Android phones. This will likely take me a few days to sort out and require an updated server container. I'm sorry for the outage :(

If you get a chance, can you update the the latest container and test it out; this should now be resolved. Thank you for reporting this!

I have updated to the latest container, along with a friend who regularly chats with me via Databag. He is receiving notifications now, but I am not.
Friend's Phone: Galaxy S21 5G - SM-G991U1 Kernel Version 5.4.254-qgki-28582291-abG991U1UEUCFXGB running Android 14
My Phone: Pixel 6 with the Jun 20th 2024 Databag Update installed

Any ideas on why myself and clients of my server specifically are still not receiving notifications?

Sorry, somehow I missed your last message.
With the firebase API changes, I now need to host a "repeater" to channel the push notifications through the firebase SDK. Do you have any firewall settings that would prevent communication to: https://repeater.coredb.org/notify?
Would you be able to look at the container logs when you write a message that should produce a push notification, and see if any errors appear?
I'm sorry for the difficulties.

I'll need some guidance on how to do that. I should have on e way communication enabled out from my server to your repeater, but i'll explicitly allow it.

I am happy to help. Assuming your are using a container, the logs should be printed as the container output. If you have trouble viewing the container output, let me know what you use for hosting and I will try and figure it out.

dblog.txt
Here's the log.

I am a bit at a loss, I don't see anything wrong with your server from the logs.
Can you test with the demo server (databag.coredb.org)? Create an account and login with the mobile app; then see if you can create a push notification from another account?

I'll try that out.
Sent with Proton Mail secure email.
On Wednesday, October 9th, 2024 at 4:15 PM, Pierre Balzack @.***> wrote:

I am a bit at a loss, I don't see anything wrong with your server from the logs.
Can you test with the demo server (databag.coredb.org)? Create an account and login with the mobile app; then see if you can create a push notification from another account?
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.Message ID: @.***>

Hello friends!
I have the same problems only on a Huawei smartphone.

@VlaK0r you are able to get notifications on other phones but not the Huawei? Is it just sound that is not working?

@balzack It is notifications that do not come.

@balzack Everything works for my wife on the iPhone.

Hrm, I know this has been reported before, but I can't reproduce this on my pixel/android phone. Is there anything special about the firmware on your phone. LineageOS for example?

@balzack I installed it for the first time. "Microg" is installed on the smartphone so that there would be support for Google services, including notifications.

All, I still need to test this.
Sent with Proton Mail secure email.
On Sunday, November 10th, 2024 at 3:54 AM, VlaK0r @.***> wrote:

@.***(https://github.com/balzack) I installed it for the first time. "Microg" is installed on the smartphone so that there would be support for Google services, including notifications.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.Message ID: @.***>


Unclear how to run rules

I'm trying to run gptlint on only one rule (use-correct-english). But for the life of me, I can't get it to work.
Here's my config: gptlint.config.mjs:
export default [
  {
    "files": [
      "**/*.{py,md,ipynb}"
    ],
    rules: {
      "use-correct-english": "error"
    }
  }
]

Here's what happens when I run --print-config
(env) (base) ➜  agentops git:(gpt-lint) ✗ npx gptlint --print-config
Ignoring invalid source file: /Users/reibs/Projects/agentops-ai/agentops/examples/langchain_examples.ipynb
Ignoring source file with 2 long lines: /Users/reibs/Projects/agentops-ai/agentops/tests/openai_handlers/_test_gpt_role_function.py (a long line has at least 1024 characters, and a file can have at most 1 long line which will be truncated)

logging resolved config and then exiting because `printConfig` is enabled

config {
  "files": [
    "**/*.{py,md,ipynb}"
  ],
  "ignores": [],
  "ruleFiles": [
    ".gptlint/**/*.md",
    "!.gptlint/readme.md",
    "!.gptlint/README.md"
  ],
  "ruleDefinitions": [],
  "rules": {
    "use-correct-english": "error"
  },
  "linterOptions": {
    "noInlineConfig": false,
    "earlyExit": false,
    "concurrency": 24,
    "debug": false,
    "printConfig": true,
    "debugModel": false,
    "debugGrit": false,
    "debugStats": true,
    "disabled": false,
    "noCache": false,
    "noGrit": false,
    "dryRun": false,
    "cacheDir": "/Users/reibs/node_modules/.cache/gptlint"
  },
  "llmOptions": {
    "apiKey": "<redacted>",
    "apiBaseUrl": "https://api.openai.com/v1",
    "model": "gpt-4-turbo",
    "weakModel": "gpt-3.5-turbo",
    "temperature": 0
  },
  "overrides": []
}

warning: no rules found

input files [
  "Untitled.ipynb",
  "TOS.md",
  "tests/test_teardown.py",
  "tests/test_session.py",
  "tests/test_record_function.py",
  "tests/test_patcher.py",
  "tests/test_events.py",
  "tests/test_canary.py",
  "tests/test_atexit.py",
  "tests/openai_handlers/_test_llm_tracker_lt_1_sync.py",
  "tests/openai_handlers/_test_llm_tracker_ge_1_sync.py",
  "tests/openai_handlers/_test_llm_tracker_ge_1_async.py",
  "tests/openai_handlers/_test_handler_openai_v1.py",
  "tests/openai_handlers/_test_handler_openai_v0.py",
  "tests/openai_handlers/_test_gpt_vision.py",
  "tests/langchain_handlers/_test_langchain_handler.py",
  "README.md",
  "examples/recording-events.ipynb",
  "examples/README.md",
  "examples/openai-gpt.ipynb",
  "examples/multi_agent_example.ipynb",
  "app.py",
  "agentops/worker.py",
  "agentops/session.py",
  "agentops/meta_client.py",
  "agentops/log_config.py",
  "agentops/llm_tracker.py",
  "agentops/langchain_callback_handler.py",
  "agentops/http_client.py",
  "agentops/host_env.py",
  "agentops/helpers.py",
  "agentops/event.py",
  "agentops/enums.py",
  "agentops/decorators.py",
  "agentops/config.py",
  "agentops/client.py",
  "agentops/agent.py",
  "agentops/__init__.py"
]

I'm running this in the AgentOps GitHub to check for spelling issues.
I'd fix the docs myself, but I'm unsure how to get this to work on my own.

Hey Alex,
The config side of gptlint definitely needs a lot of work.
I've updated gptlint to include a better error message if you specify a config file (which disables the built-in rules). In your case, you're probably better off just copying the one markdown rule file you want into your repo instead of enabling the built-in rules.
To make it easy for you to test, I created a PR: https://github.com/AgentOps-AI/agentops/pull/199 The tool didn't find any spelling / logic issues, but there may be false negatives that it's missing.

Runs! I still think I'm running into false negatives, but that's probably a prompt-fu kind of thing.


Add EGXchange.org link to readme?

You don't have to merge this, of course, but I think it's a fun addition and the transaction log might help explain to people how the balance stays the same...

Thanks for EGXchange.org - terrific work.
I was literally about to add a link myself but you've beaten me to it. Thanks also for that - much appreciated. Merging now.


NUnit TestResults.xml missing?

I need to load and convert NUnit TestResults.xml to html report. But I cannot find where is the TestResults.xml located.
Here is my project.json setting:
`
"dependencies": {
"NUnit": "3.4.1",
"dotnet-test-nunit": "3.4.0-beta-1"
},
"testRunner": "nunit",
"frameworks": {
"netcoreapp1.0": {
"imports": [
"dnxcore50"
],
"dependencies": {
"TestLibrary": { "target": "project" },
"Microsoft.NETCore.App": {
"version": "1.0.0-*",
"type": "platform"
}
}
}
`

TestResults.xml is generated after you run the tests through muxer: dotnet test command. It is located in the directory where you executed the said command. For example, if your application is following structure:
My Solution Directory
      \
       Project A
      \
       Project B
      \
       Test Project

and you run dotnet test "Test Project" while being in My Solution Directory, then you will get TestResults.xml in My Solution Directory. If you are in the Test Project directory, then after running the test, you will get the XML report in the test directory and you don't need to specify directory path when running the command (just dotnet test without any further argument would do the job).

got it now. Many thanks


8.3.0 SelfObservability DashBoard  always show zero

We used skywalking-8.3.0 . Our SelfObservability always shows zero
Our applaction.yaml like this :

Our self.yaml like this :

But our SelfObservability DashBoard always show zero

We want to know whether we configured error .

Do you send any data to OAP? Such as trace, metrica, etc.

Do you send any data to OAP? Such as trace, metrica, etc.

Do you send any data to OAP? Such as trace, metrica, etc.
We send trace data to oap server by grpc

Do you send any data to OAP? Such as trace, metrica, etc.
We send trace data to oap server by grpc

Do you send any data to OAP? Such as trace, metrica, etc.

We send trace data to oap server by grpc
And when we test using " curl http://<IP_ADDRESS>:1234 " , it can also show data like this :

Do you send any data to OAP? Such as trace, metrica, etc.

We send trace data to oap server by grpc
And when we test using " curl http://<IP_ADDRESS>:1234 " , it can also show data like this :

@ZhouZiHe  self.yaml looks like <IP_ADDRESS>:1234`, please confirm

@ZhouZiHe  self.yaml looks like <IP_ADDRESS>:1234`, please confirm

Yes, doc says so. Please fix.
https://github.com/apache/skywalking/blob/master/docs/en/setup/backend/backend-telemetry.md

Yes, doc says so. Please fix.
https://github.com/apache/skywalking/blob/master/docs/en/setup/backend/backend-telemetry.md

@ZhouZiHe self.yaml looks like <IP_ADDRESS>:1234, please confirm

@ZhouZiHe self.yaml looks like <IP_ADDRESS>:1234, please confirm
there is no problem this is our develop config
when we use online config is this :

@ZhouZiHe self.yaml looks like <IP_ADDRESS>:1234, please confirm

@ZhouZiHe self.yaml looks like <IP_ADDRESS>:1234, please confirm
there is no problem this is our develop config
when we use online config is this :

version: '3.8'
services:
  oap:
    image: apache/skywalking-oap-server:8.3.0-es7
    container_name: oap
    restart: always
    ports:
      - 12800:12800
    healthcheck:
      test: ["CMD-SHELL", "/skywalking/bin/swctl ch"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 40s
    environment:
      SW_STORAGE: h2
      SW_HEALTH_CHECKER: default
      SW_TELEMETRY: prometheus
      SW_PROMETHEUS_FETCHER: default
  ui:
    image: apache/skywalking-ui:8.3.0
    container_name: ui
    depends_on:
      - oap
    links:
      - oap
    restart: always
    ports:
      - 8080:8080
    environment:
      SW_OAP_ADDRESS: oap:12800

@ZhouZiHe Did you set the`prometheus-fetcher? https://github.com/apache/skywalking/blob/master/docs/en/setup/backend/backend-telemetry.md#self-observability

version: '3.8'
services:
  oap:
    image: apache/skywalking-oap-server:8.3.0-es7
    container_name: oap
    restart: always
    ports:
      - 12800:12800
    healthcheck:
      test: ["CMD-SHELL", "/skywalking/bin/swctl ch"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 40s
    environment:
      SW_STORAGE: h2
      SW_HEALTH_CHECKER: default
      SW_TELEMETRY: prometheus
      SW_PROMETHEUS_FETCHER: default
  ui:
    image: apache/skywalking-ui:8.3.0
    container_name: ui
    depends_on:
      - oap
    links:
      - oap
    restart: always
    ports:
      - 8080:8080
    environment:
      SW_OAP_ADDRESS: oap:12800

@ZhouZiHe Did you set the`prometheus-fetcher? https://github.com/apache/skywalking/blob/master/docs/en/setup/backend/backend-telemetry.md#self-observability

version: '3.8'
services:
  oap:
    image: apache/skywalking-oap-server:8.3.0-es7
    container_name: oap
    restart: always
    ports:
      - 12800:12800
    healthcheck:
      test: ["CMD-SHELL", "/skywalking/bin/swctl ch"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 40s
    environment:
      SW_STORAGE: h2
      SW_HEALTH_CHECKER: default
      SW_TELEMETRY: prometheus
      SW_PROMETHEUS_FETCHER: default
  ui:
    image: apache/skywalking-ui:8.3.0
    container_name: ui
    depends_on:
      - oap
    links:
      - oap
    restart: always
    ports:
      - 8080:8080
    environment:
      SW_OAP_ADDRESS: oap:12800

@ZhouZiHe Did you set the`prometheus-fetcher? https://github.com/apache/skywalking/blob/master/docs/en/setup/backend/backend-telemetry.md#self-observability
We have already change the config as doc say

This is our  config below :

But it didn't seem to work

version: '3.8'
services:
  oap:
    image: apache/skywalking-oap-server:8.3.0-es7
    container_name: oap
    restart: always
    ports:
      - 12800:12800
    healthcheck:
      test: ["CMD-SHELL", "/skywalking/bin/swctl ch"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 40s
    environment:
      SW_STORAGE: h2
      SW_HEALTH_CHECKER: default
      SW_TELEMETRY: prometheus
      SW_PROMETHEUS_FETCHER: default
  ui:
    image: apache/skywalking-ui:8.3.0
    container_name: ui
    depends_on:
      - oap
    links:
      - oap
    restart: always
    ports:
      - 8080:8080
    environment:
      SW_OAP_ADDRESS: oap:12800

@ZhouZiHe Did you set the`prometheus-fetcher? https://github.com/apache/skywalking/blob/master/docs/en/setup/backend/backend-telemetry.md#self-observability
We have already change the config as doc say

This is our  config below :

But it didn't seem to work

Yes, doc says so. Please fix.
https://github.com/apache/skywalking/blob/master/docs/en/setup/backend/backend-telemetry.md
### Is there anything wrong with docs about this when configure OAP backend Self Observability :

Yes, doc says so. Please fix.
https://github.com/apache/skywalking/blob/master/docs/en/setup/backend/backend-telemetry.md
### Is there anything wrong with docs about this when configure OAP backend Self Observability :

We declared if needed. What do you mean?

We declared if needed. What do you mean?

We declared if needed. What do you mean?

This is easy to make people ambiguous . But that is not important . After we changed the config as the doc say

However it does not work ...................
We don't know what the problem is?

We declared if needed. What do you mean?

This is easy to make people ambiguous . But that is not important . After we changed the config as the doc say

However it does not work ...................
We don't know what the problem is?

You are binding <IP_ADDRESS>, but want to access through 172...*? Why do you think it works?

You are binding <IP_ADDRESS>, but want to access through 172...*? Why do you think it works?

You are binding <IP_ADDRESS>, but want to access through 172...*? Why do you think it works?

Please do not think so ,  on the production environment , we have aleardy changed the " SW_TELEMETRY_PROMETHEUS_HOST "

You are binding <IP_ADDRESS>, but want to access through 172...*? Why do you think it works?

Please do not think so ,  on the production environment , we have aleardy changed the " SW_TELEMETRY_PROMETHEUS_HOST "

I am not sure what do you want us to provide, this is a very easy fetching. You could open OAP debug to see what is going on.
Posting screenshots with untouchable environment(even we can, we shouldn't touch), that is far from community's responsibility.

I am not sure what do you want us to provide, this is a very easy fetching. You could open OAP debug to see what is going on.
Posting screenshots with untouchable environment(even we can, we shouldn't touch), that is far from community's responsibility.

I am not sure what do you want us to provide, this is a very easy fetching. You could open OAP debug to see what is going on.
Posting screenshots with untouchable environment(even we can, we shouldn't touch), that is far from community's responsibility.
First I'm very sorry for the trouble for you .
Actually  I have to say that I don't want you to provide anything for us . I just want to ask what are the possible reasons for this kind of problem. After all, I've already seen it in previous issue .  And there is no problem for us to communicate the issue like this way in the community . Since you say so, we can not get useful suggestion . We have to check again and solve this problem by ourselves .
Thanks

I am not sure what do you want us to provide, this is a very easy fetching. You could open OAP debug to see what is going on.
Posting screenshots with untouchable environment(even we can, we shouldn't touch), that is far from community's responsibility.
First I'm very sorry for the trouble for you .
Actually  I have to say that I don't want you to provide anything for us . I just want to ask what are the possible reasons for this kind of problem. After all, I've already seen it in previous issue .  And there is no problem for us to communicate the issue like this way in the community . Since you say so, we can not get useful suggestion . We have to check again and solve this problem by ourselves .
Thanks

It is really hard to locate this kind of issue with description. Because we can't see any exception or error.

It is really hard to locate this kind of issue with description. Because we can't see any exception or error.


Direct use of curried functions is considered an error

Hello,
I was using https://github.com/felixfbecker/php-language-server when I encoutered a bug that seems to come from this project.
Using the result of a curried functions displays the following errors:

[php] ';' expected.
[php] ')' expected.

Example code:
$hello = function () {
    return function () {
        return 'hello';
	};
};

$res = $hello()();

But it is valid php. (php >= 7 at least, I didn't check for earlier versions)
https://github.com/felixfbecker/php-language-server/issues/542

This looks like a duplicate of https://github.com/Microsoft/tolerant-php-parser/issues/175

It is!
Sorry I didn't find this issue before reporting.

This can be closed. The duplicate issue was fixed, and so should this. (Fixed by https://github.com/Microsoft/tolerant-php-parser/pull/223 )


Error in delayed stream : TypeError: source.on is not a function

Hello everybody,
I am trying to upload a file from a NodeJS server to an API that requires POST form-data request :http://docs.octoprint.org/en/master/api/fileops.html#upload-file
When trying to add the file to my form with form.append, I get the following error :
TypeError: source.on is not a function
at Function.DelayedStream.create (/home/vignac/Bureau/paztech2/node_modules/delayed-stream/lib/delayed_stream.js:33:10)
at FormData.CombinedStream.append (/home/vignac/Bureau/paztech2/node_modules/combined-stream/lib/combined_stream.js:43:37)
at FormData.append (/home/vignac/Bureau/paztech2/node_modules/form-data/lib/form_data.js:68:3)
This is my code :
var value = {
'path':'./test.gcode',
'readable':fs.readFileSync('./test.gcode')
// it does not change anything if either path or readable is commented
} ;
var options = {
//'contentType':'text/plain',
'filename':"test.gcode"
};
form.append("file",value,options);
Does anybody know why I get this error ?
Thanks !

@cvignac why is value a primitive object? This should work fine in you change value to
var value = fs.createReadStream(__dirname + '/test.gcode')

Let me know if this solves your issue.

@cvignac Exact same error, here's my code, how do I fix this?
formdata.append('file', {uri: file://${uri}, type: 'video/mp4', name: 'upload.mp4'})

I am not sure at it will help you, but here is what worked for me :
let options = {         method: 'POST',         url: myurl,         headers: { 'header1': myheader},         formData: {           file: {             value: fs.createReadStream(path),             options: { fileName: req.fileName}           } }};       request(options,callback)
The tricky part is that I had to put fileName as an option, It took me a long time to find it out.

Thanks, not sure if that's right for React-Native though.
Here's my (non-test) code, using RN's native FormData, where we don't use your polyfill:
let formdata = new FormData()
formdata.append('file', {uri: `file://${uri}`, type: 'video/mp4', name: 'upload.mp4'})

That's how it works in RN. So when we do our testing, I was thinking that simply assigning your polyfill to global namespace should work the same way. No?

We have a similar symptom :
https://github.com/bitinn/node-fetch/issues/192

io.on('connection', function(socket) {
ss(socket).on('create', function(stream, data) {
var req = request.post(url);
var form = req.form();
form.append('message', data.message);
// doesn't work
form.append('file', stream);

// doesn't work
form.append('file', { value: stream, options: { filename: data.filename }});

// doesn't work
var sss = ss.createStream();
form.append('file', { value: sss, options: { filename: data.filename }});
stream.pipe(sss);

});
});
Every option I try still results in the 'source.on is not a function' problem.  It DOES work if I test using a local fs.createReadStream('test.txt')

I'm experiencing this issue as well. It was working fine a few months ago, but revisiting my code, I must have pulled in a newer version of this package, and now I get this error without having changed my usage code.

@GeoffreyPlitt afaik this module has never accepted the uri option. If someone can post an isolated repro I can take a look 😄.

I'm getting the same issue, but I'm just appending a string.
let form = new FormData();

form.append('api_key', this.key); // this.key is a string

I had a similar error when I created formData as follows:
formData: {
	key: id,
	image: fs.createReadStream(path.resolve(imagePath))
}

I thought the whole thing was in the file. But the problem was that id was not a string, but an object that was the right string only when converting. Fixed just using the toString method.
formData: {
	key: id.toString(),
	image: fs.createReadStream(path.resolve(imagePath))
}

Be considerate and check the typeof the fields sent.

I was using Multer to send a file through Express. Using File directly or its stream failed, but using File's buffer worked.

whats the solution for this guys?

The following worked for me
const servicePostData = multer();

this.app // app is express.Application
  .route('/uploadFile')
  .post(servicePostData .single('file'), async (request: express.Request, response: express.Response) => {
    const form = new FormData();
    form.append('file', request.file.buffer, {
      contentType: request.file.mimetype,
      filename: request.file.originalname,
    });
    await fetch(url, {
        body: form,

I see this is old issue, but if someone finds it, my solution was to provide string data to form.append:
form.append("credentials", { "username": uname, "password": pass });
to:
form.append("credentials", \{ "username": "${uname}", "password": "${pass}" }`);`

why form data doesn't support nested objects?


Added application/vnd.api+json media type by default

Hello, am started a new project today, slim + json api, and I had a problem because slim returned empty data, and I spent many time searching a reason, and I think you need add this format by default.

Mind adding text/javascript  as well?

Also, please update ether Error and PhpError handlers.

Needs Unit tests for these new content types

@geggleto @akrabat Fix Error and PhpError, done. Add tests, done.

I think this is a wrong way to do it. There are gazillion of custom vendor mediatypes. You cannot add them all to Slim core. The application/vnd.api+json is not an general mediatype for JSON based APIs. Instead it is specific for a vendor called JSON API.
More clear example is application/vnd.github.v3+json which can is parsed as vendor prefix vnd, vendor github version 3 datatype json.
So instead of adding all different vendor prefixed mediatypes to Slim core correct way would be to just parse the vendor mediatypes into their general mediatype equivalents. For example application/vnd.api+json becomes application/json and application/vnd.foobar+xml becomes  application/xml.

Good point.

The problem is that you can't just convert the mediatype in the request in middleware as you'll need it to understand the parsed data you get back out of getParsedBody().
Do we need to update the way the media parser works so that +json and +xml are detected and handled correctly?

@akrabat Yes, I think @tuupola offered a good option, but you need time for rewrite media parser, and why would you don't merge this commit, and start rewrite for next version?

@akrabat I think you are correct. Media parser should be updated.

@Big-Shark,
Until we sort out the way we are most happy, you can register middleware:
$app->add(function ($request, $response, $next) {
    $request->registerMediaTypeParser('tapplication/vnd.api+json', function ($input) {
        return json_decode($input, true);
    });

    return $next($request, $response);
});

@akrabat hahaha, this is my code now, but I think as long as we try to make decisions, many people may face with this problem, as jsonapi quite popular format.

Closed in favour of #1781


Stop issue

Stops during learning. Do you know why?

Hi @DeepHM ,
Sorry for the late response, I was busy for some other projects. Could you provide the settings including batch size, dataset, and environments?
Cheers,
Yuyuan


Chisel Recipe Conflict

As of Teckel Version 1.12-1.338, the Preciso Blade recipe conflicts with the Iron Chisel recipe from... well, Chisel. I've got no idea if this has since been fixed (Running it on a server, haven't got time for the mod updates yet today)
In either case, probably something you should know. Apologies if this has been fixed already.

You've now made it more likely to collide with recipes from other mods. :100:
I'd suggest changing the recipe to a brass ingot and an iron ingot diagonal.


Question about background on this approach...

Thanks for making/posting this!
Is there somewhere I can read up on this approach to authenticating API's?  I saw Facebook uses something similar with their apps) but I don't feel like I quite understand it yet.  Specifically...
If I understand the server side right...
The app_key never goes across the wire, so when it gets to the server, the server recomputes the signature using it's own app_key and makes sure it matches.
Ok I get this....but

Why is the app_key used only in the GET and not in the POST then?
Can't someone who intercepts your GET send the exact same message again, impersonating you?  Granted this isn't quite as bad - they can only do over again a post you already made (replay attack).  But still, could be used to write a bunch of stuff to someones account.

Regarding #2, I'm assuming this is why Facebook passes the microtime (num milliseconds) as an additional param?  So maybe that is an additional precaution not used here.
Finally...is this a good way to authenticate BOTH a user and an app?  I guess this is what I'm really trying to do.  Haven't quite wrapped my head around it though.
Thanks again!
Brian

Hi.
Good description on how your app works on server side.  Thats what I understood from reading the documentation for the GET method.  But I did not see the secret passed in POST requests.  How does this library verify authenticity of a POST request?  Just calc MD5 sum is not enough, anyone with appid can calc MD5 sum of post data and create a signature.
Please advise.
Thanks
Matt

Hi @mpestritto,
The secret never goes on the wire, it is used as a salt before hashed. For POST and PUT methods, the secret is added at the end of the body and this whole payload is hashed which becomes the signature. (as seen in the build_api_payload method)
But you are right, the docs doesn't mention where the secret is used for POST, I'll update that.
Best Regards


Lazily load autolink images/previews

Intersection Observer API can be used for this. https://developer.mozilla.org/en-US/docs/Web/API/Intersection_Observer_API

Terrible idea, especially for mobile with unstable connection. I really hate such stuff (on twitter for example), because you can't even properly read loaded messages without connection. Also, you need to wait for each image to load, instead of loading them in background. Strong :-1: for this.

This seems to work most of the time in vue but after loading a lot of history previews seem to stop loading.


Input fails if there are reflective surfaces visible on screen

This is the case in eg. cinema_theatron.
Caused by UI getting rendered more than twice, and the render that would pass UI justPressed test uses reflected coordinates instead of real world coordinates.

Can possibly be fixed/worked around by checking if LocalPlayer():EyePos() == EyePos()

Also can be fixed with a timeout based input (allow only 1 input per UI component per delay(2-3 frames))


[Feature Request] Statistics with graphs

It would be great to have the ability to have statistics with a graph summarizing the moods!

Ok I'll have to figure that out. Thanks for using the app and helping to improve it. Will figure it out and and it to the app. Thanks for your feature request and enjoy the app ❤

Yes i have that planned for later releases on the app. I will make that and will add day compare with graph which you can add more than one day and have the graph show the details.

@Wjxfi so for the graph view for a certain range of dates, i decided to put a date range select so that user can decide how many days to select. so the short version is that the app wont have pre-defined ranges such as year and weeks and user should select them.


Update requirements-dev.txt

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 84.937%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
2172

Relevant Lines:
2460

💛  - Coveralls


Special characters at the beginning and/or end of string placed wrongly inside of a reversed swiper

Check that this is really a bug

[X] I confirm

Reproduction link
https://codesandbox.io/s/wierd-text-issue-swiper-forked-cvvvhj?file=/index.html
Bug description
As you can see in the provided codesandbox is my swiper in reverse direction mode. In this mode i'v added the simple text &Lorem ipsum dolor sit amet, consectetur adipiscing elit. Duis ultrices vitae mauris non gravida. Cras quis pharetra ipsum. Cras eget ultrices lorem. Nulla porttitor dui quis venenatis hendrerit.. In the preview you can clearly see that the & is at the end of the first line instead of at the beginning and the . is at the beginning of the last line instead of the end. This works for any special character. When the special character is added between normal text it will render normally.
Expected Behavior
The special character added on the first position of the string (in my case the &) should be at the beginning of the string.
The special character added on the last position of the string (in my case the .) should be at the end of the string.
Actual Behavior
The special character added on the first position of the string (in my case the &) is at the end of the first line of the string.
The special character added on the last position of the string (in my case the .) is at the beginning of the last line of the string.
Swiper version
9 and previous versions (our live version is 8.4.7)
Platform/Target and Browser Versions
Latest versions of Chrome, Safari and Firefox
Validations

[X] Follow our Code of Conduct
[X] Read the docs.
[X] Check that there isn't already an issue that request the same feature to avoid creating a duplicate.
[X] Make sure this is a Swiper issue and not a framework-specific issue

Would you like to open a PR for this bug?

[ ] I'm willing to open a PR

This is how it works in browser, not related to Swiper

This is quite a blunt and very disappointing statement as it is an issue that Swiper causes. Can you elaborate a bit more as to why this is how it works in a browser?


Add proxy option

The http client now allows for setting a proxy (https://youtrack.jetbrains.com/issue/IDEA-314789/IntelliJ-HTTP-Client-ignores-proxy-settings).
I updated the docker image as well, not sure why dependabot is not doing that

Done, thanks!

@madhead
Thank you for your fix...! I can now confirm that it works from main both with and without proxy 😅


Add missing Time Evolution Tutorials

This adds more tutorials on steady state and stochastic solver.

Thanks for the review! :) I updated the tutorial accordingly.
I also reduced the number of solvers used in optomechanical-steadystate to have a more reasonable runtime.

I have moved the tutorials into the time-evolution folder, which is required later on for creating the index page.I also updated the titles of the notebooks to match the titles at the tutorials website.

Thanks for the review, @hodgestar!
The notebooks 15, 16, 17 are the Stochastic Solver notebooks in #30.
I opened an issue to create a test for empty code cells in #38.


Improve commit graph page UI alignment (#23751)

Backport #23751
Fix the UI alignment by the way (adding some gt-mr-xx)
Before:

After:

ping lg-tm


ci: 🎡 Normalize CRLFs across the project

Works! Found 7 files with CRLF endings../.github/workflows/robot_ci.yml
./CANLayout.md
./README.md
./deploy_utils/Installer-script-win/opkg_requirements.txt
./deploy_utils/Installer-script-win/robotpy_installer_script.bat
./LICENSE
./robot.py
Now this becomes my problem ;)


how can i find the proxy

I have it working using docker container and AWS instances. However it is unclear how to know which proxy instance to stop when using https. I understand the proxy cannot read request headers or change response headers. OK so the docs suggest to connect to proxy over over http. How do you do that?
Tried using python/requests; then python httpclient. Also tried raw messages using ncat and the code in the documentation as below. However I just get request error from target "socket hangup".
GET /index.html
Host: localhost:8888
Location: https://www.google.com/index.html
Accept: text/html

I'm also encountering similar issues. I understand that the documentation outlines the different modes that can be used to have Scrapoxy work as an HTTPS proxy. I've been able to get it to work using the HTTP CONNECT method fine (modes A and B), but not with mode c (the "no tunnel" mode).
The issue with using HTTP CONNECT is that:

Using MITM (mode A) triggers SSL alerts. This effectively makes that mode useless for a large number of websites.
Using CONNECT without MITM (mode B) makes it impossible to override the response headers from the target server to include to the proxy name. This makes managing the proxies difficult: for example, if I get a response from the target server that leads me to believe the proxy's IP has been banned, I want to kill that particular node in the cluster; however, without MITM, it's impossible for me to know which node needs to be killed.

This means that mode A fails for a large number of websites but makes cluster management possible, and mode B works everywhere but makes cluster management impossible. It seems to me, then, that the only practical solution is to use a solution without tunnelling, but I can't seem to get this to work with python requests and there's little documentation for mode C. I'm open to rolling my own custom solution where I manage the sockets myself, but I'd rather not reinvent the wheel unless absolutely necessary.
I did do some digging into scrapy to see where that ?noconnect URL param is being processed, and it seems like it eventually feeds into twisted. Is there something necessarily asynchronous about mode C that requires something like twisted?
I'll continue digging, but ideally I'd be able to get this working out of the box with python requests.

The simplest answer is to http to the proxy manager which then handles the
https request and response.
Looks like scrapoxy is no longer maintained. I ended up writing my own
https://github.com/simonm3/mproxy.
On Sun, 26 Apr 2020 at 16:03, Josh Baiad<EMAIL_ADDRESS>wrote:

I'm also encountering similar issues. I understand that the documentation
https://scrapoxy.readthedocs.io/en/master/advanced/understand/index.html#can-scrapoxy-relay-https-requests
outlines the different modes that can be used to have Scrapoxy work as an
HTTPS proxy. I've been able to get it to work using the HTTP CONNECT method
fine (modes A and B), but not with mode c (the "no tunnel" mode).
The issue with using HTTP CONNECT is that:

Using MITM (mode A) triggers SSL alerts. This effectively makes
that mode useless for a large number of websites.
Using CONNECT without MITM (mode B) makes it impossible to override
the response headers from the target server to include to the proxy name.
This makes managing the proxies difficult: for example, if I get a response
from the target server that leads me to believe the proxy's IP has been
banned, I want to kill that particular node in the cluster; however,
without MITM, it's impossible for me to know which node needs to be killed.

This means that mode A fails for a large number of websites but makes
cluster management possible, and mode B works everywhere but makes cluster
management impossible. It seems to me, then, that the only practical
solution is to use a solution without tunnelling, but I can't seem to get
this to work with python requests and there's little documentation for
mode C. I'm open to rolling my own custom solution where I manage the
sockets myself, but I'd rather not reinvent the wheel unless absolutely
necessary.
I did do some digging into scrapy to see where that ?noconnect URL param
is being processed, and it seems like it eventually feeds into twisted.
Is there something necessarily asynchronous about mode C that requires
something like twisted?
I'll continue digging, but ideally I'd be able to get this working out of
the box with python requests.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/fabienvauchelles/scrapoxy/issues/183#issuecomment-619566274,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAJE32PZEIIZPQMXHUI7QI3ROREK7ANCNFSM4LV6DJHA
.

@jbaiad How did u get mode A to work?

all modes are supported in Scrapoxy 4.0.0:

HTTP over HTTP
HTTPS over HTTP
HTTPS over HTTP (without MITM)
HTTPS over HTTP (with MITM)

Hey there! 🌟 Exciting news! Scrapoxy 4 is ready to rock 🚀. Check it out at Scrapoxy.io (explore the "get started" guide, deployment documentation, and more 📚).
I can't wait to hear your feedback on this new version! Send me your coolest screenshots with as many proxies as possible! 📸💻
Join the Discord community if you have any questions or just want to chat. You can also open a GitHub issue for any bug or feature request 🐞✨.
See you soon!  😎
Fabien


MTLS port override doesn't appear to work for headless services

Bug Description
This was found while working on porting security/reachability_test.go to use the echotest framework.
PeerAuthn:
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: {{ .To.ServiceName }}
spec:
  selector:
    matchLabels:
      app: {{ .To.ServiceName }}
  mtls:
    mode: STRICT
  portLevelMtls:
    {{ (.To.PortForName `http`).WorkloadPort }}:
      mode: DISABLE

Where To is a target echo instance. This seems to work fine when sending to non-headless workloads.
Version
master

Additional Information
No response

@aryan16 assigning to you since I know you've been working on security tests. Feel free to re-assign as appropriate.

I didn't get the issue here, what's the exact problem with this policy ?

@aryan16 I updated the description. The client will get 502s when sending requests a headless service, but it will work fine for non-headless.

‘PeerAuth’ policies only enforce at the workload with sidecar, hence it won’t get applied to headless services.

@aryan16 headless services had sidecars ... are you thinking of "naked" services?

@nmittler ah my bad I got confused with the naked service, looking into the issue.

One possibility for this issue - https://istio.io/latest/docs/ops/common-problems/network-issues/#503-error-while-accessing-headless-services

@aryan16 I don't think that's it (but not certain). The test framework should be setting a proper Host header, which by the docs indicates it should work for headless.


How to run an example with command line arguments?

I have an example examples/read.rs and trying to run it with flamegraph:
$ cargo flamegraph --example read
    Finished release [optimized] target(s) in 0.04s

WARNING: building without debuginfo. Enable symbol information by adding the following lines to Cargo.toml:

[profile.release]
debug = true

thread 'main' panicked at 'could not spawn perf: Os { code: 2, kind: NotFound, message: "No such file or directory" }', src/libcore/result.rs:997:5

could not spawn perf: Os { code: 2, kind: NotFound

It seems like perf is not installed. What output does perf -h give?

Oh, I missed that I needed perf. Thanks.


consider not retuning all columns from metastore when listing splits

splits has 12 columns, while we filter based on most of these, we rarely need to fetch more than split_id index_uid and possibly split_metadata_json. We might reduce some postgres + metastore load by not fetching other columns, especially tags which can grow fairly large and often uses toast when it's not empty

this seems to have little (if at all) impact on actual performance. I suspect this would have a larger impact if split_metadata_json and tags where toasted, which is generally not the case unless you have a lot of tags.
It's unclear if we'd want to lower the toast limit, that would reduce the number of blocks to read when selecting, but add new reads when fetching (which is very often for split_metadata_json). Toasting tags could also have a very adverse effect on workload heavily relying on tags (for which we still do seq scans). We'd need to go through on #4658 before we can toast it.