HADOOP-19033. S3A: disable checksums when fs.s3a.checksum.validation == false

HADOOP-19033. S3A: disable checksums when fs.s3a.checksum.validation == false
Add new option fs.s3a.checksum.validation, default false, which
is used when creating s3 clients.
There's two interceptors in fs.s3a.audit.interceptors to assist
here but it turns out they aren't needed. Retained in case they are,
but we could cut them.
How was this patch tested?
Add a test in ITestS3AOpenCost to validate that disabling works.
This has to use reflection to walk down the sdk filter stream chain; when checksums are
enabled it fails because the bottom stream in the chain is one of the two checksum
validating streams.
Expecting:<software.amazon.awssdk.services.s3.internal.checksums.S3ChecksumValidatingInputStream@60179ebf>
not to be an instance of:<software.amazon.awssdk.services.s3.internal.checksums.S3ChecksumValidatingInputStream>
at org.apache.hadoop.fs.s3a.S3ATestUtils.assertStreamIsNotChecksummed(S3ATestUtils.java:1728)
at org.apache.hadoop.fs.s3a.performance.ITestS3AOpenCost.testStreamIsNotChecksummed(ITestS3AOpenCost.java:173)

For code changes:

[X] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
[x] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[ ] If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

tested s3 london, all good.
only failure is the one I see on trunk itself, testEverything(org.apache.hadoop.fs.s3a.commit.ITestS3ACommitterFactory. I should debug and fix that.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 23s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 3 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
33m 34s

trunk passed

+1 :green_heart:
compile
0m 29s

trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
compile
0m 22s

trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
checkstyle
0m 22s

trunk passed

+1 :green_heart:
mvnsite
0m 24s

trunk passed

+1 :green_heart:
javadoc
0m 16s

trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
0m 20s

trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
spotbugs
0m 45s

trunk passed

+1 :green_heart:
shadedclient
23m 29s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 17s

the patch passed

+1 :green_heart:
compile
0m 20s

the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javac
0m 20s

the patch passed

+1 :green_heart:
compile
0m 16s

the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
javac
0m 16s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
0m 13s
/results-checkstyle-hadoop-tools_hadoop-aws.txt
hadoop-tools/hadoop-aws: The patch generated 2 new + 6 unchanged - 0 fixed = 8 total (was 6)

+1 :green_heart:
mvnsite
0m 22s

the patch passed

+1 :green_heart:
javadoc
0m 10s

the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
0m 17s

the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
spotbugs
0m 40s

the patch passed

+1 :green_heart:
shadedclient
23m 38s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m  8s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
0m 23s

The patch does not generate ASF License warnings.

91m 57s

Subsystem
Report/Notes

Docker
ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6441/2/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6441

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux 1efd73443a5d 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 43b4fb067fafa830c75b2f77884e9e8a1dde4e1f

Default Java
Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6441/2/testReport/

Max. process+thread count
559 (vs. ulimit of 5500)

modules
C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6441/2/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 32s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 3 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
41m  8s

trunk passed

+1 :green_heart:
compile
0m 42s

trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
compile
0m 34s

trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
checkstyle
0m 31s

trunk passed

+1 :green_heart:
mvnsite
0m 41s

trunk passed

+1 :green_heart:
javadoc
0m 27s

trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
0m 35s

trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
spotbugs
1m  6s

trunk passed

+1 :green_heart:
shadedclient
32m 36s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 30s

the patch passed

+1 :green_heart:
compile
0m 34s

the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javac
0m 34s

the patch passed

+1 :green_heart:
compile
0m 26s

the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
javac
0m 26s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
0m 20s
/results-checkstyle-hadoop-tools_hadoop-aws.txt
hadoop-tools/hadoop-aws: The patch generated 2 new + 6 unchanged - 0 fixed = 8 total (was 6)

+1 :green_heart:
mvnsite
0m 31s

the patch passed

+1 :green_heart:
javadoc
0m 15s

the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
0m 25s

the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
spotbugs
1m  5s

the patch passed

+1 :green_heart:
shadedclient
32m 54s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 47s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
0m 35s

The patch does not generate ASF License warnings.

122m 50s

Subsystem
Report/Notes

Docker
ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6441/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6441

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux 89c95051682f 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 43b4fb067fafa830c75b2f77884e9e8a1dde4e1f

Default Java
Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6441/1/testReport/

Max. process+thread count
699 (vs. ulimit of 5500)

modules
C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6441/1/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

LGTM +1

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 23s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 3 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
35m 30s

trunk passed

+1 :green_heart:
compile
0m 26s

trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
compile
0m 20s

trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
checkstyle
0m 20s

trunk passed

+1 :green_heart:
mvnsite
0m 24s

trunk passed

+1 :green_heart:
javadoc
0m 19s

trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
0m 21s

trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
spotbugs
0m 41s

trunk passed

+1 :green_heart:
shadedclient
22m 25s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 20s

the patch passed

+1 :green_heart:
compile
0m 23s

the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javac
0m 23s

the patch passed

+1 :green_heart:
compile
0m 15s

the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
javac
0m 15s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
0m 11s

the patch passed

+1 :green_heart:
mvnsite
0m 18s

the patch passed

+1 :green_heart:
javadoc
0m  9s

the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
0m 16s

the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
spotbugs
0m 43s

the patch passed

+1 :green_heart:
shadedclient
21m  9s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 12s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
0m 24s

The patch does not generate ASF License warnings.

90m  2s

Subsystem
Report/Notes

Docker
ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6441/4/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6441

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux 4612773d0aab 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 7d57d1f8e14245f20b10fed71d85e5480486f73f

Default Java
Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6441/4/testReport/

Max. process+thread count
745 (vs. ulimit of 5500)

modules
C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6441/4/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 23s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 3 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
36m  4s

trunk passed

+1 :green_heart:
compile
0m 23s

trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
compile
0m 18s

trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
checkstyle
0m 20s

trunk passed

+1 :green_heart:
mvnsite
0m 27s

trunk passed

+1 :green_heart:
javadoc
0m 16s

trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
0m 21s

trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
spotbugs
0m 42s

trunk passed

+1 :green_heart:
shadedclient
25m 29s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 16s

the patch passed

+1 :green_heart:
compile
0m 20s

the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javac
0m 20s

the patch passed

+1 :green_heart:
compile
0m 16s

the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
javac
0m 16s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
0m 10s
/results-checkstyle-hadoop-tools_hadoop-aws.txt
hadoop-tools/hadoop-aws: The patch generated 1 new + 6 unchanged - 0 fixed = 7 total (was 6)

+1 :green_heart:
mvnsite
0m 18s

the patch passed

+1 :green_heart:
javadoc
0m  9s

the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
0m 15s

the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
spotbugs
0m 39s

the patch passed

+1 :green_heart:
shadedclient
22m  1s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m  8s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
0m 23s

The patch does not generate ASF License warnings.

93m 58s

Subsystem
Report/Notes

Docker
ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6441/5/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6441

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux 4bcb5c822a75 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / c991f651cf66f764c5d315c877e6029594f55f19

Default Java
Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6441/5/testReport/

Max. process+thread count
620 (vs. ulimit of 5500)

modules
C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6441/5/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
16m 48s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
detsecrets
0m  1s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 3 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
47m 19s

trunk passed

+1 :green_heart:
compile
0m 43s

trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
compile
0m 33s

trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
checkstyle
0m 30s

trunk passed

+1 :green_heart:
mvnsite
0m 41s

trunk passed

+1 :green_heart:
javadoc
0m 26s

trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
0m 33s

trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
spotbugs
1m  6s

trunk passed

+1 :green_heart:
shadedclient
37m 29s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

-1 :x:
mvninstall
0m 20s
/patch-mvninstall-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch failed.

-1 :x:
compile
0m 23s
/patch-compile-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04.txt
hadoop-aws in the patch failed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04.

-1 :x:
javac
0m 23s
/patch-compile-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04.txt
hadoop-aws in the patch failed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04.

-1 :x:
compile
0m 20s
/patch-compile-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_392-8u392-ga-1~20.04-b08.txt
hadoop-aws in the patch failed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08.

-1 :x:
javac
0m 20s
/patch-compile-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_392-8u392-ga-1~20.04-b08.txt
hadoop-aws in the patch failed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08.

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
0m 20s
/results-checkstyle-hadoop-tools_hadoop-aws.txt
hadoop-tools/hadoop-aws: The patch generated 1 new + 6 unchanged - 0 fixed = 7 total (was 6)

-1 :x:
mvnsite
0m 21s
/patch-mvnsite-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch failed.

+1 :green_heart:
javadoc
0m 15s

the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
0m 25s

the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

-1 :x:
spotbugs
0m 20s
/patch-spotbugs-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch failed.

+1 :green_heart:
shadedclient
40m 20s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

-1 :x:
unit
0m 24s
/patch-unit-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch failed.

+1 :green_heart:
asflicense
0m 34s

The patch does not generate ASF License warnings.

151m 50s

Subsystem
Report/Notes

Docker
ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6441/6/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6441

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux bf1f01ef8589 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / d3172fc3abcab06e738b49aac61c4f23d5e29945

Default Java
Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6441/6/testReport/

Max. process+thread count
541 (vs. ulimit of 5500)

modules
C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6441/6/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

rebase problems; will submit fix. latest test runs works except for HADOOP-19037
S3A: S3A: ITestS3AConfiguration failing with region problems .. not this code; me changing my .aws/config defaults

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 46s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 3 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
46m 59s

trunk passed

+1 :green_heart:
compile
0m 44s

trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
compile
0m 34s

trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
checkstyle
0m 32s

trunk passed

+1 :green_heart:
mvnsite
0m 40s

trunk passed

+1 :green_heart:
javadoc
0m 26s

trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
0m 33s

trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
spotbugs
1m  8s

trunk passed

+1 :green_heart:
shadedclient
37m 39s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 30s

the patch passed

+1 :green_heart:
compile
0m 37s

the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javac
0m 37s

the patch passed

+1 :green_heart:
compile
0m 26s

the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
javac
0m 26s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
0m 20s
/results-checkstyle-hadoop-tools_hadoop-aws.txt
hadoop-tools/hadoop-aws: The patch generated 1 new + 6 unchanged - 0 fixed = 7 total (was 6)

+1 :green_heart:
mvnsite
0m 31s

the patch passed

+1 :green_heart:
javadoc
0m 15s

the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
0m 24s

the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
spotbugs
1m  6s

the patch passed

+1 :green_heart:
shadedclient
37m 58s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 50s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
0m 33s

The patch does not generate ASF License warnings.

139m 12s

Subsystem
Report/Notes

Docker
ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6441/7/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6441

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux 5f4d10f608be 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 2597fad30544d898b78d9545bd1218315d892897

Default Java
Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6441/7/testReport/

Max. process+thread count
530 (vs. ulimit of 5500)

modules
C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6441/7/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
11m 32s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 3 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
43m 14s

trunk passed

+1 :green_heart:
compile
0m 41s

trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
compile
0m 34s

trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
checkstyle
0m 32s

trunk passed

+1 :green_heart:
mvnsite
0m 41s

trunk passed

+1 :green_heart:
javadoc
0m 26s

trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
0m 33s

trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
spotbugs
1m  9s

trunk passed

+1 :green_heart:
shadedclient
32m 30s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 30s

the patch passed

+1 :green_heart:
compile
0m 33s

the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javac
0m 33s

the patch passed

+1 :green_heart:
compile
0m 26s

the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
javac
0m 26s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
0m 20s

the patch passed

+1 :green_heart:
mvnsite
0m 31s

the patch passed

+1 :green_heart:
javadoc
0m 15s

the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
0m 25s

the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
spotbugs
1m  5s

the patch passed

+1 :green_heart:
shadedclient
32m 14s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 48s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
0m 35s

The patch does not generate ASF License warnings.

134m 59s

Subsystem
Report/Notes

Docker
ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6441/8/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6441

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux 12d0f51299b5 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 5faad0a285cfd503b22884696c0aaf2436b7fdab

Default Java
Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6441/8/testReport/

Max. process+thread count
731 (vs. ulimit of 5500)

modules
C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6441/8/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

build is failing from

[2024-01-15T14:27:46.923Z] Archiving artifacts

No space left on device

this is nothing to do with this patch which has got through the maven phase without any problems as well as local testing

retested, all good except when I ran the openfile cost test with -Dprefetch, which was from #6425. Filed HADOOP-19043 for that

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 53s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 3 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
47m 58s

trunk passed

+1 :green_heart:
compile
0m 44s

trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
compile
0m 33s

trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
checkstyle
0m 31s

trunk passed

+1 :green_heart:
mvnsite
0m 40s

trunk passed

+1 :green_heart:
javadoc
0m 26s

trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
0m 33s

trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
spotbugs
1m  8s

trunk passed

+1 :green_heart:
shadedclient
37m 51s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 29s

the patch passed

+1 :green_heart:
compile
0m 35s

the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javac
0m 35s

the patch passed

+1 :green_heart:
compile
0m 25s

the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
javac
0m 25s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
0m 19s

the patch passed

+1 :green_heart:
mvnsite
0m 30s

the patch passed

+1 :green_heart:
javadoc
0m 15s

the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
0m 24s

the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
spotbugs
1m  8s

the patch passed

+1 :green_heart:
shadedclient
38m 13s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
3m  4s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
0m 35s

The patch does not generate ASF License warnings.

141m  6s

Subsystem
Report/Notes

Docker
ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6441/9/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6441

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux 0c0f5c9a8cb6 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 45adbb7fba606f60ef1adfb74af736547c22d998

Default Java
Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6441/9/testReport/

Max. process+thread count
567 (vs. ulimit of 5500)

modules
C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6441/9/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.


Take the subsystems.xml and template.xml unchanged or transformed from the depen

@objectiser this PR shows the idea of extending the subsystems.xml and template.xml from the dependency feature packs.
Note that the present PR depends on https://github.com/hawkular/hawkular-accounts/pull/137 and will fail untill https://github.com/hawkular/hawkular-accounts/pull/137 is merged or you can try it out locally with -Dsrcdeps.url.1.accounts=scm:git:https://github.com/ppalaga/hawkular-accounts.git
I did the changes only for your standalone feature pack. The integrated FP is TBD.

@ppalaga  Ok thanks Peter - when is the accounts change going to be merged?

Tested locally and it worked fine. Restarted travis build.

The Accounts change is merged. A new release will follow probably today.

Seems to be a problem running the srcdeps in travis:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-ear-plugin:2.10.1:ear (default-ear) on project hawkular-commons-embedded-cassandra-ear: Error assembling EAR: Problem creating jar: Execution exception: Java heap space -> [Help 1]
I'll merge as worked locally.


Parsing error : For Spark Log

Hi I am trying to use goaccess for first time. While trying to parse my log file getting below error.
Parsed 1 linesproducing the following errors:
Token '' doesn't match specifier '%m'
command : goaccess -f /home/demoUser/source-log.log  -o spark-log-report.html --real-time-html --ws-url=host -p config/goaccess.conf
output :
/home/demoUser/spark-log.log
Parsed 1 linesproducing the following errors:
Token '' doesn't match specifier '%m'
Format Errors - Verify your log/date/time format
Config:
time-format %H:%M:%S
date-format %Y-%m-%d
log-format W3C
Few lines of logs:
2019-02-22 08:47:59 WARN  NativeCodeLoader:62 - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
log4j2ConfigFile: bigdata-config/bigdata-app-config/log4j2.xml
Configuring the Log4j2 for the path:: bigdata-config/bigdata-app-config/log4j2.xml
08:48:01.243 [main] ERROR  - Error Message!
08:48:01.244 [main] FATAL  - Fatal Message!
[INFO ]  2019-02-22 08:48:01.244 [main] - [TDACommon]- Inside to load loadRawRecordsDefaultValues Method.
[INFO ]  2019-02-22 08:48:01.359 [main] - [TDACommon]- Exit Raw_Records_Default_Values is loaded successfully
[INFO ]  2019-02-22 08:48:01.363 [main] - [DBUtility]- update method   192.168.xx.xx  --analytics
2019-02-22 08:48:01 INFO  cluster:71 - Cluster created with settings {hosts=[192.168.xx.xx:27017], mode=SINGLE, requiredClusterType=UNKNOWN, serverSelectionTimeout='30000 ms', maxWaitQueueSize=500}
[INFO ]  2019-02-22 08:48:01.597 [main] - [DBUtility]- @@@@       date from update method         @@@@Tue Feb 22 08:48:01 EST 2039
2019-02-22 08:48:01 INFO  cluster:71 - No server chosen by WritableServerSelector from cluster description ClusterDescription{type=UNKNOWN, connectionMode=SINGLE, serverDescriptions=[ServerDescription{address=192.168.xx.xx:27017, type=UNKNOWN, state=CONNECTING}]}. Waiting for 30000 ms before timing out

I suspect somthing wrong configured.
Please provide some solution.

Seems like you are trying to parse a spark log. Please note that goaccess is a web/access log analyzer, so you won't be able to use for a spark log.


Null Reference Error When Using RawEvent in Hangfire-Scheduled _analytics.Track Call

Describe the bug
When scheduling a Hangfire job to call _analytics.Track with a RawEvent delegate, RawEvent is found to be null, resulting in a NullReferenceException. This issue does not occur when _analytics.Track is called directly, without scheduling through Hangfire, suggesting a discrepancy in how RawEvent is instantiated or accessed within Hangfire's execution context.
To Reproduce
Steps to reproduce the behavior:

Schedule ScheduleIntegrationTrialExpiringTomorrow as a background job using Hangfire.
Within ScheduleIntegrationTrialExpiringTomorrow, attempt to use _analytics.Track with a delegate that modifies RawEvent.

public class HangfireService : IHangfireService
{
    private readonly Analytics _analytics;

    // Constructor that injects an Analytics instance
    public HangfireService(Analytics analytics)
    {
        _analytics = analytics;
    }

    // Method to schedule a Hangfire job that will run after a delay
    public void ScheduleIntegrationTrialJob()
    {
        // Creating a DTO with integration details to pass to the scheduled job
        var integrationDetail = new SegmentIntegrationDetailDto
        {
            CompanyIdentifier = "Company123",
            IntegrationCustomName = "Marketing Integration",
            IntegrationGuideUrl = "https://company.com/guide",
            IntegrationMessage = "Welcome to the integration",
            IntegrationName = "HubSpot Integration",
            IntegrationType = "CRM"
        };

        // Scheduling the job to run after 60 seconds
        BackgroundJob.Schedule<HangfireService>(
            x => this.ScheduleIntegrationTrialExpiringTomorrow(integrationDetail),
            TimeSpan.FromSeconds(60)
        );
    }

    // Method called by Hangfire to track an integration trial expiring event
    public void ScheduleIntegrationTrialExpiringTomorrow(SegmentIntegrationDetailDto integrationDetail)
    {
        // First call to _analytics.Track without a RawEvent delegate - this works as expected
        _analytics.Track("integration_trial_expiring_tomorrow", integrationDetail);

        // Second call to _analytics.Track with a RawEvent delegate - this causes a NullReferenceException
        // when scheduled through Hangfire, as RawEvent is null. However, this same call works
        // without issues when executed directly (not through Hangfire).
        _analytics.Track("integration_trial_expiring_tomorrow", integrationDetail, RawEvent =>
        {
            RawEvent.UserId = "123";  // Attempting to set the UserId, but RawEvent is null in this context
            return RawEvent;
        });
    }
}

Expected behavior
When scheduled as a background job, the RawEvent parameter is null, causing a NullReferenceException. Calling _analytics.Track directly without scheduling works as expected, even with the RawEvent delegate.

hey @RaviAajugiya, the first call works as expected because the enrichment closure/delegate is null (see this line). the second call passes a closure/delegate and that closure/delegate is invoked with the enrichment result from previous plugins (see this line). I'd suggest to check if you have any plugins that returns null after processing.
if you don't have any custom plugins, there are 2 cases you might receive null values in the closure/delegate:

you have a destination plugin but it's not enabled (see this line)
analytics is not ready (see this line). this happens when analytics failed to fetch settings from network, or there are events being tracked before analytics got settings

in either case, you should do null checks yourself to prevent NullReferenceException


Refactor/convenience frames

Proposed changes

The approach frame is now used for the kicking and the ball_kick_area decision. After the changes for usage of the localization in the behavior are done, we can evaluate if we want to use other convenience frame as well or the approach frame for other actions or decisions. To not interfere with the other decisions and actions a new method in the world model capsule returning the ball position in the approach frame has been created.
Related issues

#68
Necessary checks

[x] Update package version
[x] Run catkin build
[ ] Write documentation
[ ] Create issues for future work
[x] Test on your machine
[ ] Test on the robot
[ ] Put the PR on our Project board

I wonder whether it would be better to use the approach frame generally for all move base commands. the pose in the approach frame might be more accurate.

I tried to test these changes but the branch is quite a bit behind the master now.


[Feature Request] Support for Nested APIBlueprint

Hey flask-openapi3 project :)
Are there any plans for adding support for nested APIBlueprint, similar to the Blueprint from Flask?
Link to docs: https://flask.palletsprojects.com/en/2.1.x/api/?highlight=register_blueprint#flask.Blueprint.register_blueprint
I have been trying to play around with an implementation for this. Currently, it looks like this.
# ...

from urllib.parse import urljoin

# ...

class APIBlueprint

# ...

    def register_api(self: APIBlueprint, api: APIBlueprint) -> None:
        # Don't know if this will work, have not really used tags 😅
        for tag in api.tags:
            if tag.name not in self.tag_names:
                self.tags.append(tag)

        # This seems to work, pretty well
        for path, path_item in api.paths.items():
            path: str
            path_item: PathItem
            joined_path = urljoin(self.url_prefix, path.lstrip("/"))
            self.paths[joined_path] = path_item

        # I could imagion that this could give some problems with classes name overlapping, but it is early testing
        self.components_schemas.update(**api.components_schemas)

        self.register_blueprint(api)

I need to learn before I do it.

Nice to hear @luolingchun, I would like to help, if nothing else I can do some testing. I am also planing to work more on it, but I have a 3 weeks holidays so nothing is happening in this time :sweat_smile:

I have submitted a PR #18
You can do some testing and you can fix some bug if you find it.

Nice, I will differently try that when I am back at work :wink:
For now, I will close this issue because I think it makes more sense to continue in the PR https://github.com/luolingchun/flask-openapi3/pull/18


Validate form by formlyConfig without creating component

Is there any service or class that can create a form according to the configuration by adding validators and returning it so that it can be validated without creating a component on the page.

use FormlyFormBuilder


Added deployment file for push and merge PR events

Jira issue
Successful run example: https://github.com/nitinja/aem-uix-examples/actions/runs/10606608061/job/29397668970

When commit is done in extensions that need not deployed, run will still succeed with no deployment.

[X] I have signed the Adobe Open Source CLA.
[X] I have updated the documentation accordingly.
[X] I have read the CONTRIBUTING document.

@nitinja, could you show examples or explain how the deployment will work if the deployment of the extension in AppBuilder requires certain environment variables, like the Google API key?
Thank you.

@nitinja, could you show examples or explain how the deployment will work if the deployment of the extension in AppBuilder requires certain environment variables, like the Google API key? Since some extensions (at the moment) can only be deployed manually (or something like merge option during deployment for .env)
Thank you.

Right now, we are only able to auto-deploy apps that can be deployed using simply aio app deploy command. For this command to succeed, we need certain variables in .env file in project directory. Those variables are auto populated when you run aio app use project-config.json command. If the app requires additional environment variables, we can simply add those variables as environment variables in GitHub settings and build should run (although I have not tried address-autocomplete extension, since I do not have this key. If you provide this key, I can try). As of now, I have not added merge option, but it's possible to do it.

For future reference: This is a generic build system that first detects all changed directories and individually and then runs build command in each of those directories after creating .env file according to the secret stored in GitHub with the key as directory name.

cc @slitviachenko for reviewing retrospectively.


Argument 'X' has incorrect type (expected numpy.ndarray, got csr_matrix)

Hi all,
I just installed auto_ml and tried to use for the first time.. But even in Boston_dataset, I always get the following error in Jupiter notebook..
Can anyone help me with that?
Best regards,
It’s basically is like that:
Argument 'X' has incorrect type (expected numpy.ndarray, got csr_matrix)
and the details..

TypeError                                 Traceback (most recent call last)
 in ()
11 ml_predictor = Predictor(type_of_estimator='regressor', column_descriptions=column_descriptions)
12
---> 13 ml_predictor.train(df_train)
14
15 ml_predictor.score(df_test, df_test.MEDV)
/Users/vkocaman/anaconda/envs/py36/lib/python3.6/site-packages/auto_ml/predictor.py in train(self, raw_training_data, user_input_func, optimize_final_model, write_gs_param_results_to_file, perform_feature_selection, verbose, X_test, y_test, ml_for_analytics, take_log_of_y, model_names, perform_feature_scaling, calibrate_final_model, _scorer, scoring, verify_features, training_params, grid_search_params, compare_all_models, cv, feature_learning, fl_data, optimize_feature_learning, train_uncertainty_model, uncertainty_data, uncertainty_delta, uncertainty_delta_units, calibrate_uncertainty, uncertainty_calibration_settings, uncertainty_calibration_data, uncertainty_delta_direction, advanced_analytics, analytics_config, prediction_intervals, predict_intervals, ensemble_config, trained_transformation_pipeline, transformed_X, transformed_y, return_transformation_pipeline, X_test_already_transformed)
637
638         # This is our main logic for how we train the final model
--> 639         self.trained_final_model = self.train_ml_estimator(self.model_names, self._scorer, X_df, y)
640
641         if self.ensemble_config is not None and len(self.ensemble_config) > 0:
/Users/vkocaman/anaconda/envs/py36/lib/python3.6/site-packages/auto_ml/predictor.py in train_ml_estimator(self, estimator_names, scoring, X_df, y, feature_learning, prediction_interval)
1159         # Use Case 1: Super straightforward: just train a single, non-optimized model
1160         elif (feature_learning == True and self.optimize_feature_learning != True) or (len(estimator_names) == 1 and self.optimize_final_model != True):
-> 1161             trained_final_model = self.fit_single_pipeline(X_df, y, estimator_names[0], feature_learning=feature_learning, prediction_interval=False)
1162
1163         # Use Case 2: Compare a bunch of models, but don't optimize any of them
/Users/vkocaman/anaconda/envs/py36/lib/python3.6/site-packages/auto_ml/predictor.py in fit_single_pipeline(self, X_df, y, model_name, feature_learning, prediction_interval)
835             print(start_time)
836
--> 837         ppl.fit(X_df, y)
838
839         if self.verbose:
/Users/vkocaman/anaconda/envs/py36/lib/python3.6/site-packages/auto_ml/utils_model_training.py in fit(self, X, y)
223
224                     self.model.set_params(n_estimators=num_iter, warm_start=warm_start)
--> 225                     self.model.fit(X_fit, y)
226
227                     if self.training_prediction_intervals == True:
/Users/vkocaman/anaconda/envs/py36/lib/python3.6/site-packages/sklearn/ensemble/gradient_boosting.py in fit(self, X, y, sample_weight, monitor)
1005                                     self.estimators_.shape[0]))
1006             begin_at_stage = self.estimators_.shape[0]
-> 1007             y_pred = self._decision_function(X)
1008             self._resize_state()
1009
/Users/vkocaman/anaconda/envs/py36/lib/python3.6/site-packages/sklearn/ensemble/gradient_boosting.py in _decision_function(self, X)
1123         # not doing input validation.
1124         score = self.init_decision_function(X)
-> 1125         predict_stages(self.estimators, X, self.learning_rate, score)
1126         return score
1127

That's weird. The tests still pass on my machine and on Travis, and I haven't touched that part of the codebase in a bit.
Can you tell me which versions of the packages you're using (pip freeze)? My current guess is that there's some outdated package causing it. auto_ml latest is 2.7.6 as of this morning, and sklearn latest is 0.19.0. In the meantime, try using other estimators! Try pip install lightgbm, and then feeding in model.train(df_train, model_names='LGBMRegressor')

Sorry, meant to post that earlier. I just spent the morning playing around with the code to fix an unrelated issue (lightgbm had a breaking version update), and couldn't reproduce the bug you're reporting, which makes me think the version theory is probably correct here.
Another possible solution if your local environment is weird, is to start a virtualenv (super simple if you haven't done it before), or to use docker (i've already got an image built here: https://hub.docker.com/r/climbsrocks/auto_ml_tests/).

Hi preston,
The version installed are stated below.. I tried several more times but still get same errors.. I use Anaconda with Python 3.6 on MacOS..

Reply to this email directly, view it on GitHub https://github.com/ClimbsRocks/auto_ml/issues/335#issuecomment-334852143, or mute the thread https://github.com/notifications/unsubscribe-auth/AYcwwDT1xnmD_CocwQpqaqhtEkC8vedkks5spoNKgaJpZM4Pu3Xq.

heck yeah! weird that that broke between versions. i'll try to see if i can update to be compatible with both.
thanks for filing the issue! would love to know anything else you run into


Change base api localhost in client to prod base URL

Change the Base URL : localhost:5000 to production URL: https://mydiary-backend-host.herokuapp.com
You will able to find the api base url in redux -> action -> all files

is heroku still working?? i believe it is a paid service now..

Yes. Backend is not working anymore :). Need to find some different alternative


Add University of Finance and Management in Warsaw

@sergiibrusnyk
File path in your request (lib/domains/pl/waw/vizja.txt) corresponds to non-existing domain.
~$ host vizja.waw.pl
Host vizja.waw.pl not found: 3(NXDOMAIN)

Please make sure that you are familiar with the rules of this repository located at the bottom of this page, and modify your request accordingly.

Will modify the path

@sergiibrusnyk Thank you! Could you please also remove lib/domains/pl/waw/vizja.txt to avoid adding non-existing domains to the repository?

@philipto Done :)

@sergiibrusnyk Pull request merged. Thank you!


M3-2319 Update: Summary Panels in Sidebar

Summary Sidebar
This PR implements a new UI pattern in which we feature the summary panel in a new sidebar. It was a major rework in order to move the TagsPanel in there as well.
This pattern is implemented for the following entities:

linodes
nodebalancers
domains
account

Type of Change

Non breaking change ('update')

Note to Reviewers
linodes/[id]/summary
/nodebalancers/[id]/summary
domains/[id]/records
/account/billing

Non-blocking but there is a weird bug with paypal. Steps to reproduce:

Click "PayPal" radio
Press "tab" 3 times
Click enter
Observe paypal button disappear

Another non-blocking comment is that our forms are not actually forms. It's a potential accessibility issue and also it doesn't allow you to submit form with "Enter" button. I would suggest wrap them all with forms and move submit button event to form submit event

There are two h1s on the page. This is a blocking comment.

@rulikrulit fixed the headers


Need Cancel Promotion Action functionality in UI

As a marketing manager
I need the ability to cancel a Promotion in the UI
So that I do not need to use the CLI in order to cancel existing promotions
Details and assumptions

This feature should be embedded in the UI described by #41
This feature is related to the Action described in #32 (so if we decide to modify that story, we need to modify this one accordingly)

Acceptance criteria
        When I visit the "Home Page"
        And I set the "Name" to "New customer discount"
        And I select "VIP" in the "Type" dropdown
        And I set the "Discount" to "10"
        And I set the "Customer" to "34"
        And I set the "Start Date" to "08-07-2022"
        And I set the "End Date" to "08-15-2022"
        And I press the "Create" button
        Then I should see the message "Success"
        When I copy the "Id" field
        And I press the "Clear" button
        Then the "Id" field should be empty
        And the "Name" field should be empty
        And the "Type" field should be empty
        And the "Discount" field should be empty
        When I paste the "Id" field
        And I press the "Search" button
        Then I should see the message "Success"
        When I copy the "Id" field
        And I press the "Cancel" button
        Then I should see the message "Success, promotion has been cancelled!"

modify acceptance criteria

was able to reproduce passing tests on my end; no PR associated with this story but moving to done as everything appears to be working


There are numbers of "TIMED WAITING" & "BLOCKED" threads in jstack log

[environment]
1 single virtual machine, 4C/8GB, with es & skywalking server deployed.
standalone one-node elasticsearch, assigned Xmx4g for jvm.
skywalking server deployed on the same vm, assigned Xmx2g for jvm.
2 spring boot applications (deployed on another vm) connected to skywalking server.
[symptom]

After restarting es & skywalking server, skywalking server receives continued traces from application.
It lasts 15-30 minutes. after then, no traces come into skywalking server any more.

[effort taken]

I've checked elasticsearch.log. There are no obvious error messages in elasticsearch.log.
I've checked gc.log. There are no gc overhead messages in gc.log.
I've printed stacktrace using jstack, log shows as below:

"DataCarrier.IndicatorPersistentWorker.all_p50_day.Consumser.0.Thread" #20 daemon prio=5 os_prio=0 tid=0x00007f3e9507f000 nid=0x2e99 waiting on condition [0x00007f3e58acd000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at org.apache.skywalking.apm.commons.datacarrier.consumer.ConsumerThread.run(ConsumerThread.java:72)

"DataCarrier.IndicatorPersistentWorker.endpoint_avg.Consumser.0.Thread" #43 daemon prio=5 os_prio=0 tid=0x00007f3e950e2800 nid=0x2eb0 waiting for monitor entry [0x00007f3dfb2f1000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.elasticsearch.action.bulk.BulkProcessor.internalAdd(BulkProcessor.java:286)
- waiting to lock <0x00000000813c8440> (a org.elasticsearch.action.bulk.BulkProcessor)
at org.elasticsearch.action.bulk.BulkProcessor.add(BulkProcessor.java:271)
at org.elasticsearch.action.bulk.BulkProcessor.add(BulkProcessor.java:267)
at org.elasticsearch.action.bulk.BulkProcessor.add(BulkProcessor.java:253)
at org.apache.skywalking.oap.server.storage.plugin.elasticsearch.base.BatchProcessEsDAO.lambda$batchPersistence$0(BatchProcessEsDAO.java:64)
at org.apache.skywalking.oap.server.storage.plugin.elasticsearch.base.BatchProcessEsDAO$$Lambda$278/81782502.accept(Unknown Source)
at java.lang.Iterable.forEach(Iterable.java:75)
at org.apache.skywalking.oap.server.storage.plugin.elasticsearch.base.BatchProcessEsDAO.batchPersistence(BatchProcessEsDAO.java:62)
at org.apache.skywalking.oap.server.core.analysis.worker.PersistenceWorker.onWork(PersistenceWorker.java:51)
at org.apache.skywalking.oap.server.core.analysis.worker.IndicatorPersistentWorker.onWork(IndicatorPersistentWorker.java:63)
at org.apache.skywalking.oap.server.core.analysis.worker.IndicatorPersistentWorker$PersistentConsumer.consume(IndicatorPersistentWorker.java:153)
at org.apache.skywalking.apm.commons.datacarrier.consumer.ConsumerThread.consume(ConsumerThread.java:101)
at org.apache.skywalking.apm.commons.datacarrier.consumer.ConsumerThread.run(ConsumerThread.java:68)

"DataCarrier.IndicatorAggregateWorker.instance_jvm_cpu.Consumser.0.Thread" #152 daemon prio=5 os_prio=0 tid=0x00007f3e951c1000 nid=0x2f1d sleeping[0x00007f3df4584000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at org.apache.skywalking.apm.commons.datacarrier.buffer.Buffer.save(Buffer.java:64)
at org.apache.skywalking.apm.commons.datacarrier.buffer.Channels.save(Channels.java:52)
at org.apache.skywalking.apm.commons.datacarrier.DataCarrier.produce(DataCarrier.java:88)
at org.apache.skywalking.oap.server.core.analysis.worker.IndicatorPersistentWorker.in(IndicatorPersistentWorker.java:68)
at org.apache.skywalking.oap.server.core.analysis.worker.IndicatorTransWorker.in(IndicatorTransWorker.java:89)
at org.apache.skywalking.oap.server.core.analysis.worker.IndicatorTransWorker.in(IndicatorTransWorker.java:32)
at org.apache.skywalking.oap.server.core.remote.client.SelfRemoteClient.push(SelfRemoteClient.java:55)
at org.apache.skywalking.oap.server.core.remote.RemoteSenderService.send(RemoteSenderService.java:51)
at org.apache.skywalking.oap.server.core.analysis.worker.IndicatorRemoteWorker.in(IndicatorRemoteWorker.java:51)
at org.apache.skywalking.oap.server.core.analysis.worker.IndicatorRemoteWorker.in(IndicatorRemoteWorker.java:33)
at org.apache.skywalking.oap.server.core.analysis.worker.IndicatorAggregateWorker.lambda$sendToNext$0(IndicatorAggregateWorker.java:93)
at org.apache.skywalking.oap.server.core.analysis.worker.IndicatorAggregateWorker$$Lambda$203/1364775767.accept(Unknown Source)
at java.util.HashMap$Values.forEach(HashMap.java:981)
at org.apache.skywalking.oap.server.core.analysis.worker.IndicatorAggregateWorker.sendToNext(IndicatorAggregateWorker.java:88)
at org.apache.skywalking.oap.server.core.analysis.worker.IndicatorAggregateWorker.onWork(IndicatorAggregateWorker.java:73)
at org.apache.skywalking.oap.server.core.analysis.worker.IndicatorAggregateWorker.access$100(IndicatorAggregateWorker.java:38)
at org.apache.skywalking.oap.server.core.analysis.worker.IndicatorAggregateWorker$AggregatorConsumer.consume(IndicatorAggregateWorker.java:131)
at org.apache.skywalking.apm.commons.datacarrier.consumer.ConsumerThread.consume(ConsumerThread.java:101)
at org.apache.skywalking.apm.commons.datacarrier.consumer.ConsumerThread.run(ConsumerThread.java:68)

I've checked es cluster stat and node stat.
heap percent is in normal range, and cluster keeps yellow.

[target]
I wonder if you could give some hint on:

if there is recommended hardware for skywalking deployment. Like recommend 1C/2GB for single application tracing.
if there is any log or troubleshooting methods I could take for more deeply looking into the problem.

Have you checked the ElasticSearch log? Is there GC log there?
15 mins are very short, even can't be affected by TTL.
@peng-yongsheng suggestion?

2019-05-14T11:51:35.050+0800: 61631.320: Total time for which application threads were stopped: 0.0357366 seconds, Stopping threads took: 0.0001109 seconds
2019-05-14T11:51:35.056+0800: 61631.326: Total time for which application threads were stopped: 0.0005567 seconds, Stopping threads took: 0.0000694 seconds
2019-05-14T11:51:35.058+0800: 61631.328: Total time for which application threads were stopped: 0.0003282 seconds, Stopping threads took: 0.0000618 seconds
2019-05-14T11:52:13.062+0800: 61669.332: [GC (Allocation Failure) 2019-05-14T11:52:13.062+0800: 61669.332: [ParNew
Desired survivor size 17432576 bytes, new threshold 6 (max 6)
- age   1:     637648 bytes,     637648 total
- age   2:      49376 bytes,     687024 total
- age   3:        664 bytes,     687688 total
- age   4:      92232 bytes,     779920 total
- age   5:         96 bytes,     780016 total
- age   6:      39008 bytes,     819024 total
: 273784K->1174K(306688K), 0.0515317 secs] 851262K->578652K(4160256K), 0.0518978 secs] [Times: user=0.13 sys=0.00, real=0.05 secs]
2019-05-14T11:52:13.114+0800: 61669.384: Total time for which application threads were stopped: 0.0527399 seconds, Stopping threads took: 0.0000840 seconds
2019-05-14T11:52:13.115+0800: 61669.385: Total time for which application threads were stopped: 0.0004761 seconds, Stopping threads took: 0.0000548 seconds
2019-05-14T11:52:13.116+0800: 61669.386: Total time for which application threads were stopped: 0.0003557 seconds, Stopping threads took: 0.0000546 seconds
2019-05-14T11:52:13.116+0800: 61669.386: Total time for which application threads were stopped: 0.0002822 seconds, Stopping threads took: 0.0000583 seconds
2019-05-14T11:52:13.117+0800: 61669.387: Total time for which application threads were stopped: 0.0002746 seconds, Stopping threads took: 0.0000556 seconds
2019-05-14T11:52:50.811+0800: 61707.081: [GC (Allocation Failure) 2019-05-14T11:52:50.811+0800: 61707.081: [ParNew
Desired survivor size 17432576 bytes, new threshold 6 (max 6)
- age   1:     632360 bytes,     632360 total
- age   2:       2792 bytes,     635152 total
- age   3:      47128 bytes,     682280 total
- age   4:        472 bytes,     682752 total
- age   5:      92104 bytes,     774856 total
- age   6:         64 bytes,     774920 total
: 273814K->1246K(306688K), 0.0606944 secs] 851292K->578763K(4160256K), 0.0609771 secs] [Times: user=0.12 sys=0.00, real=0.06 secs]

This is GC log:
2019-05-14T11:51:35.050+0800: 61631.320: Total time for which application threads were stopped: 0.0357366 seconds, Stopping threads took: 0.0001109 seconds
2019-05-14T11:51:35.056+0800: 61631.326: Total time for which application threads were stopped: 0.0005567 seconds, Stopping threads took: 0.0000694 seconds
2019-05-14T11:51:35.058+0800: 61631.328: Total time for which application threads were stopped: 0.0003282 seconds, Stopping threads took: 0.0000618 seconds
2019-05-14T11:52:13.062+0800: 61669.332: [GC (Allocation Failure) 2019-05-14T11:52:13.062+0800: 61669.332: [ParNew
Desired survivor size 17432576 bytes, new threshold 6 (max 6)
- age   1:     637648 bytes,     637648 total
- age   2:      49376 bytes,     687024 total
- age   3:        664 bytes,     687688 total
- age   4:      92232 bytes,     779920 total
- age   5:         96 bytes,     780016 total
- age   6:      39008 bytes,     819024 total
: 273784K->1174K(306688K), 0.0515317 secs] 851262K->578652K(4160256K), 0.0518978 secs] [Times: user=0.13 sys=0.00, real=0.05 secs]
2019-05-14T11:52:13.114+0800: 61669.384: Total time for which application threads were stopped: 0.0527399 seconds, Stopping threads took: 0.0000840 seconds
2019-05-14T11:52:13.115+0800: 61669.385: Total time for which application threads were stopped: 0.0004761 seconds, Stopping threads took: 0.0000548 seconds
2019-05-14T11:52:13.116+0800: 61669.386: Total time for which application threads were stopped: 0.0003557 seconds, Stopping threads took: 0.0000546 seconds
2019-05-14T11:52:13.116+0800: 61669.386: Total time for which application threads were stopped: 0.0002822 seconds, Stopping threads took: 0.0000583 seconds
2019-05-14T11:52:13.117+0800: 61669.387: Total time for which application threads were stopped: 0.0002746 seconds, Stopping threads took: 0.0000556 seconds
2019-05-14T11:52:50.811+0800: 61707.081: [GC (Allocation Failure) 2019-05-14T11:52:50.811+0800: 61707.081: [ParNew
Desired survivor size 17432576 bytes, new threshold 6 (max 6)
- age   1:     632360 bytes,     632360 total
- age   2:       2792 bytes,     635152 total
- age   3:      47128 bytes,     682280 total
- age   4:        472 bytes,     682752 total
- age   5:      92104 bytes,     774856 total
- age   6:         64 bytes,     774920 total
: 273814K->1246K(306688K), 0.0606944 secs] 851292K->578763K(4160256K), 0.0609771 secs] [Times: user=0.12 sys=0.00, real=0.06 secs]

I used the command line "top -HP PID" to print some of the information associated with Elasticsearch:
top - 17:18:45 up 116 days,  3:10,  2 users,  load average: 1.97, 2.10, 1.88
Threads:  77 total,   0 running,  77 sleeping,   0 stopped,   0 zombie
%Cpu(s): 19.7 us,  3.0 sy,  0.0 ni, 77.3 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem :  8010576 total,   195028 free,  6484028 used,  1331520 buff/cache
KiB Swap:  8257532 total,  7053024 free,  1204508 used.  1127316 avail Mem

@chenmutime Please try the 6.3.0, which including

ElasticSearch storage implementation totally changed, much less ES cluster payload
Unexpected TTL payload issue has been fixed.
FAQ doc about how to set up several key configurations in ES cluster.


Problem with GTDB database  at add_names step

Thank you for adding support for the GTDB database via the new download command. Though I am running into an error (which is probably a simple fix), but rather than me monkeying around with the code any more than I already have. If you have time and could look at the following error and attached names.dmp file or provide any other guidance for troubleshooting that would be greatly appreciated.
names.txt
When I run the following (which worked with the nr database, wildcards are relevant place holders)
Note: same version of diamond (2.0.14) was used for all steps as far as I am aware.

database and taxonomy point to the relevant GTDB folders.

CAT_pack/CAT contigs -c {input.contigs} -p {input.faa} --force -n {threads} -d {DBDIR}/CAT_database -t {DBDIR}/CAT_taxonomy --out_prefix {wildcards.sample}/CAT/{wildcards.sample}
CAT_pack/CAT add_names -i {wildcards.sample}/CAT/{wildcards.sample}.contig2classification.txt -o {wildcards.sample}/CAT/{wildcards.sample}.contig2classification.official_names.txt -t {DBDIR}/CAT_taxonomy --only_official --force
I get the following error....
`[2022-03-31 17:52:09] Loading file ~/databases/CAT_taxonomy/names.dmp.
Traceback (most recent call last):
File "~/CAT_pack/CAT", line 88, in
main()
File "~/CAT_pack/CAT", line 74, in main
add_names.run()
File "~/CAT_pack/add_names.py", line 74, in run
taxid2name = tax.import_names(args.names_dmp, args.log_file, args.quiet)
File "~/tax.py", line 39, in import_names
if line[6] == 'scientific name':
IndexError: list index out of range`
I expected this to complete and to get the following file with names.
{wildcards.sample}/CAT/{wildcards.sample}.contig2classification.official_names.txt output.
Thanks!

I missed that diamond is finishing abnormally above the quoted error which may in fact be the root error.

Bumping this as I'm running into the same issue with the latest version (v5.3). @Kurtj-hub, were you able to figure out the problem?

Hi there!
I've just bumped into the same problem. CAT runs flawlessly with the nr data but GTDB is tripping it up. I downloaded and decompressed the pre-prepared GTDB database (20231120_CAT_gtdb.tar.gz).
Contents of std err:
Traceback (most recent call last):
  File "/mnt/nasapps/development/python/miniconda3/3.10/envs/CAT/bin/CAT", line 84, in <module>
    main()
  File "/mnt/nasapps/development/python/miniconda3/3.10/envs/CAT/bin/CAT", line 64, in main
    contigs.run()
  File "/mnt/nasapps/development/python/miniconda3/3.10/envs/CAT/share/cat-5.2.3-1/CAT_pack/contigs.py", line 310, in run
    lineage = tax.star_lineage(
              ^^^^^^^^^^^^^^^^^
  File "/mnt/nasapps/development/python/miniconda3/3.10/envs/CAT/share/cat-5.2.3-1/CAT_pack/tax.py", line 147, in star_lineage
    questionable_taxids = find_questionable_taxids(lineage,
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/nasapps/development/python/miniconda3/3.10/envs/CAT/share/cat-5.2.3-1/CAT_pack/tax.py", line 139, in find_questionable_taxids
    taxid_parent = lineage[i + 1]
                   ~~~~~~~^^^^^^^
IndexError: list index out of range
Traceback (most recent call last):
  File "/mnt/nasapps/development/python/miniconda3/3.10/envs/CAT/bin/CAT", line 84, in <module>
    main()
  File "/mnt/nasapps/development/python/miniconda3/3.10/envs/CAT/bin/CAT", line 70, in main
    add_names.run()
  File "/mnt/nasapps/development/python/miniconda3/3.10/envs/CAT/share/cat-5.2.3-1/CAT_pack/add_names.py", line 74, in run
    taxid2name = tax.import_names(args.names_dmp, args.log_file, args.quiet)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/nasapps/development/python/miniconda3/3.10/envs/CAT/share/cat-5.2.3-1/CAT_pack/tax.py", line 39, in import_names
    if line[6] == 'scientific name':
       ~~~~^^^
IndexError: list index out of range

Hi all,
I'm sorry for the late reply! This bug has now been fixed in #115. The solution involves downloading the latest code (not released as of yet) but also remaking the GTDB database files. With the release of v6.0.2 we will post those database files on tbb.bio.uu.nl/tina/CAT_pack_prepare/](https://tbb.bio.uu.nl/tina/CAT_pack_prepare/.
I hope this helps, at least in the very near future. :) I'm closing this issue for now, let us know if you walk into other problems, we're currently in active development again and ironing out bugs (@thauptfeld).
Best wishes,
Bastiaan

Thanks Bastiaan! CAT is an awesome tool - looking forward to the new release :)


Fixes for Geneve option flags/length field

This fixes two bugs in the Geneve implementation:

The decoder was using an incorrect number of bits for the option length field
The encoder was generating corrupt packets on serialisation due to uninitialised buffer use

hey mate. is this a draft or ready to be merged?

hey mate. is this a draft or ready to be merged?

G'day 😉 - ready to be merged!


use kernelspec to pass IPython kernel args

in IPython >= 3
This is not necessarily a proposal for merge, but an illustration for discussion of one of the possibilities in #631

kernelspec is always created as 'django', so multiple django apps will all write the same file. Alternately, a kernelspec could be created in a distinct location for each django app.
separates notebook and IPython args
specifies --notebook-dir explicitly, in case a user has a default notebook path in config
behavior should be unchanged for IPython < 3

Thanks ! Looks like a great basis for IPython 3 support.


Added NameTitle class, and helper functions for player name titles.

Titles added: (some of them don't fully work yet)

Legendary
Doctor
Max Level
Mod
Developer
Owner
World Access
Game
Mentor
Grow4Good
Thanksgiving
Youtube Creator
Tiktok Creator

No way you actually formatted before pushing!! So proud of you!

No way you actually formatted before pushing!! So proud of you!

Did that yesterday too nub


Feature: improved logging

I found this far more practical for debugging issues:
Celluloid.logger.formatter = proc do |severity, datetime, progname, msg|
	name = sprintf("%08x", Thread.current.object_id)
	if Thread.current.respond_to? :actor
		name = Thread.current.actor.subject.class.to_s
	end

	"#{severity[0...1]} [#{Time.now.strftime('%M:%S')}] #{name.rjust(30)}: #{msg}\n"
end

We're coming up on a year. @ioquatix are you able to polish this off sooner than later?

Bumping a version.

Debugging and tracing in Celluloid should be reworked. I really want better debugging in Celluloid, so I might be working on it.
(Currently, I have a horrible ocean of custom debugs, traces, dump() methods that I frequently ehm "rebase against" - all just to fix unit tests. This is madness).
There are plans to make set_trace_func be zero-overhead in JRuby, so hopefully there might be more complete tracing available without any performance impact for users who don't want it.
I even think that tracing should replace logging (and Probe) ... and logging should be handled by external gems. Celluloid simply needs a "strace" - and logging can be built on top.

Boop. I might take a look at this.

@ioquatix I was going to put this together as celluloid-logger since I have like 4 loggers which each do the same thing differently right now. Care to put together a gem together?

@digitalextremist I'm actually working on celluloid-dns and so I don't have much available time between that and commercial work so if you are already planning on doing that, could you do it please and thanks?


Fix paths in types for nodenext

Initial checklist

[x] I read the support docs
[x] I read the contributing guide
[x] I agree to follow the code of conduct
[x] I searched issues and couldn’t find anything (or linked relevant results below)
[x] If applicable, I’ve added docs and tests

Description of changes
In TS 4.7, the new moduleResolution: NodeNext mode would determine whether declaration files are ESM or CJS as well. If a declaration is thought to be ESM (e.g. because the package.json contains type: module), all imports must have .js extensions.
We are already doing a great job and most import paths have .js extensions. The only offending one is "..". This blocks the build without using skipLibCheck, which causes https://github.com/facebook/docusaurus/pull/7586 to fail. Docusaurus is not adding skipLibCheck for our website because we have some hand-written declaration files, and in order to ensure their correctness we have to check them when compiling. This is the limitation of TS because TS does not allow excluding some declaration files while including others.

Thanks! I was working on the docs locally, which I’ll bundle with this. So expect a release this evening or tomorrow!

Thanks! I'll hold off https://github.com/facebook/docusaurus/pull/7586 till then.

Thanks, released!


Does not work on L1 Huawei inverters

I have an Huawei inverter, SUN2000-6KTL-L1, with this version of firmware: V200R001C00SPC146
502 port is not opened in this inverter, only 6607. I already setup my Huawei integration, which works fine from march, this year.
However, I wanted to add an energy diverter, which also must connect to inverter to get its production. Because of lacking on connections on this port (only one at a time) I needed a solution, and I found a friend on internet which made the connection successfully, but with an M1 inverter, which is three phases inverter, not mono phase like mine.
I tried to setup the modbus-proxy, changing the ports from 502 to mine, 6607. However, even if it seems to starts well, I cannot make the integration to connect to my HA instance.
these are the parameters of modbus-proxy:
upstreamhost: <IP_ADDRESS>
upstreamport: 6607
listenport: 6607
connection_time: 0.1
timeout: 1
loglevel: INFO

103 is my inverter... So, now I want to change the Huawei integration to connect to my HA ip, which I think must get information from modbus proxy:
Ha integration connection:
host: <IP_ADDRESS>
port: 6607
slave ID: 0

I tried different settings, no one worked.
Please, help!!!

Any error message in the logs?
 Did you check if the port 6607 is open and accessible on your HA Server ?

No error.
I am using already the integration of Huawei inverter on port 6607, so the port is ok.

Anyone please?


perf(all): avoid creating composition layers

Short description of what this resolves:
Changes proposed in this pull request:

Ionic Version:
Fixes: #

There are reasons this CSS was added. Why are they being removed?

It was added a long time ago, maybe some of that is not longer needed. The issue is that firefox is reporting some warning about us using to much composition layers, which is not recommended. Submitted the PR to see the screenshots.
No worries, i will not merge it


Client#options are missing

Describe the bug
Hey, seems you accidently removed the Client#options value from the Client class. Is that intended?

The options is still there, but I refactored it, the latest options look like this

Ok, my bad. Thanks.


Fallback configuration activated by timer or missing commit.

Important notices
Before you add a new report, we ask you kindly to acknowledge the following:
[-] I have read the contributing guide lines at https://github.com/opnsense/core/blob/master/CONTRIBUTING.md
[-] I have searched the existing issues and I'm convinced that mine is new.
Is your feature request related to a problem? Please describe.
If changes on remote firewalls are necessary that possibly could cause a not working configuration or a lock out situation. It would be nice to be able to have a mechanism to revert to a marked configuration automatically if no commit or approve action is taken by the user.
Describe the solution you'd like
It would be nice to lock the firewall in a "major change" mode where only one session is able to do changes until the major change mode is exited. This mode should be able to define a working configuration from the backup config history or the current configuration when this specific mode is activiated. It should be possible to set a timer for change commitment. Now the administrator can for exampel make significant changes to routing or rules that possible could lock him out of the firewall. If he does not approve that his change was sucessfull and workes as intended the firewall roles back to the defined configuration. This way the administrator can log in an "try" again or rethink his change.
Describe alternatives you've considered
No other alternatives where discussed up to now.
Additional context
This feature would make administration of remote sites easier and helps to do critical changes if necessary. If the config roles back in a case of missconfiguration the firewall is available again.

Hi Dominik,
This loosely relates to #3042 and I see this one here as a more useful approach than the other, but that's only personal taste. To ensure consistency a discussion should be started over there as to how much intersection is there between what people require vs. what your solution could offer.
Would you mind to do this to gather more information?
Cheers,
Franco

I agree, i will let this one open for reference but join the discussion over there. Thank you for the hint, and sorry for the double posting.

This issue has been automatically timed-out (after 180 days of inactivity).
For more information about the policies for this repository,
please read https://github.com/opnsense/core/blob/master/CONTRIBUTING.md for further details.
If someone wants to step up and work on this issue,
just let us know, so we can reopen the issue and assign an owner to it.


element-desktop: 1.7.14 -> 1.7.15

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://repology.org/metapackage/element-desktop/versions.
meta.description for element-desktop is: "A feature-rich client for Matrix.org"
meta.homepage for element-desktop is: "https://element.io/"
meta.changelog for element-desktop is: ""
Updates performed

Version update

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact

Checks done (click to expand)

built on NixOS

The tests defined in passthru.tests, if any, passed

0 of 0 passed binary check by having a zero exit code.

0 of 0 passed binary check by having the new version present in output.

found 1.7.15 with grep in /nix/store/5ms3qri3a64lgswnm5am56ybcipwvgz0-element-desktop-1.7.15

directory tree listing: https://gist.github.com/b273cf86a7219c0efd4bdba4c9ec676c

du listing: https://gist.github.com/54e2713c82d00a9bb440479845c469f8

Rebuild report (if merged into master) (click to expand)

4 total rebuild path(s)

1 package rebuild(s)

1 x86_64-linux rebuild(s)
1 i686-linux rebuild(s)
1 x86_64-darwin rebuild(s)
1 aarch64-linux rebuild(s)

First fifty rebuilds by attrpath
element-desktop

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/5ms3qri3a64lgswnm5am56ybcipwvgz0-element-desktop-1.7.15 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A element-desktop https://github.com/r-ryantm/nixpkgs/archive/4f9e67fe452c79e056cfc2183b9a5526f6970867.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/5ms3qri3a64lgswnm5am56ybcipwvgz0-element-desktop-1.7.15
ls -la /nix/store/5ms3qri3a64lgswnm5am56ybcipwvgz0-element-desktop-1.7.15/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review 1

1 package built:

element-desktop

Maintainer pings
cc @ma27 @pacien @fadenb @mguentner @ekleog @ralith for testing.

This update is bogus since it neither updates element-web nor does it regenerate the yarn2nix dependencies. Will do the update by myself now.
cc @ryantm the bot should probably exclude this package :)

@Ma27  element-desktop is skipped https://github.com/ryantm/nixpkgs-update/commit/78236b74a53af1c38d17c22636dbf6e0bd896515


Initial drop frame support as discussed in issue #123

Hi!
Here is my initial implementation of drop frame support in from_frames, to_timecode and from_timecode functions.
I've added a decorator function to validate passed frame rates to these functions. I guess it could be useful elsewhere but will most likely need some refactoring to support all use cases. It can probably wait until the choice is made regarding int's vs float's for frame rates.
The tests that involve passing odd frame rates to the above mentioned functions have been updated and I've added a new test case for the decorator as well.
Please let me know if you disagree with any of my choices or find my tests lacking. I should probably add tests for all frame rates supported, but wanted to get some feedback before I move on.

Thanks for your constructive feedback guys! I'll work on these notes and get  back to you. Should I continue working in my dropframe branch or would that mess up things?

Keep pushing to the same branch!  Your changes will update the PR and we can follow along here.  Thanks @apetrynet!

Great!
I'll begin working on updates this evening (my time).
On Fri, May 18, 2018, 17:03 Stephan Steinbach<EMAIL_ADDRESS>wrote:

Keep pushing to the same branch! Your changes will update the PR and we
can follow along here. Thanks @apetrynet https://github.com/apetrynet!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/PixarAnimationStudios/OpenTimelineIO/pull/258#issuecomment-390235738,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABFv4SE6MXUpG_arJ0FFPkmFHdUoDqT7ks5tzuKogaJpZM4UAZaK
.

Hey!
I've done revisions on most notes above. Still need to work on updating the unit-tests as requested, but might need to do that tomorrow.

Notes adressed

Thanks @apetrynet !


Add interval for use without canvas?

Allow the user to set an interval with flo to allow animating dom elements etc without creating their own draw loop.

...and or have a callback at the end of each frame.