YARN-11736. Enhance MultiNodeLookupPolicy to allow configuration of extended comparators for better usability.

Description of PR
Please refer to JIRA: YARN-11736
How was this patch tested?
UT
For code changes:

[x] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
[ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[ ] If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m  0s

Docker mode activated.

-1 :x:
docker
1m  8s

Docker failed to build run-specific yetus/hadoop:tp-12082}.

Subsystem
Report/Notes

GITHUB PR
https://github.com/apache/hadoop/pull/7121

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7121/1/console

versions
git=2.34.1

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@yangwwei @sunilgovind @shameersss1  Could you please help to review this PR? Thanks!

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m  0s

Docker mode activated.

-1 :x:
docker
1m 18s

Docker failed to build run-specific yetus/hadoop:tp-9856}.

Subsystem
Report/Notes

GITHUB PR
https://github.com/apache/hadoop/pull/7121

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7121/2/console

versions
git=2.34.1

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

Hi, @yangjiandan  It seems that you were using and contributing for multi-nodes mechanism recently, could you please help to review this PR? Thanks.

Sure, will review this week.

Please add the modification about the configs and new class here : https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html (you can find the changes to be done for this page in the hadoop code base itself) so that the end users know about its existence.

@shameersss1 There is no introduce about multi-node policy in the document yet, therefore I am unable to update or add any further information.
I would like to add document for multi-node mechanism, but it's not suitable in this PR, we may need to create another JIRA ticket for the documentation. Does that sound good?

@TaoYang526 @shameersss1 I am eager to help, even though I’m not very familiar with this part of YARN. I will do my best. If we can confirm together that the modified code is fine, we can proceed with merging it.

@shameersss1 Thanks for the review. I have added javadoc for key fields in the last commit.
Please take another look and let me know if there’s anything else that needs attention.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
11m 56s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
detsecrets
0m  1s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 3 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
50m  3s

trunk passed

-1 :x:
compile
0m 36s
/branch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt
hadoop-yarn-server-resourcemanager in trunk failed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.

+1 :green_heart:
compile
1m 11s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
checkstyle
1m  3s

trunk passed

+1 :green_heart:
mvnsite
1m 13s

trunk passed

+1 :green_heart:
javadoc
1m 11s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
1m  3s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
spotbugs
2m 31s

trunk passed

-1 :x:
shadedclient
8m 19s

branch has errors when building and testing our client artifacts.

_ Patch Compile Tests _

-1 :x:
mvninstall
1m  2s
/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
hadoop-yarn-server-resourcemanager in the patch failed.

-1 :x:
compile
0m 10s
/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt
hadoop-yarn-server-resourcemanager in the patch failed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.

-1 :x:
javac
0m 10s
/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt
hadoop-yarn-server-resourcemanager in the patch failed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.

-1 :x:
compile
0m 17s
/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkPrivateBuild-1.8.0_422-8u422-b05-1~20.04-b05.txt
hadoop-yarn-server-resourcemanager in the patch failed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05.

-1 :x:
javac
0m 17s
/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkPrivateBuild-1.8.0_422-8u422-b05-1~20.04-b05.txt
hadoop-yarn-server-resourcemanager in the patch failed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05.

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
0m 57s
/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 36 new + 139 unchanged - 0 fixed = 175 total (was 139)

-1 :x:
mvnsite
0m  9s
/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
hadoop-yarn-server-resourcemanager in the patch failed.

-1 :x:
javadoc
0m 55s
/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt
hadoop-yarn-server-resourcemanager in the patch failed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.

-1 :x:
javadoc
0m 51s
/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkPrivateBuild-1.8.0_422-8u422-b05-1~20.04-b05.txt
hadoop-yarn-server-resourcemanager in the patch failed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05.

-1 :x:
spotbugs
2m 36s
/new-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.html
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)

-1 :x:
shadedclient
36m 41s

patch has errors when building and testing our client artifacts.

_ Other Tests _

-1 :x:
unit
0m 27s
/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
hadoop-yarn-server-resourcemanager in the patch failed.

+0 :ok:
asflicense
0m 29s

ASF License check generated no output?

122m 30s

Reason
Tests

SpotBugs
module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager

org.apache.hadoop.yarn.server.resourcemanager.scheduler.placement.policy.CompositeComparator implements Comparator but not Serializable  At MultiComparatorPolicy.java:Serializable  At MultiComparatorPolicy.java:[lines 334-359]

Subsystem
Report/Notes

Docker
ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7121/3/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/7121

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux aed8082e3e2d 5.15.0-117-generic #127-Ubuntu SMP Fri Jul 5 20:13:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 95ef881da9f990132dd30eb4396dae2ceeff11e9

Default Java
Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7121/3/testReport/

Max. process+thread count
91 (vs. ulimit of 5500)

modules
C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7121/3/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
11m 44s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 3 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
43m 21s

trunk passed

-1 :x:
compile
0m 33s
/branch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt
hadoop-yarn-server-resourcemanager in trunk failed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.

+1 :green_heart:
compile
0m 55s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
checkstyle
0m 59s

trunk passed

+1 :green_heart:
mvnsite
1m  1s

trunk passed

+1 :green_heart:
javadoc
1m  0s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
0m 50s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
spotbugs
2m  0s

trunk passed

+1 :green_heart:
shadedclient
34m 36s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 48s

the patch passed

-1 :x:
compile
0m 24s
/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt
hadoop-yarn-server-resourcemanager in the patch failed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.

-1 :x:
javac
0m 24s
/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt
hadoop-yarn-server-resourcemanager in the patch failed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.

+1 :green_heart:
compile
0m 46s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
javac
0m 46s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
0m 43s
/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 36 new + 139 unchanged - 0 fixed = 175 total (was 139)

+1 :green_heart:
mvnsite
0m 50s

the patch passed

-1 :x:
javadoc
0m 44s
/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt
hadoop-yarn-server-resourcemanager in the patch failed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.

-1 :x:
javadoc
0m 43s
/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkPrivateBuild-1.8.0_422-8u422-b05-1~20.04-b05.txt
hadoop-yarn-server-resourcemanager in the patch failed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05.

-1 :x:
spotbugs
2m  0s
/new-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.html
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)

+1 :green_heart:
shadedclient
34m 41s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
105m 44s

hadoop-yarn-server-resourcemanager in the patch passed.

+1 :green_heart:
asflicense
0m 38s

The patch does not generate ASF License warnings.

244m 25s

Reason
Tests

SpotBugs
module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager

org.apache.hadoop.yarn.server.resourcemanager.scheduler.placement.policy.CompositeComparator implements Comparator but not Serializable  At MultiComparatorPolicy.java:Serializable  At MultiComparatorPolicy.java:[lines 333-358]

Subsystem
Report/Notes

Docker
ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7121/4/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/7121

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux dd81502eb1c3 5.15.0-117-generic #127-Ubuntu SMP Fri Jul 5 20:13:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 938e0d6cca4ddb04291741595d971ee1ff3058c0

Default Java
Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7121/4/testReport/

Max. process+thread count
954 (vs. ulimit of 5500)

modules
C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7121/4/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@slfan1989 Could you please help to review this PR? Thanks.

Oh, I will we encoutered the similar problems about multi node placement about different resource spec. And I found some bugs about this features, please refer to: https://zuston.vercel.app/publish/hadoop-yarn/

@zuston Thanks for the feedback.

Oh, I will we encoutered the similar problems about multi node placement about different resource spec. And I found some bugs about this features, please refer to: https://zuston.vercel.app/publish/hadoop-yarn/
You are right, I have proposed to fix those bugs in YARN-9598, but it was in dispute after some discussions, and part of it were merged into community in YARN-11573 which you mentioned in your article.
FYI, when the scheduler found another node can place the pending request, reserved container for this request can be unreserved before assigning, you can see the details in RegularContainerAllocator#assignContainer.

BTW, I think the node sorting policy could be extended by ourself, there is no necessary to change the default policy.
This PR doesn't change the default policy, just add a new policy can be configured to use.

FYI, when the scheduler found another node can place the pending request, reserved container for this request can be unreserved before assigning

Thanks for pointing out this, let me take a deep look

@TaoYang526  - Isn't this - https://issues.apache.org/jira/browse/YARN-11728 Same problem as well ?

@TaoYang526 - Isn't this - https://issues.apache.org/jira/browse/YARN-11728 Same problem as well ?

Yes, this issue is created by me.

@zuston Thanks for the feedback.

Oh, I will we encoutered the similar problems about multi node placement about different resource spec. And I found some bugs about this features, please refer to: https://zuston.vercel.app/publish/hadoop-yarn/

You are right, I have proposed to fix those bugs in YARN-9598, but it was in dispute after some discussions, and part of it were merged into community in YARN-11573 which you mentioned in your article. FYI, when the scheduler found another node can place the pending request, reserved container for this request can be unreserved before assigning, you can see the details in RegularContainerAllocator#assignContainer.

BTW, I think the node sorting policy could be extended by ourself, there is no necessary to change the default policy.

This PR doesn't change the default policy, just add a new policy can be configured to use.

After reading the RegularContainerAllocator#assignContainer, I think some bugs still exist like described in my article or YARN-11728.
In the below refered code, it just will pickup reserved container from another node, but obviously it don't unreserve this in the commit phase, that's the root cause of this bug.
From my sight, this bug also exist in the case of multi node placement disable, which is just covered up by the normal node round-robin strategy.
https://github.com/apache/hadoop/blob/964e0897d4ad828834bbb81c9f82ba92834d994c/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/allocator/RegularContainerAllocator.java#L581-L621

@zuston This FYI is just a kind reminder because I thought you are trying to find out the logic about how to allocate on another node for reserved container, according to your article. The bug is still there, and I also metioned that YARN-9598 tried to fix it but only part of it is merged. It's great if you can help to solve that bug.
Could you please take the further discussion in YARN-11728? I'd like to keep this PR focusing on the improvement of the sorting policy. Thanks so much for your understanding!

@TaoYang526 Please take care of checkstyle and javadoc issues. Is UT failures related?

Thanks @shameersss1 for the reminder. The UT failures seems irrelevant (Fatal error compiling: error: release version 1.8 not supported). I will merge some optimized code on the performance improvement for large scale cluster and address some check-style issues later.

I have just committed optimized code to help this policy achieving high scheduling quality and meanwhile ensuring high performance with cache enabled.
There are 2 important improvements:

Introduce PreferredIterator with next-fit policy to avoid always choosing the first node when cache enabled.
Separate sorted nodes to 3 zones: preferred, normal and ignored to achieve high scheduling quality, front nodes in preferred zone will have more opportunities to be chosen then nodes in normal zone, and rear nodes in ignored zone can be ignored in the scheduling process for now to avoid make things worse.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 34s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 4 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
35m 39s

trunk passed

+1 :green_heart:
compile
1m  1s

trunk passed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04

+1 :green_heart:
compile
0m 54s

trunk passed with JDK Private Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga

+1 :green_heart:
checkstyle
0m 55s

trunk passed

+1 :green_heart:
mvnsite
0m 56s

trunk passed

+1 :green_heart:
javadoc
0m 58s

trunk passed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04

+1 :green_heart:
javadoc
0m 49s

trunk passed with JDK Private Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga

+1 :green_heart:
spotbugs
2m  2s

trunk passed

+1 :green_heart:
shadedclient
34m  7s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

-1 :x:
mvninstall
0m 22s
/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
hadoop-yarn-server-resourcemanager in the patch failed.

-1 :x:
compile
0m 22s
/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04.txt
hadoop-yarn-server-resourcemanager in the patch failed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04.

-1 :x:
javac
0m 22s
/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04.txt
hadoop-yarn-server-resourcemanager in the patch failed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04.

-1 :x:
compile
0m 22s
/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkPrivateBuild-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga.txt
hadoop-yarn-server-resourcemanager in the patch failed with JDK Private Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga.

-1 :x:
javac
0m 22s
/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkPrivateBuild-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga.txt
hadoop-yarn-server-resourcemanager in the patch failed with JDK Private Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga.

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
0m 45s
/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 41 new + 139 unchanged - 0 fixed = 180 total (was 139)

-1 :x:
mvnsite
0m 24s
/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
hadoop-yarn-server-resourcemanager in the patch failed.

-1 :x:
javadoc
0m 23s
/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04.txt
hadoop-yarn-server-resourcemanager in the patch failed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04.

-1 :x:
javadoc
0m 22s
/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkPrivateBuild-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga.txt
hadoop-yarn-server-resourcemanager in the patch failed with JDK Private Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga.

-1 :x:
spotbugs
0m 22s
/patch-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
hadoop-yarn-server-resourcemanager in the patch failed.

-1 :x:
shadedclient
11m 34s

patch has errors when building and testing our client artifacts.

_ Other Tests _

-1 :x:
unit
0m 24s
/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
hadoop-yarn-server-resourcemanager in the patch failed.

-1 :x:
asflicense
0m 35s
/results-asflicense.txt
The patch generated 1 ASF License warnings.

90m 57s

Subsystem
Report/Notes

Docker
ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7121/5/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/7121

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux 066885d9e077 5.15.0-124-generic #134-Ubuntu SMP Fri Sep 27 20:20:17 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 8fec29258d8bb1847d9306ebbf91dda0e023b32d

Default Java
Private Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7121/5/testReport/

Max. process+thread count
554 (vs. ulimit of 5500)

modules
C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7121/5/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 34s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 4 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
35m 22s

trunk passed

+1 :green_heart:
compile
0m 59s

trunk passed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04

+1 :green_heart:
compile
0m 53s

trunk passed with JDK Private Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga

+1 :green_heart:
checkstyle
0m 56s

trunk passed

+1 :green_heart:
mvnsite
0m 57s

trunk passed

+1 :green_heart:
javadoc
0m 57s

trunk passed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04

+1 :green_heart:
javadoc
0m 48s

trunk passed with JDK Private Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga

+1 :green_heart:
spotbugs
1m 57s

trunk passed

+1 :green_heart:
shadedclient
33m 44s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

-1 :x:
mvninstall
0m 41s
/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
hadoop-yarn-server-resourcemanager in the patch failed.

-1 :x:
compile
0m 50s
/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04.txt
hadoop-yarn-server-resourcemanager in the patch failed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04.

-1 :x:
javac
0m 50s
/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04.txt
hadoop-yarn-server-resourcemanager in the patch failed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04.

-1 :x:
compile
0m 42s
/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkPrivateBuild-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga.txt
hadoop-yarn-server-resourcemanager in the patch failed with JDK Private Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga.

-1 :x:
javac
0m 42s
/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkPrivateBuild-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga.txt
hadoop-yarn-server-resourcemanager in the patch failed with JDK Private Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga.

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
0m 41s

the patch passed

-1 :x:
mvnsite
0m 43s
/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
hadoop-yarn-server-resourcemanager in the patch failed.

-1 :x:
javadoc
0m 44s
/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04.txt
hadoop-yarn-server-resourcemanager in the patch failed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04.

-1 :x:
javadoc
0m 41s
/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkPrivateBuild-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga.txt
hadoop-yarn-server-resourcemanager in the patch failed with JDK Private Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga.

-1 :x:
spotbugs
0m 43s
/patch-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
hadoop-yarn-server-resourcemanager in the patch failed.

-1 :x:
shadedclient
13m 21s

patch has errors when building and testing our client artifacts.

_ Other Tests _

-1 :x:
unit
0m 46s
/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
hadoop-yarn-server-resourcemanager in the patch failed.

+1 :green_heart:
asflicense
0m 35s

The patch does not generate ASF License warnings.

93m 28s

Subsystem
Report/Notes

Docker
ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7121/6/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/7121

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux 0719a4554168 5.15.0-124-generic #134-Ubuntu SMP Fri Sep 27 20:20:17 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / d80c66d0eb22f1aa88b27e3416b934d453b0f25a

Default Java
Private Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7121/6/testReport/

Max. process+thread count
556 (vs. ulimit of 5500)

modules
C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7121/6/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 35s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
detsecrets
0m  1s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 4 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
34m 33s

trunk passed

+1 :green_heart:
compile
0m 59s

trunk passed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04

+1 :green_heart:
compile
0m 51s

trunk passed with JDK Private Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga

+1 :green_heart:
checkstyle
0m 55s

trunk passed

+1 :green_heart:
mvnsite
0m 58s

trunk passed

+1 :green_heart:
javadoc
0m 57s

trunk passed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04

+1 :green_heart:
javadoc
0m 49s

trunk passed with JDK Private Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga

+1 :green_heart:
spotbugs
1m 55s

trunk passed

+1 :green_heart:
shadedclient
33m 31s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 44s

the patch passed

+1 :green_heart:
compile
0m 50s

the patch passed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04

+1 :green_heart:
javac
0m 50s

the patch passed

+1 :green_heart:
compile
0m 45s

the patch passed with JDK Private Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga

+1 :green_heart:
javac
0m 45s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
0m 43s

the patch passed

+1 :green_heart:
mvnsite
0m 47s

the patch passed

-1 :x:
javadoc
0m 43s
/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04.txt
hadoop-yarn-server-resourcemanager in the patch failed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04.

-1 :x:
javadoc
0m 40s
/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkPrivateBuild-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga.txt
hadoop-yarn-server-resourcemanager in the patch failed with JDK Private Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga.

-1 :x:
spotbugs
1m 53s
/new-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.html
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)

+1 :green_heart:
shadedclient
33m 32s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
104m  0s

hadoop-yarn-server-resourcemanager in the patch passed.

+1 :green_heart:
asflicense
0m 37s

The patch does not generate ASF License warnings.

221m  0s

Reason
Tests

SpotBugs
module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager

org.apache.hadoop.yarn.server.resourcemanager.scheduler.placement.policy.CompositeComparator implements Comparator but not Serializable  At MultiComparatorPolicy.java:Serializable  At MultiComparatorPolicy.java:[lines 419-444]

Subsystem
Report/Notes

Docker
ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7121/7/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/7121

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux 480c8778b84a 5.15.0-124-generic #134-Ubuntu SMP Fri Sep 27 20:20:17 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 3b75308a879fc9396d2c44d35783163def366e16

Default Java
Private Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7121/7/testReport/

Max. process+thread count
964 (vs. ulimit of 5500)

modules
C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7121/7/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 33s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
detsecrets
0m  1s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 4 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
34m 12s

trunk passed

+1 :green_heart:
compile
1m  0s

trunk passed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04

+1 :green_heart:
compile
0m 52s

trunk passed with JDK Private Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga

+1 :green_heart:
checkstyle
0m 53s

trunk passed

+1 :green_heart:
mvnsite
0m 57s

trunk passed

+1 :green_heart:
javadoc
0m 57s

trunk passed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04

+1 :green_heart:
javadoc
0m 49s

trunk passed with JDK Private Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga

+1 :green_heart:
spotbugs
1m 55s

trunk passed

+1 :green_heart:
shadedclient
36m 57s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 47s

the patch passed

+1 :green_heart:
compile
0m 51s

the patch passed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04

+1 :green_heart:
javac
0m 51s

the patch passed

+1 :green_heart:
compile
0m 46s

the patch passed with JDK Private Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga

+1 :green_heart:
javac
0m 46s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
0m 45s

the patch passed

+1 :green_heart:
mvnsite
0m 46s

the patch passed

+1 :green_heart:
javadoc
0m 44s

the patch passed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04

+1 :green_heart:
javadoc
0m 39s

the patch passed with JDK Private Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga

-1 :x:
spotbugs
1m 56s
/new-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.html
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)

+1 :green_heart:
shadedclient
33m 43s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
103m 31s

hadoop-yarn-server-resourcemanager in the patch passed.

+1 :green_heart:
asflicense
0m 38s

The patch does not generate ASF License warnings.

223m 47s

Reason
Tests

SpotBugs
module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager

org.apache.hadoop.yarn.server.resourcemanager.scheduler.placement.policy.Comparator is Serializable; consider declaring a serialVersionUID  At MultiComparatorPolicy.java:a serialVersionUID  At MultiComparatorPolicy.java:[lines 336-355]

The field org.apache.hadoop.yarn.server.resourcemanager.scheduler.placement.policy.Comparator.calculator is transient but isn't set by deserialization  In MultiComparatorPolicy.java:but isn't set by deserialization  In MultiComparatorPolicy.java

Subsystem
Report/Notes

Docker
ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7121/8/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/7121

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux 41ebfc405d42 5.15.0-124-generic #134-Ubuntu SMP Fri Sep 27 20:20:17 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / f774f3eecdc799d29925a0bc99e052b8a3d66372

Default Java
Private Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7121/8/testReport/

Max. process+thread count
1017 (vs. ulimit of 5500)

modules
C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7121/8/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
11m 58s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 4 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
35m  6s

trunk passed

+1 :green_heart:
compile
0m 59s

trunk passed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04

+1 :green_heart:
compile
0m 52s

trunk passed with JDK Private Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga

+1 :green_heart:
checkstyle
0m 54s

trunk passed

+1 :green_heart:
mvnsite
0m 59s

trunk passed

+1 :green_heart:
javadoc
0m 56s

trunk passed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04

+1 :green_heart:
javadoc
0m 47s

trunk passed with JDK Private Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga

+1 :green_heart:
spotbugs
1m 56s

trunk passed

+1 :green_heart:
shadedclient
34m  5s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 53s

the patch passed

+1 :green_heart:
compile
0m 51s

the patch passed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04

+1 :green_heart:
javac
0m 51s

the patch passed

+1 :green_heart:
compile
0m 43s

the patch passed with JDK Private Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga

+1 :green_heart:
javac
0m 43s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
0m 44s

the patch passed

+1 :green_heart:
mvnsite
0m 45s

the patch passed

+1 :green_heart:
javadoc
0m 44s

the patch passed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04

+1 :green_heart:
javadoc
0m 40s

the patch passed with JDK Private Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga

+1 :green_heart:
spotbugs
1m 51s

the patch passed

+1 :green_heart:
shadedclient
33m 27s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
104m 15s

hadoop-yarn-server-resourcemanager in the patch passed.

+1 :green_heart:
asflicense
0m 35s

The patch does not generate ASF License warnings.

233m 45s

Subsystem
Report/Notes

Docker
ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7121/9/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/7121

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux e1c82a246065 5.15.0-124-generic #134-Ubuntu SMP Fri Sep 27 20:20:17 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / a7419eff1a492500a933e40a246c83a1a198d3c1

Default Java
Private Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7121/9/testReport/

Max. process+thread count
944 (vs. ulimit of 5500)

modules
C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7121/9/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.


The labels are not working

Hi,
You have a cool package but the labels are not working...

I suppose that the unitSteps are not working either: take a look at augury's config on your component: ( Not all the config are there... )

I tried your localdev setup and it works in it. So it seems that there that the issue is in the packaged version. ( If I add "ngx-duration-picker": "^3.2.0" in your repo instead of doing ng build ngx-duration-picker --watch the labels are not working)

@megametres thanks for your feedback. Have you tried packaging the latest revision and see if that works?

Yes, I just figured that out, the labels are not in the 3.2.0 version... I'll do that! thanks for the quick reply!

@megametres thanks for your feedback. Have you tried packaging the latest revision and see if that works?

@FrancescoBorzi
Could you release the latest version so it would come out of the box?

@PpesR done! 3.3.0 is out

@PpesR FYI: I published 3.3.1 to solve an issue with Ivy compatibility

Thanks, much appreciated!


A password with a trailing whitespace doesn't work in ROPC flow

Azure AD allows leading or trailing whitespaces in a password. The password works on an interactive sign in page.
However, in the ROPC flow, it results in AADSTS50126 error. I confirmed this is not a client issue.
So, it would be helpful if the limitation is stipulated on the doc.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: b3eac288-4ddd-1099-1c86-885aa42a566f
Version Independent ID: 190585ca-8337-0644-2182-75b77f53c175
Content: Sign in with resource owner password credentials grant - Microsoft identity platform
Content Source: articles/active-directory/develop/v2-oauth-ropc.md
Service: active-directory
Sub-service: develop
GitHub Login: @nickludwig
Microsoft Alias: ludwignick

@tomatsue Thank you for your feedback . We will investigate and update the thread.

@tomatsue I have tried password with trailing whitespace to get the access token using ROPC flow through postman and its returning the response successfully.  Could you please share the code snippet where you are configuring the ROPC flow and getting the error.

@shwetamathur7 Could you try the following repro steps?
Repro steps

Create a test user  with a password with a trailing whitespace.

# Define a password with a trailing whitespace
$PasswordProfile = @{
    password = 'xWwvJ6NMwbWHd ';
    forceChangePasswordNextSignIn = 'false'
}

# Create a new user with the password
New-MgUser -DisplayName 'testuser001' -PasswordProfile $PasswordProfile -AccountEnabled -MailNickName 'testuser001' -UserPrincipalName<EMAIL_ADDRESS>

Confirm the password works on normal interactive sign-in pages.
Try to get a access token in the ROPC flow.

$tenantId = ""
$body = @{ grant_type = "password"
    client_id = ""
    scope = "openid https://graph.microsoft.com/.default"
    username =<EMAIL_ADDRESS>    password = $PasswordProfile.password
}
$uri = "https://login.microsoftonline.com/$tenantId/oauth2/v2.0/token"
$contentType = 'application/x-www-form-urlencoded'
$result = Invoke-RestMethod -Method POST -Uri $uri -body $body -ContentType $contentType

The output is below.
Invoke-RestMethod : {"error":"invalid_grant","error_description":"AADSTS50126: Error validating credentials due to invalid use
rname or password.\r\nTrace ID: e9517fd5-2677-455a-a989-159577582700\r\nCorrelation ID: 01cae8ef-f0b2-48e2-baac-62a8ed249558\r
\nTimestamp: 2022-05-23 11:50:36Z","error_codes":[50126],"timestamp":"2022-05-23 11:50:36Z","trace_id":"e9517fd5-2677-455a-a98
9-159577582700","correlation_id":"01cae8ef-f0b2-48e2-baac-62a8ed249558","error_uri":"https://login.microsoftonline.com/error?c
ode=50126"}

I confirmed this can be reproduced through postman. You can use Fiddler to see the actual value in the request.

@tomatsue Thanks for the above snippet and apologies for the last answer as well. I was not able to repro the issue correctly through the postman
There is a known issue where password with leading or trailing whitespaces are not allowed and got trimmed in ROPC flow. I have updated the document. Changes will be reflected after author's approval.
Thanks again for your contribution to enhance the document.


v18.0 branch not working in Tour De Jewel project

Hi Josh,
I just compiled locally v0.18 to try it and while in my real Apache Royale project it works ok, in Tour de Jewel is not working. Maybe some update to "asconfig.json" is needed, but I'm trying to figure what could be without luck. Hope you could give me some hint to try
Thanks in advance

Please ensure that you have my latest commit from about an hour ago. It fixes a serious freeze/hang bug.
For reference, I am able to open TourDeJewel on my machine, and code intelligence is working correctly. No configuration changes required.

Hi Josh,
today I check that I had the latest code (I was up to date yesterday). I rebuild again and updated VSCode. Reinitialize and the problem persist. Can you try again TDJ with latest updates?
For now I switched back to 17.2 until we see what could be the problem.
If there's something I can do to find the problem let me know
Thanks

I tested the TourDeJewel project on both Windows and macOS with the latest versions of everything in Royale. Code intelligence is working correctly for me on both computers.
Is there any kind of error message that you can share? "Not working" is very vague and could mean anything. An error message from VSCode or any kind of compiler error/warning would probably provide some clues.

Hi Josh,
the problem is that in status bar appears a message "Iniitalizaing Actionscript & MXML..." and a moving icon of two circular arrows moving, and while in my real project this happens few seconds and then all works, in TDJ this message remains and never disappear, and extension seems blocked.
Hope this helps to reach to the problem
thanks!

If you open up the Output view and select ActionScript & MXML Language Server, do you see any output there?
If you open the Command Palette and run the Developer: Toggle Developer Tools command, do you see any output there?

HI Josh,
I think I got it: The problem is that I'm using a workspace not just TDJ alone. If I open just TDJ it works, but open a workspace with:
-Jewel
-JewelTheme
-TourDeJewel
-Core
-Basic
-BE0004_Adding_an_item_to_a_Jewel_List
Is where I have the problem. Removing this 3 solves the problem:
-Core
-Basic
-BE0004_Adding_an_item_to_a_Jewel_List

Thanks for the info! I can now reproduce this issue.

Hey Carlos, this issue should now be resolved. I've been working on fixing some multi-threading issues recently, and I ended up introducing a new one!
Sorry for the trouble!

Many thanks Josh for the quick fix! :)


minimum iOS support version

For my project
It's platform :ios, '9.0' in podfile
I want to install and enable this library on iOS 11 +, is there any way ?

@iShawnWang, LayoutInspector's minimum deployment target is iOS 11.0
To enable this library on a project with minimum deployment target iOS 9 using CocoaPods:

Fork this repo
Set s.ios.deployment_target = '9.0' in LayoutInspector.podspec
Point LayoutInspector in your Podfile to the forked branch (example)
Then, you will have to fix errors due to some APIs unavailability in iOS 9, one of them will be UIGraphicsImageRenderer which is available only since iOS 10.

Alternatively, you can manually add LayoutInspector to your project and jump straight to the step #4 described above

The deployment target is iOS 13 now, and i want to support from iOS 9  ~ iOS 13

ok, so you should try steps described above


closure compiler does not allow me to declare a variable named 'sun'.

Attempting to compile a script whose sole contents is just (minimal reproducible issue -- my real file was much longer of course)
let sun;

Results in the following error:
Number of errors: 1
JSC_REDECLARED_VARIABLE_ERROR: Illegal redeclared variable: sun at line 1 character 4
let sun;
    ^

Whereas for example
let foo;

compiles just fine.

This is because the compiler includes var sun; in the default externs to support Firefox's LiveConnect.
https://github.com/google/closure-compiler/blob/15e8c339cb9d6d88ca16181c529f809b233f077f/externs/browser/window.js#L125
I think LiveConnect is now just a plugin, not built into Firefox - anyone know if we still need externs for it?


Fix XML examples that should use actual strings, not constant references

It goes without saying that the XML files cannot use the Java constants for these strings.
The example projects were fixed in https://github.com/opencabstandard/opencab/commit/8126a2288ce602881b683f7dc02186026acfff28

I tend to think the example should now include
<action android:name="org.opencabstandard.ACTION_IDENTITY_INFORMATION_CHANGED" />

too for completeness.

cc @sebastian-valentin who I finally added to maintainers list.


Re run last test

You can do this in the test buffer by pressing g, but a command to do it would be nice.


Way to use webpack flag profile with stats-json flag

🚀 Feature request
Command (mark with an x)

- [ ] new
- [x] build
- [ ] serve
- [ ] test
- [ ] e2e
- [ ] generate
- [ ] add
- [ ] update
- [ ] lint
- [ ] xi18n
- [ ] run
- [ ] config
- [ ] help
- [ ] version
- [ ] doc

Description
Running ng build [app name] --stats-json outputs a stats.json just like webpack --json > stats.json would. The profile flag for the webpack cli adds profile section to modules of the outputted json. Can we get the same output as we would with webpack --profile --json > stats.json from ng build ...
see https://webpack.js.org/api/stats/
repost of #10028
Describe the solution you'd like
Some flag or solution to make ng build [app name] --stats-json output stats.json with profile sections like webpack --profile --json > stats.json would.
Describe alternatives you've considered
ejecting to webpack
ng v
Angular CLI: 7.3.4
Node: 10.14.1
OS: win32 x64
Angular: 7.2.7
... animations, common, compiler, compiler-cli, core, forms
... http, language-service, platform-browser
... platform-browser-dynamic, router

Package                            Version
------------------------------------------------------------
@angular-devkit/architect          0.13.3
@angular-devkit/build-angular      0.13.4
@angular-devkit/build-ng-packagr   0.13.3
@angular-devkit/build-webpack      0.13.4
@angular-devkit/core               7.3.3
@angular-devkit/schematics         7.3.4
@angular/cdk                       7.3.3
@angular/cli                       7.3.4
@angular/flex-layout               7.0.0-beta.23
@angular/material                  7.3.3
@ngtools/json-schema               1.1.0
@schematics/angular                7.3.4 (cli-only)
@schematics/update                 0.13.4
ng-packagr                         4.7.1
rxjs                               6.4.0
typescript                         3.2.4
webpack                            4.29.0

Hi @gatimus, we will be enabling this by default when using the stats-json flag


Differences in method names between huawei and google maps sdk

While converting a project from google maps to huawei maps (thanks for your excellent work btw), I noticed following differences on the MarkerOptions object:
Position instead of SetPosition
Title instead of SetTitle
Snippet instead of SetSnippet
I would prefer those functions to be replaced with properties in both sdk's, but as that will not happen it might be better to follow the xamarin google maps bindings?
Anyway, thanks for your work, this has made my job incredibly easier!

Hi, thanks for your comment.
I was actually busy working on bindings for the HMS v5 SDKs.
While I was at it, I applied your suggestions in the rc1 of those packages :-)
Please give it a try and tell me what you think about them. (Do not forget to activate nuget previews).


Train Station Disassemble() reverts train blocks to their initial states

Description
I have a train with a set of Immersive Engineering MV Accumulators, designed to carry power to a factory. It is decorated with various blocks from Framed Blocks. At the factory train station I have a CC computer running code pasted below (and also double checked with a simplified version) that will automatically disassemble any train that arrives at the station, so that the accumulators revert to blocks and connect to the power cables.
This idea works perfectly fine when disassembling the train through the station UI, but when using the computer seemingly all blocks get 'reset'. Accumulators lose their charge and all faces but the top disable their connections and the framed blocks lose their set texture. Also, despite being blocks and having a connection on the top where the cable is, no connection is made between the power line and the accumulator despite this also working when using the system manually.
To Reproduce
Steps to reproduce the behaviour:

Create a train with customisable blocks (such as IE accumulators or FramedBlocks slopes), with obvious customisations made
Assemble the train through the train station UI
Run either script below
Observe the blocks revert to their newly placed, newly crafted state

Expected behaviour
I would expect disassembly through a CC:Tweaked computer to work exactly the same as disassembly through the train station UI, putting all blocks of the contraption back into their block state without any loss.
Screenshots
Before disassembly, pre-arrival:

After Disassembly, post-arrival:

Full Script:
station = peripheral.wrap('left')
isNewTrain = true
while true do
  isTrainPresent = station.getPresentTrain()
  print(isTrainPresent)

  if not isTrainPresent and not isNewTrain then
    isNewTrain = true
  end
  if isTrainPresent and isNewTrain then
    station.disassemble()
    os.pullEvent('redstone')
    station.assemble()
  end
  sleep(10)
end

Shortened Script:
station = peripheral.wrap('left')
station.disassemble()

Software (please complete the following information):

Minecraft Version 1.18.2
Mod Version 1.3.1
ModLoader Forge

This is running on a server, as part of the FTB Plexiglass Mountain Modpack. CC:C Bridge is the only custom mod (aside from FTB Backups 2, on the server only). I thought I had this working in a single player world, but testing again this occurs there too.

Hey! Thanks for reporting this.
This sounds very odd. The Train Stations currently do for some reason much shenanigans.
We will take a look at this.

Hi, sorry for the problems. Just to make sure for my Documentation do you have this problem too when doing the same but with the station GUI ?

This idea works perfectly fine when disassembling the train through the station UI, but when using the computer seemingly all blocks get 'reset'.

Nope, manually pressing disassemble causes no problems. Everything works as expected through the station's GUI.

Hi i tried fixing this. Could you try using this version ? [https://github.com/tweaked-programs/cccbridge/files/11347687/JarFiles.zip]

Hi, has been a busy week sorry. Hopefully I'll get the chance to try it this weekend. Thanks ^-^!

And as an addition, you could tweak your code to make it listen for the "arrival" event so you can check easier when a train arrive :)

Cause you got a preview test version to validate that everything works, it's not yet documented

Mm, I misunderstood, thought you were saying it was something pre-existing I'd missed. I'll try check out the fix as soon as possible!

Finally got around to checking this out (exam season is the worst), but unless I'm being dumb I think the files you gave are for the wrong version for the pack we're playing on. The pack we're using is for 1.18.2 but the .jar file says 1.19.2.

Sorry, I've read your issue wrong.
But I may take a lot longer to make a new 1.18.2 version because it includes many 1.19.2 only stuff so

Not a problem, would have spotted it earlier if I'd managed to test it so it's no worries.

Hey i had the same issue and i tried your version @Luke1505 but it crashed my game when i launched
2023-06-10-3.log.gz

Yeah,my Version was fucked up cause it is 1.19.2 and had some bugs on Different machines but not mine I am currently working on fixing that

Okay Thanks

Hello again, because the Train assembly and disassembly was taken care of by Create Themselves all our colliding functions will be replaced by theirs but that does not mean we are not adding additional functions to it in the future.
Kind Regards Luke.

Closed (removed function)


Use count from forceUpdate to determine if components areEqual

So feel free to reject this and we can just keep things simple, it's not a large performance increase.
I do believe there is a slight performance increase. It is hard to measure because I think it's about 5%, but there is more variation in the runtime of the updates than that. I'd like to think it shouldn't be slower than comparing the node objects, but could be wrong :-X
Maybe there is a better way to measure specifically the memoization equality check?
Also think it's kind of cool to have available the # of updates a component has had, but don't actually have any plans of what we would do with that haha.

Oops. I somehow overlooked this, and now its git history is all a mess (Probably from the force push I did to remove the large assets I accidentally committed).

Oops. I somehow overlooked this, and now its git history is all a mess (Probably from the force push I did to remove the large assets I accidentally committed).

Eh, it was just a hack and we can figure out whats right when we pick this back up, no worries


Add support to SignalR for Session

What's the context here? SignalR doesn't really have a way of clearly tracking the Session since it involves multiple requests, this is why Context.Items exists, for storing per-Connection metadata.

Is Context.Items not sufficient for this? That's a dictionary that is maintained across the entire life of the connection. Or do you need direct access to the Session data?

We'd like a way to maintain state across connections.

We've had some discussions about this, but all of them require fairly significant work we don't have resources for in 2.2, so I'm putting this in 3.0 for now.


Build fails due to multiple implementations of attract

Fuse version 1.2.0 (build 13948) (current QA release)
E8001: Multiple implementations of the UX function 'attract' found in project: Fuse.Animations.Attract and Fuse.Animations.Attract.

Seems like this might be related to premiumlibs being out of sync with the QA release.


community: add AzureContentSafetyTextTool

community: add AzureContentSafetyTextTool

Description: With the ever-growing number of Azure AI services available, not all have been integrated with LangChain yet. This tool aims to expose one part of the Azure AI content safety service and allows customers to have input text get assessed against four criteria,  based on Azure's Content Safety Policy. As not all AI models may have their own inbuilt content safety feature this tool can be used in conjuction with other models as an agent, to assess and determine what to do with harmful content.

Dependencies: no new dependencies for the tool, but azure.ai.contentsafety has been added to the extended-tests dependencies file.

3 unit tests have been supplied, mocking the underlying client. One testing the tools initialisation, one testing when harmful content is passed and one testing when no harmful content is passed.
an example notebook showing its use has been created in the appropriate docs/tools folder, using headers enforced by the LangChain CI.

Lint and test: All linting passed.

@Sheepsta300 would you be able to make the changes?

Sorry for taking a while, things have been busy lately! I've made all the changes you suggested I believe. By removing the ##Chaining header from the docs this does cause a linting error in the integration docs lint as it expects this header.
Let me know if anything else needs to be changed.

@Sheepsta300 thank you for iterating on this.
We are actively moving away from adding new integrations to community in favor of separating them into integration packages. A package for Azure was recently added (langchain-azure).
Advantages of doing this include:

Dependencies can be managed instead of catching import errors and recommending pip installs.
Changes can be properly versioned.
We can run integration tests.

I think the right thing to do is to implement this in langchain-azure. If we merge it into community it is liable to be deprecated once there's equivalent functionality in langchain-azure.
I notice you've made a few contributions around Azure into LangChain. I think these efforts could do a lot to boost the functionality of the Azure integration package.
Let me know what you think!

Totally understand. I'll open a PR to the langchain-azure integration package soon, thanks for the info!


docs: add issue templates and contributing.md

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 94.222%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
212

Relevant Lines:
225

💛  - Coveralls


Revise the naming of all and full methods

Fixes #16
Rename:

all -> all_bits
is_all -> is_all_bits
full -> all_variants
is_full -> is_all_variants

Deprecates the old names, while still allowing their use for code not to break
I didn't use deprecate's since argument, because the deprecated methods live in the user crate and won't actually work as intended.

After looking at it, I think I prefer flags over variants, so all_flags instead of all_variants. What do you think?
And I would change the note in the #[deprecated] macro to Please use xxx instead., surround the method name with `, and write it like ::all_flags() for constructors and .is_all_flags() for methods.

Sorry for disappearing like that, I applied the suggestions


 I cannot find it.

See https://github.com/samber/awesome-prometheus-alerts/commit/10b810fd6e0fe92500279366dd3b6d2309be8def


CODEGEN: C++ text control using dot instead of ->

Two issues:
First issue:
C++ code generation is using dot instead of -> for wx/srchctrl
    text_ctrl_srch = new wxSearchCtrl(panel_3, wxID_ANY, wxEmptyString);
     : : :
    text_ctrl_srch.ShowCancelButton(true);

Should be:
    text_ctrl_srch = new wxSearchCtrl(panel_3, wxID_ANY, wxEmptyString);
     : : :
    text_ctrl_srch->ShowCancelButton(true);

Second issue:
srch ctl doesn't emit the needed includes.
Has to be added manually.
#include <wx/srchctrl.h>

Thanks for reporting. This should be fixed now in the repository.

Released v1.0.3


Quesiton: Pull events, no poll?

The only event triggers are based on push notifications from a webhook (i.e. github), correct?
If I have brigade behind my firewall I would need a public reverse proxy of some sort to get in.
No polling mechanisms?

The design of Brigade goes like this:

There's a controller that listens for events
There's a gateway that generates the events

The gateway system is designed to be pluggable, so you can write your own gateways fairly easily. It should be possible to write one that polls an external source, or listens to a message queue or whatever you want.
I'll see if we can brush up the documentation on that. We've written some that are just simple shell scripts, and they work just fine. So hopefully the barrier of entry for writing a completely custom gateway is not too high.

A custom gateway sounds great. I appreciate any documentation that can be put together.

Here's a sample custom gateway that listens to trello webhook events: https://github.com/technosophos/brigade-trello

Here's a shell script that I use inside of a Git receive hook for a version of Git that I run in cluster. It's really basic, but performs as a gateway. https://gist.github.com/technosophos/bd13903c536e7a07dd1a3c58aedea2c7

I'm closing this since I don't think there's more to do. Feel free to re-open if necessary.


Merge Branch Branch origin/WDRT_Py3

@zmorrell-sand It looks like this branch can be merged. It has a couple of print statements and changes to the way data is loaded.
Can you review the 2 commits and let me know if they should be merged?

Hi Sterling,
It would be better to Merge WDRT_Py3 rather than origin/WDRT_Py3.  I merged those two branches a while back, and if I recall Aubrey's most recent push fixes a bug that was in origin/WDRT_Py3.  The print statements were put in to help find what was causing the bug.

Perfect I am merging WDRT_Py3 now.
Can I delete origin/WDRT_Py3?

Relevant code from this branch is included in PR #36


Delayed message delivery intermittently fails to deliver messages until consumers reconnect when a KeyShared consumer is used and delays are randomized

Describe the bug
When consuming from a topic with a KeyShared consumer, messages with delayed delivery are often not returned from the consumer. This doesn't happen with all delayed messages, but seems to happen with around half.
This issue doesn't occur when using Shared subscriptions.
After the consumer is restarted, all messages that should have been delivered but were not are immediately delivered.
This only happens when delayed messages are not sequentially scheduled; i.e. it won't happen if I publish messages in order with equal or increasing delays, since those messages "come due" in publish order. For example, it will happen if I publish two messages, the first with a delay of 10sec and the second with a delay of 5sec.
To Reproduce

In one terminal, run the below code snippet with the command line argument consume. Verify it connects and waits for messages.
In another terminal, run the below snippet with the command line argument produce; observe that it prints out messages that it published and their delays.
In the first terminal, wait for 12 seconds (the maximum possible delay published by the producer script).
In the first terminal, observe that all messages were not delivered.
Restart (CTRL+C and run again) the consumer in the first terminal. Observe that it immediately receives all of the messages that were missed in step 4.

Code to reproduce:
import sys
import time
from datetime import timedelta, datetime
import random
import pulsar

topic = "persistent://public/default/foobar"

def main():
    client = pulsar.Client(service_url='pulsar://localhost:6650')
    if sys.argv[-1] == "produce":
        p = client.create_producer(topic=topic)
        for i in range(10):
            data = f"msg-{i}".encode()
            delay = timedelta(milliseconds=random.randint(1000, 2000))
            rv = p.send(data, deliver_after=delay, partition_key=str(i))
            print("sent", data, "delay:", delay, rv)
            time.sleep(1)
    if sys.argv[-1] == "consume":
        c = client.subscribe(
            topic=topic,
            subscription_name="chariot_subscription-heartbeat-heartbeat",
            consumer_type=pulsar.ConsumerType.KeyShared
        )

        while True:
            try:
                msg = c.receive(timeout_millis=1000)
            except pulsar.Timeout:
                pass
            else:
                c.acknowledge(msg)
                print(datetime.now(), msg.message_id(), msg.data())

if __name__ == "__main__":
    main()

Expected behavior
All messages should print out as received in step 3 above.
Desktop (please complete the following information):

OS: MacOS monterey.
Pulsar: 2.9.2 via Homebrew; also repro'd on latest standalone Docker image as of 4/29.

Additional context
KeyShared consumers are necessary for our use case: topics sometimes contain delayed messages, and sometimes contain non-delayed messages with a partition key set across groups of messages for ordered, single-consumer delivery.
We know that combining delays with a partition key can cause problems; so we enforce that delayed messages have UUID partition keys at our client layer.
To facilitate delayed delivery, we could work around the problem by periodically reconnecting consumers (hacky, induces extra delays) or by running a second Shared subscriber next to our KeyShared subscribers (complicated and potentially expensive in terms of consumer memory). We would prefer to keep the system simple and use a single KeyShared consumer for both types of messages.

Additional interesting info: if I call redeliver_unacknowledged_messages() on the consumer periodically (when there are no unacknowledged messages--i.e. if I replace the pass in the except Timeout in the example code with c.redeliver_unacknowledged_messages()), the consumer gets more, but not all, delayed messages. In the above test, periodically calling redeliver regularly gets 7-9/10 messages, where omitting that call sometimes only gets 1-3/10.

Also reproduced on 2.10

This looks to be caused by #12890.
We're calling this.getMessagesToReplayNow(1) (which  pops the message out of the delayed-delivery tracker) but then we're not doing anything with that message because minReplayedPosition == null

Thanks, I will test against a master build and respond here with results.


Support cyclic union types

Purpose
Fixes #23691 - Improve support for clone, cloneReadOnly
Fixes https://github.com/ballerina-platform/ballerina-lang/issues/23690 - Allow cyclic reference in unions typedef
Fixes #22429 - Remove error from anydata
Fixes #24337 - Replace error detail type from (anydata|readonly) to Cloneable
Fixes #24339 - Replace type of annotation tag from (anydata|readonly) to Cloanable
Fixes #24338 - Replace static type of send action from (anydata|readonly) to Cloneable
Approach

Describe how you are implementing the solutions along with the design details.

Samples

Provide high-level details about the samples related to this feature.

Remarks
Done a hack to finish the implementation.
https://github.com/ballerina-platform/ballerina-lang/issues/27786 TypeParam flag is propagated to original type
Check List

[x] Read the Contributing Guide
[ ] Updated Change Log
[ ] Checked Tooling Support (#)
[x] Added necessary tests

[x] Unit Tests
[ ] Spec Conformance Tests
[ ] Integration Tests
[ ] Ballerina By Example Tests

[ ] Increased Test Coverage
[ ] Added necessary documentation

[ ] API documentation
[ ] Module documentation in Module.md files
[ ] Ballerina By Examples

Create an issue for the PR for
@typeParam
type CloneableType Cloneable|never;		// hack
//type CloneableType Cloneable;			// expected

https://github.com/ballerina-platform/ballerina-lang/issues/27786

Create an issue for the PR for
@typeParam
type CloneableType Cloneable|never;		// hack
//type CloneableType Cloneable;			// expected

https://github.com/ballerina-platform/ballerina-lang/issues/27786


Refactor find the last punctuation mark

When truncating the token, we need to find the last punctuation mark. But in our current code, we're looping over each punctuation mark. If it can't find that punctuation mark, it will loop the length of the chunkText. I added something to define the punctuation as an enum and terminate the loop as soon as that punctuation is found.
Thanks.

Closing as the current functionality has passing tests  and no feedback on questions.  Please resubmit if you want and thanks in any case.


No settings window

The settings window does not open from the menu, affects: 6.99+, 7.0+.
MacOS: Monterey 12.7.5

In the new release 7.0.2


hypnotix 3.5 -> 3.6

https://github.com/linuxmint/hypnotix/compare/3.5...3.6
Description of changes

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Ha I actually have opened #250369


Any way for the software to auto search best connection and connect without any user intervention?

Love the program, great work!
As per title, how can I:

Have the program auto start at login
Automatically search best connection and connect to it

Thank you!

Also another ask, as I'm pretty new to linux as a whole, is there a way to make a simple script that I can click to terminate my OpenVPN connection?
I know the temrinal command sudo killall openvpn will do it, but not sure how to make a clickable file that can run

Hi @andyktaylor , about closing your connection using clicks, it depends on the Desktop Environment you are running. With Gnome, you can right-click on the top right corner of the screen to disable the VPN:

For auto-start, take a look at https://github.com/sebasmonia/nordvpn-client#installation and the .desktop file included in the repository
For start and auto-connect, would adding a parameter work? nordvpn-client auto or nordvpn --auto to trigger that.

Hi @andyktaylor , about closing your connection using clicks, it depends on the Desktop Environment you are running. With Gnome, you can right-click on the top right corner of the screen to disable the VPN:
For auto-start, take a look at https://github.com/sebasmonia/nordvpn-client#installation and the .desktop file included in the repository
For start and auto-connect, would adding a parameter work? nordvpn-client auto or nordvpn --auto to trigger that.

nordvpn-client auto doesn't seem to do anything, it opens the window, but the detect best local server is still selectable and it doesn't populate anything. Any other suggestions please?

Hey @andyktaylor sorry for the long wait. Life got crazy :)
I was asking if you think that would be a good way to handle it, adding the --auto parameter.
Will take a look at it tomorrow and report back!

If you try the latest version, it will auto-connect with --auto.
Although since I don't have a NordVPN account anymore, I could do the test 100%, I was prompted for a password, and the UI completed all the steps. I am optimistic :)


uuid versions into separate modules, per #154

👍 overall, just need to remove the version bump (FYI use the defunctzombie/changelog module to manage the changelog and release versions separately from PRs once I know what will go into the version). Also the addition of the new isCrypto api should be a separate PR.


GPT Engineer will not save individual files when given specs that result in many files.

The generated code goes into the logfile however it would be more useful if the tool could make all those files automatically.

It should always generate all files. This seems to happen sometimes, we should improve the prompt.

Happy to take a well prompt engineered PR on the generate and run_clarified steps

#94 should make sure the file creation works.

I have solved these issues and made a PR #120 with my improved parser.

Files are not being written. Did the html prompt and it managed to write the html file and another file called python with part of the game code but missing all the rest

I have solved these issues and made a PR #120 with my improved parser.

Do I modify the chat_to_file.py code with the green highlighted code? Also your gpt-engineer/test  test_chat_parser code does that need to be a new file in that directory with your written code?

Many changes have been applied since you posted this, give the latest version a try.
If you wish to reopen the issue please do following the new issue template.


Complete game logic have been done

Added Jest for implementing tests, babel.config to run the tests
And included all the necessary logic to make the game work. Additionally, functions for user interaction were implemented.

LGTM!!!


WIP: Update outdated docs

#20

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Changes look good 👍


Use async version of elm compiler to be able to get the error message

I want to be able to display the error message on the screen, for that I need to capture the error message produced during a compilation error, but this is not possible because the sync function from node-elm-compiler returns just a hardcoded string, as you can see here: https://github.com/rtfeldman/node-elm-compiler/pull/110
Since we are already in an async function, we might as well just use await here and get the proper error message

Unfotunately, as mentioned in #2, this can result in compilation failing occasionally due to corrupt builds.

I want to be able to display the error message on the screen
Can you elaborate a bit on this? The error messages generated by elm are piped through to stderr so not sure what you mean by this.

@phenax when I'm focusing on build the UI, I rely on hot reloading to refresh the screen, having just the code and the browser open side by side, so I wanted to display the errors in the DOM directly, with an overlay, not having to switch to the terminal

@rogeriochaves As mentioned in #2, this could lead to broken builds so I feel this issue should be dealt with upstream instead.


Upgrade third_party/protobuf to v3.14.0

Some of the failures might already been investigated as part of https://github.com/grpc/grpc/pull/24723.

Problems that likely need to be solved (the rest looks like preexisting issues)
Python27 on windows:
https://source.cloud.google.com/results/invocations/1ff03886-f487-4466-9149-67aca05767e8/targets/github%2Fgrpc/tests
Python bazel tests:
https://source.cloud.google.com/results/invocations/0fda9ba0-4fd9-4a50-9af1-41cc753f04d3/log
Ruby native gem on macos:
https://source.cloud.google.com/results/invocations/7a195775-9293-4177-a79e-370af526a00f/targets/github%2Fgrpc/tests
@gnossen @lidizheng  can you take a look at the python-related issues?

C:\tools\msys64_win7\mingw64\bin\gcc.exe -mdll -O -Wall -DWIN32_LEAN_AND_MEAN=1 -DMS_WIN64=1 -I. -Igrpc_root -Igrpc_root\include -Ithird_party\protobuf\src -IC:\Python27\include -IC:\Python27\PC -c third_party\protobuf\src\google\protobuf\util\internal\json_escaping.cc -o build\temp.win-amd64-2.7\Release\third_party\protobuf\src\google\protobuf\util\internal\json_escaping.o -std=c++11 -D_ftime=_ftime64 -D_timeb=__timeb64 -D_hypot=hypot
C:\tools\msys64_win7\mingw64\bin\gcc.exe -mdll -O -Wall -DWIN32_LEAN_AND_MEAN=1 -DMS_WIN64=1 -I. -Igrpc_root -Igrpc_root\include -Ithird_party\protobuf\src -IC:\Python27\include -IC:\Python27\PC -c third_party\protobuf\src\google\protobuf\compiler\cpp\cpp_map_field.cc -o build\temp.win-amd64-2.7\Release\third_party\protobuf\src\google\protobuf\compiler\cpp\cpp_map_field.o -std=c++11 -D_ftime=_ftime64 -D_timeb=__timeb64 -D_hypot=hypot
In file included from third_party\protobuf\src\google\protobuf\util\message_differencer.cc:50:0:
third_party\protobuf\src/google/protobuf/map_field.h: In constructor 'constexpr google::protobuf::internal::MapFieldBase::MapFieldBase(google::protobuf::internal::ConstantInitialized)':
third_party\protobuf\src/google/protobuf/map_field.h:336:37: error: call to non-constexpr function 'google::protobuf::internal::WrappedMutex::WrappedMutex()'
         state_(STATE_MODIFIED_MAP) {}
                                     ^

The Python compilation error seems caused by the new ProtoBuf.

The build failure looks like a legitimate build failure in protobuf that they've already fixed on master. The error indicates that the WrappedMutex constructor is not constexpr, which it isn't on 3.14.0. Some compilers are lenient on this and will add the constexpr for you if possible. Our Windows compiler appears not to be one of them.
I think our two options are:

Upgrade the compiler to something more lenient on constexpr marking.
Backport this fix to v3.14.0

P.S.: I'm secretly rooting for the backport, because that gets us one digit closer to this being the pi release.

FTR the python bazel build failure is:
==================== Test output for //src/python/grpcio_tests/tests_aio/interop:local_interop_test:
Traceback (most recent call last):
  File "/root/.cache/bazel/_bazel_root/d2dc70c3d9da3fab488ba0dcbbd35051/sandbox/processwrapper-sandbox/2815/execroot/com_github_grpc_grpc/bazel-out/k8-fastbuild/bin/src/python/grpcio_tests/tests_aio/interop/local_interop_test.runfiles/com_github_grpc_grpc/src/python/grpcio_tests/tests_aio/interop/local_interop_test.py", line 24, in <module>
    from tests_aio.interop import methods
  File "/root/.cache/bazel/_bazel_root/d2dc70c3d9da3fab488ba0dcbbd35051/sandbox/processwrapper-sandbox/2815/execroot/com_github_grpc_grpc/bazel-out/k8-fastbuild/bin/src/python/grpcio_tests/tests_aio/interop/local_interop_test.runfiles/com_github_grpc_grpc/src/python/grpcio_tests/tests_aio/interop/methods.py", line 29, in <module>
    from google import auth as google_auth
ImportError: cannot import name 'auth'
================================================================================
INFO: From Linking tools/distrib/python/grpcio_tools/grpc_tools/_protoc_compiler.so:
clang-12: warning: '-fuse-ld=' taking a path is deprecated. Use '--ld-path=' instead
INFO: From Linking tools/distrib/python/grpcio_tools/libprotoc_lib.so:
clang-12: warning: '-fuse-ld=' taking a path is deprecated. Use '--ld-path=' instead
FAIL: //src/python/grpcio_tests/tests_aio/status:grpc_status_test (see /root/.cache/bazel/_bazel_root/d2dc70c3d9da3fab488ba0dcbbd35051/execroot/com_github_grpc_grpc/bazel-out/k8-fastbuild/testlogs/src/python/grpcio_tests/tests_aio/status/grpc_status_test/test.log)
INFO: From Testing //src/python/grpcio_tests/tests_aio/status:grpc_status_test:
==================== Test output for //src/python/grpcio_tests/tests_aio/status:grpc_status_test:
Traceback (most recent call last):
  File "/root/.cache/bazel/_bazel_root/d2dc70c3d9da3fab488ba0dcbbd35051/sandbox/processwrapper-sandbox/2955/execroot/com_github_grpc_grpc/bazel-out/k8-fastbuild/bin/src/python/grpcio_tests/tests_aio/status/grpc_status_test.runfiles/com_github_grpc_grpc/src/python/grpcio_tests/tests_aio/status/grpc_status_test.py", line 22, in <module>
    from google.rpc import code_pb2, error_details_pb2, status_pb2
ModuleNotFoundError: No module named 'google.rpc'
================================================================================

they've already fixed on master

Looks like that fix is coming from the internal codebase (cl/341698875)
@acozzette   can we backport cl/341698875 into 3.14.x branch?

@jtattermusch I just sent out https://github.com/protocolbuffers/protobuf/pull/8115 to pull that fix into 3.14.x.

@stanley-cheung  Ideally protobuf should be upgraded before cutting the 1.35.x branch (according to our release process).

@stanley-cheung  Ideally protobuf should be upgraded before cutting the 1.35.x branch (according to our release process).

@stanley-cheung Ideally protobuf should be upgraded before cutting the 1.35.x branch (according to our release process).

Sorry I missed this. Should have made this a release blocker. What's the remedy now? Should we also backport this to the v1.35.x branch?

@stanley-cheung Ideally protobuf should be upgraded before cutting the 1.35.x branch (according to our release process).

Sorry I missed this. Should have made this a release blocker. What's the remedy now? Should we also backport this to the v1.35.x branch?

@stanley-cheung Ideally protobuf should be upgraded before cutting the 1.35.x branch (according to our release process).

Sorry I missed this. Should have made this a release blocker. What's the remedy now? Should we also backport this to the v1.35.x branch?

I created https://github.com/grpc/grpc/pull/25131 hopefully all the tests pass.

@stanley-cheung Ideally protobuf should be upgraded before cutting the 1.35.x branch (according to our release process).

Sorry I missed this. Should have made this a release blocker. What's the remedy now? Should we also backport this to the v1.35.x branch?

I created https://github.com/grpc/grpc/pull/25131 hopefully all the tests pass.

Superseded by https://github.com/grpc/grpc/pull/25183 (which brings in #25131 from v1.35.x)

Superseded by https://github.com/grpc/grpc/pull/25183 (which brings in #25131 from v1.35.x)


when running prompty through the vs code extension, can't use a json file as a sample

I'm running prompty through the vs code extension. My prompty configuration has a complex input which I want to load from a json file. This doesn't seem to be supported by the extension

Great question - have you tried something like this?

It doesn't work. Interestingly enough when running the same .prompty file through the promptflow VS code extension through the test button, it works:

Can you describe what you are doing and what error you are getting?

I found that prompflow tries to search for files located in folders with lowercase, so, if you have a project in prompflow and the json files are located in folder with some uppercase prompflow can't file the file to be included (replaced) in prompty. I solve the problem renaming my folder to lowercase (I'm using WSL 2).

Marking as closed - happy to reopen if there are isssues.


Notebook: HDFS as a backend storage (Read & Write Mode)

What is this PR for?
Zeppelin currently supports many backends for storing notes through Apache Commons VFS.
Apache Commons VFS supports HDFS in readonly mode.
This PR makes HDFS a first class citizen by allowing users to load notes from / save notes to HDFS.
What type of PR is it?
Improvement
Todos

[ ] - Task

What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1515
How should this be tested?
Update zeppelin.notebook.dir property to a value like hdfs://localhost:9000/tmp/notebook and the property zeppelin.notebook.storage to the value org.apache.zeppelin.notebook.repo.HdfsNotebookRepo
check that your notes are loaded from and stored to HDFS by listing notes using the command :
hdfs dfs -ls /tmp/notebook
Screenshots (if appropriate)
Questions:

Does the licenses files need update? No
Is there breaking changes for older versions? No
Does this needs documentation? Yes

Thanks for the contribution.
How about let user set not only HADOOP_CONF_DIR but also HADOOP_HOME in conf/zeppelin-env.sh to use HDFS notebook storage?
Then hadoop dependencies in zeppelin-zengine can be provided scope, and that'll reduce binary package size. Also that'll allow user set different hadoop version without rebuild.
what do you think?

@Leemoonsoo  Just pushed the changes to make hadoop lib provided.

@Leemoonsoo
Test testClose in org.apache.zeppelin.python.PythonInterpreterTest. Can't understand why it is failing on travis CI but not failing on my laptop.

@hayssams Why did you removed provided scope? If it doesn't, all kind of hadoop dependencies will be included in zengine.

@jongyoul
When using provided and bringing in all hadoop dependencies to zengine by adding hadoop lib in HADOOP_CONF_DIR to the classpath, Zeppelin raise an exception at runtime.
However by bringing explicitely in scope to the zengine only hadoop-hdfs and hadoop-common, everything works as intended.

@hayssams AFAIK, those hadoop-* have a lot of dependencies by themselves and it's possible to make a problem like version mismatch even in case that we don't use that feature. What do you think of it?

Agree with @jongyoul we should make hadoop dependency as provided, @hayssams  Could you paste the errors you can see when using provided ? So that someone else can help that.

@jongyoul @zjffdu
This issue is with hadoop 2.7 only. With hadoop 2.6 making hadoop dependency as provided works well.
I get the following exception
Caused by: java.lang.NoSuchMethodError: org.apache.hadoop.tracing.SpanReceiverHost.get(Lorg/apache/hadoop/conf/Configuration;Ljava/lang/String;)Lorg/apache/hadoop/tracing/SpanReceiverHost;
at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:634)
at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:619)
at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:149)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2596)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:91)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2630)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2612)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:370)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:169)
at org.apache.zeppelin.util.HdfsSite.mkdirs(HdfsSite.java:98)

@hayssams Do you have any clue that you mentioned? I don't know why two different jars while building Zeppelin.

You know, there is WebHDFS Rest API in hadoop.
If we use this, we don’t need to care about library dependency and configuration.
What do you think using WebHDFS Rest API?
HDFS interpreter already use it, so I added HDFSNotebookRepo using this function.
Please check this code : https://github.com/aspen01/zeppelin/commit/4b57a2faefdd34f4675e1acd78b25677111a5e57

I made a pull request : https://github.com/apache/zeppelin/pull/1600

Replaced by PR #2333


Questions about LLMCompiler

Hi, team.
I am developing a brand new agent framework instinct.cpp. In latest version, I tried to implement parallel function calling with the idea from your paper. And it actually works well.
After checking some bad cases, although no further experiments are done due my limited energy, some concerns are raised:

LLMCompiler requires LLM to have exceptional reasoning and instruct following capabilities at least on par with gpt-3.5-turbo, or it many be almost unusable. And to have such traits, more often, 70B models seem to be a must.
Replan seems to be unreliable. In original paper, the efficiency of re-planing is not discussed in details. In my experiments, if the model failed to produce good plan in first plan, it's unlikely it would have better result in second round.
In the process of dependency resolution, joiner plays an important role to format former answers to an entity of single word. This simplifies the argument substitution for downstream function calls which depend on those results, but it has many limitations. For example:

What's the result of temperature in New York yesterday raised to power of two?

Planner would give a task graph similar to this one, with tools of web search and a math calculator:
1. search("temperature in New York yesterday")
2. math("$1 ^ 2")
3. join()

While first call will succed with result like 21°C, the second call would fail with a math expression evaluator as it's undefined behavior to calculate power of two with21°C.
So here are my questions:

What opensource models are recommending for tool agent with LLMCompiler?
What could be future improvement about replan and join?

in addition to that, IYO, Is there any way to plan using multi-arg tools?

Could you provide some examples for the second point(Replan seems to be unreliable......)?


We can’t give specialist document finders a parent taxonomy topic, because there’s no route back

This is a constraint someone raised with us in the process of documenting the breadcrumbs component as part of the design documentation work.
We can’t give specialist document finders a parent taxonomy topic at the moment. Because there’s no route back the other way from taxonomy topic to document finder.
We think a principle of good breadcrumbs is that they can be followed in both directions: up and down.
Note: I think this would probably have to start with research. Is this the place to log this?

@ChristineWuerth it's not really the place to log this, no - this is more for technical issues with components, this sounds more like something that would require research.


Added release notes for 2.3 (#250)

Added release notes for 2.3

Signed-off-by: Chris Hesterman<EMAIL_ADDRESS>
Remove commit 221, move 229 to Maintenance, add 250 to Documentation

Signed-off-by: Chris Hesterman<EMAIL_ADDRESS>
Move 230 to Maintenance, 229 back to Bug fixes

Signed-off-by: Chris Hesterman<EMAIL_ADDRESS>
Move 229 back to Maintenance

Signed-off-by: Chris Hesterman<EMAIL_ADDRESS>Signed-off-by: Chris Hesterman<EMAIL_ADDRESS>(cherry picked from commit 248a2fe16d05a307837107419cbc4de8d765b6a7)
Description
[Describe what this change achieves]
Issues Resolved
[List any issues this PR will resolve]
Check List

[ ] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Codecov Report

Merging #253 (435408f) into 2.3 (2824645) will not change coverage.
The diff coverage is n/a.

:exclamation: Current head 435408f differs from pull request most recent head 6e4adb3. Consider uploading reports for the commit 6e4adb3 to get more accurate results

@@           Coverage Diff           @@
##              2.3     #253   +/-   ##
=======================================
  Coverage   51.54%   51.54%
=======================================
  Files         183      183
  Lines        5805     5805
  Branches      888      888
=======================================
  Hits         2992     2992
  Misses       2592     2592
  Partials      221      221

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


[Ruby] Specify module init in README

README should include information on cloning submodule repositories as the instructions for building the Ruby GRPC gem from source won't work.
PR: Adds the following command into the steps to building the gem from source.
git submodule update --init

Thank you for your pull request. Before we can look at your contribution, we need to ensure all contributors are covered by a Contributor License Agreement.
After the following items are addressed, please respond with a new comment here, and the automated system will re-verify.

User @frankyn isn't covered by a CLA. They will need to complete the form at https://identity.linuxfoundation.org/projects/cncf

Regards,
The Linux Foundation CLA GitHub bot

Please review my CLA status again.

@apolcyn could you review this?

waiting for tests to run and then merging