HDFS-13274. RBF: Extend RouterRpcClient to use multiple sockets

Description of PR
Jira: HDFS-13274
HADOOP-13144 introduces the ability to create multiple connections for the same user and use different sockets. The RouterRpcClient should use this approach to get a better throughput.
In my practice, I configured dfs.federation.router.connection.pool-size=6 and found some abnormal cases:

RBF frequently to create socket with NN during forwarding request.
There are a large number of connection threads with name node. (There are 70+ downstream name services)

After tracing the code and found that we can optimize the mechanism of round-robin obtaining connection.

One ConnectionContext can support handler some concurrent requests at the same time, such as 5
We can mark one ConnectionContext usable when the numThreads is less than maxConcurrencyPerConn
About getConnection, we can returns the previous usable connection in order
If there no usable connections, we can return one connection round-robin

After using the above solution,  we effectively controlled the number of connections between RBF and NameNode, and there is almost no the phenomenon of creating sockets during forwarding requests.

@goiri I have raised a new PR-4542 to push HADOOP-13144 forward. And after it, then I will continue push this issue forward.

@goiri I have raised a new PR-4542 to push HADOOP-13144 forward. And after it, then I will continue push this issue forward.

I left a few minor comments; it would be good for others to review this too.

@goiri I have updated this patch base on the trunk, please help me reivew it again. Thanks~

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 57s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
detsecrets
0m  1s

detect-secrets was not available.

+0 :ok:
xmllint
0m  1s

xmllint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
37m 54s

trunk passed

+1 :green_heart:
compile
1m  8s

trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
compile
0m 58s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
checkstyle
0m 52s

trunk passed

+1 :green_heart:
mvnsite
1m  3s

trunk passed

+1 :green_heart:
javadoc
1m  8s

trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
javadoc
1m 14s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
1m 49s

trunk passed

+1 :green_heart:
shadedclient
21m 27s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 43s

the patch passed

+1 :green_heart:
compile
0m 45s

the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
javac
0m 45s

the patch passed

+1 :green_heart:
compile
0m 40s

the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
javac
0m 40s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
0m 27s

the patch passed

+1 :green_heart:
mvnsite
0m 43s

the patch passed

+1 :green_heart:
javadoc
0m 41s

the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
javadoc
1m  0s

the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
1m 29s

the patch passed

+1 :green_heart:
shadedclient
20m 36s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
33m 40s

hadoop-hdfs-rbf in the patch passed.

+1 :green_heart:
asflicense
0m 52s

The patch does not generate ASF License warnings.

132m 12s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4531/3/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/4531

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint

uname
Linux fb953da2d1da 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 79ef02b09c2fa7ec15ca78e1e7b36f0a90b01947

Default Java
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS> /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4531/3/testReport/

Max. process+thread count
2205 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4531/3/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@ayushtkn @ferhui @Hexiaoqiao Can you help me review this patch?

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 58s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
xmllint
0m  0s

xmllint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
37m 58s

trunk passed

+1 :green_heart:
compile
1m  2s

trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
compile
0m 57s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
checkstyle
0m 51s

trunk passed

+1 :green_heart:
mvnsite
1m  3s

trunk passed

+1 :green_heart:
javadoc
1m  7s

trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
javadoc
1m 15s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
1m 47s

trunk passed

+1 :green_heart:
shadedclient
21m 25s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 42s

the patch passed

+1 :green_heart:
compile
0m 45s

the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
javac
0m 45s

the patch passed

+1 :green_heart:
compile
0m 39s

the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
javac
0m 39s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
0m 27s

the patch passed

+1 :green_heart:
mvnsite
0m 43s

the patch passed

+1 :green_heart:
javadoc
0m 41s

the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
javadoc
0m 58s

the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
1m 27s

the patch passed

+1 :green_heart:
shadedclient
20m 35s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
35m 36s

hadoop-hdfs-rbf in the patch passed.

+1 :green_heart:
asflicense
0m 53s

The patch does not generate ASF License warnings.

133m 56s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4531/4/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/4531

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint

uname
Linux bccc8cf47db9 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 9deba067bfd69972199bd3bbc70ed5bf6d95b220

Default Java
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS> /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4531/4/testReport/

Max. process+thread count
2211 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4531/4/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
1m 22s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
xmllint
0m  0s

xmllint was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
39m 17s

trunk passed

+1 :green_heart:
compile
1m  2s

trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
compile
0m 59s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
checkstyle
0m 52s

trunk passed

+1 :green_heart:
mvnsite
1m  4s

trunk passed

+1 :green_heart:
javadoc
1m  9s

trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
javadoc
1m 14s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
1m 46s

trunk passed

+1 :green_heart:
shadedclient
21m 13s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 42s

the patch passed

+1 :green_heart:
compile
0m 43s

the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
javac
0m 43s

the patch passed

+1 :green_heart:
compile
0m 40s

the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
javac
0m 40s

the patch passed

-1 :x:
blanks
0m  0s
/blanks-eol.txt
The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply

+1 :green_heart:
checkstyle
0m 26s

the patch passed

+1 :green_heart:
mvnsite
0m 43s

the patch passed

+1 :green_heart:
javadoc
0m 41s

the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
javadoc
1m  1s

the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
1m 31s

the patch passed

+1 :green_heart:
shadedclient
20m 52s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
36m 12s

hadoop-hdfs-rbf in the patch passed.

+1 :green_heart:
asflicense
0m 53s

The patch does not generate ASF License warnings.

136m 32s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4531/5/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/4531

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint markdownlint

uname
Linux 100e3f4ef007 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 072786b7b8daf2e7ec79951d015c619b5ac15e0b

Default Java
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS> /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4531/5/testReport/

Max. process+thread count
2212 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4531/5/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 59s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
xmllint
0m  0s

xmllint was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
38m 38s

trunk passed

+1 :green_heart:
compile
1m  2s

trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
compile
0m 58s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
checkstyle
0m 52s

trunk passed

+1 :green_heart:
mvnsite
1m  2s

trunk passed

+1 :green_heart:
javadoc
1m  8s

trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
javadoc
1m 14s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
1m 49s

trunk passed

+1 :green_heart:
shadedclient
21m 25s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 42s

the patch passed

+1 :green_heart:
compile
0m 44s

the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
javac
0m 44s

the patch passed

+1 :green_heart:
compile
0m 40s

the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
javac
0m 40s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
0m 26s

the patch passed

+1 :green_heart:
mvnsite
0m 44s

the patch passed

+1 :green_heart:
javadoc
0m 41s

the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
javadoc
0m 58s

the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
1m 27s

the patch passed

+1 :green_heart:
shadedclient
20m 35s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
35m 39s

hadoop-hdfs-rbf in the patch passed.

+1 :green_heart:
asflicense
0m 50s

The patch does not generate ASF License warnings.

134m 45s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4531/6/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/4531

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint markdownlint

uname
Linux 81f28fd5f6f7 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / b2dabf627d663a0f0b55c175a165e27eaeda8fb5

Default Java
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS> /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4531/6/testReport/

Max. process+thread count
2205 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4531/6/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@goiri Sir, thanks very much for your careful review, I have updated this patch, please help me review it again. Thanks

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m  0s

Docker mode activated.

-1 :x:
patch
0m 26s

https://github.com/apache/hadoop/pull/4531 does not apply to trunk. Rebase required? Wrong Branch? See https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute for help.

Subsystem
Report/Notes

GITHUB PR
https://github.com/apache/hadoop/pull/4531

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4531/7/console

versions
git=2.17.1

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m  0s

Docker mode activated.

-1 :x:
patch
0m 21s

https://github.com/apache/hadoop/pull/4531 does not apply to trunk. Rebase required? Wrong Branch? See https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute for help.

Subsystem
Report/Notes

GITHUB PR
https://github.com/apache/hadoop/pull/4531

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4531/8/console

versions
git=2.17.1

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m  0s

Docker mode activated.

-1 :x:
patch
0m 20s

https://github.com/apache/hadoop/pull/4531 does not apply to trunk. Rebase required? Wrong Branch? See https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute for help.

Subsystem
Report/Notes

GITHUB PR
https://github.com/apache/hadoop/pull/4531

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4531/9/console

versions
git=2.17.1

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

LGTM. It would be good to get a little more feedback from people working on this.

@ayushtkn @goiri Master, can you help merge this PR into trunk? Thanks.

Can we retrigger the build?

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 51s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
xmllint
0m  0s

xmllint was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

-1 :x:
mvninstall
21m  5s
/branch-mvninstall-root.txt
root in trunk failed.

+1 :green_heart:
compile
1m 10s

trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
compile
0m 52s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
checkstyle
0m 39s

trunk passed

+1 :green_heart:
mvnsite
0m 57s

trunk passed

+1 :green_heart:
javadoc
1m  9s

trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
javadoc
1m 15s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
2m  5s

trunk passed

-1 :x:
shadedclient
27m  1s

branch has errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 52s

the patch passed

+1 :green_heart:
compile
0m 51s

the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
javac
0m 51s

the patch passed

+1 :green_heart:
compile
0m 44s

the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
javac
0m 44s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
0m 27s

the patch passed

+1 :green_heart:
mvnsite
0m 48s

the patch passed

+1 :green_heart:
javadoc
0m 49s

the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

-1 :x:
javadoc
0m 27s
/patch-javadoc-hadoop-hdfs-project_hadoop-hdfs-rbf-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt
hadoop-hdfs-rbf in the patch failed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.

-1 :x:
spotbugs
0m 34s
/patch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-rbf.txt
hadoop-hdfs-rbf in the patch failed.

-1 :x:
shadedclient
9m 33s

patch has errors when building and testing our client artifacts.

_ Other Tests _

-1 :x:
unit
0m 33s
/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt
hadoop-hdfs-rbf in the patch failed.

+0 :ok:
asflicense
0m 34s

ASF License check generated no output?

70m 55s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4531/10/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/4531

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint markdownlint

uname
Linux 57abd26ef8cd 4.15.0-169-generic #177-Ubuntu SMP Thu Feb 3 10:50:38 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / ab301ace71af42c430b6e8e1e4e377345e74543d

Default Java
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS> /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4531/10/testReport/

Max. process+thread count
92 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4531/10/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
1m  0s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
detsecrets
0m  1s

detect-secrets was not available.

+0 :ok:
xmllint
0m  1s

xmllint was not available.

+0 :ok:
markdownlint
0m  1s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
38m 20s

trunk passed

+1 :green_heart:
compile
1m  2s

trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
compile
0m 57s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
checkstyle
0m 51s

trunk passed

+1 :green_heart:
mvnsite
1m  3s

trunk passed

+1 :green_heart:
javadoc
1m  8s

trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
javadoc
1m 14s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
1m 47s

trunk passed

+1 :green_heart:
shadedclient
21m 28s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 42s

the patch passed

+1 :green_heart:
compile
0m 44s

the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
javac
0m 44s

the patch passed

+1 :green_heart:
compile
0m 39s

the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
javac
0m 39s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
0m 26s

the patch passed

+1 :green_heart:
mvnsite
0m 42s

the patch passed

+1 :green_heart:
javadoc
0m 40s

the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
javadoc
0m 59s

the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
1m 26s

the patch passed

+1 :green_heart:
shadedclient
20m 44s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
33m 34s

hadoop-hdfs-rbf in the patch passed.

+1 :green_heart:
asflicense
0m 52s

The patch does not generate ASF License warnings.

132m 31s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4531/11/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/4531

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint markdownlint

uname
Linux ccdb928d81c5 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 44e9844132c2ac13c2f5d8986224455393905a35

Default Java
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS> /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4531/11/testReport/

Max. process+thread count
2104 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4531/11/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@goiri Master, I rebased it on the latest branch and triggered a building.
If nothing else, please helping me merge it into trunk. Thank you very much.

@goiri @Hexiaoqiao Master, thank you very much for helping me review this patch and very nice suggestion.


Memory leak on OSX

See https://github.com/PistonDevelopers/glium_graphics/issues/125
Here is a screen shot from Instruments:

Seems to be this method http://tomaka.github.io/glium/glium/backend/trait.Backend.html#tymethod.is_current.

I've tracked the leak down to this method in Glutin https://github.com/tomaka/glutin/blob/master/src/api/cocoa/mod.rs#L739.


Debug settings - out of date

The directions and screenshots in this section appear to be out of date with the current UI.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 97b73eb5-3313-6dbd-2dc3-56ef526ac29a
Version Independent ID: 0d2d41da-442b-39f4-d101-0fe495291692
Content: Mapping data flow Debug Mode - Azure Data Factory
Content Source: articles/data-factory/concepts-data-flow-debug-mode.md
Service: data-factory
GitHub Login: @kromerm
Microsoft Alias: makromer

The issue here may be that the settings are only shown when debugging is enabled. Perhaps this can be made clear in the description.

@VitalyMCT  Thanks for bringing this to our attention. Your feedback  has been shared with the content owner for further review.

@djpmsft

Fixed, should be live soon
#please-close


how to get 32FC1 from /camera/depth/image_rect_raw

hi, I want to get sensor_msgs/Imagic from  /camera/depth/image_rect_raw with 32FC1, I have changed the rs_rgbd.launch, set align_depth = false. There are some new topics of sw_registered, but on them the image_encodings is still 16UC1.So where is my fault?

have you found a solution?

[Please Ignore - RealSense system comment]

@gongyue666 What's the realsense ROS version when you get the issue? What camera and FW version? Did you still get it with the latest version? Looking forward to your update. Thanks!

@gongyue666 Could you please provide more information so that we can further investigate? Thanks!

@gongyue666 Haven't got update from you for long time. If no other questions, will close the ticket accordingly.

@RealSenseCustomerSupport
Hi,
I am facing the same problem. I am using Ubuntu 18.04 and ROS Melodic. The realsense camera I used is D415. The image published by "/camera/depth/image_rect_raw" topic launching by rs_rgbd.launch is in 16UC1, but what I need is 32FC1 format.
Any suggestion on the format conversion?
Thanks

Each pixel is depth in millimeters.

@doronhi Thanks for mentioning the pixel is in millimeters. I might have missed it, but could you put this information somewhere more obvious? It took me a little digging to find this. Thanks!

Thanks for the remark.
realsense2_camera follows OpenNI Raw Representation as the ROS convention mentioned here.
However, I accept that it is not clear for the realsense user and will add it to the README.md file.


blindex:0.1.0

I am submitting

[ X ] a new package
[ ] an update for a package

Description: Index-making of Biblical literature citations in Typst. The functionality achieved by this package is unique in the Typst ecosystem and useful for theologians or anyone citing biblical literature.

I have read and followed the submission guidelines and, in particular, I

[ X ] selected a name that isn't the most obvious or canonical name for what the package does
[ X ] added a typst.toml file with all required keys
[ X ] added a README.md with documentation for my package
[ X ] have chosen a license and added a LICENSE file or linked one in my README.md
[ X ] tested my package locally on my system and it worked
[ X ] excluded PDFs or README images, if any, but not the LICENSE anda small thumbnail.

[ X ] ensured that my package is licensed such that users can use and distribute the contents of its template directory without restriction, after modifying them through normal use.

Test scope seem to include other packages, which is quite unexpected. Probably because I kept my fork in sync with upstream (are devs not supposed to do that?)
My package is failing, but I want to apply a fix in an unsync'd fork, so I guess the best course of action is to close this PR and start anew once I'm ready... sorry for the dust.

Test scope seem to include other packages, which is quite unexpected. Probably because I kept my fork in sync with upstream (are devs not supposed to do that?)
My package is failing, but I want to apply a fix in an unsync'd fork, so I guess the best course of action is to close this PR and start anew once I'm ready... sorry for the dust.

No worries :) The CI runs for all packages, this is expected behavior. It is better to keep your fork in sync with upstream, but not mandatory for this repository (as you are only supposed to create new files when submitting a package, which should always be trivial to merge and should not conflict, regardless of the point of history you are based on).

No worries :) The CI runs for all packages, this is expected behavior. It is better to keep your fork in sync with upstream, but not mandatory for this repository (as you are only supposed to create new files when submitting a package, which should always be trivial to merge and should not conflict, regardless of the point of history you are based on).

Thank you for the clarifications!!


fix(slideshow): improve accessibility

General summary

Fixed keyboard navigation and screen reader compatibility for slideshows

Non visible slides now can't be navigated to using the keyboard.
Non visible slides are hidden from screen readers.
When slides become visible, the focus is restored and so is their screen reader visibility.

Added roving tab index to controls

Slides grouped together are now wrapped inside individual divs.

Added roles and attributes to improve a11y complience.

Screenshots

Converting to draft as I am working on other improvements that may interfer.

Test

Slideshow focus order ✔
Slideshow hidden slides are removed from focus order ✔
Slideshow slide shortcut buttons  implement roving tab index ✔

Non regression test

Dialog focus trap works like before ✔
Slideshow looks and work as expected ✔


Create a web3-like js lib

Have necessary APIs on server
Create a web3-like lib for client

Implement in icetea instead.


Get IntoJsValue, FromJsValue, and derives into Boa and get back on upstream

We don't really need this functionality urgently anymore, and indeed Boa has some pull requests open to address these kinds of things. I'll bump this to a low priority for now:

[ ] https://github.com/boa-dev/boa/pull/2276
[ ] https://github.com/boa-dev/boa/pull/1993
[ ] https://github.com/boa-dev/boa/pull/1995


Resource/Learn card animation looks odd

All cards slightly shifts the rest card on righton hover but course card on hover not only shifts rest rest to right and also goes up.

I think animation shoud be same for all the cards. If you want me to work on this, I can do that

@shreyshreyansh Okay


fix: Repeating HashTags

Fixes #1653
Changes:

calling chipGroup.removeAllView/() before adding chips

Screenshots for the change:

@aggarwalpulkit596  @angmas1 peer review

@iamareebjamal  please review


Refactor: cubewrite() extract conversion from 64 to 32 bit values

Tasks for this block of code:
https://github.com/ACCESS-NRI/um2nc-standalone/blob/70a890cc9434a501810d549e308668f59d21eb00/umpost/um2netcdf.py#L162-L166

[x] Extract to function
[x] Add unit tests
[ ] Does rounding need to be handled explicitly for float values?
[ ] Are any int64 values likely to overflow when converted to int32?

[ ]  Can numpy be toggled to warn of under/overflows in arrays?

[ ] Move conversion call into process()?

This relates to work under the parent issue #12.


Coord name not set when concating along a DataArray

from xarray import DataArray, concat
a = DataArray([0], dims='a')
out = concat([a, a], dim=DataArray([0, 1], dims='b'))
print(out.coords)

Coordinates:

None     (b) int32 0 1

I would've expected the name of the new coordinate to be the name of the DataArray variable but instead it's None.
Environment:
Output of xr.show_versions()
INSTALLED VERSIONS
commit: None
python: 3.9.2 | packaged by conda-forge | (default, Feb 21 2021, 04:59:43) [MSC v.1916 64 bit (AMD64)]
python-bits: 64
OS: Windows
OS-release: 10
machine: AMD64
processor: Intel64 Family 6 Model 142 Stepping 12, GenuineIntel
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: English_United States.1252
libhdf5: None
libnetcdf: None
xarray: 0.17.0
pandas: 1.2.4
numpy: 1.20.2
scipy: None
netCDF4: None
pydap: None
h5netcdf: None
h5py: None
Nio: None
zarr: None
cftime: None
nc_time_axis: None
PseudoNetCDF: None
rasterio: None
cfgrib: None
iris: None
bottleneck: None
dask: None
distributed: None
matplotlib: None
cartopy: None
seaborn: None
numbagg: None
pint: None
setuptools: 49.6.0.post20210108
pip: 21.1.1
conda: None
pytest: None
IPython: 7.23.0
sphinx: None

would've expected the name of the new coordinate to be the name of the DataArray variable

that's exactly what it's doing: it uses the name of DataArray([0, 1], dims='b'), which is None. To fix that, assign a name to the DataArray:
In [3]: a = xr.DataArray([0], dims='a')
   ...: out = xr.concat([a, a], dim=xr.DataArray([0, 1], dims='b', name="x"))
   ...: out
Out[3]:
<xarray.DataArray (b: 2, a: 1)>
array([[0],
       [0]])
Coordinates:
    x        (b) int64 0 1
Dimensions without coordinates: b, a

Since None is not a particularly good name, we could raise for unnamed DataArray objects (not sure if falling back to dims for 1D DataArray objects would be a good idea)

Thanks @keewis! I was confusing dimension names and variable names. I would support raising or falling back to a reasonably sane default -- the reason I stumbled on this was having a None coord name was breaking the _repr_html in Jupyter and causing a much more confusing error message

The repr will probably be fixed by #5149, but we should probably still fix that. Since we use dim anyways I guess we should fall back to that (not sure though). Thoughts, @pydata/xarray?
For reference, the conversion happens in _calc_concat_dim_coord ("else"): https://github.com/pydata/xarray/blob/04acabb50bea9f48224464850ae5a918e57406e5/xarray/core/concat.py#L244-L265

Agree that this should coerce to the name of the dim if the array has no name.
Technically should the kwarg of concat be coord rather than dim? I generally supply dim names there, rather than coords vars. (though I'm not suggesting we actually change it given the backward-compat).
Note that we already do this in the DataArray constructor:

In [12]: a = xr.DataArray([0,1], coords=[xr.DataArray([1,2])], dims='b')   # No name on the coord

In [13]: a
Out[13]:
<xarray.DataArray (b: 2)>
array([0, 1])
Coordinates:
  * b        (b) int64 1 2    # Yes name on the coord


Named generics (or generic maps) for partial generic application (and inference for those that aren't passed)

Suggestion
🔍 Search Terms

#26242
Partial Generics
Generic maps
Generic objects
Generic inference

✅ Viability Checklist
My suggestion meets these guidelines:

[x] This wouldn't be a breaking change in existing TypeScript/JavaScript code
[x] This wouldn't change the runtime behavior of existing JavaScript code
[x] This could be implemented without emitting different JS based on the types of the expressions
[x] This isn't a runtime feature (e.g. library functionality, non-ECMAScript syntax with JavaScript output, new syntax sugar for JS, etc.)
[x] This feature would agree with the rest of TypeScript's Design Goals.

⭐ Suggestion
We have all heard or seen this PR: #26242 which outlines the technical merits of allowing inference on some, but not all generics. What I am proposing in addition to that core concept is either

A new way of partially applying generics via an object-like syntax
Better support for generic inference in generic objects

📃 Motivating Example
Here is a tweet thread (with codesandbox examples) detailing some of the motivations for this feature: https://twitter.com/tannerlinsley/status/1363926547489456129
💻 Use Cases

Allowing users to customize only some generics for a type or function without needing to know the number of generics, eg. no LibraryType<*, *, *, *, *, *, TData> // Oops! LibraryType expected 5 generic arguments, not 7.
In the tweets and codesandboxes in that tweet, you can see some potential workarounds for allowing this type of behavior, but both are limited in either TS's ability to infer types nested in an object literal, or the ability for Ts to infer all generics after user-supplied ones (#26242 would fix this, but would still have the issue of Type<*,*,*,*,*,TDData>)

It appears this has a PR already here: #23696. Closing!


Implicit exist condition for imports

Right now a lot of times you see something like this
<Import Project="path\to\foo.targets" Condition="Exists('path\to\foo.targets')" />

But is there a reason not to always have an implicit check if an import exist instead of failing the build if it's not found? Perhaps there should be a log message or something, but not having that exist condition would make things a lot less verbose :)

I don't think such a check should be the default as you propose. In fact, I think many of the cases that do have this check are wrong, and should cause failures.
A danger of the if-exists check is that in many (most?) cases, the build that happens with chunks of its build logic missing is wrong, but there's no obvious way to know that. If you fail to import the props file that sets the FIX_ALL_BUGS preprocessor flag, the build might complete without errors but be producing the wrong thing.
I would be more amenable to reducing the verbose and error-prone nature of this check by adding an attribute along the lines of IfExists="true", though. That might also be the best way to do this for #1400.

But there are already instances where imports are loaded dynamically with wildcards, is that so different? a nugget restore will place a bunch of targets on disk that are required for the build to work, but not there by default and not checked for,
Having a less verbose ThrowIfMissing attribute would be a step in the right direction though.

My workaround below:
    <PropertyGroup>         <JenskinsTarget>$([MSBuild]::GetPathOfFileAbove('Directory.Build.targets', '$(MSBuildThisFileDirectory)../'))</JenskinsTarget>     </PropertyGroup>     <Import Project="$(JenskinsTarget)" Condition="exists('$(JenskinsTarget)')" />

Duplicate of #822


Add github action for upload to pypi

@jcohenadad Could you add the TWINE_USERNAME and TWINE_PASSWORD to the repo's secrets section?

@Drulex i've done it in the past and then forgot-- quick google sent me there for setting up the values, but my question is: how do i get the values?

The owner/creator of the pypi account should have them. Maybe copy them from the ivadomed secrets?

OK, I remember what i did now. I've created a PYPI_PASSWORD at the organizational level (neuropoly/), which is valid for all repos inside it. So you should be able to use that.

Hurray first release https://pypi.org/project/changelog-neuropoly/1.0.0/


Fix for the nextChangeIndex invariant violation in the light mode

It turns out that address discovery in the light-mode did not update the list of pending indexes. We came to the conclusion that instead of adding fixing this in the light-mode, we change the behavior in node-mode too — specifically, we change the semantics of nextChangeIndex to automatically drop pending indexes that are too old. The idea is that PendingIxs now behaves more like a random seed rather than a specific list of addresses; by making this type abstract, we can switch out its implementation later (if desired).

[x] I have re-formatted code to add readability and code-style compliance in a separate commit;
[x] changed the code such that pending indexes are not updated when a new address discovered, but are updated right before the next change index is about to be generated. This way its easier to maintain pending indexes invariant as the update logic isn't scattered across modules.

Issue Number
ADP-2154

[x] BlockArguments removed.
[x] Style suggestion applied.

I use (`function` secondArg) instead of flip

[x] Ineger overflow issue addressed.

bors merge

bors merge

bors try

I've been doing some e2e tests using tests from piotr/light-asset-balance against this branch. I haven't noticed the original error, however at times on submitting transaction I see "Something went wrong" from the API and this Blockfrost exception in the log:
[cardano-wallet.wallet-engine:Error:11215] [2022-08-30 13:21:11.59 UTC] 28b6bc20: submitTx:
BlockfrostExce  - inputs: txid: 881f4c87
  - exception: BlockfrostException (ClientError (FailureResponse (Request {requestPath = (BaseUrl {baseUrlScheme = Https, baseUrlHost = "cardano-testnet.blockfrost.io", baseUrlPort = 443, baseUrlPath = ""},"/api/v0/tx/submit"), requestQueryString = fromList [], requestBody = Just ((),application/cbor), requestAccept = fromList [application/json;charset=utf-8,application/json], requestHeaders = fromList [("User-Agent","blockfrost-haskell/<IP_ADDRESS>"),("project_id","4mTH5uHPjqSK3iAg0clysnY93Fso7B3p")], requestHttpVersion = HTTP/1.1, requestMethod = "POST"}) (Response {responseStatusCode = Status {statusCode = 400, statusMessage = "Bad Request"}, responseHeaders = fromList [("Date","Tue, 30 Aug 2022 13:21:11 GMT"),("Content-Type","application/octet-stream"),("Content-Length","468"),("Connection","keep-alive"),("vary","Origin"),("access-control-allow-origin","*"),("submit-server","submit2"),("CF-Cache-Status","DYNAMIC"),("Expect-CT","max-age=604800, report-uri=\"https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct\""),("Report-To","{\"endpoints\":[{\"url\":\"https:\\/\\/a.nel.cloudflare.com\\/report\\/v3?s=kE0NGuE0H%2FSaNDx8gk8wW6%2BTWLkeQeosy7RGqkXDjSvvuQvyQlQawa0UfuCBoJwd6RiRiZcoWljTJhX8NqubxOhaEO%2FR5%2F5222FEE%2BwMbe3N0oCuseUFtnMhlgdZfIr5e5Wq1Aag7tgWbVBRzET9\"}],\"group\":\"cf-nel\",\"max_age\":604800}"),("NEL","{\"success_fraction\":0,\"report_to\":\"cf-nel\",\"max_age\":604800}"),("Server","cloudflare"),("CF-RAY","742dd3fd39d3b912-AMS")], responseHttpVersion = HTTP/1.1, responseBody = "{\"error\":\"Bad Request\",\"message\":\"\\\"transaction submit error ShelleyTxValidationError ShelleyBasedEraBabbage (ApplyTxError [UtxowFailure (UtxoFailure (BabbageOutputTooSmallUTxO [((Addr Testnet (KeyHashObj (KeyHash \\\\\\\"fe2e6bcc8f3ebf44e84d2c2e316a2ebca3cb8d15e7bc7a24a9d14d6e\\\\\\\")) (StakeRefBase (KeyHashObj (KeyHash \\\\\\\"c301819a18e29debb03a635bf98a3ca63263e285c7ba73df0b516fe2\\\\\\\"))),Value 775519 (fromList []),NoDatum,SNothing),Coin 969750)]))])\\\"\",\"status_code\":400}"})))

bors merge


Updated tree-sitter, added clarification regarding the build process to Readme, added test script entry

Updated to tree-sitter 0.20.6, which is the latest version of tree-sitter at the time of this pull request, runs now with Node 20.10.0 (LTS).
I have added a test script entry in package.json that runs "jest" in order for "npm test" to work. This should be changed to the new method one #20 is implemented.
Added entry for the supported node version in package.json

I tried updating all tree-sitter dependencies to their newest versions, however, updating "tree-sitter-php" reproducibly leads to the failure of the test case "parses PHP McCabeComplexity metric => should count if statements correctly". This change also increases the deviation to the snapshot in the test case "parsing PHP dependencies".

Since this is about updating tree-sitter, I guess we could also update all other tree-sitter dependencies.
Could you please open a ticket to install renovate for automated dependency updates? :-)

I tried updating all tree-sitter dependencies to their newest versions, however, updating "tree-sitter-php" reproducibly leads to the failure of the test case "parses PHP McCabeComplexity metric => should count if statements correctly". This change also increases the deviation to the snapshot in the test case "parsing PHP dependencies".
The renovate-thing should be already covered with #15 I guess?

Since the project is not installable at all with current versions of node.js without this change here and there seems to be more to updating these other tree-sitter dependencies (as I understand: grammars for the respective languages) than just changing the version number, we might open a new issue for updating these other dependencies and fixing this PHP issue that occurs by doing so?

Since the project is not installable at all with current versions of node.js without this change here and there seems to be more to updating these other tree-sitter dependencies (as I understand: grammars for the respective languages) than just changing the version number, we might open a new issue for updating these other dependencies and fixing this PHP issue that occurs by doing so?

I have opened issue #24 for updating the other tree-sitter dependencies/grammars now.

I would update all tree-sitter ones that are not causing any issues and merge it afterwards. You could also remove one unused library, if I am not mistaken. Otherwise I would forget about it later.

Thinking about these dependencies/grammars, the Readme states that there are actions to be performed in case the grammars are updated, including the manual mapping of expressions (see https://github.com/MaibornWolff/metric-gardener#maintenance-of-tree-sitter-grammars). This explains why the program fails when updating the grammar without updating these mappings. I am therefore hesitant to change any of them until clarifying with @ce-bo how to update the existing mappings efficiently. I have added the issues #24 for this.
Regarding the unused library, I have opened issue #27 to clarify that.


taox11/tao/x11/system_exception.h:271:5: error: '_info' is a protected member of 'taox11::CORBA::SystemException' [clang-diagnostic-error]

Version
/* -------------------------------------------------------------------
 * @file    versionx11.h
 * @author  Johnny Willemsen
 *
 * @brief   TAOX11 Version information file
 *
 * @copyright Copyright (c) Remedy IT Expertise BV
 * -------------------------------------------------------------------*/
#define TAOX11_MAJOR_VERSION 1
#define TAOX11_MINOR_VERSION 8
#define TAOX11_MICRO_VERSION 1
#define TAOX11_VERSION "1.8.1"

Describe the bug
/home/klein_cl/Workspace/cpp/taox11/axcioma/taox11/tao/x11/system_exception.h:271:5: error: '_info' is a protected member of 'taox11::CORBA::SystemException' [clang-diagnostic-error]
    TAOX11_STANDARD_SYSTEM_EXCEPTION_LIST
    ^
/home/klein_cl/Workspace/cpp/taox11/axcioma/taox11/tao/x11/system_exception.h:159:5: note: expanded from macro 'TAOX11_STANDARD_SYSTEM_EXCEPTION_LIST'
    TAOX11_SYSTEM_EXCEPTION (MARSHAL) \
    ^
/home/klein_cl/Workspace/cpp/taox11/axcioma/taox11/tao/x11/system_exception.h:249:12: note: expanded from macro 'TAOX11_SYSTEM_EXCEPTION'
        x_._info (os_); \
           ^
/home/klein_cl/Workspace/cpp/taox11/axcioma/taox11/tao/x11/system_exception.h:95:12: note: declared protected here
      void _info (std::ostream& strm) const override;
           ^
28559 warnings and 20 errors generated.
Error while processing /home/klein_cl/Workspace/cpp/lca-adapter/src/ConnectionHandler.cpp.
Suppressed 28529 warnings (28529 in non-user code).
Use -header-filter=.* to display errors from all non-system headers. Use -system-headers to display errors from system headers as well.
Found compiler error(s).
make: *** [GNUmakefile:44: check] Error 1

Clang is not a formally supported compiler, we haven't tested with clang for a long time anymore.

Clang is not a formally supported compiler, we haven't tested with clang for a long time anymore.

You don`t understand the problem.
That our application code, compiling with gcc under Linux.
But we want to make static code analysis of our hand written code, that include your headers!
This is a requirement of our customer to do this with clang-tidy, or other tools based on this.


ENH: support all areaDetector plugins versions 1-9-1 to R3-4

I may have forgotten some details here, as this branch was done around the time of the proposal of #662. Apologies if anything is unclear.
Notes

Largely auto-generated
Allows specification of version_type only, for use in base plugin classes
AddsCommonPlugin class hierarchy, allowing for easy access to common plugins specified by commonPlugins.cmd (or EXAMPLE_commonPlugins.cmd in some versions)
See PV coverage below
~* Adds PluginVersion class hierarchy, allowing for easy access to specific versions of plugins compatible with an ADCore release~

ophyd.areadetector.plugins cleanup

Removes the mistake that was the C abbreviation of Component (now Cpt)
Adds the required versioning
Uses kind instead of attr lists
Refactors ad_group into much more readable DDC_SignalWithRBV-style partials

TODO

Detectors/cams to follow in another PR

What a messy class hierarchy, ew!
To those who have that comment, this solution:

Allows for code re-use such that methods can be implemented at the correct points in the versioning scheme for either (1) a specific plugin, (2) a specific file plugin, or (3) all plugins of a version.
The style lends toward seeing an easy "diff" of what changed from one version to the next in a fairly succinct manner
The versioning scheme makes selection of these classes significantly easier from a user or coding standpoint

If the above isn't convincing enough, and you have a better idea and time to implement it, by all means go for it.
Coverage
Coverage of PVs versus those in a standard AreaDetector IOC using the example common plugins (as is fairly standard practice) are as follows:
R2-0 total pvs in ophyd 2102 of 2102 : PVs from IOC 100.0 %
R2-1 total pvs in ophyd 2193 of 2193 : PVs from IOC 100.0 %
R2-2 total pvs in ophyd 2868 of 2868 : PVs from IOC 100.0 %
R2-3 total pvs in ophyd 2913 of 2913 : PVs from IOC 100.0 %
R2-4 total pvs in ophyd 2913 of 2913 : PVs from IOC 100.0 %
R2-5 total pvs in ophyd 3039 of 3039 : PVs from IOC 100.0 %
R2-6 total pvs in ophyd 4099 of 4099 : PVs from IOC 100.0 %
R3-1 total pvs in ophyd 4746 of 4746 : PVs from IOC 100.0 %
R3-2 total pvs in ophyd 4757 of 4757 : PVs from IOC 100.0 %
R3-3 total pvs in ophyd 5555 of 5555 : PVs from IOC 100.0 %
R3-4 total pvs in ophyd 5982 of 5982 : PVs from IOC 100.0 %

One possibility is a plugins package with a module per plugin. I think that would work out fairly cleanly. (We would need the requisite back-compat imports into plugins/__init__.py of course.
But given the auto-generated nature of this code, "one giant module" is also fine with me.

Thanks, @danielballan - I am definitely looking for opinions on organization. new_plugins is merely a placeholder.
So far, it seems like the votes are:
One giant plugins file

@ZLLentz
maybe @danielballan

Separate file per plugin

maybe @danielballan

Undecided (or no comment)

@klauer
@tacaswell

Correct summary. I think either option is fine — and we can change our mind without breaking API if we really want to!

To check my understanding:

common_plugins -> Devices per version that you could mix into a user-space Device to get a functioning AD
new_plugins.py -> the actual implementation of the plugin classes
plugin_versions.py -> namespaces of all the plugins grouped by version.
plugins.py -> the existing source of plugins (which were implicitly 1.9.1)

I think new_plugins.py should be made private, plugin_versions should be renamed to versioned_plugins.py, plugins.py should be deprecated (that is going to be fun!), and we need a better name for common_plugins (as the same suggests it contains the common plugins, not Devices that have all the common plugins as children)

This isn't tested, but I think something like the following will also work:
from ophyd import Component

class VersionedComponent(Component):
    def __set_name__(self, owner, attr_name):
        super().__set_name__(owner, attr_name)
        self.cls = _select_version(self.cls, owner._class_info_['version'])

That way on the floor when the IOC gets updated it is (ideally) a 1 line change to update the version of all of the plugins.
How do we want to fold in the changes to the trigger method and the changed default value of 'blocking_callbacks' to this (a-la https://github.com/NSLS-II/nslsii/blob/master/nslsii/ad33.py )?

Do you really think the dynamic version you've provided is clearer, @tacaswell? I personally like being able to look at the source code and see what it's doing, which your version completely obfuscates.
VersionedComponent was going to be something that came out of the versioning PR (it was part of my original proposal), but that discussion took so long that it never got that far.

I don't find many screens of auto generated almost the same but different in subtle ways code particularly clear either.  At least with for loops you can see what the patterns are instead of trying to reverse engineer them.
I'm glad we all agree VersionComponent is a good idea.
Is
from ophyd import Component

class VersionedComponent(Component):
    def __set_name__(self, owner, attr_name):
        super().__set_name__(owner, attr_name)
        try:
            self.cls = _select_version(self.cls, owner._class_info_['version'])
        ValueError:
            setattr(owner, attr_name, None)

to evil / magical?

Taking most of @tacaswell's suggestions; do not merge as there are massive PV coverage-related issues.

OK, I believe I have fixed @tacaswell's dynamic generation of common plugins/plugin versions. Since I was going over all of this again, I've also increased the common plugin PV coverage to 100% across the board (it required adding a PluginBase for R2-5, as that's when QueueSize was introduced - not R2-6 as erroneously noted before).

I am :+1: on this going in on the technical side of things, but I think there are a few API questions that should get more eyes than @klauer and I
how to get access to a 'versioned namespace' of the plugins
06d0055 implements a dict-of-SimpleNamespace and 5892bc1 implements a versioned ophyd object that is the namespace.
In either case, the imagining use would be

plugin_ns = select_version(Plugins, (3, 2)
plugin_ns = versioned_plugins[(3, 2)]

class MyAD(ADBase):
    tiff = Cpt(plugin_ns.TIFFPlugin)
    ....

names of CommonPlugins and Plugins
The first is a proper Device with a coherent set of plugins an components that you can mix into your own device, the second is a namespace of coherently versioned plugins. The names collide too much in (at least my) head space.  CommenPlugins matches some of the naming schemes

class MyDet(SomeDetWithCam, CommonPlugins):
   ....

should CommonPlugins include the file writing ones?
The classes for the file plugins do not include the Asset Document minting logic
how to expose access to the *_VXXX classes
Currently systematically named Plugins and CommonPlugins are exposed in __all__ all the way up through to the top-level ophyd namespace, where as for the individual plugins, only the non-qualified base class (which is implicitly v1.9.1) is.
Do we want to:

leave it as is
only expose the un-qualified base class of CommonPlugins and Plugins (and force the use of select_version on them)
expose all of the VXX classes out.

Addressed @tacaswell's comments out of order because it will be easier to follow:

names of CommonPlugins and Plugins

I have renamed Plugins to PluginNamespace. I hope that is satisfactory. It's a sensible but potentially confusing aspect of the design that will require good documentation, and optimizing its name is only going to do so much good here.
Commit pushed by Dan

how to get access to a 'versioned namespace' of the plugins

We will support these ways of getting, for example, StatsPlugin_V34:

I have the base class; I want a specific version.

In [2]: from ophyd import select_version

In [3]: from ophyd import StatsPlugin

In [4]: select_version(StatsPlugin, (3, 4))
Out[4]: ophyd.areadetector.plugins.StatsPlugin_V34

I want to directly import a class from the module where it is defined.

In [5]: from ophyd.areadetector.plugins import StatsPlugin_V34
# not available from ophyd or ophyd.areadetector though

I have a versioned PluginNamespace_V34; I want a specific plugin from it.

In [6]: from ophyd import PluginNamespace_V34

In [7]: PluginNamespace_V34.StatsPlugin
Out[7]: ophyd.areadetector.plugins.StatsPlugin_V34

4.I have PluginNamespace, and I want to get a specific version and plugin.
In [8]: from ophyd import PluginNamespace

In [9]: select_version(PluginNamespace, (3, 4)).StatsPlugin
Out[9]: ophyd.areadetector.plugins.StatsPlugin_V34

This is a lot of ways to do it, but I can think of arguments for why each one is a reasonable way to do it, so I don't think there is a very good reason to making any of them not possible.
Commit coming by Ken

how to expose access to the *_VXXX classes

They will be importable from ophyd.areadetector.plugins but they will not be hoisted into ophyd.areadetector or ophyd. in ophyd/areadetector/__init__.py we will explicitly hoist the old (1.9.1) classes for back-compat. We will rely on documentation to make the new classes discoverable.
Commit coming by Ken

should CommonPlugins include the file-writing ones?

To be clear, the CommonPlugins currently do contain file plugins --- but those file plugins do not include the asset registry integration. The original reasons for keep the asset registry integration separate no longer apply, and it can probably be pushed into the file plugins contained in CommonPlugins now. This will require significant additional tests and likely some discussion. Let's handle it in a separate PR.
No action required for this PR


Adds external governance links

Fixes #9

needs some context about cascading trusted_governance links

needs hashlink or other method to make sure the file you pull is verified? maybe include the publishing DID?

needs discussion of trust implications by publisher  when external governance is linked to.

Discussed WG 20230925. No objections to merging.

@TelegramSam do you need some help merging this?


License type not automatically detected by GitHub.

Monitor to see if GitHub automatically refreshes the license type after the most recent release of the package.

GitHub did not automatically refresh the license type after a long time.

The following procedure worked to refresh the license type.

Remove LICENSE file from repository.
Add a License file using GitHub repository UI

"Add file" > "Create new file".
Naming the file "LICENSE" activates the "Choose a license template" button.
Select a license template from the options and add it to the repository.
Follow the instructions to create a pull request and merge the update into "main".

This process will cause GitHub to rescan the license and update the license type displayed in the GitHub UI.


your installer script, sir

https://github.com/coolaj86/caddy-install-script
On OS X and most *nixes:
curl -fsSL bit.ly/install-caddy | bash

On Ubuntu and minimal *nix installations:
wget -nv bit.ly/install-caddy -O - | bash

I think it'll work on all existing supported OSes and platforms except arm
We still need:

caddy_linux_armv6l-hf.zip
caddy_linux_armv6l.zip
caddy_freebsd_armv6l-hf.zip
caddy_freebsd_armv6l.zip

Cool! I've been working on lots of deployment/ops stuff lately, so I'll come back to this when I'm ready to use it.

Ported it to getcaddy.com for some trial runs. Thanks!


Warn against erasing array elements while iterating.

Erasing array elements while iterating does not appear to be safe.
For example, the following prints nothing:
var a := [0,1,2,3,4,5,6]
for i in a:
        if i % 2 == 0:
                a.erase(i)
        else:
                print(i)

While this is often true for array implementations, it still seems worth
documenting explicitly. I copied the wording from Dictionary.xml.

Thanks!


Vendor files

Hi,
We are using vendor files which rarely change (twice a month),
We need them to load before our application code.
Is that possible ?
Taking from default_index.html:
	{% for (var chunk in o.htmlWebpackPlugin.assets) { %}
	<script src="{%=o.htmlWebpackPlugin.assets[chunk]%}"></script>
	{% } %}

loads all the files without order.

Ok, solved it:
<script src="{%=o.htmlWebpackPlugin.assets.libs%}"></script>
<script src="{%=o.htmlWebpackPlugin.assets.app%}"></script>

And in webpack.config.js:
module.exports = {
	entry: {
		app: './entry.js',
		libs: [
			'./bower_components/mithril/mithril.min.js',
			'./bower_components/pure/pure-min.css',
			'./bower_components/pure/grids-responsive-min.css'
		]
	}
}


ComposeMenu [nfc]: Convert to functional component.

This was step one in a UX change I was planning to make. I got sidetracked due to problems with the library I was planning to use (callstack/react-native-paper#2769), but I figure I'd push this refactor commit anyways.

Thanks for this refactor! I agree with Chris's small comment, and everything else looks good.
Other than cutting that useCallback which isn't doing anything useful, the other direction to go would be to apply useCallback to all the named functions there, so that the callbacks passed as props to child components would be made stable. That's potentially an optimization. But we had a chat discussion a couple of weeks ago about when to use useCallback:
https://chat.zulip.org/#narrow/stream/243-mobile-team/topic/React.2EuseCallback/near/1226466
and one conclusion was that that optimization may often not be effective at all. If it is effective here, it's unlikely to be meaningful -- it doesn't seem like these icon components (which are what ultimately get some of these callbacks as props) should have much work at all to do in their render functions.


Both tensors are on same device but raises an error both aren't

Discussed in https://github.com/PyTorchLightning/metrics/discussions/607

Originally posted by ThiruRJST November  5, 2021
"""This module does training and validation of the model."""

import random

import numpy as np
import torch
import torchmetrics
import wandb
from torch import nn
from torch.cuda.amp import GradScaler, autocast
from torch.utils.data import DataLoader
from tqdm import tqdm

from constants import BATCHSIZE, EPOCHS
from preprocess import AudioDataset
from utils.build_model import CNN14_mod
from utils.callbacks import EarlyStopping
from pytorch_lightning import seed_everything

def train_loop(epoch, dataloader, scaler, model, loss_fn, optim, device='cuda:0'):
    """Train loop that trains the model."""
    model.train()
    epoch_loss = 0
    metric = torchmetrics.Accuracy()
    step = None

    pbar = tqdm(enumerate(dataloader), total=len(dataloader))
    for step, (img, label) in pbar:
        optim.zero_grad()

        img = img.to(device).float()
        label = label.to(device).long()

        with autocast():
            yhat = model(img)
            # Loss Calculation
            train_loss = loss_fn(input=yhat, target=label)
            out = torch.softmax(yhat, dim=1)

        train_step_acc = metric(out, label)

        scaler.scale(train_loss).backward()
        scaler.step(optim)
        scaler.update()

        epoch_loss += train_loss.item()

    train_epoch_loss = epoch_loss / len(dataloader)
    train_acc = metric.compute()

    wandb.log({'Train_Epoch_Loss': train_epoch_loss, 'Epoch': epoch})
    wandb.log({'Train_Epoch_Acc': train_acc, 'Epoch': epoch})
    wandb.log({'Train_Step_Acc': train_step_acc, 'Step': step})

    metric.reset()

    return train_epoch_loss, train_acc

def val_loop(epoch, dataloader, model, loss_fn, device='cuda:0'):
    """Validation loop to validate the trained model and save it based on performance."""
    model.eval()
    val_epoch_loss = 0
    metric = torchmetrics.Accuracy()
    pbar = tqdm(enumerate(dataloader), total=len(dataloader))
    step = None

    with torch.no_grad():
        for step, (img, label) in pbar:
            img = img.to(device).float()
            label = label.to(device).long()

            yhat = model(img)
            val_loss = loss_fn(input=yhat, target=label)
            out = torch.softmax(yhat, dim=1)

            val_epoch_loss += val_loss.item()

            val_step_acc = metric(out, label)

        val_lossd = val_epoch_loss / len(dataloader)
        val_acc = metric.compute()

        wandb.log({'Val_Epoch_Loss': val_epoch_loss, 'Epoch': epoch})
        wandb.log({'Val_Epoch_Acc': val_acc, 'Epoch': epoch})
        wandb.log({'Val_Epoch_Acc': val_step_acc, 'Step': step})

        metric.reset()

        return val_lossd, val_acc

I'm using torchmetrics to calculate the accuracy of model in pytorch but it raises an error that the tensors aren't in the same device. Actually both the tensors are in same device.
ERROR: RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!
Is this a bug or have I made any mistakes?

pls see #340 we would recommend to use Pytorch-Lightning which does this device sync for you
cc: @SkafteNicki

Just going to add that the solution is to move your metrics to the correct device (similar to you moving model and data). So instead of
metric = torchmetrics.Accuracy()

do
metric = torchmetrics.Accuracy().to(device)


Method to access data in GArray?

I see there are ways to add and insert things into GArray ( and other collections types), but there doesn't seem to be a way to access that information besides using search etc. Is there a way to access information like
let data: GArray
let value = data[ArrayMesh.ARRAY_VERTEX] or more generically let value = data[5]

oh wait I see there is a subscript, it just doesn't infer that because I probably have to change ARRAY_VERTEX to something


Provide id parameter in delete requests

It looks like the params there were a placeholder, likely copied from a previous use. I updated it to use the fullname for the id param, as described by the api documentation here:
https://www.reddit.com/dev/api/#POST_api_del

Thanks so much for catching and fixing this!


Decide on enabling persistent storage by default for Grafana

While we should promote immutable approach, with fully automated dashboards (via ConfigMaps etc.), it might be annoying for the users, which create their dashboards by hand to lose them, when Grafana Pod is being removed.
We should consider enabling persistence for Grafana by default and document our recommendation:

that we don't enable it, so users must be careful. Perhaps some reference to the guide of how to create automated dashboards will be then useful
that we enable it for convenience, but we highly encourage users to disable it. Again we should point to the guide of how to automate them

We have a how to guide that explains how to create dashboards with configmaps: https://kinvolk.io/docs/lokomotive/0.8/how-to-guides/monitoring-with-prometheus-operator/#add-custom-grafana-dashboards. Closing.


Cancel ongoing runs of long-running workflows under a PR when new changes are pushed (OSOE-743)

This is particularly interesting for long-running workflows (project/solution builds) to reduce resource consumption, but small things add up for the shorter ones (e.g., spell-checking) as well. Example: When a commit in a PR triggers the solution build and another commit is pushed while the previous one's build is still running, we can cancel the now outdated build using concurrency groups - this is directly from the documentation:
concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

Untested idea/theory: We can take it one step further and make it optional by using an intermediary input parameter and the fallback value:
concurrency:
  group: ${{ inputs.concurrency-group || github.run_id }}
  cancel-in-progress: true

on:
  workflow_call:
    inputs:
      concurrency-group: ${{ github.workflow }}-${{ github.ref }}

When concurrency-group is passed in by the caller workflow as an empty string (or some unique value), then github.run_id (which is globally unique) or that value will take over and make the concurrency group have no effect.
Jira issue

I frequently push to have multiple builds in flight to test out multiple things so I wouldn't welcome this being always the case. dev builds should not cancel out each other either. Optionally, sure.

Also, we could have the option on by default.

It's meant to run for PRs only, yes, see the description. Also, Copilot helped confirming that it can be made fully dynamic, so I'm updating the examples, it should be fairly simple overall.

Done in https://github.com/Lombiq/GitHub-Actions/pull/296.


responsiveness of the website fixed

Closes #557

Description
I have fixed the website responsiveness and  now it is completely responsive , it will work fine on almost all devices.
Screenshots

hi @JasonDsouza212 I have created a pull request for this issue , had to do a lot of work on this , as the css written for this
section was very wrong and I had fixed the css as well as  made it responsive  , made navbar icons responsive as well
and the footer as well. Thankyou...

The design is not proper and has many bugs.

@JasonDsouza212 done

As the issue was of high priority, it has been taken care of.

Thank you for your time

@JasonDsouza212 will I get no points for this one ??

@JasonDsouza212 The UI is still not responsive


TLS certificate on federation server is not valid

There are two problems with the TLS cert served by the federation server started by federation.NewServer:

it is not signed by the complement CA
The subject lacks a CN (which should be host.docker.internal)

For reference, the cert looks something like:
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            ba:6e:e1:1d:07:7f:4d:a8:27:a6:b8:9d:e4:a1:23:58
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: C = GB, ST = London, L = London, street = 123 Street, postalCode = 12345, O = matrix.org
        Validity
            Not Before: Apr  6 21:50:06 2022 GMT
            Not After : Apr  6 22:50:06 2022 GMT
        Subject: C = GB, ST = London, L = London, street = 123 Street, postalCode = 12345, O = matrix.org
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                RSA Public-Key: (4096 bit)
                Modulus:
                    00:cc:6f:2a:8c:2f:07:d1:51:d7:5d:cf:aa:52:6a:
                    38:74:83:91:87:09:9d:ce:9c:34:02:3b:ca:b8:34:
                    80:64:07:ab:d0:d3:35:65:4d:2b:84:cd:8c:aa:34:
                    e2:3f:06:2d:f7:16:f8:7b:c7:ee:db:3c:4d:d4:ca:
                    2f:aa:d1:20:2b:0f:f9:d9:43:1a:c7:53:47:aa:47:
                    9c:86:00:27:8b:b6:36:d9:2d:c4:d6:4c:2b:b8:ca:
                    2b:ec:e1:11:bc:4a:43:04:1c:2d:09:d8:8c:92:a6:
                    0c:4a:e8:f5:d8:cb:c5:9a:d6:6f:a9:fd:ce:17:6f:
                    91:00:77:52:f6:34:13:e3:48:55:2a:d3:e7:25:c3:
                    a0:be:ba:5b:fe:e8:d1:de:4f:f7:f0:12:fe:c8:41:
                    0a:a7:7f:65:d1:4b:76:16:c7:91:6f:ae:f1:91:16:
                    1f:22:ca:dc:c8:9c:20:ea:cc:cc:a9:a1:28:8f:27:
                    d0:90:68:f3:de:28:af:22:05:12:aa:22:1b:c8:4c:
                    27:57:ce:4b:fd:93:d9:f0:ff:98:e1:d0:95:d0:d6:
                    36:12:04:c1:90:88:41:9b:d1:45:2e:9b:52:41:9d:
                    83:48:42:eb:bf:3f:5d:9e:eb:2c:a3:32:a2:d1:9b:
                    3c:3e:e1:bc:cf:ef:af:6f:7f:19:5c:05:d9:a4:c0:
                    52:d9:ec:87:f6:3e:a3:47:a8:06:bb:80:b5:61:17:
                    b6:56:b9:74:fd:32:b1:4f:69:1f:ef:c4:5e:f1:f6:
                    22:b5:56:9e:20:00:27:d0:c3:a3:30:9b:1b:7d:f3:
                    af:40:2c:1b:82:e4:d7:23:38:04:ee:b6:06:49:27:
                    bc:6e:04:16:f1:33:f6:cc:0f:00:44:93:f9:4d:4d:
                    b1:4d:78:a7:71:40:a0:bf:ad:77:68:ae:6e:9b:83:
                    e0:7c:c5:6a:fd:75:ee:d6:35:da:7b:90:dd:d3:66:
                    0c:e7:8a:56:57:1d:74:17:c3:be:2f:41:53:e5:d9:
                    13:3c:88:a3:c2:0e:4e:9e:b8:b9:15:6b:18:aa:f1:
                    d5:76:15:dc:5e:ca:2a:aa:24:ca:74:d8:10:bc:a1:
                    81:2c:8d:91:ca:82:c5:0e:7b:d4:89:2f:83:60:d9:
                    49:81:6f:ac:f5:ec:66:81:65:4b:f0:fe:bd:e5:bf:
                    13:1b:e5:f2:3a:ba:6f:77:67:ef:ff:2c:d9:db:da:
                    bf:a3:0c:65:27:8e:52:af:56:03:cc:47:09:33:9c:
                    7b:2e:66:ed:41:0f:34:6e:cf:b2:37:28:50:b9:57:
                    af:e6:f1:76:ea:7f:41:a1:58:df:40:a1:ea:99:d7:
                    4d:99:38:de:ec:e3:27:fe:85:89:1e:69:1c:46:7d:
                    51:52:13
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Key Usage: critical
                Digital Signature, Key Encipherment
            X509v3 Extended Key Usage:
                TLS Web Server Authentication
            X509v3 Basic Constraints: critical
                CA:FALSE
            X509v3 Subject Alternative Name:
                DNS:host.docker.internal
    Signature Algorithm: sha256WithRSAEncryption
         22:2d:0c:4a:af:08:d2:3f:b6:8d:c7:d8:58:60:24:cc:2d:5f:
         a6:7e:9d:53:da:eb:8a:7e:50:2b:53:06:1e:45:f4:80:a6:28:
         79:90:0d:17:74:90:cc:f6:8e:01:d3:be:9d:91:63:f6:17:88:
         24:ec:54:5a:24:82:9e:cd:ee:1a:bb:a3:04:ec:8a:48:11:4b:
         59:4b:cb:3f:8f:4f:3d:db:2b:25:e9:f0:5d:ec:e5:99:7d:4b:
         39:04:ec:72:a0:5e:7c:dc:e6:46:a9:59:9f:1b:f4:4f:a7:4a:
         28:ca:46:eb:34:aa:c7:5f:2b:31:05:be:e1:37:a5:42:3f:f0:
         ad:c4:00:6e:fb:aa:54:22:d1:5f:9b:02:e7:02:55:26:62:b9:
         7e:35:0b:88:d6:59:bf:d2:e7:f6:29:f8:33:7b:58:e2:93:80:
         d9:bc:35:6a:7e:9e:2e:8d:c5:44:59:92:09:7b:31:42:fb:b3:
         46:65:5b:42:0a:91:00:27:95:4c:b1:67:db:1e:1d:a7:b0:23:
         68:09:0b:f6:23:66:90:a0:de:ac:15:a7:2b:af:bd:bb:90:4d:
         5a:11:b0:02:14:5b:9a:84:de:d0:5b:f3:4b:c1:09:eb:f8:88:
         eb:48:cf:59:53:71:c5:c2:a6:a0:80:24:67:32:5c:a5:68:94:
         20:69:b5:2f:17:9e:a0:f8:41:07:9c:59:ef:25:8c:44:1b:68:
         b1:50:6c:58:cd:e4:d2:3e:85:25:83:c8:32:d2:bb:ef:2f:4d:
         0c:06:d7:10:60:f2:4f:2d:d2:81:63:90:6c:3d:21:2b:10:5e:
         0c:c4:31:45:b5:cd:e1:e2:5c:e3:a7:ac:0c:26:12:64:07:e2:
         3c:fb:33:98:81:ae:0d:c6:1a:51:a3:f9:3b:2e:1f:14:c2:f0:
         0d:1e:ed:5c:a2:07:91:54:b5:41:20:a0:36:13:51:eb:69:fd:
         18:06:b5:73:22:1f:6a:5a:40:a2:36:0d:7b:d9:f9:9d:cf:d5:
         04:a0:82:20:5d:f0:ef:a0:7e:ab:b1:78:20:1e:a5:65:9d:da:
         e5:26:c5:46:b9:a0:20:ad:c8:e2:19:a0:fb:32:09:2b:58:ac:
         d6:87:ec:73:ef:bf:07:b7:ac:da:e7:65:1f:99:5b:1c:fa:b1:
         fd:75:42:82:d5:ef:4b:2f:a5:1e:df:6f:1d:e1:04:bb:c8:72:
         be:e0:3a:2e:3c:18:7e:34:01:f9:90:55:3a:0f:84:66:a4:3a:
         70:da:05:bc:cf:d0:59:5d:d8:5f:bd:a7:11:6e:c2:d9:e7:a2:
         cb:ae:57:48:82:63:c8:99:ce:12:46:2b:93:46:25:ef:6f:d7:
         b3:71:21:a4:31:22:4a:cc
-----BEGIN CERTIFICATE-----

I obtained this by:

adding a defer time.Sleep(time.Hour) (after any defer cancel()) to a federation test (specifically I used TestEventAuth), and running that test
Inspecting the HS output with docker logs complement_fed_1_fed.one_to_one_room.hs1_1, to get the port number it's trying to connect to.
docker exec complement_fed_1_fed.one_to_one_room.hs1_1 openssl s_client -connect host.docker.internal:33381 -showcerts | openssl x509 -text

it is not signed by the complement CA

Apologies, this was incorrect. It is signed by the CA; however the output from openssl x509 is confusing because the Subject of both certificates is the same ("C = GB, ST = London, L = London, street = 123 Street, postalCode = 12345, O = matrix.org") - in particular the CA certificate doesn't identify itself as such; hence, the cert looks like a self-signed one.


perf: make to_py_scalar 3x faster when argument is str

pretty consistent difference here:
main:
In [2]: %timeit nw.to_py_scalar('foo')
468 ns ± 67.9 ns per loop (mean ± std. dev. of 7 runs, 1,000,000 loops each)

In [3]: %timeit nw.to_py_scalar('foo')
338 ns ± 37.6 ns per loop (mean ± std. dev. of 7 runs, 1,000,000 loops each)

here:
In [1]: import narwhals as nw

In [2]: %timeit nw.to_py_scalar('foo')
154 ns ± 23.6 ns per loop (mean ± std. dev. of 7 runs, 10,000,000 loops each)

In [3]: %timeit nw.to_py_scalar('foo')
131 ns ± 6.37 ns per loop (mean ± std. dev. of 7 runs, 10,000,000 loops each)

What type of PR is this? (check all applicable)

[ ] 💾 Refactor
[ ] ✨ Feature
[ ] 🐛 Bug Fix
[ ] 🔧 Optimization
[ ] 📝 Documentation
[ ] ✅ Test
[ ] 🐳 Other

Related issues

Related issue #
Closes #

Checklist

[ ] Code follows style guide (ruff)
[ ] Tests added
[ ] Documented the changes

If you have comments or can explain your changes, please do so below.

@EdAbati fancy taking a look?

really useful comments here, thanks both for excellent reviews 🙏


Resolves issue #130

Pull Request Title

Related Issue

#130
Description

Type

[x] Bug Fix
[ ] Feature Enhancement
[ ] Documentation Update
[ ] Code Refactoring
[ ] Other (please specify):

Proposed Changes

Screenshots / Code Snippets (if applicable)

How to Test

Checklist

[x] The code compiles successfully without any errors or warnings
[x] The changes have been tested and verified
[x] The documentation has been updated (if applicable)
[x] The changes follow the project's coding guidelines and best practices
[ ] The commit messages are descriptive and follow the project's guidelines
[ ] All tests (if applicable) pass successfully
[ ] This pull request has been linked to the related issue (if applicable)

Additional Information

I've made the changes, I could install qdrant_client by keeping the version > 1.2 . If anyone gets an issue I think its better to install using pip and specifying the version
pip install 'qdrant-client>=1.2' --force-reinstall

There are still JSON changes being proposed here for some reason

There are still JSON changes being proposed here for some reason

That shouldn't be a problem, you can take reference from the README.md in the commit .


Updates

Transferred logic to entities
Added settings architecture

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.0 out of 2 committers have signed the CLA.:x: prostoleonid:x: Gid733You have signed the CLA already but the status is still pending? Let us recheck it.


sentence case and style uppercase headings

e.g. here https://github.com/bvn-architecture/styleguide/blob/gh-pages/_includes/cart-philosophies/dimensioning/00-block.markdown

I tried to close this like a spazz with 198


More jest integration

First of all I would like to thank you for your work!
It allowed me to decreased snapshot file size from 10k lines to 1.3k!
Add custom matcher
e.g. https://github.com/americanexpress/jest-image-snapshot#usage
See https://facebook.github.io/jest/docs/expect.html#expectextendmatchers
Usage:
expect(nextState).toMatchDiffSnapshot(prevState, { contextLines: 2 })

Add snapshot serializer based on input type
e.g. https://github.com/prettier/prettier/blob/master/tests_config/raw-serializer.js
Current output for object diffs is too noisy, so I had do implement custom serializer in my project, and now output changed  from:
 Object {
   \\"bail\\": false,
   \\"context\\": \\".\\",
   \\"devServer\\": Object {
     \\"disableHostCheck\\": true,
-    \\"host\\": \\"localhost\\",
+    \\"host\\": \\"local.host\\",
     \\"hotOnly\\": true,
     \\"inline\\": true,
     \\"noInfo\\": true,
     \\"overlay\\": Object {
       \\"errors\\": true,
       \\"warnings\\": true,
     },
     \\"port\\": 3000,

To:
 Object {
   "bail": false,
   "context": ".",
   "devServer": Object {
     "disableHostCheck": true,
-    "host": "localhost",
+    "host": "local.host",
     "hotOnly": true,
     "inline": true,
     "noInfo": true,
     "overlay": Object {
       "errors": true,
       "warnings": true,
     },
     "port": 3000,

I would love to add PR with this changes.

Hey! Thanks for posting. And I'd love to review the PR :)
cc @ferrannp

As a side note, I forgot about custom matcher and serializer but this seems like a fine way to go.
Instead I thought about fixing the issue upstream in Jest – here's a proposal https://github.com/facebook/jest/pull/4183.

@thymikee should I close this and wait for https://github.com/facebook/jest/pull/4183 or proceed to second step?

I'd go with snapshot serializer now, and switch to facebook/jest#4183 when it's merged and new Jest is released. You can mimic the output I guess.

Fixed with #12


[Icon Requests from Feather] boat / ship

As per https://github.com/lucide-icons/lucide/issues/119 and https://github.com/feathericons/feather/issues/631:

@karsa-mistmere I think the left one is not really in the "lucide" style. The left sail of the second one looks a bit too dense i think.

Yeah, you're probably right in both cases, here's a looser version of sailboat:

Suggesting a few modifications:

If the two sails are distinguished from each other by their size (which means the boat now has a moving direction from right to left), I feel it would be nice to distinguish the front and the rear of the boat as well. The front is pointier, the rear is usually dull or flat.
I also adjusted the size of the larger sail to have nicer proportions.

![sailboat](https://user-images.githubusercontent.com/236343/170493309-d7b6e1ab-820d-4956-845d-51c37a699007.svg)


Add documentation and targets to help build project locally for macOS

This adds documentation and attempts to clean up some of the scripting (phase 1)

/hold

/hold cancel


Added new reference / Added Hardlink support / Performance considerations

ReFS can now support Hardlinks under certain conditions
added ReFS version reference

About the link in the last line of the page:

ReFS Version history (unofficial) (https://gist.github.com/0xbadfca11/da0598e47dd643d933dc#file-windows-refs-versions-en-md)

Is that a temporary solution, until a docs.microsoft.com page is created containing the same information?

@jasongerend  Please review these changes and let  me know when this is ready to publish. Thanks!

@illfated yes it is. So far there no changelog or matrix like this for ReFS officially.
Of course it we might think about such a paragraph and it would be helpful.

@Karl-WE : Thanks for your contribution! The author(s) have been notified to review your proposed change. @gawatu

@Karl-WE : Will you be able to resolve the file conflict?
Page location & history sources:

MS Docs: https://docs.microsoft.com/en-us/windows-server/storage/refs/refs-overview
GitHub: https://github.com/MicrosoftDocs/windowsserverdocs/blob/master/WindowsServerDocs/storage/refs/refs-overview.md
History: https://github.com/MicrosoftDocs/windowsserverdocs/commits/master/WindowsServerDocs/storage/refs/refs-overview.md

Looks like you have contributed quite a lot to this page yourself since this PR was created, do you know if any of your previous commits overlap the changed content in this PR?

@Karl-WE : Thanks for your contribution! The author(s) have been notified to review your proposed change. @gawatu

@illfated @gawatu the conflict has been resolved. I have also removed the mention of Windows 10 Pro for ReFS 3.5 as this version is not able to newly format ReFS drives. I hope the current version is now clear and ready to merge.

@Karl-WE : Thanks for your contribution! The author(s) have been notified to review your proposed change. @gawatu

@illfated  @gawatu this PR has a subsequent optional PR I did not merge it intentionally to give you time for review.
https://github.com/Karl-WE/windowsserverdocs/pull/1
I am not sure how I could mention / point you to this otherwise. Mention was not possible from there.

Fair enough, thank you for telling me about the PR upon which this PR relies. I will make a comment in the prioritized PR to keep track of it.

@Karl-WE : Thanks for your contribution! The author(s) have been notified to review your proposed change. @gawatu

@Karl-WE : Thanks for your contribution! The author(s) have been notified to review your proposed change. @gawatu

now contains new chapter performance considerations that i previously handled in a private branch for discussion. as suggestion by @illfated I merged it with #4092 as it is unlikely the other branch would be read, due to time contraints from the docs team.
for more information on the idea of adding this whole new chapter please read my notes here
https://github.com/MicrosoftDocs/windowsserverdocs/pull/4092/commits/13175bbaca89de05e9477850d8c35efc7d6d63d0

@Karl-WE : Thanks for your contribution! The author(s) have been notified to review your proposed change. @gawatu

@Karl-WE : Thanks for your contribution! The author(s) have been notified to review your proposed change. @gawatu

@Karl-WE : Thanks for your contribution! The author(s) have been notified to review your proposed change. @gawatu

@Karl-WE : Thanks for your contribution! The author(s) have been notified to review your proposed change. @gawatu

@Karl-WE : Thanks for your contribution! The author(s) have been notified to review your proposed change. @gawatu

@Anhansen @nedpyle  Can either of your confirm whether this ready to publish? Or if not, point me to who can make the call. Thanks!

@Karl-WE : Thanks for your contribution! The author(s) have been notified to review your proposed change. @gawatu

@Karl-WE : Thanks for your contribution! The author(s) have been notified to review your proposed change. @gawatu

@eross-msft with help from @illfated, I think from our end we have finished polishing the grammar and typos.

Seems that none of these folks are around anymore, and the entire file is a merge conflict.


WIP: include internal introspection types

WIP to solve this issue: https://github.com/dotansimha/graphql-code-generator/issues/643
So there are 2 issues here:

[X] We need to keep types starts with __
[ ] We need to make the introspection types available.

So far, I managed to keep the introspection types in the schema, and added a cli flag for it.
My problem now is with matching the __schema type inside Query type.
When the codegen builds the schema context, in this line: https://github.com/dotansimha/graphql-code-generator/blob/master/packages/graphql-codegen-core/src/utils/get-field-def.ts .
When we are running parentType.getFields, it returns only the regular Query fields, without the introspection fields, so when we are trying to do parentType.getFields()[name], it resolves to undefined.
@FredyC @kamilkisiela @ardatan - any ideas? :)

@dotansimha Is this still going? How can I generate this with a workaround?


Doesn't work with "Tsukihime -A piece of blue glass moon-"

Hi, I installed and tried the patch, it worked with Smash Ultimate but for this game it didn't, the game came out yesterday (august 25) I've read the other open issue so, here's the info if anyone can take a look, thanks
Name: 月姫 -A piece of blue glass moon-
TID: 01001DC01486A000
Update: v65536 (<IP_ADDRESS>) (01001DC01486A800)

Finally got some time to work on this, i may have an idea on how to fix this but i don't have the game to test, if you're still interested could you test a new patch for me ? Need to know your firmware version.

Sure thing, i'm using Firmware version 12.1.0 and AMS 1.0.0

Can you try if this patch works ? extract and replace the one with the same name you already have then reboot the console.

I deleted the old "am" folder and replaced with this patch you send me and IT WORKED!, tested it with Tsukihime and Smash Ultimate, worked on both, thanks

Great, will make a proper release tomorrow then.


Added Single method and tests.

Added Single methods and test, reusing Where().

Closing this one as I have to clean up my fork. Will open another one with the changes.


change 'Export YAML' to 'Download YAML'

Minor fix for export custom roles

LGTM. Is this live and ready to merge?


Weird issue when parent re-renders with map child.

Haven't setup a demo project to reproduce as it seems very easy to trigger.
Basically, i have some complex map drawing logic setup but it seems every time i rerender the parent of my map component (through changing state) the map will zoom back to its original center/zoom levels regardless of where the user has moved the map.
TL;DR
Load Map.
Move Map zoom/center point.
Trigger rerender in parent.
Map will automatically re-zoom / re-center on original center point (undesired)
Any ideas on how to fix?

Are you providing zoom and center values to the Map component? If so, are you keeping them up to date using events such as onZoomEnd so that they don't revert when you the component re-renders?
Can you provide a code sample of how you are using the react-mapbox-gl component? I think it would be helpful to fully understand what default values you are providing as well as what events you are listening to.

how can i make zoom/center controlled using onZoomEnd? where on that object is the current zoom level (same with center).
Don't have time to make demo right now maybe in a few days slammed atm with this geosspatial project

Are you setting zoom/center on the MapGL component? If so, you need to use events to update the values you are using. For example:
        <MapGl
          center={[viewport.longitude, viewport.latitude]}
          zoom={[viewport.zoom]}
          onMoveEnd={this._onViewportChange}
          onZoomEnd={this._onViewportChange}
          onPitchEnd={this._onViewportChange}
        >

Then in the function _onViewportChange, you'd need to do something like:
  _onViewportChange = (updated) => {
    setViewport({
      ...viewport,
      width: updated._container.clientWidth,
      height: updated._container.clientHeight,
      longitude: updated.getCenter().lng,
      latitude: updated.getCenter().lat,
      zoom: updated.getZoom(),
    });
  };

This will then update the state to reflect the new values and if a re-render triggers, will use the updated values.


allow the alignment of the snackbar to be configured

I've added some styled properties to SnackbarHost to allow specifying the alignment of the snackbar. By default it's in the bottom left but this will allow a user to move it around, for example to the bottom right, or top center.

Thanks for contributing


No Scripts Exported!!!!

Whenever I use UtinyRipper, It exports everything nicely EXCEPT for the scripts!  For some reason, the scripts don't export at all, even though they show as MonoScripts in the UI before exporting.
This is so annoying, because now I can't see the different things that are supposed to happen within the decompiled scenes! (I've been exploring games made in Unity for learning purposes)

MonoScript is an asset type name. It contains meta information about script, such as assembly name, namespace and script name. It doesn't contain actual script data. Google scripting backend (Mono/.NET and Il2Cpp) to learn how unity work with scripts. Ripper work only with Mono but you can generate stub Mono assemblies with Il2CppDumper