Brave browser not discoverable

I'm running Brave on my Windows computer (Version 1.26.67, Chromium: 91.0.4472.114) as well as Android phone (Version 1.25.73, Chromium: 91.0.4472.106). But both my devices cannot discover each other in sharedrop. I opened the site in Google Chrome on both devices and they didn't have any trouble. I checked my public IP on both devices through Brave on some site and it shows same IP address.
I have seen that Brave isn't mentioned in supported browsers but since both Edge and Google Chrome seems to work fine, I was expecting Brave too. They all are Chromium.
Is this a bug? Can it be resolved?

Hey!
Just to clarify - the devices can see each other when you use Chrome on both, but don't when you use Brave on one of them?
Does it work when you use Brave and Chrome on the same WIndows device?
Does it work if you create a custom room and open it on both devices?

I tried to open both browsers on both the devices and they can discover each other just fine. I don't know why I couldn't reproduce this bug today. Initially not all users seem to discover but then I refreshed the pages and they could discover each other.
You can close this issue.


HDFS-17001. Support getStatus API in WebHDFS

JIRA: HDFS-17001. Support getStatus API in WebHDFS

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 35s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
18m 12s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
19m 36s

trunk passed

+1 :green_heart:
compile
5m 14s

trunk passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1

+1 :green_heart:
compile
5m  5s

trunk passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09

+1 :green_heart:
checkstyle
1m 19s

trunk passed

+1 :green_heart:
mvnsite
2m 56s

trunk passed

+1 :green_heart:
javadoc
2m 32s

trunk passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1

+1 :green_heart:
javadoc
3m 18s

trunk passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09

+1 :green_heart:
spotbugs
6m 53s

trunk passed

+1 :green_heart:
shadedclient
20m 33s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 28s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
2m 20s

the patch passed

+1 :green_heart:
compile
5m  9s

the patch passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1

+1 :green_heart:
javac
5m  9s

the patch passed

+1 :green_heart:
compile
4m 50s

the patch passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09

+1 :green_heart:
javac
4m 50s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
1m  6s
/results-checkstyle-hadoop-hdfs-project.txt
hadoop-hdfs-project: The patch generated 1 new + 154 unchanged - 0 fixed = 155 total (was 154)

+1 :green_heart:
mvnsite
2m 30s

the patch passed

+1 :green_heart:
javadoc
2m  1s

the patch passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1

+1 :green_heart:
javadoc
2m 52s

the patch passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09

+1 :green_heart:
spotbugs
6m 49s

the patch passed

+1 :green_heart:
shadedclient
20m 35s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 22s

hadoop-hdfs-client in the patch passed.

-1 :x:
unit
207m 25s
/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
hadoop-hdfs in the patch passed.

+1 :green_heart:
unit
20m 58s

hadoop-hdfs-rbf in the patch passed.

+1 :green_heart:
asflicense
0m 51s

The patch does not generate ASF License warnings.

370m 23s

Reason
Tests

Failed junit tests
hadoop.hdfs.server.datanode.TestDirectoryScanner

Subsystem
Report/Notes

Docker
ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5628/2/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/5628

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint

uname
Linux b33ff0c007b5 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3 19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 945c5453390bb23845404327e9f7d6cec8a1211f

Default Java
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5628/2/testReport/

Max. process+thread count
3209 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5628/2/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 34s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
16m 21s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
19m 46s

trunk passed

+1 :green_heart:
compile
5m 15s

trunk passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1

+1 :green_heart:
compile
5m  3s

trunk passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09

+1 :green_heart:
checkstyle
1m 21s

trunk passed

+1 :green_heart:
mvnsite
2m 51s

trunk passed

+1 :green_heart:
javadoc
2m 33s

trunk passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1

+1 :green_heart:
javadoc
3m 13s

trunk passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09

+1 :green_heart:
spotbugs
6m 54s

trunk passed

+1 :green_heart:
shadedclient
20m 35s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 30s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
2m 22s

the patch passed

+1 :green_heart:
compile
5m  4s

the patch passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1

+1 :green_heart:
javac
5m  4s

the patch passed

+1 :green_heart:
compile
4m 52s

the patch passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09

+1 :green_heart:
javac
4m 52s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
1m  6s

the patch passed

+1 :green_heart:
mvnsite
2m 28s

the patch passed

+1 :green_heart:
javadoc
1m 59s

the patch passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1

+1 :green_heart:
javadoc
2m 52s

the patch passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09

+1 :green_heart:
spotbugs
6m 49s

the patch passed

+1 :green_heart:
shadedclient
20m 38s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 23s

hadoop-hdfs-client in the patch passed.

+1 :green_heart:
unit
202m 52s

hadoop-hdfs in the patch passed.

+1 :green_heart:
unit
20m 57s

hadoop-hdfs-rbf in the patch passed.

+1 :green_heart:
asflicense
0m 53s

The patch does not generate ASF License warnings.

363m 51s

Subsystem
Report/Notes

Docker
ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5628/3/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/5628

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint

uname
Linux 76df79e93db0 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3 19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 343e54bdd321d346a8844bf975a212e6cfacd2da

Default Java
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5628/3/testReport/

Max. process+thread count
3562 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5628/3/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@ayushtkn @slfan1989 Would you be so kind as to review my pull request, please?   Thank you very much!

@zhtttylz Thanks for the contribution! We need to modify the code as suggested by @ayushtkn.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 36s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
detsecrets
0m  1s

detect-secrets was not available.

+0 :ok:
markdownlint
0m  1s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
16m 19s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
19m 53s

trunk passed

+1 :green_heart:
compile
5m 13s

trunk passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1

+1 :green_heart:
compile
5m  5s

trunk passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09

+1 :green_heart:
checkstyle
1m 20s

trunk passed

+1 :green_heart:
mvnsite
2m 54s

trunk passed

+1 :green_heart:
javadoc
2m 33s

trunk passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1

+1 :green_heart:
javadoc
3m 18s

trunk passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09

+1 :green_heart:
spotbugs
6m 55s

trunk passed

+1 :green_heart:
shadedclient
20m 28s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 29s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
2m 18s

the patch passed

+1 :green_heart:
compile
5m 11s

the patch passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1

+1 :green_heart:
javac
5m 11s

the patch passed

+1 :green_heart:
compile
4m 55s

the patch passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09

+1 :green_heart:
javac
4m 55s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
1m  6s

the patch passed

+1 :green_heart:
mvnsite
2m 28s

the patch passed

+1 :green_heart:
javadoc
2m  2s

the patch passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1

+1 :green_heart:
javadoc
2m 51s

the patch passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09

+1 :green_heart:
spotbugs
6m 53s

the patch passed

+1 :green_heart:
shadedclient
21m 20s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 21s

hadoop-hdfs-client in the patch passed.

-1 :x:
unit
208m 33s
/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
hadoop-hdfs in the patch passed.

-1 :x:
unit
21m  7s
/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt
hadoop-hdfs-rbf in the patch passed.

+1 :green_heart:
asflicense
0m 52s

The patch does not generate ASF License warnings.

370m 21s

Reason
Tests

Failed junit tests
hadoop.hdfs.server.datanode.TestDirectoryScanner

hadoop.hdfs.server.federation.router.TestRouterRPCMultipleDestinationMountTableResolver

Subsystem
Report/Notes

Docker
ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5628/4/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/5628

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint

uname
Linux f7b23d8406e0 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3 19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / b43141a201e722390653e2fc7a48e5f09b63bdb8

Default Java
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5628/4/testReport/

Max. process+thread count
3329 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5628/4/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@ayushtkn @slfan1989 Could you please help review this PR again? The Junit Test error is not caused by our pr.

@ayushtkn Thank you very much for your valuable suggestion. We will create a ticket to add this feature to HTTPFs!

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 33s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
16m 10s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
19m 34s

trunk passed

+1 :green_heart:
compile
5m 16s

trunk passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1

+1 :green_heart:
compile
5m  1s

trunk passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09

+1 :green_heart:
checkstyle
1m 21s

trunk passed

+1 :green_heart:
mvnsite
2m 54s

trunk passed

+1 :green_heart:
javadoc
2m 27s

trunk passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1

+1 :green_heart:
javadoc
2m 52s

trunk passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09

+1 :green_heart:
spotbugs
6m 53s

trunk passed

+1 :green_heart:
shadedclient
20m 42s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 28s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
2m 19s

the patch passed

+1 :green_heart:
compile
5m  5s

the patch passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1

+1 :green_heart:
javac
5m  5s

the patch passed

+1 :green_heart:
compile
4m 58s

the patch passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09

+1 :green_heart:
javac
4m 58s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
1m  7s

the patch passed

+1 :green_heart:
mvnsite
2m 30s

the patch passed

+1 :green_heart:
javadoc
1m 54s

the patch passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1

+1 :green_heart:
javadoc
2m 26s

the patch passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09

+1 :green_heart:
spotbugs
6m 50s

the patch passed

+1 :green_heart:
shadedclient
20m 40s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 23s

hadoop-hdfs-client in the patch passed.

-1 :x:
unit
204m 33s
/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
hadoop-hdfs in the patch passed.

+1 :green_heart:
unit
21m  1s

hadoop-hdfs-rbf in the patch passed.

+1 :green_heart:
asflicense
0m 52s

The patch does not generate ASF License warnings.

364m 20s

Reason
Tests

Failed junit tests
hadoop.hdfs.server.datanode.TestDirectoryScanner

Subsystem
Report/Notes

Docker
ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5628/6/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/5628

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint

uname
Linux 9b141d6440cb 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3 19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / f53d5abb39bf0b50ef6ecc2a25f9a8ad7bb1c43a

Default Java
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5628/6/testReport/

Max. process+thread count
3021 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5628/6/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@slfan1989 Would you kindly assist in reviewing this PR once again?The Junit Test error is not caused by our pr.

@ayushtkn @slfan1989 Thank you for your assistance in reviewing the code!


Update site to new theme - just-the-docs

Important information!
The updated theme uses Actions to deploy the site, so you need to set it in the settings.

go to the Settings tab -> Pages -> Build and deployment, then select Source: GitHub Actions.
if there were any failed Actions, go to the Actions tab and click on Re-run jobs.

The main changes made:

Images moved to the assets folder
main information is placed in the footer (license)
table of contents is now created automatically
general design changes
users can now use search

Other information you are interested in can be found in the documentation

@maks-oleksyuk As you can see I merged the MR but I get the following error during build:

Branch "gh-pages" is not allowed to deploy to github-pages due to environment protection rules.
The deployment was rejected or didn't satisfy other protection rules.

Any thoughts?  I'm looking around at docs to see if I can figure it out.

Ah, I think I got it.  I'm going to try change a file and make sure it builds ok.  I think I might prefer a light theme but I will keep it and see if I grow to love it.  Thanks for all the hard work!  Please add yourself to the attributions section.

hmmm. Still seeing some errors:
https://github.com/selwynpolit/d9book/actions/runs/4709155962/jobs/8352042683

Some settings need to be changed (I wrote about it in the pull request)
go to the Settings tab -> Pages -> Build and deployment, then select Source: GitHub Actions.
This error occurs during the process of publishing a file from GitHub, the current theme uses actions for this

Yeah, I did that.  Then changed it back, then selected Github actions again.  I see this:
Warning:  github-pages can't satisfy your Gemfile's dependencies.
/usr/local/bundle/gems/jekyll-3.9.3/lib/jekyll/theme.rb:84:in `rescue in gemspec': The just-the-docs theme could not be found. (Jekyll::Errors::MissingDependencyException)
	from /usr/local/bundle/gems/jekyll-3.9.3/lib/jekyll/theme.rb:81:in `gemspec'
	from /usr/local/bundle/gems/jekyll-3.9.3/lib/jekyll/theme.rb:19:in `root'

I had a similar error.
If the settings are changed to actions, then only action will be run on the next commit (it works successfully).

Ah, I did another commit and it got better.  ok, now I can finish up the twig chapter.  Lots of work to do.  Is it difficult to try the light theme?

Never mind. I figured it out.  That was easy.

@maks-oleksyuk Do you notice that leading backslashes (e.g. as in \Drupal) show up as highlighted.  See a real example at https://selwynpolit.github.io/d9book/reaching-out#guzzle-post-example
Any ideas how to fix that?

That's how I saw it. I was thinking of trying to add some custom css files to fix it, or changing the code highlighting to a different theme entirely.
I will try to fix it in the near future


Start rewriting the front-end in React

This branch contains the work I've been doing the last 48 hours or so to write a new front-end for smallboard using React. I'm still working on it but I want to share it right now to open it up for comment, to let other people get involved, to make people aware I am doing this, and to try to merge things incrementally.
Why do this? The current front-end works pretty well and I don't want to rush into replacing it until the alternative is stable and mature. With that said, we've had performance issues with the current front-end and while I'm pretty sure a large part of the slowness I experienced during last year's mystery hunt was from the regression fixed in #211 the existing DataTables + TreeGrid libraries are in part just fundamentally slow. Using the data from last year's Mystery Hunt, the actual table rendering currently takes 1.1s on my machine, of which 600ms is from TreeGrid and the other 500ms come from DataTables, and that renders only 50 puzzles. The React version only takes 460ms to render all 175 puzzles, and while that will go up some once I add the remaining functionality and styling, it should still come in a lot faster.
Beyond performance, React is now ubiquitous in modern front-end codebases and should be easier to work with, maintain, and extend. It has good tooling and extensive library support.
This branch currently changes the following:

Adds a new Django app called api with the beginnings of a REST api for smallboard. At the moment this API has two endpoints: one to get info about a hunt and one to get a list of all puzzles of a hunt. It uses Django's sessions authentication but only checks right now that you are logged in.

Adds a new view, at /hunts/react/:id, to render the all puzzles page with React.

Adds some React code to call the API endpoints and use the data to render a barebones table. The table supports expand/collapse and filtering in the same ways as existing smallboard, handles the puzzle ordering and the tree construction itself, and renders all puzzles at once (no pagination).

Adds new dependencies on django rest framework, yarn (for the JS dependencies), babel & webpack (transpilation and bundling), react, and react-table (which I have had to modify/extend to get filtering to work correctly).

Update the Docker configs to handle JS compilation and reloading.

At this point I think the riskiest/most complicated pieces are done and I feel pretty good that this is going to produce something viable, but there are still a lot of tasks to get it production-ready and at parity with or better than the existing front-end:

[ ] Real-time table updates

[ ] Poll the puzzles API every 10s, as the current UI does.
[ ] Make the collapse logic preserve collapse state across updates and not re-expand everything. Current smallboard doesn't do this; it's going to be complicated to get fully correct since we may need to map rows pre-update to rows post-update and that's not always going to be trivial: there may be duplicate rows (puzzles assigned to multiple metas) in different collapsed states, puzzles could be deleted or have their meta relationships change, etc, but we should be able to get the common cases correct at least.

[ ] Port functionality (UI + API)

[ ] Adding a puzzle
[ ] Deleting a puzzle
[ ] Editing a puzzle
[ ] Editing puzzle tags
[ ] Meta assignment
[ ] Answer submission
[ ] Active puzzler display

[ ] Improve UI

[ ] Style everything
[ ] Render the correct table entries: tags, linkify URLs, remove extra debugging columns.
[ ] Display some basic stats (number of puzzles visible/total, num solved/pending/stuck/etc)

[ ] Productionize

[ ] Integrate webpack into the deployment pipeline
[ ] Make the JS prettier
[ ] Add testing, especially for the API
[ ] Update instructions and documentation

For next steps, I'm going to work on the real-time updates and persisting the collapse/expand when they happen, since I think it's the most complicated and will require doing more work with react-table, then some of the UI improvements, then work through porting the other pieces of functionality. At this point a lot of stuff is parallelizable if anyone else is interested in contributing as well. I'm planning on continuing to work on this extensively for the rest of the week and hoping to demo it at our next sync this Sunday.

Thanks Ryan - should we track those outstanding tasks you listed in some new Issues to help facilitate splitting up the work?
Since this PR is already quite large, I assume the plan is to first merge the changes in this PR to get the basic structure, then followup with the PRs for the improvements you listed above - does that make sense here?

Thanks Ryan - should we track those outstanding tasks you listed in some new Issues to help facilitate splitting up the work?
Since this PR is already quite large, I assume the plan is to first merge the changes in this PR to get the basic structure, then followup with the PRs for the improvements you listed above - does that make sense here?

Or is the plan to put all of those changes on this branch/PR? Seems like that might be harder to merge cleanly :open_mouth:

@pcvera PTAL too :smile:

+1 for prettier, +1 for ts

would suggest using some abstraction over webpack if we can get away with it, not sure if that would be more or less difficult to fit into existing pipeline

Thanks for the comments, I'm glad that you guys are interested and invested 😄 I'll try to incorporate some of the feedback.
Re: prettier, it's on the list! There are some other QOL changes that you guys have noticed too which I think are also important, eg I started writing this with class-based components since that's what I'm familiar with but react-table forced me to use hooks so that's why both are there; probably the class one should use hooks for consistency, although i do personally find them much less clear. Similarly while I thought of using typescript I haven't actually used it before and wanted to focus on productivity vs learning a new language (and I was also a little worried that it might make it harder for other people to contribute or fix things).

im with you on typescript raising the difficulty level, so probably off the table. side comment, if you want to make these changes to the react-table library without this "-fixed" business, you could use patch-package

side comment, if you want to make these changes to the react-table library without this "-fixed" business, you could use patch-package

Wow, this looks fantastic, thanks! Will definitely look into it.

Avoiding camel case names is fine too as long as we've got a defined style
IMO. Should we just pick someone's style guide (maybe airbnb or standard
js?) and plug it into prettier?
On Fri, Dec 4, 2020, 3:18 PM rgossiaux<EMAIL_ADDRESS>wrote:

@rgossiaux commented on this pull request.
In hunts/src/hunt-view.js
https://github.com/cardinalitypuzzles/smallboard/pull/237#discussion_r536440003
:

@@ -0,0 +1,161 @@
+import React from "react";

yeah, I've always disliked camelcase in file names but i'll change it
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/cardinalitypuzzles/smallboard/pull/237#discussion_r536440003,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABTQKHMR2B63DBVDHM5W6H3STFU2TANCNFSM4UJZMINQ
.

That’s more of an eslint thing

Which we definitely should have (eslint)

Ok, I tried to address most of the comments above, feel free to take a look.
I did switch to patch-package for patching react-table though it was surprisingly nontrivial-- since we need to patch the dist/ files from the package, i needed to rebuild react-table first to regenerate them, and you wouldn't believe how many dependencies it has (well, you probably would, because that's what the npm world looks like these days).
Very amusingly I needed to patch-package patch-package itself because it doesn't use a big enough buffer to handle the patch.

are you saying it takes running yarn install 2x to have patch package work? i just ran npm install and had it work no problem afaict.

are you saying it takes running yarn install 2x to have patch package work? i just ran npm install and had it work no problem afaict.

You're talking about the Dockerfile? I structured it that way so that adding patches doesn't require re-installing all the npm dependencies from scratch. Docker images are built in cached layers, so if we first copied both package.json and the patches and then ran yarn install in one go, changes to the patches would invalidate the cache for all the node_modules. As it is the second yarn install doesn't do anything besides applying the patches and takes less than a second on my machine. With that said I'm not sure if this is better overall or not but that was the idea at least.

ok yeah figured it was just for the layers

Ok, just added in the live updating logic and got the table to preserve state across updates (our default-expanded behavior meant I had to make another patch to react-table, which defaults to collapsed). I think that should be just about all of the hairy table things done, so going forward should now just be writing standard UI/API code and no more source diving into react-table internals.
I'd like to merge this into master soon since the PR is already kind of large and so people can work on it without having to merge into my branch. We can then address any follow-ups & future work separately. Let me know what you think; @pcvera since you requested changes I'll need you to approve to merge.


Fixing implied internals

Fixes the following:

#6 (Initializer not specifically marked public)
Several methods that were implicitly interpreted as internal

Also bumps podspec to 0.1.1
Note to @jogendra: anything not explicitly marked public or private is interpreted by the compiler as internal :)

this pull request is missing some func that need also to change the access modifier level to be public
like moveCircleBy

I updated the ones that seemed specifically to be public. The project is structured a little weird for my liking, so I was unsure of what else to set explicitly to public

Hi @MatrixSenpai Thank you for opening the PR. I will make the required changes and publish a new version.

@MatrixSenpai I Made a new release with 0.1.1 tag https://github.com/jogendra/BadgeHub/tree/0.1.1


"Limits of Lifetimes" example now compiles

struct Foo;

impl Foo {
    fn mutate_and_share(&mut self) -> &Self { &*self }
    fn share(&self) {}
}

fn main() {
    let mut foo = Foo;
    let loan = foo.mutate_and_share();
    foo.share();
}

The above example, from https://doc.rust-lang.org/stable/nomicon/lifetime-mismatch.html, should now (as of Rust 2018), compile. I guess it is because of NLL, but I am not 100% sure.
Example ran on Rust Playground compiles with Rust 2018 but doesn't on Rust 2015 (with the error explained in the book).

I think this should be fixed by #101.

I see! Closing now.


Correct Service parameterization article - it is outdated

https://jdi-docs.github.io/jdi-dark/#service-parameterization

done


Programmatically set the value of a time-picker not working

Describe the bug
I try to set the value programmatically to a time-picker, however the value is not visible until I click on the input.
To Reproduce
Please see the example here https://codesandbox.io/s/vue3-date-time-picker-demo-forked-7zney?file=/src/components/Demo.vue
Expected behavior
I expect that the value in the field to be updated without necessary clicking on the input.

This is working only if I do a rerender of the component, but is not quite nice.
Please check here:
https://codesandbox.io/s/vue3-date-time-picker-demo-forked-0rfv1?file=/src/components/Demo.vue
Or do I miss something?

The issue was fixed (#20). Please update the component to at least 2.1.8.
On the latest, working fine: https://codesandbox.io/s/vue3-date-time-picker-demo-forked-9h5cd

@Jasenkoo I just did it however still not working plase check here : https://codesandbox.io/s/vue3-date-time-picker-demo-forked-7zney?file=/src/components/Demo.vue

@ekkode Sorry, my bad, I checked on working sandbox. I will fix it in the next patch.

Thanks!
In the meantime I will use the rerender workaround as I said here: https://github.com/Vuepic/vue3-date-time-picker/issues/37#issuecomment-958755398

The issue is fixed in the new version 2.3.3

Hello @Jasenkoo
I think this issue is still present. If you use only timepicker and you have a custom #dp-input slot and then set model value, you have to click into the custom input. Only after that, the value is updated in the input.

@videk11 Will check


[BEAM-664] Update examples archetype

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

[x] Make sure the PR title is formatted like:
[BEAM-<Jira issue #>] Description of pull request
[x] Make sure tests pass via mvn clean verify. (Even better, enable
Travis-CI on your fork and ensure the whole test matrix passes).
[x] Replace <Jira issue #> in the title with the actual Jira issue
number, if there is one.
[x] If this contribution is large, please file an Apache
Individual Contributor License Agreement.

@davorbonaci

Jenkins errors:
[WARNING] The requested profile "apex-runner" could not be activated because it does not exist.
Waiting for Jenkins to finish collecting data
[ERROR] Failed to execute goal org.apache.rat:apache-rat-plugin:0.12:check (default) on project beam-parent: Too many files with unapproved license: 1 See RAT report in<EMAIL_ADDRESS>-> [Help 1]

Note that all new Java is autogenerated, so all comments should be addressed in terms of bugs in the shell script.

LGTM.


[BUG] Nav menu on mobile not opening for homepage (Website)

Is there an existing issue for this? No

[X] I have searched the existing issues

Current Behavior
On the website, , on mobile devices, the hamburger menu button on the homepage doesn't open the sidebar like it's supposed to. It works on every other page except the homepage.
Expected Behavior
It should open the sidebar on the homepage so users on mobile can properly navigate the page. I get that most people are probably visiting the website on desktop considering Altair is a desktop app, but I believe its a bug that should still be fixed, for user experience.
Steps To Reproduce

Open  on a mobile device or responsive design mode on desktop browsers.
Try clicking the hamburger menu while on the homepage

Environment
- OS: MacOS ventura, iOS 17
- Browser: Safari, Firefox
- Platform: Mobile, Desktop
- Version:

Additional context
I am brand new to working on open source, so I was wondering if I can get some guidance on how to run the website on a local host in order to see my changes to the vue files. This seems like a simple issue to solve and I would love to help out with it. But I haven't been able to figure out how to run the webpage on localhost. Also if there's some sort of other communication channel I would love to be apart of that. Thanks - Akim.

Hey! The code for the site is located in https://github.com/altair-graphql/altair/tree/master/site

The site has been revamped and this should no longer be an issue


aws s3 does not provide --output json

Nice tool.
Anyway, the aws.s3 cannot work as it does not provide the --output json option, it outputs only text.
For this reason would propose removing the s3 from supported commands.
It is possible (but not much likely), that there might be other subcommands of this nature (but I am not aware of any more).

Thanks for bringing this issue to my attention @vlcinsky. I think I have a solution for this. I'll take care of it during the week. Basically I'll make __get_base_args function return --output json as an argument only for commands that support it.


Can't use subquery result at on: join

Elixir version
Elixir 1.14.3 (compiled with Erlang/OTP 25)
Database and Version
PostgreSQL 13-alpine
Ecto Versions
ecto 3.9.2, ecto_sql 3.9.1
Database Adapter and Versions (postgrex, myxql, etc)
postgrex 0.16.5
Current behavior
Having a subquery returning a list from an aggregate function, and using its return to write a new join is totally feasible in PostgreSQL. Here's a dummy example (n:n) of a relation Author <-> Pair (book authors) <->  Books
Author
|> join(:left, [a], pair in subquery(pair_query), as: :pair, on: a.id == pair.author_id)
|> join(:left, [a, pair], cp in Pair, on: cp.book_id == any(pair.books)) # <- fails to parse any(pair.books)
|> ...

In this example the subquery pair_query is returning a list of ids: select array_agg(book_id) as books, this works in postgresql, however, the elixir code above throws the following ArgumentError:
** (ArgumentError) invalid expression for join `:on`, subqueries aren't supported
    (ecto 3.9.2) lib/ecto/query/builder/join.ex:210: Ecto.Query.Builder.Join.build_on/7
    (ecto 3.9.2) lib/ecto/query/builder/join.ex:187: Ecto.Query.Builder.Join.build/10
    (ecto 3.9.2) expanding macro: Ecto.Query.join/5

ref: https://github.com/elixir-ecto/ecto/blob/master/lib/ecto/query/builder/join.ex#L215
Expected behavior
As the subquery's join is aliased, it should converge to something like this:
LEFT JOIN pairs ON pairs.book_id = ANY(pair.books)

Hi,
Thanks for the report. The any you are using is actually not the same as Postgres operator. It means something else in Ecto:
Tests whether one or more values returned from the provided subquery match in a comparison operation.

What you want to do should be achievable by something like this:
s = from p in Post, select: %{agg: fragment("array_agg(?)", p.title)}
q = from p in Post, join: s in subquery(s), on: p.title in s.agg

This will generate the SQL you are expecting here:
LEFT JOIN pairs ON pairs.book_id = ANY(pair.books)

Please give it a try and let me know if you still have an issue.

I should clarify, the subquery any is not a new construct by Ecto. It is something Postgres and other databases have: https://www.postgresql.org/docs/current/functions-subquery.html#FUNCTIONS-SUBQUERY-ANY-SOME

Oh, you're right @greg-rychlewski! Thank you so much for the clarification, indeed the query worked like a charm by using the in operator instead :D


SNBT List Tag Float Parsing

Was going to mention, I had to write a script to do the NBT editing, rather than just use SNBT, because it looks like my SNBT parser currently doesn't handle (not sure which it is) either really long lists, which breaks the rest of the parsing, or the use of bigger numbers inside of a List tag, say like highly-precise Float tag values. Something about the SNBT parser reading those isn't working right, and I have to figure out why reading it as SNBT isn't working out correctly. It's not NBTify's binary NBT implementation, because that can read and write the same file no issue, it's just going from the SNBT output back to JS that seems to error. The JS to SNBT output also seems to be working great, so I don't know of any issues here.


[ListItem] Add primaryTextStyle and secondaryTextStyle options

The limitation to secondaryText is not necessary. It makes me more difficult to modify the style of ListItem.

This will be fixed in next with the new style approach. (PS. Please use the issue template when submitting issues.)


I have issue while integrating vercel blobs using server actions.

Hello, there!
Here is my approach to save file.
- frontend

Heading

      Subheading
      <input
        id="subheading"
        name="subheading"
        type="text"
        autoComplete="subheading"
        placeholder="subheading"
        required
        onChange={updateBanner}
        className="block w-full rounded-md border-0 py-1.5 text-gray-900 shadow-sm ring-1 ring-inset ring-gray-300 placeholder:text-gray-400 focus:ring-2 focus:ring-inset focus:ring-indigo-600 sm:text-sm sm:leading-6 px-3"
      />

      Image
      <input
        id="image"
        name="image"
        type="file"
        autoComplete="image"
        placeholder="Image"
        required
        onChange={imageCallback}
        className="block w-full rounded-md border-0 py-1.5 text-gray-900 shadow-sm ring-1 ring-inset ring-gray-300 placeholder:text-gray-400 focus:ring-2 focus:ring-inset focus:ring-indigo-600 sm:text-sm sm:leading-6 px-3"
      />

      Link
      <input
        id="link"
        name="link"
        type="text"
        autoComplete="link"
        placeholder="Link"
        required
        onChange={updateBanner}
        className="block w-full rounded-md border-0 py-1.5 text-gray-900 shadow-sm ring-1 ring-inset ring-gray-300 placeholder:text-gray-400 focus:ring-2 focus:ring-inset focus:ring-indigo-600 sm:text-sm sm:leading-6 px-3"
      />

      CTA
      <input
        id="cta"
        name="cta"
        type="text"
        autoComplete="cta"
        placeholder="CTA"
        required
        onChange={updateBanner}
        className="block w-full rounded-md border-0 py-1.5 text-gray-900 shadow-sm ring-1 ring-inset ring-gray-300 placeholder:text-gray-400 focus:ring-2 focus:ring-inset focus:ring-indigo-600 sm:text-sm sm:leading-6 px-3"
      />

      <Button
        onClick={() => router.back()}
        className="bg-rose-600 text-white px-5 me-3"
        type="button"
        >
        Back
      </Button>
      <Button
        onClick={() => {}}
        className="bg-cyan-400 text-white px-5"
        type="submit"
        >
        Add
      </Button>

  </form>

- backend
const saveBanner = async (formData: FormData) => {
'use server';
const imageFile = formData.get('image') as File;
console.log('here', imageFile.name, imageFile);
const readStream = await imageFile.stream();
const blob = await put(imageFile.name, readStream, {
  access: 'public',
});
console.log('here1')
revalidatePath('/');
return blob;

}
After it, i got this image url but I can't visit the url
https://tq4op7s2k335xqoq.public.blob.vercel-storage.com/wallpaperflare.com_wallpaper (23)-j4BVhNk3RoP2tCjv3zV0E5qVjIMjEu.jpg
Thanks in advance.

Hey @build-green-earth, it seems this Vercel Blob store has been deleted already, did you delete it?

Yes, I already deleted but the url didn't work at that time.

Now that I look again, I see you have literal % in the pathname, this could have been the issue. I'll close for now, if you want to try again let me know. We recommend you normalize pathnames to avoid URL conflicting characters like % or #. We'll probably do this automatically in future versions of Vercel Blob.

Okay, thanks for your reply.
But I couldn't upload images to blob in the vercel admin dashboard.


Port boolean* functions and tests

[ ]  booleanClockwise
[ ]  booleanConcave
[ ]  booleanContains
[ ]  booleanCrosses
[ ]  booleanDisjoint
[ ]  booleanEqual
[ ]  booleanIntersects
[ ]  booleanOverlap
[ ]  booleanParallel
[ ]  booleanPointInPolygon
[ ]  booleanPointOnLine
[ ]  booleanWithin

Inspiration:
https://github.com/mfogel/polygon-clipping

@lukas-h,
what do you think about implementing 'properties' for FeatureCollection class?
During working with unit tests I saw a few occasions where it was used in the geojson files for featureCollections, and I think it could be in some cases beneficial.

@armantorkzaban Are you referring to a properties member for FeatureCollection? I would try to avoid that, since that is not part of the GeoJSON-RFC.

Yes, let's avoid it for now, but in the future, #44 will be implemented.

GitHub projects, I started one, but didn't have the time to complete it. We will have a backlog and sprints in separate boards there.

https://github.com/orgs/dartclub/projects/1

After the discovery another blocking dependencies (rbush) we decided to separated the dependant libraries from this PR and work on them separately. Therefore the task list is shortened and PR will be marked as ready for review.

the issue definition is updated.


Nuke GuardDuty (#175)

Add support for removing GuardDuty as requested in #175
Update README with new functionality

Updated the PR based on your feedback. Let me know if I missed anything.

hey @jdpleiness thanks for raising this PR & addressing the feedback. Do you mind getting it up to speed with the latest from master, and we can hopefully help moving it forward faster?

@ina-stoyanova I happened to implement my own before I realized there was an open PR and since this one is stale by a couple months, I went ahead and opened my PR https://github.com/gruntwork-io/cloud-nuke/pull/239

@ekristen @ina-stoyanova I got sidetracked and forgot about this for a while. It should be back up to speed with master now if you would like to merge it. If you would like to use @ekristen's version of the feature instead, feel free to close close the PR. Either is fine with me.

Hey @jdpleiness & @ekristen thank you both so much for the contributions! I'll take a look as soon as possible, and we can see how we go about merging both! :)

Hey @jdpleiness & @ekristen thank you both so much for the contributions! I'll take a look as soon as possible, and we can see how we go about merging both! :)

just a heads up - that's looking like next week :)

@jdpleiness thanks for raising this PR! I already commented on @ekristen's, but to be transparent - we're currently ramping up someone to help us maintain this repo and hopefully, we'll take a look at merging your PRs and adding the support for Guardduty as soon as possible. This is more likely to be pushed back by another week while the ramping up is still going on.

@ina-stoyanova Are there any updates on this? I know these things take time, but it's been 7 months and other PRs have been merged as well as a new release. It would be great to have support for GuardDuty. I don't see any updates on @ekristen's PR either at this point.


OpenSSL Deprecations

The more recent Ruby documentation for OpenSSL::SSL::SSLContext says:

The cert, key, and extra_chain_cert attributes are deprecated. It is recommended to use add_certificate instead.

Review the gem's SSL support with the idea of addressing these deprecations.

Just a couple of notes for now.
The add_certificate method has been added recently.  Not sure exactly when,  It is in Ruby 2.5.1, but not Ruby 2.3.2.
The gem attempts to support very old Ruby versions, back to and including 1.8.7.  There is no intent to change that.
One strategy might be to use a construct like:
    if ctx.respond_to?(:add_certificate)
        # New code here
    else
        # Old code here
    end

This will result in even more ugly code in netio.rb, but is likely the best implementation route.

OK, done.  See: cd19c71


Exclude muted users from more UI parts.

This deals with-

Typing notifications
PM recipients typeahead
@ mentions typeahead
settings/user-groups typeahead when adding a new member to a user group

Most of this is simple array filtering, and I've tested everything manually as well as written node tests.

@m-e-l-u-h-a-n Since you worked on compose typeahead recently, could you review this?

I went through the changes, and posted a couple of minor comment changes to user_pill.js rest of the changes appear fine 🙂.

Thanks a lot for the review! I replied with the reasoning inline, and also mentioned it in the commit message.
I also added a commit here for the stream edit typeahead, which I had missed earlier.

Thanks for reviewing! Changes in the last push are-

Removed the couple of commits dealing with the stream/user group settings pages.
Added some helpers in muting.js for filtering muted users so that we don't duplicate the same code (and also added a commit to clean up code from buddy data and pm_conversations which had already been merged). I preferred this approach over something like people.get_active_unmuted_users() because many modules use custom data sources, rather than directly doing people.get_active_users().
The typing logic is now in typing_data instead of typing_events and also covered by node tests 🙂
The first two commits are kind of unrelated to the rest of the PR now, but I didn't drop them because I think they are a good cleanup.

This looks good, except maybe we should rename the functions to filter_users_for_muting and filter_user_ids_for_muting?  get I usually expect to not return a subset of what was passed in, but instead query somewhere else.

Changed the function names to filter_muted_users and filter_muted_user_ids

Merged as the series ending with 65249d90e9fc2cac8fbf8b745a5a7012eef90180, thanks @abhijeetbodas2001!
I think we're almost there!


Add placement group to payload

Description 📝
The main concept with this PR is to include the details of the selected Placement Group as part of the Create Linode workflow.
Changes  🔄
List any change relevant to the reviewer.

The LabelAndTagsPanel has been renamed to DetailsPanel since the panel had been updated to include the PlacementGroupSelect component. Accordingly, the Details title has been added to the panel.
If the selected PlacementGroup does not have any capacity, an error Notice is displayed within the panel.
Updates to the Linodes schema to include Placement Group details.

Preview 📷

How to test 🧪
(How to setup test environment)

Enable the following using the Cloud Manager Dev Tools:

Placement Groups feature flag
MSW

Verification steps
Success

Go through the Create Linode workflow and select Newark, NJ (us-east) for the Region.
Select any of the listed Placement Groups.
Complete the remaining fields and click on the Create Linode button.
Verify that the request was successful.

Error

Update the placementGroupFactory by adding an additional element to the linode_ids array. This will make the Placement Group be at full capacity.
Go through the Create Linode workflow as above.
Verify that clicking on the Create Linode button results in an error and the Notice is displayed.

As an Author I have considered 🤔
Check all that apply

[ ] 👀 Doing a self review
[ ] ❔ Our contribution guidelines
[ ] 🤏 Splitting feature into small PRs
[ ] ➕ Adding a changeset
[ ] 🧪 Providing/Improving test coverage
[ ] 🔐 Removing all sensitive information from the code and PR description
[ ] 🚩 Using a feature flag to protect the release
[ ] 👣 Providing comprehensive reproduction steps
[ ] 📑 Providing or updating our documentation
[ ] 🕛 Scheduling a pair reviewing session
[ ] 📱 Providing mobile support
[ ] ♿  Providing accessibility support

The placement_group payload is still sent when the flag is OFF, which we want to avoid

@abailly-akamai
This makes sense, however, if the feature flag is OFF; should the value of placement_group be null or undefined?

@carrillo-erik it should be undefined

@carrillo-erik When the error appears the tooltip icon shifts down. We should remove the tooltip icon here and just pass textFieldProps to <PlacementGroupsSelect />:
textFieldProps={{tooltipText}}

Obviously when you call the underlying <Autocomplete /> component you'll need to spread the props there {...textFieldProps} and maybe even add a tooltipText prop to the <TextField /> component itself in Autocomplete.tsx.


x/tools/gopls: Exhausted failed to load view

Issue
I use multi-module workspace (kinda monorepo). I'm getting an error from gopls that says -
Error loading workspace folders (expected 11, got 9) failed to load view for file:///Users/sram/ram/project: exhausted failed to load view for file:///Users/sram/ram/project/node: exhausted

gopls is looking for my node directory in my project workspace.
Below is the directory structure of my workspace:
my-workspace
~
 | /.config
 | /.cabal/
 | /.nvm/
 | .bash_profile
 | .bashrc
 | /project/
 |  | /Clojure/
 |  |  | /allmycljprojects
 |  | /Shell/
 |  |  | /allmybashprojects
 |  | /rust/
 |  |  | /allmyrustprojects
 |  | /haskell/
 |  |  | /allmyhaskellprojects
 |  | /python/
 |  |  | /allmypythonprojects
 |  | /go/
 |  |  | /bin
 |  |  | /pkg
 |  |  | /src
 |  |  |  | /github.com
 |  |  |  | /gitlab.com
 |  |  |  | /golang.org
 |  |  |  | /gopkg.in
 |  | /scala/
 |  |  | /allmyscalaprojects
 |  | /node/
 |  |  | /allmynodeprojects

Fix
I would want to enable directory filters (directoryFilters) to explicitly say gopls to stop looking at node directory or node_modules in specific project path OR I want gopls to look at only the go directory in my project folder
Could you please help if below is right format? Will below help in specifying gopls to look only at go directory
"gopls": {
     // "experimentalWorkspaceModule": true,
     "directoryFilters": ["+/~/project/go/"],
      ....
  }

Or Would I be able to use
"gopls": {
     // "experimentalWorkspaceModule": true,
     "directoryFilters":["-/Users/s0v00nr/ram/project/node/","-node_modules"],
      ....
  }

to make gopls to ignore all node directory?
***** Other details ******
What version of Go are you using (go version)?

$ go version
go version go1.15.2 darwin/amd64

Does this issue reproduce with the latest release?
Yes. I was seeing this issue even in earlier versions of gopls
What operating system and processor architecture are you using (go env)?
go env
$ go env
GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/sram/Library/Caches/go-build"
GOENV="/Users/sram/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/sram/ram/project/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/sram/ram/project/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/sn/pw7xq23n2vj_wnq16lkgdrxh0000gn/T/go-build150007759=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you expect to see?
No errors from gopls.

I believe you would want something like the following:
"gopls": {
     "directoryFilters":["-/Users/s0v00nr/ram/project","+/Users/s0v00nr/ram/project/go"],
}

if you only want gopls to look at the go directory.
/cc @heschi for confirmation

Thanks! That worked. Below is my current config -
    "gopls": {
        "ui.documentation.hoverKind": "SynopsisDocumentation",
        "build.directoryFilters":["-/Users/sram/ram/project","+/Users/sram/ram/project/go"],
        "build.experimentalWorkspaceModule": true,
     }


updating version to beta.7

:shipit:
note: we should probably switch this from tsd to typings too at some point...


[graphql/rpc] Auto create markdown doc from examples

Description
As discussed with @stefan-mysten this starts the auto doc workflow
It could be prettier but it gets the ball rolling
Test Plan
Manual

If your changes are not user-facing and not a breaking change, you can skip the following section. Otherwise, please indicate what changed, and then add to the Release Notes section as highlighted during the release process.
Type of Change (Check all that apply)

[ ] protocol change
[ ] user-visible impact
[ ] breaking change for a client SDKs
[ ] breaking change for FNs (FN binary must upgrade)
[ ] breaking change for validators or node operators (must upgrade binaries)
[ ] breaking change for on-chain data layout
[ ] necessitate either a data wipe or data migration

Release notes

Overall this looks good, great work. If we could fix all those unwraps (I know, it's annoying), that'd be great.

hmm since its mostly tests and not hot path code I think the unwraps are okay. Creating errors for this will be quite a task.
I'll do some cleanup but wont wanna over-invest in it

Overall this looks good, great work. If we could fix all those unwraps (I know, it's annoying), that'd be great.

Overall this looks good, great work. If we could fix all those unwraps (I know, it's annoying), that'd be great.

hmm since its mostly tests and not hot path code I think the unwraps are okay. Creating errors for this will be quite a task. I'll do some cleanup but wont wanna over-invest in it

All done

Overall this looks good, great work. If we could fix all those unwraps (I know, it's annoying), that'd be great.

Overall this looks good, great work. If we could fix all those unwraps (I know, it's annoying), that'd be great.

hmm since its mostly tests and not hot path code I think the unwraps are okay. Creating errors for this will be quite a task. I'll do some cleanup but wont wanna over-invest in it

All done

Lovely, there are still some anyhow::anyhow! that should be replaced with anyhow! as you already imported the macro.


Floating base to main merge

This pull request merges all floating base changes compatible with main into the main branch of A2R Lab GRiD repo.
Summary

This PR includes updates to each of the GRiD submodules (URDFParser, GRiDCodeGen, RBDReference) as well as several associated testing files. This PR also merges in auto-add-documentation bot, the sphinx documentation generator, and github workflows associated with badges and git pages deployment.
This change is necessary to have all our updated working code in a single branch.

Changes

Updates latest submodules (GRidCodeGen, URDFParser, RBDReference)
Added auto contribution workflows and code to credit commits.
Adds sphinx documentation and workflows
Adds unfinished unit test workflow which needs to be updated. [UNFINISHED]
Updates README with badges and contributors
Updated printGRiD.py and gitmodules to be compatible for merge (includes FILE_NAMESPACE parameter in parse inputs option)****

Testing

Testing remains the same with printReferenceValues.py with updated testGRiD added by @Z4KH for Codegen testing.
In the github actions tag, the workflow progress and status can be monitored for testing.
Submodules and documentation house testing instructions.

Additional Notes

Assuming that submodules are up to date, if there are compatibility issues check version control.

I've gone through and adapted for any needed changes for floating base -> main merge. Resolving for conflicts and doing the PR.


[BUG]:

How are you running AnythingLLM?
AnythingLLM desktop app
What happened?
Good afternoon! Just started with a fresh install of AnythingLLM, downloaded and activated the Llama3.1 8B model and tried to prompt it but keep receiving the following error: "llama runner process has terminated: CUDA error".
I have not been able to find any posts on the web regarding the exact same error. Is there anything I can do to fix this? Note: I have been able to get this model to work in a different tool called GPT4all, so it should work.
extra device info:
Windows 10, using latest installation version of AnythingLLM
GPU = AMD 6650XT

Are there known steps to reproduce?
No response

GPU = AMD 6650XT

We dont currently ship the ROCM bindings for AMD cards (>1GB) so instead of trying to patch this in the app, which will be frustrating you can install Ollama and connect to that as an external LLM.
Not ideal, we know about this bug though and it is tracked elsewhere and will be patched soon


Split off widgets

widgets can be separated from the gui. Could add to iced_aw since it was inspired by a crash with the Card in iced_aw.

Likely just use iced since it seems to behave better in 0.13


Dask hangs in random moment, hard to reproduce

I have quite a big dask graph to compute. It operates on multiple parquet files from s3, processes the data and saves the result to local disk. The problem is, the process can stop in a random moment.
I'm not sure what causes it, so I don't know if I should submit the issue here or in distributed repo.
Setup:

AWS EC2 machine with 64 cores and 500 GB of RAM

docker container running dask-scheduler

docker container running dask-worker [scheduler-address] --nprocs 64 --memory-limit 0.3

docker container running python code, connecting to the scheduler

Python 3.5

dask 0.15.2 (same thing happens with newer version)

distributed 1.18.3 (same thing happens with newer version)

tornado 4.5.2

fastparquet 0.1.2

When the computation starts, I can see progress bars in scheduler dashboard. Everything seems to be fine, tasks are being computed, until suddenly everything stops. CPU usage drops to almost zero, progress bars stop, worker's bokeh page stops responding. When I restart the whole process, it will most probably stop again, but in totally different moment. I couldn't find any rule on when the progress stops.
I would love to include a minimum example, but I have no idea what causes this behaviour. As I said, it may happen in different moments, so I can't even tell whether it's beginning or end of my pipeline. So, I don't know how to reproduce it.
I tried looking at logs. When the computations stop, there's no sign of anything going wrong either in worker or scheduler. After almost 1 hour of doing nothing, last lines from worker and scheduler look like this:
dask-scheduler    | 2017-11-05T18:16:56.929566066Z distributed.scheduler - INFO - Starting worker compute stream, tcp://<IP_ADDRESS>:37075
dask-scheduler    | 2017-11-05T18:16:56.932564278Z distributed.scheduler - INFO - Register tcp://<IP_ADDRESS>:58982
dask-scheduler    | 2017-11-05T18:16:56.933666514Z distributed.scheduler - INFO - Register tcp://<IP_ADDRESS>:41864
dask-scheduler    | 2017-11-05T18:16:56.935390112Z distributed.scheduler - INFO - Register tcp://<IP_ADDRESS>:57315
dask-scheduler    | 2017-11-05T18:16:56.937407410Z distributed.scheduler - INFO - Register tcp://<IP_ADDRESS>:33051
dask-scheduler    | 2017-11-05T18:16:56.939018686Z distributed.scheduler - INFO - Register tcp://<IP_ADDRESS>:39833
dask-scheduler    | 2017-11-05T18:16:56.940121438Z distributed.scheduler - INFO - Register tcp://<IP_ADDRESS>:59492
dask-scheduler    | 2017-11-05T18:16:56.940805054Z distributed.scheduler - INFO - Starting worker compute stream, tcp://<IP_ADDRESS>:58982
dask-scheduler    | 2017-11-05T18:16:56.941496080Z distributed.scheduler - INFO - Starting worker compute stream, tcp://<IP_ADDRESS>:41864
dask-scheduler    | 2017-11-05T18:16:56.941522107Z distributed.scheduler - INFO - Starting worker compute stream, tcp://<IP_ADDRESS>:57315
dask-scheduler    | 2017-11-05T18:16:56.942195523Z distributed.scheduler - INFO - Starting worker compute stream, tcp://<IP_ADDRESS>:59492
dask-scheduler    | 2017-11-05T18:16:56.942201960Z distributed.scheduler - INFO - Starting worker compute stream, tcp://<IP_ADDRESS>:33051
dask-scheduler    | 2017-11-05T18:16:56.942719202Z distributed.scheduler - INFO - Starting worker compute stream, tcp://<IP_ADDRESS>:39833
dask-scheduler    | 2017-11-05T18:16:56.951637056Z distributed.scheduler - INFO - Register tcp://<IP_ADDRESS>:38661
dask-scheduler    | 2017-11-05T18:16:56.952947498Z distributed.scheduler - INFO - Starting worker compute stream, tcp://<IP_ADDRESS>:38661
dask-scheduler    | 2017-11-05T18:16:56.957188880Z distributed.scheduler - INFO - Register tcp://<IP_ADDRESS>:40078
dask-scheduler    | 2017-11-05T18:16:56.958534533Z distributed.scheduler - INFO - Starting worker compute stream, tcp://<IP_ADDRESS>:40078
dask-scheduler    | 2017-11-05T18:16:57.202564703Z distributed.scheduler - INFO - Receive client connection: Client-82484af4-c255-11e7-8082-0242ac130005
dask-scheduler    | 2017-11-05T18:19:44.344417691Z distributed.core - WARNING - Event loop was unresponsive for 1.05s.  This is often caused by long-running GIL-holding functions or moving large chunks of data. This can cause timeouts and instability.
dask-scheduler    | 2017-11-05T18:20:08.078638857Z distributed.core - WARNING - Event loop was unresponsive for 23.08s.  This is often caused by long-running GIL-holding functions or moving large chunks of data. This can cause timeouts and instability.

dask-worker_1     | 2017-11-05T18:21:32.362563969Z distributed.worker - DEBUG - future state: ('read-parquet-b2c5abe77c11876195eb6a77738d1492', 19) - RUNNING
dask-worker_1     | 2017-11-05T18:21:32.362579815Z distributed.worker - DEBUG - Execute key: ('concat-7075105cf1c6acffe24d76c23229da72', 4361) worker: tcp://<IP_ADDRESS>:53559
dask-worker_1     | 2017-11-05T18:21:32.393695151Z distributed.worker - DEBUG - Deleted 1 keys
dask-worker_1     | 2017-11-05T18:21:32.394078549Z distributed.worker - DEBUG - Deleted 1 keys
dask-worker_1     | 2017-11-05T18:21:32.394140791Z distributed.worker - DEBUG - Send compute response to scheduler: ('dropna-de531fe104bb0718a633ac3ba57e44c9', 4505), {'thread':<PHONE_NUMBER>39744, 'status': 'OK', 'op': 'task-finished', 'type': <class 'pandas.core.frame.DataFrame'>, 'key': "('dropna-de531fe104bb0718a633ac3ba57e44c9', 4505)", 'stop':<PHONE_NUMBER>.38893, 'start':<PHONE_NUMBER>.7096455, 'nbytes': 347313008}
dask-worker_1     | 2017-11-05T18:21:32.395095639Z distributed.worker - DEBUG - Execute key: ('astype-fa084f475d61990b4586d370bacd818e', 4324) worker: tcp://<IP_ADDRESS>:44932
dask-worker_1     | 2017-11-05T18:21:32.401399080Z distributed.worker - DEBUG - Send compute response to scheduler: ('assign-1d9eb7afe69be0e3579b6a8be9c6b291', 4455), {'thread':<PHONE_NUMBER>77600, 'status': 'OK', 'op': 'task-finished', 'type': <class 'pandas.core.frame.DataFrame'>, 'key': "('assign-1d9eb7afe69be0e3579b6a8be9c6b291', 4455)", 'stop':<PHONE_NUMBER>.3977563, 'start':<PHONE_NUMBER>.1396115, 'nbytes': 322818495}
dask-worker_1     | 2017-11-05T18:21:32.402049239Z distributed.worker - DEBUG - Execute key: ('astype-fa084f475d61990b4586d370bacd818e', 4398) worker: tcp://<IP_ADDRESS>:59993
dask-worker_1     | 2017-11-05T18:21:32.431883216Z distributed.worker - DEBUG - Send compute response to scheduler: ('read-parquet-b2c5abe77c11876195eb6a77738d1492', 19), {'thread':<PHONE_NUMBER>95360, 'status': 'OK', 'op': 'task-finished', 'type': <class 'pandas.core.frame.DataFrame'>, 'key': "('read-parquet-b2c5abe77c11876195eb6a77738d1492', 19)", 'stop':<PHONE_NUMBER>.4114537, 'start':<PHONE_NUMBER>.2736945, 'nbytes': 365677731}
dask-worker_1     | 2017-11-05T18:21:32.433350396Z distributed.worker - DEBUG - Execute key: ('astype-867f8e9773a47c02dc913a3001779e23', 4419) worker: tcp://<IP_ADDRESS>:47891
dask-worker_1     | 2017-11-05T18:21:32.433365801Z distributed.worker - DEBUG - Send compute response to scheduler: ('astype-955e0ccfc224ca95aa0211b2986fc390', 4434), {'thread':<PHONE_NUMBER>46784, 'status': 'OK', 'op': 'task-finished', 'type': <class 'pandas.core.series.Series'>, 'key': "('astype-955e0ccfc224ca95aa0211b2986fc390', 4434)", 'stop':<PHONE_NUMBER>.4325304, 'start':<PHONE_NUMBER>.3020687, 'nbytes': 4140533}
dask-worker_1     | 2017-11-05T18:21:32.434231696Z distributed.worker - DEBUG - Execute key: ('getitem-936f98cb941dae846251947cdf358006', 4388) worker: tcp://<IP_ADDRESS>:38661
dask-worker_1     | 2017-11-05T18:21:32.435753372Z distributed.worker - DEBUG - Send compute response to scheduler: ('getitem-936f98cb941dae846251947cdf358006', 4388), {'thread':<PHONE_NUMBER>46784, 'status': 'OK', 'op': 'task-finished', 'type': <class 'pandas.core.series.Series'>, 'key': "('getitem-936f98cb941dae846251947cdf358006', 4388)", 'stop':<PHONE_NUMBER>.4348502, 'start':<PHONE_NUMBER>.4345806, 'nbytes': 56286028}
dask-worker_1     | 2017-11-05T18:21:32.436108651Z distributed.worker - DEBUG - Execute key: ('assign-7e08af70bcc5505618642c0209ce5970', 4339) worker: tcp://<IP_ADDRESS>:38661
dask-worker_1     | 2017-11-05T18:21:32.438114434Z distributed.worker - DEBUG - Deleted 1 keys
dask-worker_1     | 2017-11-05T18:21:32.456848909Z distributed.worker - DEBUG - Send compute response to scheduler: ('concat-7075105cf1c6acffe24d76c23229da72', 4273), {'thread':<PHONE_NUMBER>57504, 'status': 'OK', 'op': 'task-finished', 'type': <class 'pandas.core.frame.DataFrame'>, 'key': "('concat-7075105cf1c6acffe24d76c23229da72', 4273)", 'stop':<PHONE_NUMBER>.3513696, 'start':<PHONE_NUMBER>.1742594, 'nbytes': 362452000}
dask-worker_1     | 2017-11-05T18:21:32.458012439Z distributed.worker - DEBUG - Execute key: ('getitem-8e7fcdbbeeef95f9da75ec0adbd9adc3', 4477) worker: tcp://<IP_ADDRESS>:48040
dask-worker_1     | 2017-11-05T18:21:32.458026177Z distributed.worker - DEBUG - future state: ('read-parquet-248809b98a4a58f4cebbf22d9fc05dfe', 47) - RUNNING
dask-worker_1     | 2017-11-05T18:21:32.461667639Z distributed.worker - DEBUG - Send compute response to scheduler: ('getitem-8e7fcdbbeeef95f9da75ec0adbd9adc3', 4477), {'thread':<PHONE_NUMBER>57504, 'status': 'OK', 'op': 'task-finished', 'type': <class 'pandas.core.series.Series'>, 'key': "('getitem-8e7fcdbbeeef95f9da75ec0adbd9adc3', 4477)", 'stop':<PHONE_NUMBER>.4587786, 'start':<PHONE_NUMBER>.458351, 'nbytes': 35910430}
dask-worker_1     | 2017-11-05T18:21:32.462353208Z distributed.worker - DEBUG - Execute key: ('concat-7075105cf1c6acffe24d76c23229da72', 4422) worker: tcp://<IP_ADDRESS>:48040

Here you can see the whole logs:
worker.log
scheduler.log
After I kill the client, following lines appear:
worker_clientKilled.log
scheduler_clientKilled.log
Output from the client is like:
distributed.comm.tcp - DEBUG - Setting TCP keepalive: nprobes=10, idle=10, interval=2
distributed.comm.tcp - DEBUG - Setting TCP user timeout: 30000 ms
distributed.comm.tcp - DEBUG - Setting TCP keepalive: nprobes=10, idle=10, interval=2
distributed.comm.tcp - DEBUG - Setting TCP user timeout: 30000 ms
distributed.client - DEBUG - Started scheduling coroutines. Synchronized
distributed.client - DEBUG - Waiting on futures to clear before gather
<here it hangs forever and I have to kill it>

I will be grateful for any help with debugging this issue.

2017-11-05T18:20:08.078638857Z distributed.core - WARNING - Event loop was unresponsive for 23.08s.  This is often caused by long-running GIL-holding functions or moving large chunks of data. This can cause timeouts and instability.

This is odd.  My first guess would be that you're trying to move a large amount of data from workers to the client, such as might be caused by calling df.compute() on a very large result.
Alternatively, you might want to check the memory and CPU use of the scheduler to make sure that it isn't becoming overwhelmed.  This is available on the System tab of the diagnostic dashboard.

Thank you for your response.
All docker containers run on the same machine. CPU and memory usage of this machine are low when this happens.
Very large result may be the case here, but I guess it's not the problem, as today I got the same issue on a much smaller dataframe.

Something is causing the scheduler to become unresponsive for 20 seconds.  All the scheduler does is accept messages and pass bytes around.
Another option is if your graph were well above a million tasks large then I can imagine it slowing down while waiting setting up the new tasks, but even in that case I would expect the hang to occur before execution began.  Other than that no common causes come to mind.

Although as always if you're able to provide a simple reproducible example that is similar to your problem then it becomes much easier to diagnose things.  I appreciate that this can be quite challenging to produce though.

FWIW, I have seen behavior like this when I misconfigure the memory limit of my worker. In my case, I had a heterogeneous cluster, but set the memory limit to the largest value available over all workers.

Thank you @caseyjlaw for your comment. I specify memory limit as fraction and tried to lower it to avoid memory errors.
It's a bit unclear to me: is --memory-limit set per process? If I have --nprocs 10, should I also set --memory-limit 0.1?

I've been a bit unclear on this, too. It is set per worker and the worker can have multiple cores. But it isn't clear how to me how the scheduler considers memory per core.

To give some more light on the issue, it doesn't seem to happen when I use LocalCluster instead of scheduler and worker in different docker containers.

--memory-limit does get divided per process.
FWIW in production I rarely use the --nprocs keyword just to avoid ambiguities like this.  Instead I typically start many dask-worker processes.  There is no performance change here, I just find it to be simpler overall.

I've migrated toward using the "resources" argument to manage memory available to submitted tasks. It seemed like the standard memory estimationt of some of my tasks were too low. (I can't reproduce it well enough to submit an issue.)
Caveat: My work flow uses lots of numba, numpy, and other GIL-releasing functions, so threading works well for me.

Those are just warnings.  Things should evaluate fine.
Do we know if rasterio releases the GIL?  It might be worth asking upstream.

Conversation seems to have died here so I'm going to close.  Happy to reopen if things start up again.
For others who come along, some common causes for an unresponsive cluster:

Your software versions are mismatched
You are sending a lot of data in the task graph itself.  It's always wiser to load data as part of a task, rather than to include large local objects directly in function calls.
Your own code has some sort of deadlock in it unrelated to dask

Common things you can do to check what is wrong

call client.get_versions(check=True)
Look at the dashboard to see how workers are doing
Look at the call stacks of your worker threads to see what they're spending time on with client.call_stack()

I ran into random hangs too; but my case was pretty clear what is happening:

dask pauses worker under memory pressure.
there is only one active worker with remaining work and it is under memory pressure.

I am not sure how to recover from this. I tried to kill the hanging worker but it seems to fail my entire job calculation.
I suspect the problem is because one of the library I use has pretty bad leaks; even when dask reports there is nothing in memory, the worker's memory usage is still huge after a task graph is computed.

Here is why. I have two types of tasks: compute and IO-write. IO-write tasks are much slower than compute tasks. each compute task is followed by an IO-write task. There is no dependence between different IO-write task / compute tasks.
dask tries to hold the result from the computing tasks in memory as it runs through as many of them quickly. It then realizes memory is used up, and dask pauses the workers. Sometimes the memory is used up so quickly that before the work actually pauses, it already uses too much memory and the scheduler starts to issue kills.
The IO-write tasks, which reduces memory usage never gets called because
The scheduler should prioritize tasks that depend on objects that reduces memory usage. I suppose there is no easy way to notify the scheduler a task will reduce memory usage. Perhaps I can fiddle with the priority of the IO tasks.

Since an entire graph is submitted by dask.compute(), specifying the priority with dask.compute() will not help: IO and compute are on the same graph, and receive the same priority.
I suggest we add a way of controlling priority of dask objects as they are created, e.g.
da.from_array(...., priority=-10) # reading gets the lowest priority
da.map_blocks(..., priority=10) # writing gets the highest priority

You can supply the priority= keyword with a dictionary of different
priorities.
Unfortunately we can't currently provide the priority to dask array
operations because it is specific to the distributed scheduler.  Users of
the single-machine scheduler would be confused why their priorities weren't
being respected.
On Thu, Dec 13, 2018 at 7:07 PM Yu Feng<EMAIL_ADDRESS>wrote:

Since an entire graph is submitted by dask.compute(), specifying the
priority with dask.compute() will not help: IO and compute are on the same
graph, and receive the same priority.
I suggest we add a way of controlling priority of dask objects as they are
created, e.g.
da.from_array(...., priority=-10) # reading gets the lowest priority
da.map_blocks(..., priority=10) # writing gets the highest priority
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/dask/dask/issues/2866#issuecomment-447166643, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AASszIMQ8J8Lvm0HGN7XWALE0PTSQcIVks5u4uu0gaJpZM4QSgFG
.

Perhaps a hint/category/type keyword that describes the type of work the
function does,
which can be totally neglected by the single machine scheduler, and
translated to some sort of priority on the distributed scheduler.
This can also help the web interface / performance analysis.
On Tue, Dec 18, 2018 at 5:42 AM Matthew Rocklin<EMAIL_ADDRESS>wrote:

You can supply the priority= keyword with a dictionary of different
priorities.
Unfortunately we can't currently provide the priority to dask array
operations because it is specific to the distributed scheduler. Users of
the single-machine scheduler would be confused why their priorities weren't
being respected.
On Thu, Dec 13, 2018 at 7:07 PM Yu Feng<EMAIL_ADDRESS>wrote:

Since an entire graph is submitted by dask.compute(), specifying the
priority with dask.compute() will not help: IO and compute are on the
same
graph, and receive the same priority.
I suggest we add a way of controlling priority of dask objects as they
are
created, e.g.
da.from_array(...., priority=-10) # reading gets the lowest priority
da.map_blocks(..., priority=10) # writing gets the highest priority
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/dask/dask/issues/2866#issuecomment-447166643, or
mute
the thread
<
https://github.com/notifications/unsubscribe-auth/AASszIMQ8J8Lvm0HGN7XWALE0PTSQcIVks5u4uu0gaJpZM4QSgFG
.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/dask/dask/issues/2866#issuecomment-448224751, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAIbTMJotQU1y4iyCZBWq-EJw7ZebQokks5u6PC-gaJpZM4QSgFG
.

I am experiencing the very same issue and I do not understand how can i solve it.
Just to give a bit of context: I have a big file in S3 that I already partitioned in 500 .csv.gz files of size around 1MB (compressed). Now I have a transformation function that given an integer loads the i-th partition in memory in a pandas.DataFrame, transforms it and then writes it at a new location in S3 (using boto3), in chunks of 10e4 rows at a time (so the I/O is within the function, it is not a separate task). It returns 0 if everything is ok, -1 otherwise. I am running it on an AWS EC2 instance with 4 cores with 64GB RAM.
Since each partition is independent from the others, I use dask to parallelize their transformation.
Now the code looks a bit like:
with Client() as client:
    tran_ops = [dask.delayed(func)(i) for i in range(500)]
    tran_op = client.compute(dask.delayed(sum)(tran_ops)
    if client.gather(tran_op) != 0:
        print('nay')
    else:
        print('yay')

I also tried to split the 500-items list in sub-lists of 100 elements and compute them consecutively like so
for j in range(5):
    start = j*100
    stop = start + 100
    tran_ops = [dask.delayed(func)(i) for i in range(start, stop)]
    tran_op = client.compute(dask.delayed(sum)(tran_ops)
    if client.gather(tran_op) != 0:
        print('nay')
        break
    print('yay')

In both cases transformation hangs for 2 hours giving no sign of life whatsoever and then resumes normally and completes the transformation with no errors. Funny enough, in the former scenario the transformation goes on for around 25mins and transforms around 60-80% of the partitions before the break and the remaining 40-20% after; in the latter the transformation interrupts after 10mins when only 10-20% of the partitions have been processed and then goes on with the remaining 90-80% without further breaks.
I understand that this may be due to GIL-holding functions and processing of large chunks of data but:

I have not very clear what GIL really is and which will be the GIL-holding function here; I read that pandas, numpy, etc. release the GIL, so maybe boto3?
large chunk is like saying big data, what's big here?

Did all of your workers turn orange in the monitor UI, when their is no
progress?
On Wed, Jul 17, 2019 at 1:58 AM Massimiliano R<EMAIL_ADDRESS>wrote:

I am experiencing the very same issue and I do not understand how can i
solve it.
Just to give a bit of context: I have a big file in S3 that I already
partitioned in 500 .csv.gz files of size around 1MB (compressed). Now I
have a transformation function that given an integer loads the i-th
partition in memory in a pandas.DataFrame, transforms it and then writes it
at a new location in S3 (using boto3), in chunks of 10e4 rows at a time (so
the I/O is within the function, it is not a separate task). It returns 0 if
everything is ok, -1 otherwise. I am running it on an AWS EC2 instance with
4 cores with 64GB RAM.
Since each partition is independent from the others, I use dask to
parallelize their transformation.
Now the code looks a bit like:
with Client() as client:
tran_ops = [dask.delayed(func)(i) for i in range(500)]
tran_op = client.compute(dask.delayed(sum)(tran_ops)
if client.gather(tran_op) != 0:
print('nay')
else:
print('yay')
I also tried to split the 500-items list in sub-lists of 100 elements and
compute them consecutively like so
for j in range(5):
start = j*100
stop = start + 100
tran_ops = [dask.delayed(func)(i) for i in range(start, stop)]
tran_op = client.compute(dask.delayed(sum)(tran_ops)
if client.gather(tran_op) != 0:
print('nay')
break
print('yay')
In both cases transformation hangs for 2 hours giving no sign of life
whatsoever and then resumes normally and completes the transformation with
no errors. Funny enough, in the former scenario the transformation goes on
for around 25mins and transforms around 60-80% of the partitions before the
break and the remaining 40-20% after; in the latter the transformation
interrupts after 10mins when only 10-20% of the partitions have been
processed and then goes on with the remaining 90-80% without further breaks.
I understand that this may be due to GIL-holding functions and processing
of large chunks of data but:

I have not very clear what GIL really is and which will be the
GIL-holding function here; I read that pandas, numpy, etc. release the GIL,
so maybe boto3?
large chunk is like saying big data, what's big here?

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/dask/dask/issues/2866?email_source=notifications&email_token=AABBWTHAUYC6KSIVNQIQHB3P73NMBA5CNFSM4ECKAFDKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2DQZIA#issuecomment-512167072,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AABBWTAT3VM2ZQ2OKOLYRD3P73NMBANCNFSM4ECKAFDA
.

No. When I monitored the whole process through the UI it executed 473 tasks before interrupting and it happened that only one worker got stucked when it had 27 task left, while the others went on with theirs until they finished.
Speaking about the color, I saw the bars turning red-ish (I say "ish" because I use a dark theme that slightly changes the color) when they where about to finish, like with 5 tasks left or less, while the one that got stucked I remember it being coloured in green. However, I do not remember if the other workers have been green for a while, too (for example when having less than 50 tasks).

I have often had similar bogdown issues, with the scheduler giving GIL warnings.  I'm using dask-gateway and launched two graphs of ~10k tasks each.  The bulk of the processing that its currently doing are ~4k tasks that take 5-10 minutes each.  Not a lot for the scheduler to do!  But still the scheduler ie pegged  100% cpu!  The memory is at 1.9GiB, bandwidth at ~2MiB/s, and # file descriptors at ~1900.    The logs are littered with "Event loop was unresponsive in Scheduler for 5.04s.  This is often caused by long-running GIL-holding functions or moving large chunks of data. This can cause timeouts and instability.".


Clarfiy that --{test,bench} select targets

Fixes #1767

r? @wycats
(rust_highfive has picked a reviewer for you, use r? to override)

@bors: r+ 355e7fb81a53fbc58885f501b68e8b6239964a33

:hourglass: Testing commit 355e7fb with merge c7e7e35...

:sunny: Test successful - cargo-linux-32, cargo-linux-64, cargo-mac-32, cargo-mac-64, cargo-win-gnu-32, cargo-win-gnu-64, cargo-win-msvc-32, cargo-win-msvc-64


Inconsistent error field type in lagoon logs

Describe the bug
harbor-clair logs an error field as a string, while something else in lagoon logs an error field as an object.
This causes issues on log ingestion:
["org.elasticsearch.index.mapper.MapperParsingException: object mapping for [error] tried to parse field [error] as object, but found a concrete value"

And also causes issues in the logs-forwarder which gets a 400 back from Elasticsearch.
This occurs on e.g. container-logs-lagoon-dev-develop-* indices.
An example event which causes this error looks like this (edited):
{"Event":"could not parse package version. skipping","Level":"warning","Location":"ubuntu.go:316","Time":"2020-03-23 11:36:01.416499","error":"invalid version","version":"0.27-1+deb7u1build<IP_ADDRESS>, 0.28-1+deb8u1","docker":{},"kubernetes":{"container_name":"harbor-clair","namespace_name":"lagoon-dev-develop","level":"info"}

To Reproduce
Steps to reproduce the behavior:

Go to an affected elasticsearch node
View its logs
See error

Expected behavior
The fields of an index should be consistently typed, otherwise Elasticsearch can't index them.
Screenshots
n/a
Additional context
I'm not sure, but maybe this should be fixed by some filtering in logstash?

I think this is all handled int he logs-dispatcher/logs-concentrator now?