WSL 2: ReactJS not reloading after saved. [Solved]

Is your proposal related to a problem?
WSL2 : ReactJS not reloading after saved.

I recently installed WSL 2 but when I create an app using create-react-app and use npm start the app was’t reloaded when I edited and saved some file
Describe the solution you'd like
I want to use Hot Reloading on my Projects located in Windows file system.

WSL2 changed the file sharing protocol, from using their own custom developed protocol using the 9P protocol, which does not support file changes event for Create React App.
Describe alternatives you've considered
To FIX this issue : You can put your code on the Linux file system (example : in your user’s home directory ), and access these files through the WSL share, \wsl$\DISTRO_NAME from Windows.
Additional context

I wrote everything properly again here
https://prajjwaldatir.medium.com/create-react-app-not-working-62fc9fc4368b

Did you try it out with the CHOKIDAR_USEPOLLING environment variable?

@bennettdams Wait I'll try and update here.

@bennettdams  CHOKIDAR_USEPOLLING is if I'm running my react app inside a VM right!?
I tried all other troubleshooting methods, none of them worked.
I'm not running my project inside VM so why & how do I use CHOKIDAR_USEPOLLING=true  ?

@PrajjwalDatir "VM" is just the example the CRA team used for their documentation there. Really, Chokidar is a file watcher for Node, so it handles the "proxy" part of your development setup (remote container, WSL, VM, etc.).
How do I know? Just last week I set up a project with VS Code's Remote Containers, which had exactly the same problem you described here. With Chokidar, my changes are actually noticed by VS Code.

Thanks! It works now 🕺
#10197  is the parent issue of this, so I'm closing this issue.
For future readers,
Inside your package.json
find
    "start": "react-scripts start",
    "build": "react-scripts build",
    "test": "react-scripts test",
    "eject": "react-scripts eject"
  }

and replace it with
    "start": "CHOKIDAR_USEPOLLING=true react-scripts start",
    "build": "react-scripts build",
    "test": "react-scripts test",
    "eject": "react-scripts eject"
  }

not working if you want to use https=true yarn start

Thanks! It works now 🕺 #10197 is the parent issue of this, so I'm closing this issue.
For future readers, Inside your package.json
"scripts": {
    "start": "CHOKIDAR_USEPOLLING=true react-scripts start",
  }

This didn't solve the issue for me. My situation remains same as described in the original issue description.

Thanks! It works now 🕺 #10197 is the parent issue of this, so I'm closing this issue.
For future readers, Inside your package.json
"scripts": {
    "start": "CHOKIDAR_USEPOLLING=true react-scripts start",
  }

This didn't solve the issue for me. My situation remains same as described in the original issue description.
I have also tried the CHOKIDAR_USEPOLLING=true in a .env file (technically the same thing as this solution) to no effect.

I am facing the Same Issue.

Thanks! It works now 🕺 #10197 is the parent issue of this, so I'm closing this issue.
For future readers, Inside your package.json
"scripts": {
    "start": "CHOKIDAR_USEPOLLING=true react-scripts start",
  }

This didn't solve the issue for me. My situation remains same as described in the original issue description.
I have also tried the CHOKIDAR_USEPOLLING=true in a .env file (technically the same thing as this solution) to no effect.

I am facing the Same Issue.

Same Issue here as well. Are you on Windows 11?

Thanks! It works now 🕺 #10197 is the parent issue of this, so I'm closing this issue.
For future readers, Inside your package.json
"scripts": {
    "start": "CHOKIDAR_USEPOLLING=true react-scripts start",
  }

This didn't solve the issue for me. My situation remains same as described in the original issue description.
I have also tried the CHOKIDAR_USEPOLLING=true in a .env file (technically the same thing as this solution) to no effect.

I am facing the Same Issue.

Same Issue here as well. Are you on Windows 11?

Yes I am. My projects folder was placed on Desktop means in C: drive after some research I found out I need to place the project in linux folder itself. So I moved the project to linux folder i.e., "wsl\Ubuntu\home{$username}" and it worked even without CHOKIDAR_USEPOLLING=true . hope it helps.

Thanks! It works now 🕺 #10197 is the parent issue of this, so I'm closing this issue.
For future readers, Inside your package.json
"scripts": {
    "start": "CHOKIDAR_USEPOLLING=true react-scripts start",
  }

This didn't solve the issue for me. My situation remains same as described in the original issue description.
I have also tried the CHOKIDAR_USEPOLLING=true in a .env file (technically the same thing as this solution) to no effect.

I am facing the Same Issue.

Same Issue here as well. Are you on Windows 11?

Yes I am. My projects folder was placed on Desktop means in C: drive after some research I found out I need to place the project in linux folder itself. So I moved the project to linux folder i.e., "wsl\Ubuntu\home{$username}" and it worked even without CHOKIDAR_USEPOLLING=true . hope it helps.

Thanks! But I think it should have worked just using CHOKIDAR_USEPOLLING=true as many have said this solves the issue. But as pointed out here - https://stackoverflow.com/questions/69759267/docker-not-detecting-changes-for-react-app-in-windows, maybe this issue might be specific to windows 11

Thanks! But I think it should have worked just using CHOKIDAR_USEPOLLING=true as many have said this solves the issue. But as pointed out here - https://stackoverflow.com/questions/69759267/docker-not-detecting-changes-for-react-app-in-windows, maybe this issue might be specific to windows 11

I am using Windows 10 with WSL2/Ubuntu 20.04 and experiencing the same issue. So it might not be specific to W11. As we see from others' comments, using CHOKIDAR_USEPOLLING=true should have fixed it but it hasn't for us. Which tells me the issue may have a different origin for some of the cases.

Thanks! It works now 🕺 #10197 is the parent issue of this, so I'm closing this issue.
For future readers, Inside your package.json
find
"scripts": {
    "start": "react-scripts start",
    "build": "react-scripts build",
    "test": "react-scripts test",
    "eject": "react-scripts eject"
  }

and replace it with
"scripts": {
    "start": "CHOKIDAR_USEPOLLING=true react-scripts start",
    "build": "react-scripts build",
    "test": "react-scripts test",
    "eject": "react-scripts eject"
  }

Does not work :(

+1 none of the proposed solutions are working for me on Win 11, node v17.4.0, npm v8.3.2, create-react-app v5.0.0, react v17.0.2

Can confirm for Windows 11 with WSL2; having a react project; that the Chokidar thing works. Once I added it, I did npm start again. Adding changes and then saving automatically brings the changes. Relatively faster than restarting the whole thing. Many thanks.

I had the same problem with hot reload on WSL even after changing package.json ("start": "CHOKIDAR_USEPOLLING=true react-scripts start").
After changing react-scripts version (and then npm install) hot reload works again.
"dependencies": {
//...
"react-scripts": "4.0.3",  // previously I had 5.0.0
//...
}

Good evening everyone,
Tonight I bring you an alternative to the madness.
If running Win10/Win11 and running Ubuntu WSL2, you will find that the Windows file system has a specific file directory that acts as this VM.
Open file explorer and you can navigate to this directory with \\wsl$ in the search.
You will notice this directory has a folder titled the Ubuntu you are running. Within this folder will be a ./mnt directory. Inside, once more, is ./wsl.
This is the folder you should run your project from.
Copy the contents of your project here, open it with your IDE to verify you have permission to do so.
Once it's open, you will need to recursively update permissions on the folder to ensure you have privilege to run your react scripts start command using npm run start.
To do this:
Open Powershell with elevated privilege on your local system. Run your Ubuntu exe. For me it was:

Navigate to the wsl directory, example:

Run this command to update rights, recursively on the directory:
sudo chmod -R +x ./<YourProjectName>
You'll be asked to authenticate, do as the computer asks 👌
Presto.
docker-compose up your project from VSCode or where ever you are deving. If your docker-compose.yml is configured with the correct chokidar_usepolling environment variable, you're good to go.
Hope this helps us Windows users.

Good evening everyone, Tonight I bring you an alternative to the madness.
If running Win10/Win11 and running Ubuntu WSL2, you will find that the Windows file system has a specific file directory that acts as this VM.
Open file explorer and you can navigate to this directory with \\wsl$ in the search.
You will notice this directory has a folder titled the Ubuntu you are running. Within this folder will be a /mnt directory. Inside, once more, is ./wsl.
This is the folder you should run your project from.
Copy the contents of your project here, open it with your IDE to verify you have permission to do so.
Once it's open, you will need to recursively update permissions on the folder to ensure you have privilege to run your react scripts start command using npm run start.
To do this: Open Powershell with elevated privilege on your local system. Run your Ubuntu exe. For me it was:
Navigate to the wsl directory, example:
Run this command to update rights, recursively on the directory: sudo chmod -R +x ./<YourProjectName> You'll be asked to authenticate, do as the computer asks 👌
Presto.
docker-compose up your project from VSCode or where ever you are deving. If your docker-compose.yml is configured with the correct chokidar_usepolling environment variable, you're good to go.
Hope this helps us Windows users.

This helped for me on Windows 11, Thank you so much!
But i also needed to create project again to run server.

I had the same problem with hot reload on WSL even after changing package.json ("start": "CHOKIDAR_USEPOLLING=true react-scripts start").
After changing react-scripts version (and then npm install) hot reload works again.
"dependencies": { //... "react-scripts": "4.0.3", // previously I had 5.0.0 //... }

I have been debugging for 4 hours, downgrading rom 5.0 5o 4.0.3 solved the problem. I cant thank you enough. I am 4 whiskeys in pulling my hair.

I had the same problem with hot reload on WSL even after changing package.json ("start": "CHOKIDAR_USEPOLLING=true react-scripts start").
After changing react-scripts version (and then npm install) hot reload works again.
"dependencies": { //... "react-scripts": "4.0.3", // previously I had 5.0.0 //... }

This is what ultimately solved it for me, after trying trying both CHOKIDAR_USEPOLLING=true and FAST_REFRESH=false.

Good evening everyone, Tonight I bring you an alternative to the madness.
If running Win10/Win11 and running Ubuntu WSL2, you will find that the Windows file system has a specific file directory that acts as this VM.
Open file explorer and you can navigate to this directory with \\wsl$ in the search.
You will notice this directory has a folder titled the Ubuntu you are running. Within this folder will be a /mnt directory. Inside, once more, is ./wsl.
This is the folder you should run your project from.
Copy the contents of your project here, open it with your IDE to verify you have permission to do so.
Once it's open, you will need to recursively update permissions on the folder to ensure you have privilege to run your react scripts start command using npm run start.
To do this: Open Powershell with elevated privilege on your local system. Run your Ubuntu exe. For me it was:
Navigate to the wsl directory, example:
Run this command to update rights, recursively on the directory: sudo chmod -R +x ./<YourProjectName> You'll be asked to authenticate, do as the computer asks 👌
Presto.
docker-compose up your project from VSCode or where ever you are deving. If your docker-compose.yml is configured with the correct chokidar_usepolling environment variable, you're good to go.
Hope this helps us Windows users.

This helped for me on Windows 11, Thank you so much! But i also needed to create project again to run server.

Moving everything to the /mnt/wsl folder did the trick. Man things started loading crazy fast when I moved them in there.
Thanks a lot!

Thanks! It works now 🕺 #10197 is the parent issue of this, so I'm closing this issue.
For future readers, Inside your package.json
"scripts": {
    "start": "CHOKIDAR_USEPOLLING=true react-scripts start",
  }

This didn't solve the issue for me. My situation remains same as described in the original issue description.
I have also tried the CHOKIDAR_USEPOLLING=true in a .env file (technically the same thing as this solution) to no effect.

I am facing the Same Issue.

Same Issue here as well. Are you on Windows 11?

Yes I am. My projects folder was placed on Desktop means in C: drive after some research I found out I need to place the project in linux folder itself. So I moved the project to linux folder i.e., "wsl\Ubuntu\home{$username}" and it worked even without CHOKIDAR_USEPOLLING=true . hope it helps.

this is basically my fix. I moved everything to wsl home

I don't get it. If there's a /mnt/wsl folder as part of the mounted filesystem, why do I get a /home/me for ~? Like why does it have to go in /mnt/wsl, and not ~?

Good evening everyone, Tonight I bring you an alternative to the madness.
If running Win10/Win11 and running Ubuntu WSL2, you will find that the Windows file system has a specific file directory that acts as this VM.
Open file explorer and you can navigate to this directory with \\wsl$ in the search.
You will notice this directory has a folder titled the Ubuntu you are running. Within this folder will be a /mnt directory. Inside, once more, is ./wsl.
This is the folder you should run your project from.
Copy the contents of your project here, open it with your IDE to verify you have permission to do so.
Once it's open, you will need to recursively update permissions on the folder to ensure you have privilege to run your react scripts start command using npm run start.
To do this: Open Powershell with elevated privilege on your local system. Run your Ubuntu exe. For me it was:
Navigate to the wsl directory, example:
Run this command to update rights, recursively on the directory: sudo chmod -R +x ./<YourProjectName> You'll be asked to authenticate, do as the computer asks 👌
Presto.
docker-compose up your project from VSCode or where ever you are deving. If your docker-compose.yml is configured with the correct chokidar_usepolling environment variable, you're good to go.
Hope this helps us Windows users.

I am now personally extremely skeptical of this appproach. Would-be adherents beware. Twice now upon resetting my computer, it's completely blown away the directory I put in /mnt/wsl/. The first time I didn't quite realize it, because I used File Explorer to copy files from my Documents folder. The second time I just did a git clone to instantiate the files. I just restarted my machine and now the git clone is gone along with all of my work. 👎

The reason that version 5.x.x of react-scripts does not work with CHOKIDAR_USEPOLLING is because there was a breaking change in webpack and that variable is no longer supported. The new variable WATCHPACK_POLLING has to be used instead.
Inside your package.json find
"scripts": {
    "start": "react-scripts start",

and replace it with
"scripts": {
    "start": "WATCHPACK_POLLING=true react-scripts start",

See https://github.com/facebook/create-react-app/issues/12397 for more info

Good evening everyone, Tonight I bring you an alternative to the madness.
If running Win10/Win11 and running Ubuntu WSL2, you will find that the Windows file system has a specific file directory that acts as this VM.
...

I am now personally extremely skeptical of this appproach. Would-be adherents beware. Twice now upon resetting my computer, it's completely blown away the directory I put in /mnt/wsl/. The first time I didn't quite realize it, because I used File Explorer to copy files from my Documents folder. The second time I just did a git clone to instantiate the files. I just restarted my machine and now the git clone is gone along with all of my work. 👎
-- edit -- could be a "me" problem, I didn't run chmod -R +x.  Additionally, I'm no linux expert, the only chmodding I ever do is to make scripts executable. If you're like me, be careful.
-- edit 2 -- The chmod didn't help. Long live /home/user_name.
$ cd /mnt/wsl
$ mkdir was_it_my_fault
$ chmod -R +x ./was_it_my_fault

# restarts machine

$ cd /mnt/wsl
$ ls -la
/mnt/wsl > la
total 0
drwxr-xr-x 4 root root 100 Apr 24 02:03 docker-desktop
drwxr-xr-x 3 root root  60 Apr 24 02:03 docker-desktop-bind-mounts
drwxr-xr-x 6 root root 120 Apr 24 02:03 docker-desktop-data
# guess not maybe

Agree -- I don't think is the best approach. According to this post, /mnt/wsl is mounted as tmpfs so it will get wiped everytime you restart your computer. You probably shouldn't be using this directory long-term for development.

Also, see https://github.com/facebook/create-react-app/issues/10253#issuecomment-1127340307 for a solution that worked for me.
Adding WATCHPACK_POLLING=true worked while modifying CHOKIDAR_USEPOLLING and FAST_REFRESH flags (like previous comments suggested) did not.

"scripts": {

Fiz isso e deu certo no meu tbm

Although WATCHPACK_POLLING=true works, i still have to refresh the page to see the results. Any ideas?

Version 5.x.x of react-scripts does not work with CHOKIDAR_USEPOLLING because there was a breaking change in webpack and that variable is no longer supported. The newly added variable WATCHPACK_POLLING has to be used instead.
Inside your package.json find
"scripts": {
    "start": "react-scripts start",

and replace it with
"scripts": {
    "start": "WATCHPACK_POLLING=true react-scripts start",

See #12397 for more info

this worked for me, last time hot reload stopped working, the fix for me was reinstalling WSL ubuntu.

@michalbidzinski1

Although WATCHPACK_POLLING=true works, i still have to refresh the page to see the results. Any ideas? Im running my react + backend on docker with nginx and docker-compose btw.

What I am experiencing is that refreshing on changes takes a little longer (about 3-5 sec).
Could you check if that's the case with you?

I had the same problem with hot reload on WSL even after changing package.json ("start": "CHOKIDAR_USEPOLLING=true react-scripts start").
After changing react-scripts version (and then npm install) hot reload works again.
"dependencies": { //... "react-scripts": "4.0.3", // previously I had 5.0.0 //... }

Dude man was searching for a solution, since 4 hrs, finally this worked. Thanks man.

CHOKIDAR_USEPOLLING=true

I had the same problem with hot reaload on WSL, after using the "CHOKIDAR_USEPOLLING=true" on package-json:
"dev": "CHOKIDAR_USEPOLLING=true vue-cli-service serve",
It Works, althought it's taking a lot of time to show the changes on screen.
P.S. I'm using vue
but thanks anyway!

I moved my code from the mounted file system to ubuntu file system in ~/home and it worked!

how do I do this?

@danielcaze
Not sure about that, but you can try this: https://github.com/facebook/create-react-app/issues/10253#issuecomment-1127340307
It worked for me👍

```json
"start": "WATCHPACK_POLLING=true react-scripts start",

After a couple of hours of headscratching this solved the issue for me in my Homestead VM. Thank you!

Thank you so much, it's finally working again

Phiên bản của không hoạt động với vì đã có một thay đổi đột phá trong webpack và biến đó không còn được hỗ trợ. Biến mới được thêm vào phải được sử dụng thay thế.5.x.x``react-scripts``CHOKIDAR_USEPOLLING``WATCHPACK_POLLING
Bên trong tìm kiếm của bạnpackage.json
"scripts": {
    "start": "react-scripts start",

và thay thế nó bằng
"scripts": {
    "start": "WATCHPACK_POLLING=true react-scripts start",

Xem #12397 để biết thêm thông tin

Thank you so much, it's finally working again

Version 5.x.x of react-scripts does not work with CHOKIDAR_USEPOLLING because there was a breaking change in webpack and that variable is no longer supported. The newly added variable WATCHPACK_POLLING has to be used instead.
Inside your package.json find
"scripts": {
    "start": "react-scripts start",

and replace it with
"scripts": {
    "start": "WATCHPACK_POLLING=true react-scripts start",

See #12397 for more info

Thank you so much, it's finally working again ^^

What worked the best for me was to reinstall react-scripts, I first tried to replace the latest version with the 4.0.3, it worked, then I tried the latest (5.0.1) again and it also worked. However, make sure to MANUALLY trigger the first refresh of your app after reinstalling react-scripts, otherwise the auto refresh will not work.

due to some answer in stackoverflow, that I'm so sorry for forgetting who answered that, I could fix this bug because I set up my wsl from version 2 to version 1.
Here is my command in Powershell (with administrator, absolutely)
wsl --set-version ubuntu 1

Version 5.x.x of react-scripts does not work with CHOKIDAR_USEPOLLING because there was a breaking change in webpack and that variable is no longer supported. The newly added variable WATCHPACK_POLLING has to be used instead.
Inside your package.json find
"scripts": {
    "start": "react-scripts start",

and replace it with
"scripts": {
    "start": "WATCHPACK_POLLING=true react-scripts start",

See #12397 for more info

Thank you! Working now.

Version 5.x.x of react-scripts does not work with CHOKIDAR_USEPOLLING because there was a breaking change in webpack and that variable is no longer supported. The newly added variable WATCHPACK_POLLING has to be used instead.
Inside your package.json find
"scripts": {
    "start": "react-scripts start",

and replace it with
"scripts": {
    "start": "WATCHPACK_POLLING=true react-scripts start",

See #12397 for more info

If anyone is using WSL 2 and react-scripts, then THIS IS THE WAY!

Is your proposal related to a problem?
WSL2 : ReactJS not reloading after saved.
I recently installed WSL 2 but when I create an app using create-react-app and use npm start the app was’t reloaded when I edited and saved some file
Describe the solution you'd like
I want to use Hot Reloading on my Projects located in Windows file system.
WSL2 changed the file sharing protocol, from using their own custom developed protocol using the 9P protocol, which does not support file changes event for Create React App.
Describe alternatives you've considered
To FIX this issue : You can put your code on the Linux file system (example : in your user’s home directory ), and access these files through the WSL share, \wsl$\DISTRO_NAME from Windows.
Additional context
I wrote everything properly again here
https://prajjwaldatir.medium.com/create-react-app-not-working-62fc9fc4368b

WATCHPACK_POLLING=true in .env files

Version 5.x.x of react-scripts does not work with CHOKIDAR_USEPOLLING because there was a breaking change in webpack and that variable is no longer supported. The newly added variable WATCHPACK_POLLING has to be used instead.
Inside your package.json find
"scripts": {
    "start": "react-scripts start",

and replace it with
"scripts": {
    "start": "WATCHPACK_POLLING=true react-scripts start",

See #12397 for more info

Thx so much! This worked using colima on Mac


Adding content and warning to the README.md

I tried to keep it short to highlight the warning.

Docs Build status updates of commit 6d5fe9d:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

README.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Docs Build status updates of commit 4d3d122:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

README.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Docs Build status updates of commit 8165bf4:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

README.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Docs Build status updates of commit 6d1589d:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

README.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

@KittyYeungQ can you review this one?

Looks good.

Docs Build status updates of commit a1a8f5d:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

README.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Docs Build status updates of commit 746364f:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

README.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Docs Build status updates of commit e8af688:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

README.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

@geduardo Thank you for adding the warning. Shall we let people know though if their intended changes are only editorial, without association to the API, they can still directly edit the .md and .yml files here?

Done! @KittyYeungQ

Docs Build status updates of commit eb796ae:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

README.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

@geduardo Thank you for adding the warning. Shall we let people know though if their intended changes are only editorial, without association to the API, they can still directly edit the .md and .yml files here?

To clarify, what is an editorial change to the *.yml files? Thanks!

@geduardo Thank you for adding the warning. Shall we let people know though if their intended changes are only editorial, without association to the API, they can still directly edit the .md and .yml files here?

To clarify, what is an editorial change to the *.yml files? Thanks!

We've edited 'TOC.yml's before to rearrange the orders in the TOC.


WDI garden step connects to grapher DB

Problem
The general public can no longer build our garden dataset, since it requires a MySQL database for the WDI step for garden.
Proposed solutions

Delay any grapher data until a grapher step; OR
Do a once-off metadata query to grapher, and check it in as a JSON file as part of the recipe for this dataset

Full error
Detecting which steps need rebuilding...
Running 2 steps:
1. data://garden/worldbank_wdi/2022-05-26/wdi...
/Users/lars/Documents/owid/etl/etl/steps/data/garden/worldbank_wdi/2022-05-26/wdi.py:54: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  df["country"] = countries
2022-09-05 11:40:33,711 - [bugsnag] WARNING - No API key configured, couldn't notify
Traceback (most recent call last):
  File "/Users/lars/Documents/owid/etl/.venv/bin/run_python_step", line 6, in <module>
    sys.exit(main())
  File "/Users/lars/Documents/owid/etl/.venv/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/Users/lars/Documents/owid/etl/.venv/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/Users/lars/Documents/owid/etl/.venv/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/lars/Documents/owid/etl/.venv/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/Users/lars/Documents/owid/etl/etl/run_python_step.py", line 26, in main
    _import_and_run(path, dest_dir)
  File "/Users/lars/Documents/owid/etl/etl/run_python_step.py", line 44, in _import_and_run
    module.run(dest_dir)  # type: ignore
  File "/Users/lars/Documents/owid/etl/etl/steps/data/garden/worldbank_wdi/2022-05-26/wdi.py", line 67, in run
    tb_garden = add_variable_metadata(tb_garden)
  File "/Users/lars/Documents/owid/etl/etl/steps/data/garden/worldbank_wdi/2022-05-26/wdi.py", line 368, in add_variable_metadata
    vm = VariableMatcher()
  File "/Users/lars/Documents/owid/etl/etl/steps/data/garden/worldbank_wdi/2022-05-26/wdi.py", line 531, in __init__
    self.grapher_variables = self.fetch_grapher_variables()
  File "/Users/lars/Documents/owid/etl/etl/steps/data/garden/worldbank_wdi/2022-05-26/wdi.py", line 581, in fetch_grapher_variables
    df_vars = pd.read_sql(query, get_connection())
  File "/Users/lars/Documents/owid/etl/etl/db.py", line 22, in get_connection
    return MySQLdb.connect(
  File "/Users/lars/Documents/owid/etl/.venv/lib/python3.10/site-packages/MySQLdb/__init__.py", line 123, in Connect
    return Connection(*args, **kwargs)
  File "/Users/lars/Documents/owid/etl/.venv/lib/python3.10/site-packages/MySQLdb/connections.py", line 185, in __init__
    super().__init__(*args, **kwargs2)
MySQLdb.OperationalError: (2002, "Can't connect to server on '<IP_ADDRESS>' (36)")

I can take this one. The first proposed solution won't work because of this issue, but generating JSON file as a recipe should be straightforward.


webui not deploy containers

roy@ubuntu:~$ sudo docker images
REPOSITORY              TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
ubuntu                  latest              91e54dfb1179        12 days ago         188.4 MB
base_t3                 latest              f70ff9052b7e        4 weeks ago         577.5 MB
shipyard/rethinkdb      latest              01d0c7f830ab        11 weeks ago        296.2 MB
shipyard/deploy         latest              ba94b7a205d6        5 months ago        6.514 MB
shipyard/shipyard       latest              f1be70df06c1        6 months ago        36.26 MB
shipyard/shipyard-cli   latest              b73f9452c2ff        8 months ago        146.4 MB
roy@ubuntu:~$ sudo docker run shipyard/shipyard
time="2015-09-02T10:08:18Z" level="info" msg="shipyard version 2.0.10"
time="2015-09-02T10:08:18Z" level="fatal" msg="factory is not able to fill the pool: gorethink: dial tcp <IP_ADDRESS>:28015: connection refused"
roy@ubuntu:~$ sudo docker version
Client:
Version:      1.8.1
API version:  1.20
Go version:   go1.4.2
Git commit:   d12ea79
Built:        Thu Aug 13 02:35:49 UTC 2015
OS/Arch:      linux/amd64
Server:
Version:      1.8.1
API version:  1.20
Go version:   go1.4.2
Git commit:   d12ea79
Built:        Thu Aug 13 02:35:49 UTC 2015
OS/Arch:      linux/amd64
I use webui deploy containers the post url is
http://<IP_ADDRESS>:8080/api/containers?count=1&pull=true
always error
get the message
Network timed out while trying to connect to https://index.docker.io/v1/repositories/library/base_t3/images. You may want to check your internet connection or if you are behind a proxy.
I manual post the url http://<IP_ADDRESS>:8080/api/containers is OK ,not args
maybe something is wrong。
apologize for my english  poor

First, it doesn't look like rethinkdb is running.  You will want to check
that container as Shipyard needs it.  Second, it looks like you are having
network issues as there is a timeout connecting to a remote host.
On Wed, Sep 2, 2015 at 6:20 AM, gregorianzhang<EMAIL_ADDRESS>wrote:

roy@ubuntu:~$ sudo docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
ubuntu latest 91e54dfb1179 12 days ago 188.4 MB
base_t3 latest f70ff9052b7e 4 weeks ago 577.5 MB
shipyard/rethinkdb latest 01d0c7f830ab 11 weeks ago 296.2 MB
shipyard/deploy latest ba94b7a205d6 5 months ago 6.514 MB
shipyard/shipyard latest f1be70df06c1 6 months ago 36.26 MB
shipyard/shipyard-cli latest b73f9452c2ff 8 months ago 146.4 MB
roy@ubuntu:~$ sudo docker run shipyard/shipyard
time="2015-09-02T10:08:18Z" level="info" msg="shipyard version 2.0.10"
time="2015-09-02T10:08:18Z" level="fatal" msg="factory is not able to fill
the pool: gorethink: dial tcp <IP_ADDRESS>:28015: connection refused"
roy@ubuntu:~$ sudo docker version
Client:
Version: 1.8.1
API version: 1.20
Go version: go1.4.2
Git commit: d12ea79
Built: Thu Aug 13 02:35:49 UTC 2015
OS/Arch: linux/amd64
Server:
Version: 1.8.1
API version: 1.20
Go version: go1.4.2
Git commit: d12ea79
Built: Thu Aug 13 02:35:49 UTC 2015
OS/Arch: linux/amd64
I use webui deploy containers the post url is
http://<IP_ADDRESS>:8080/api/containers?count=1&pull=true
always error
get the message
Network timed out while trying to connect to
https://index.docker.io/v1/repositories/library/base_t3/images. You may
want to check your internet connection or if you are behind a proxy.
I manual post the url http://<IP_ADDRESS>:8080/api/containers is OK ,not
args
maybe something is wrong。
apologize for my english poor
—
Reply to this email directly or view it on GitHub
https://github.com/shipyard/shipyard/issues/586.

roy@ubuntu:/$ sudo docker ps -a
CONTAINER ID        IMAGE                      COMMAND                  CREATED             STATUS              PORTS                                                                         NAMES
08cf9df82b9d        shipyard/shipyard:latest   "/controller"            4 days ago          Up 50 minutes       <IP_ADDRESS>:8080->8080/tcp                                                        shipyard
b88ccec09a88        shipyard/rethinkdb         "/usr/bin/rethinkdb -"   4 days ago          Up 51 minutes       <IP_ADDRESS>:32770->8080/tcp, <IP_ADDRESS>:32769->28015/tcp, <IP_ADDRESS>:32768->29015/tcp   shipyard-rethinkdb
7860d5613b40        shipyard/rethinkdb         "/bin/bash -l"           4 days ago          Up 51 minutes       8080/tcp, 28015/tcp, 29015/tcp                                                shipyard-rethinkdb-data
rethinkdb is running.
images name is base_t3 is local images.
I use firefox open url http://<IP_ADDRESS>:8080/#/containers/deploy
Shift + Ctrl + I to developer tools .
deploy new container get 500 error
the post url is http://<IP_ADDRESS>:8080/api/containers?count=1&pull=true
I click the 500 POST and click headers ,check edit the resend
change the url is http://<IP_ADDRESS>:8080/api/containers
and click send ,response 200 and create new container.
network is Local Network
roy@ubuntu:/$ ifconfig eth0
eth0      Link encap:Ethernet  HWaddr da:ef:bd:0a:1d:1d
inet addr:<IP_ADDRESS>  Bcast:<IP_ADDRESS>  Mask:<IP_ADDRESS>

What is the output of docker logs shipyard

Sorry. I forget error message and I success deply containers message.
time="2015-09-07T02:39:47Z" level="warning" msg="unable to detect docker version: Not found"
time="2015-09-07T02:39:49Z" level="warning" msg="unable to detect docker version: Not found"
time="2015-09-07T02:39:57Z" level="warning" msg="unable to detect docker version: Not found"
time="2015-09-07T02:39:59Z" level="warning" msg="unable to detect docker version: Not found"
time="2015-09-07T02:40:03Z" level="warning" msg="error running container: Error: image library/base_t3: not found"
time="2015-09-07T02:40:07Z" level="warning" msg="unable to detect docker version: Not found"
time="2015-09-07T02:40:09Z" level="warning" msg="unable to detect docker version: Not found"
time="2015-09-07T02:40:17Z" level="warning" msg="unable to detect docker version: Not found"
time="2015-09-07T02:40:19Z" level="warning" msg="unable to detect docker version: Not found"
time="2015-09-07T02:40:27Z" level="warning" msg="unable to detect docker version: Not found"
time="2015-09-07T02:40:29Z" level="warning" msg="unable to detect docker version: Not found"
time="2015-09-07T02:40:37Z" level="warning" msg="unable to detect docker version: Not found"
time="2015-09-07T02:40:39Z" level="warning" msg="unable to detect docker version: Not found"
time="2015-09-07T02:40:47Z" level="warning" msg="unable to detect docker version: Not found"
time="2015-09-07T02:40:49Z" level="warning" msg="unable to detect docker version: Not found"
time="2015-09-07T02:40:57Z" level="warning" msg="unable to detect docker version: Not found"
time="2015-09-07T02:40:59Z" level="warning" msg="unable to detect docker version: Not found"
time="2015-09-07T02:41:02Z" level="info" msg="event: date=Mon Sep 07 02:41:02 +0000 2015 type=create image=base_t3 container=43a719f411db"
time="2015-09-07T02:41:02Z" level="info" msg="event: date=Mon Sep 07 02:41:02 +0000 2015 type=create image=base_t3 container=43a719f411db"
time="2015-09-07T02:41:02Z" level="info" msg="event: date=Mon Sep 07 02:41:02 +0000 2015 type=start image=base_t3 container=43a719f411db"
time="2015-09-07T02:41:02Z" level="info" msg="event: date=Mon Sep 07 02:41:02 +0000 2015 type=start image=base_t3 container=43a719f411db"
time="2015-09-07T02:41:07Z" level="warning" msg="unable to detect docker version: Not found"
time="2015-09-07T02:41:09Z" level="warning" msg="unable to detect docker version: Not found"
time="2015-09-07T02:41:17Z" level="warning" msg="unable to detect docker version: Not found"
time="2015-09-07T02:41:19Z" level="warning" msg="unable to detect docker version: Not found"

what version of docker are you using?


Update readme and complete doc

No code modification.

The current description field of github is:
Python package for covariance matrices manipulation and Biosignal classification with application in Brain Computer interface
I suggest an update:
Python machine learning package for biosignals processing and covariance matrices classification with Riemannian geometry
What dou you think? @sylvchev @agramfort @plcrodrigues
If you are ok, @agramfort, can you do the update (because I can't edit this field)?

Can't we say just symmetric positive definite matrices instead of covariance matrices ?

'Python machine learning package based on sklearn API for biosignals processing and statistical analysis of symmetric positive definite matrices using Riemannian geometry'

'Python machine learning package based on sklearn API for biosignals
processing and statistical analysis of symmetric positive definite matrices
using Riemannian geometry'Message ID:
@.***>

+1

Nice! We could update the description.

This description is a bit long, but ok for me!

symmetric positive definite matrices instead of covariance matrices
I agree to replace covariance matrices by positive definite matrices as much as possible in the doc.

Python machine learning package based on sklearn API for multivariate signal processing and statistical analysis of symmetric positive definite matrices via Riemannian geometry

Since we all agree, can you update the description @agramfort?

done !

Message ID: @.***>

Thanks @agramfort !
Do we update keywords? for eg, we could add statistics and riemannian-geometry.

@qbarthelemy I gave you admin rights so you can do the updates


[1/2]: Read in case data

[!IMPORTANT]
Do not merge. Using a stacked PR workflow.

Read the case data from a local parquet file in our standard ETL output schema. The various errors & warnings are linked to the tests via classes.
This approach preserves our existing handling of the data with respect to exclusions: it aggregates before there have been any exclusions. Applying exclusions to the aggregated data will be handled in the second stack in this PR.
I also add in here the data from Gostic, 2020. I use it in the unit tests to practice reading in data from tests/testthat/data/ but I also add it as package data (with documentation). We use this dataset in our end-to-end testing, so its addition here is also meant with an eye to that future use.

/document

/document

/document

So calling /document causes it to run the documentation CI for you?

Yes!

/style

/style

/document

This approach preserves our existing handling of the data with respect to exclusions: it aggregates before there have been any exclusions. Applying exclusions to the aggregated data will be handled in the second stack in this PR.

Just to note without context here that this sounds odd to me. If you aggregate data it'd seem to be harder to then retroactively go in and remove data points from it, rather than do the removing data first then to aggregate. I guess you must have a reason to be doing this. Since I guess now you'd have to be tracking somehow in the code all of the entries that went into a particular aggregate so that if you remove data from them you can remove data from the aggregate. (As in this seems more challenging than it need be.)

Yeah this is a good flag. I don't know if this repo is the right place to document it, but here's the gist of our ongoing discussion around data exclusions:

We receive and store facility-level data. These facilities are tagged with their corresponding state.
We want to produce state- and national-level estimates, so on-the-fly aggregate our stored data up to the desired level (this PR!)
We include right-truncation in reporting in the model. We provide a right-truncation PMF based on historical geographic-aggregate-level right-truncation.
Sometimes one geographic aggregate or another will have a one-off week where right-truncation empirically deviates from historical trend (e.g. 0 cases reported when we normally see ~some)
In those cases, we consult with the dataset expert and sometimes drop the single day with anomalous reporting for that particular geographic aggregate by converting the aggregate point to NA.
Sometimes the anomalous reporting is widespread enough that we judge that the modeling is not reflective of disease trends and exclude the whole state's model fit for the week.
This change does not propagate to other geographic aggregates (e.g. the US overall aggregate) or future report dates.

With the key question being, should the NA propagate? We've landed on "No, we should make judgement calls about each time series individually." But this discussion is still live and we're trying to get to a better place from both the data review/cleaning side and the modeling side.

/document


verify@x.com/http://Iheyreally.org

verify@x.com/http://iheyreally.org   > _ No description provided. _

#83

Originally posted by @Heyitsquoracom in https://github.com/microsoft/garnet/issues/83#issuecomment-2014514407
### Tasks
- [x] https://github.com/microsoft/garnet/issues/110

### Tasks

### Tasks

<EMAIL_ADDRESS>### Tasks
- [x] Add a draft title or issue reference here

### Tasks
- [x] Add a draft title or issue reference here

mhm spam


update border color

👷 Deploy Preview for fancy-puffpuff-df775f processing.

Name
Link

🔨 Latest commit
1836e3965c84cd20d4544e3777db4a562c06298f

🔍 Latest deploy log
https://app.netlify.com/sites/fancy-puffpuff-df775f/deploys/665b3959175834000863bc8d


Workspace for course on advanced modelling and Python's basics

The goal of the issue is to create a folder for the participants of the course consisting of all of the guides required for the course.

The code was checked with the following notebook: https://cocalc.com/projects/b51ce971-5b39-4911-ad97-ef59f15f0039/files/README FIRST.ipynb#id=894929


Issue with using single quotes

There seems to be an issue when using single quotes and inserting a URL, it leave the first single quote alone but changes the last to a double quote, results in errors.
e.g.
@import url('https://fonts.googleapis.com/css?family=Open+Sans');
gets changed to
@import url('https://fonts.googleapis.com/css?family=Open+Sans");
macOS 10.14
VSCode 1.27.2
Sass Formatter 1.1.6
Other extensions I have installed for sass are SCSS IntelliSense and stylelint. I have tried disabling these, but it is only when I disable the Sass Formatter plugin that it stops doing this.
Thanks in advance.

I'm no longer maintain this extension. You can find more details in the project README. So I'll close this issue.


[BUG] Cannot run DeepSpeed with transformers on NVIDIA Tesla T4 GPU

Describe the bug
Cannot run DeepSpeed with transformers on the ubuntu 20.04 with single GPU. GPU: NVIDIA T4
Error:
[2022-06-08 01:32:19,612] [INFO] [engine.py:132:__init__] Place model to device: 0
Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.
!!!! kernel execution error. (m: 7680, n: 3, k: 2560, error: 13)
Traceback (most recent call last):
  File "test.py", line 19, in <module>
    string = generator("DeepSpeed is", do_sample=True, min_length=50)
  File "/usr/local/lib/python3.8/dist-packages/transformers/pipelines/text_generation.py", line 175, in __call__
    return super().__call__(text_inputs, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/transformers/pipelines/base.py", line 1026, in __call__
    return self.run_single(inputs, preprocess_params, forward_params, postprocess_params)
  File "/usr/local/lib/python3.8/dist-packages/transformers/pipelines/base.py", line 1033, in run_single
    model_outputs = self.forward(model_inputs, **forward_params)
  File "/usr/local/lib/python3.8/dist-packages/transformers/pipelines/base.py", line 943, in forward
    model_outputs = self._forward(model_inputs, **forward_params)
  File "/usr/local/lib/python3.8/dist-packages/transformers/pipelines/text_generation.py", line 213, in _forward
    generated_sequence = self.model.generate(input_ids=input_ids, **generate_kwargs)  # BS x SL
  File "/usr/local/lib/python3.8/dist-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/transformers/generation_utils.py", line 1310, in generate
    return self.sample(
  File "/usr/local/lib/python3.8/dist-packages/transformers/generation_utils.py", line 1926, in sample
    outputs = self(
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1128, in _call_impl
    result = forward_call(*input, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/transformers/models/gpt_neo/modeling_gpt_neo.py", line 739, in forward
    transformer_outputs = self.transformer(
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/transformers/models/gpt_neo/modeling_gpt_neo.py", line 618, in forward
    outputs = block(
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/deepspeed/ops/transformer/inference/transformer_inference.py", line 647, in forward
    self.attention(input,
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/deepspeed/ops/transformer/inference/transformer_inference.py", line 390, in forward
    output = DeepSpeedSelfAttentionFunction.apply(
  File "/usr/local/lib/python3.8/dist-packages/deepspeed/ops/transformer/inference/transformer_inference.py", line 322, in forward
    output, key_layer, value_layer, context_layer, inp_norm = selfAttention_fp()
  File "/usr/local/lib/python3.8/dist-packages/deepspeed/ops/transformer/inference/transformer_inference.py", line 277, in selfAttention_fp
    qkv_out = qkv_func(input,
ValueError: Specified device cuda:0 does not match device of data cuda:-2

To Reproduce
Create container
docker pull nvidia/cuda:11.3.0-cudnn8-devel-ubuntu20.04

nvidia-docker run -it -d nvidia/cuda:11.3.0-cudnn8-devel-ubuntu20.04
docker exec -it <your conatiner>  /bin/bash

Inside container
sudo apt-get update
apt-get install python3-dev python3-pip vim curl
pip3 install --upgrade pip
pip3 install torch --extra-index-url https://download.pytorch.org/whl/cu113
pip3 install deepspeed transformers triton==1.0.0

Code to run
Grab the code from tutorial: https://www.deepspeed.ai/tutorials/inference-tutorial/#:~:text=DeepSpeed-Inference introduces several features,to reduce latency for inference.
run
deepspeed --num_gpus 1 gpt-neo-2.7b-generation.py

import os
import deepspeed
import torch
from transformers import pipeline

local_rank = int(os.getenv('LOCAL_RANK', '0'))
world_size = int(os.getenv('WORLD_SIZE', '1'))
generator = pipeline('text-generation', model='EleutherAI/gpt-neo-2.7B',
                     device=local_rank)

generator.model = deepspeed.init_inference(generator.model,
                                           mp_size=world_size,
                                           dtype=torch.float,
                                           replace_method='auto',
					   replace_with_kernel_inject=True)

string = generator("DeepSpeed is", do_sample=True, min_length=50)
if not torch.distributed.is_initialized() or torch.distributed.get_rank() == 0:
    print(string)

ds_report output
Please run ds_report to give us details about your setup.
root@e463e31466e3:/# ds_report
--------------------------------------------------
DeepSpeed C++/CUDA extension op report
--------------------------------------------------
NOTE: Ops not installed will be just-in-time (JIT) compiled at
      runtime if needed. Op compatibility means that your system
      meet the required dependencies to JIT install the op.
--------------------------------------------------
JIT compiled ops requires ninja
ninja .................. [OKAY]
--------------------------------------------------
op name ................ installed .. compatible
--------------------------------------------------
cpu_adam ............... [NO] ....... [OKAY]
cpu_adagrad ............ [NO] ....... [OKAY]
fused_adam ............. [NO] ....... [OKAY]
fused_lamb ............. [NO] ....... [OKAY]
sparse_attn ............ [NO] ....... [OKAY]
transformer ............ [NO] ....... [OKAY]
stochastic_transformer . [NO] ....... [OKAY]
 [WARNING]  async_io requires the dev libaio .so object and headers but these were not found.
 [WARNING]  async_io: please install the libaio-dev package with apt
 [WARNING]  If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
async_io ............... [NO] ....... [NO]
utils .................. [NO] ....... [OKAY]
quantizer .............. [NO] ....... [OKAY]
transformer_inference .. [NO] ....... [OKAY]
--------------------------------------------------
DeepSpeed general environment info:
torch install path ............... ['/usr/local/lib/python3.8/dist-packages/torch']
torch version .................... 1.11.0+cu113
torch cuda version ............... 11.3
torch hip version ................ None
nvcc version ..................... 11.3
deepspeed install path ........... ['/usr/local/lib/python3.8/dist-packages/deepspeed']
deepspeed info ................... 0.6.5, unknown, unknown
deepspeed wheel compiled w. ...... torch 1.11, cuda 11.3

Hi @lanking520, I just tried all your repro steps above and was not able to repro the stack trace.  Can you confirm what transformers version you are using? I tried w. 4.19.2. I don't currently have access to a T4 to test there but I tried on A6000, A100, and V100 and all ran okay. I am trying to get a T4 to test there and will report back.

Also, just to double check, you can run fine if you remove deepspeed.init_inference right?

@jeffra Here is the pip list
root@b963730b133d:/# pip3 list
Package            Version
------------------ ------------
certifi            2<IP_ADDRESS>
charset-normalizer 2.0.12
deepspeed          0.6.5
filelock           3.7.1
hjson              3.0.2
huggingface-hub    0.7.0
idna               3.3
ninja              <IP_ADDRESS>
numpy              1.22.4
packaging          21.3
pip                20.0.2
psutil             5.9.1
py-cpuinfo         8.0.0
pyparsing          3.0.9
PyYAML             6.0
regex              2022.6.2
requests           2.27.1
setuptools         45.2.0
tokenizers         0.12.1
torch              1.11.0+cu113
tqdm               4.64.0
transformers       4.19.2
triton             1.0.0
typing-extensions  4.2.0
urllib3            1.26.9
wheel              0.34.2

After commenting the deepspeed.init_inference, the problem just gone
Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.
[{'generated_text': 'DeepSpeed is a simple, clean and powerful app for controlling the speed of your phone, tablet or laptop.\n\nIn the past we have added more features to the simple to use, clean and powerful SpeedControl app. For the first version we'}]

Do you think this might be related to the NVLink support?

@jeffra I will also try to switch to V100 and see if I can reproduce the same issue

@lanking520,
Thanks for trying this on multiple GPUs, I will try this on my end to see what the issue is. I will let you know once I fix it.
Regarding T4 machine, could you please try this with half-precision too and see if the issue persists? This kind of issue normally happens when there is some memory-allocation issue. I will look closer and let you know.
Best,
Reza

@RezaYazdaniAminabadi
I'm observing the same behavior on an A100-SXM4-40GB system.  M so de]," better"). S not like or E en],\u2013 n better so]," better E]," de second?" P'}]
I've tried running different versions of torch and cuda without any change in behavior.

Just raised a separate issue here: https://github.com/microsoft/DeepSpeed/issues/2113 Still reproducible on 0.6.7

@lanking520, I think there are two separate issues in this thread.

single T4 inference w. deepspeed causing a kernel crash in your first post.
multi-gpu inference producing garbage results for certain models.

Let's focus this issue on (1) and focus on (2) in #2113.
I finally have access to a T4 and I am still unable to reproduce the original issue. I've tried the same torch/deepspeed/transformers versions as you originally reported and also the latest version of each. I am also using your same docker container and setup.
Can you confirm that original issue is still reproduceable?
root@fd8d0eb38aec:/# nvidia-smi
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.82.01    Driver Version: 470.82.01    CUDA Version: 11.4     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Tesla T4            Off  | 00000001:00:00.0 Off |                  Off |
| N/A   37C    P8    11W /  70W |      0MiB / 16127MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

root@fd8d0eb38aec:/# ds_report
--------------------------------------------------
DeepSpeed C++/CUDA extension op report
--------------------------------------------------
NOTE: Ops not installed will be just-in-time (JIT) compiled at
      runtime if needed. Op compatibility means that your system
      meet the required dependencies to JIT install the op.
--------------------------------------------------
JIT compiled ops requires ninja
ninja .................. [OKAY]
--------------------------------------------------
op name ................ installed .. compatible
--------------------------------------------------
cpu_adam ............... [NO] ....... [OKAY]
cpu_adagrad ............ [NO] ....... [OKAY]
fused_adam ............. [NO] ....... [OKAY]
fused_lamb ............. [NO] ....... [OKAY]
sparse_attn ............ [NO] ....... [OKAY]
transformer ............ [NO] ....... [OKAY]
stochastic_transformer . [NO] ....... [OKAY]
 [WARNING]  async_io requires the dev libaio .so object and headers but these were not found.
 [WARNING]  async_io: please install the libaio-dev package with apt
 [WARNING]  If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
async_io ............... [NO] ....... [NO]
utils .................. [NO] ....... [OKAY]
quantizer .............. [NO] ....... [OKAY]
transformer_inference .. [NO] ....... [OKAY]
--------------------------------------------------
DeepSpeed general environment info:
torch install path ............... ['/usr/local/lib/python3.8/dist-packages/torch']
torch version .................... 1.11.0+cu113
torch cuda version ............... 11.3
torch hip version ................ None
nvcc version ..................... 11.3
deepspeed install path ........... ['/usr/local/lib/python3.8/dist-packages/deepspeed']
deepspeed info ................... 0.6.5, unknown, unknown
deepspeed wheel compiled w. ...... torch 1.11, cuda 11.3

root@fd8d0eb38aec:/# pip3 list
Package            Version
------------------ ------------
certifi            2022.6.15
charset-normalizer 2.1.0
deepspeed          0.6.5
filelock           3.7.1
hjson              3.0.2
huggingface-hub    0.8.1
idna               3.3
ninja              <IP_ADDRESS>
numpy              1.23.1
packaging          21.3
pip                20.0.2
psutil             5.9.1
py-cpuinfo         8.0.0
pyparsing          3.0.9
PyYAML             6.0
regex              2022.7.25
requests           2.28.1
setuptools         45.2.0
tokenizers         0.12.1
torch              1.11.0+cu113
tqdm               4.64.0
transformers       4.19.2
triton             1.0.0
typing-extensions  4.3.0
urllib3            1.26.11
wheel              0.34.2

Here's the entire log of the run using your code snippet as well: https://gist.github.com/jeffra/b6966e155a57ec388444e13dd8b66402
The generated text is:
[{'generated_text': 'DeepSpeed is one of the most prominent and recognized companies in the world, with its presence on the world stage and a highly recognizable brand name. Fast forward twenty years and FastCompany.com has documented the evolution of FAST as a company, from'}]

@jeffra  Just tested again with single GPU, the error is gone. I also tested multi-gpu and facing some NCCL issue. I think this might leads to the machine is not installed with NVLink. I will close this issue and lets leave the rest in #2113


[Question] Is This an Anti-Pattern?

I have a promise middleware that handle promises, this middleware dispatches a new action with the same type but with a different payload on success/failure.
I want to know if it's a anti-pattern or is not recommended:
export function saveState() {
  return (dispatch, getState) => {
    return dispatch({
      type: SAVE_STATE,
      payload: localforage.setItem('state', getState())
    })
  }
}

For this use case, dispatching an action without an action creator is ok?

Dispatching an action without an action creator is always OK. Action creators are just a popular way of organizing the code. If you don’t like them or feel they are not needed in a particular case, don’t use them. It is completely up to you.

Thank you @gaearon !

While not an anti-pattern, I tend to do the following...  All action types are defined in a constants module, this way I'm less inclined to mis-type in the action creator, or reducer side... I also use redux-actions to create simple action creators, this way I always dispatch(actionName(args)) even when using redux-thunk, which I use with async actions (via babel).
I also break my code up into ./feature/feature-name/subfeature-name... in that I have base reducers.js that then references each feature's reducers... from there, each feature has it's own constants, these I prefix...
// feature/foo/constants.js
const DO_SOMETHING = 'FOO_DO_SOMETHING';

So that my action types won't conflict between features...  each feature may also have an api.js module, so that the action creators' logic is separated from any calls into foreign data/systems.
It does take a little discipline to do this, but in the end, adding features is a manner of mostly repeating these patterns...  components/pages/controls are under their respective feature, and I also have some common bits under feature/shared/*


[BUG] unable to save shapes on max edges

Added some leeway so it does not reach the exact max value on the x or y coordinates.
Suspect error was due to floating point numbers actually exceeding the canvas
tested:

drew a horizontal OBB that reaches the btm or right side of canvas
drew a OBB within canvas and moved its vertex to the btm and right side of canvas
drew a obb within canvas and rotate its vertex such that a corner of the bb would be touching the btm or right side of canvas

Those are the scenarios that had trouble before and they were solved. please test it and see if you could think of other scenarios

tested additional situation
drew a OBB within canvas and moved it to the btm and right side of canvas

Mass move will now limit box movement to within the canvas

Tested multiple boxes;
Mouse over, drawing and loading of annotations;
Seems great!


Can't install with new version of vuex-orm-next

Describe the bug
cannot install this plugin because it depends on an older version
Steps to reproduce the bug
npm i @vuex-orm/plugin-axios-next
Expected behavior
installs
Versions

Vuex ORM Axios: 1.0.0-draft-2
Vuex ORM: 1.0.0-draft-10

Link to minimal reproduction
Additional context
Any chance this will get updated to work with newer versions?

yarn add @vuex-orm/plugin-axios@next
and
npm i @vuex-orm/plugin-axios@next
seem to work :)

yarn add @vuex-orm/plugin-axios@next and npm i @vuex-orm/plugin-axios@next seem to work :)

As you can see in https://github.com/vuex-orm/plugin-axios-next/blob/0fd3fdf70a77c1868ed38f68037ec418bd76ac7c/package.json#L63 there's a dev dependency on "@vuex-orm/core": "1.0.0-draft.4".
Trying to install this with vuex-orm-next "@vuex-orm/core": "^1.0.0-draft.14" results in this error:
npm ERR! Found<EMAIL_ADDRESS>npm ERR! node_modules/@vuex-orm/core
npm ERR<EMAIL_ADDRESS>from the root project
npm ERR!
npm ERR! Could not resolve dependency:
npm ERR! peer<EMAIL_ADDRESS>from<EMAIL_ADDRESS>npm ERR! node_modules/@vuex-orm/plugin-axios
npm ERR<EMAIL_ADDRESS>from the root project
npm ERR!
npm ERR! Fix the upstream dependency conflict, or retry
npm ERR! this command with --force, or --legacy-peer-deps
npm ERR! to accept an incorrect (and potentially broken) dependency resolution.

I don't think it's an advisable approach to use the --force option, is it?
I don't understand why it works for you. Maybe an already installed older version of vuex-orm where you tried it?
I'm quite sure if you try this with a new project, you'll get the same error.

@danger89
Nothing is done in this project yet. The first draft you see here is just a shell.


Getting celery task ID

Hi, I'm wrapping my function to on_commit function from this project.
I had an issue to get celery task ID from this transaction hook.
auto = connection.on_commit(
    lambda: auto_received.apply_async(
        args=[rent_order.id],
        eta=booking.return_date + timedelta(hours=49)
    )
)

the auto variable is None, and I want to get the celery task id.
Thank in advance

As of Django 1.9, the on_commit function has been added to core Django, so this third-party package will not receive any changes other than possibly backports of critical fixes made to on_commit in core Django (for as long as Django 1.8 is supported). So if you want to propose a new feature for on_commit, it needs to be proposed on the core Django mailing list / issue tracker.
It's not possible to do what you want to do here; your on_commit callback isn't called until later, so it's simply not possible to return the Celery task id when you call on_commit, as it doesn't even exist yet. You will need to update your on_commit callback itself to store the Celery task ID in some other storage where you can retrieve it later.


Lockfile breaks walker options

When walker --help is used it becomes impossible to use walker again without manually removing /tmp/walker.lock. Suggested fix:

Don't create a lockfile for options like --help
If there is a lockfile, users should still be able to use options like --help

should be fixed.

Fixed, thanks!

Broken again in 0.0.11

should be fixed.. again.


Fixing broken AKS/AGIC sample

@lonegunmanb When I try that AG definition with the 3.x provider, I get syntax errors:

It looks like user_assigned_identity_id is no longer available, it should be identity_ids now.
Same issue to role_based_access_control block, it should be role_based_access_control_enabled now.
AzureRM provider 3.0.0 has deprecated those two arguments and nested blocks.

@TomArcherMsft is there an update on this? is it still not working

I don't know how to resolve this. I need help to get it to work. @lonegunmanb, is this something with which you or one of your devs can help me?


WIP: Yi Home Cameras

Description:
Adds support for YI Home Cameras running the alternative firmware for Xiaomi Cameras based on Hi3518e Chipset. Awaiting some underlying library changes before pressing forward.
Related issue (if applicable): N/A
Pull request in home-assistant.github.io with documentation (if applicable): TBD
Example entry for configuration.yaml (if applicable):
camera:
  - platform: yi
    name: Kitchen Camera
    host: '<IP_ADDRESS>'
    username: root
    password: my_password_123
    ffmpeg_arguments: '-vf scale=800:450'

Checklist:
If user exposed functionality or configuration variables are added/changed:

[ ] Documentation added/updated in home-assistant.github.io

If the code communicates with devices, web services, or third-party tools:

[x] Local tests with tox run successfully. Your PR cannot be merged unless tests pass
[x] New dependencies have been added to the REQUIREMENTS variable (example).
[x] New dependencies are only imported inside functions that use them (example).
[x] New dependencies have been added to requirements_all.txt by running script/gen_requirements_all.py.
[x] New files were added to .coveragerc.

@balloob: I've rewritten this using ftplib. One area of concern I have has to do with image retrieval. Since I'm a relative newbie to async programming, I'd appreciate your input.
I'm using @pvizeli's [ha-ffmpeg](https://github.com/pvizeli/ha-ffmpeg) library to process the MP4, which, I believe, necessitates using the async_camera_image method (instead of camera_image). Unless I'm mistaken, doesn't line 122 represent IO going on inside the event loop?
I'm using this as a custom component in my HASS install and everything appears fine (camera is updating, no timer desynchronization, etc.), but would love to be sure.

Whoa – not sure what happened with that rebase. Going to close this out and submit a new PR (with the latest code).


SVProgressHUDDidReceiveTouchEventNotification is not fired

I use alert this way:
[SVProgressHUD showErrorWithStatus:@"Bla bla" maskType:SVProgressHUDMaskTypeClear];

and I don't receive SVProgressHUDDidReceiveTouchEventNotification notification because self.overlay.userInteractionEnabled depends on maskType https://github.com/TransitApp/SVProgressHUD/blob/master/SVProgressHUD/SVProgressHUD.m#L630
But given maskType is lost here. https://github.com/TransitApp/SVProgressHUD/blob/master/SVProgressHUD/SVProgressHUD.m#L159

It is working for me.

I reproduced this issue here:
https://github.com/kalys/SVProgressHUD-bug-reproduce/blob/master/ololo/ViewController.m#L30

I recently rechecked. The SVProgressHUDDidReceiveTouchEventNotification is firing, also integrated for test in the demo app.

I think it's only fire if maskType is set to SVProgressHUDMaskTypeNone
SVProgressHUD.setDefaultMaskType(.none)

see here: https://github.com/SVProgressHUD/SVProgressHUD/blob/master/SVProgressHUD/SVProgressHUD.m#L889-L897


topic addition for better community reach

Hey,
for better reach of the community, I would like to suggest to add the following topic to the repository: motorola-68000
How to do this

Just added, thanks for the suggestion :)


Embassy book?

One of the things I liked very much with RTIC was that it came with a nicely written and very pedagogical book that I found very useful and taught me a lot, both technical / practical, but also 'philosophical' / how to think about embedded.
While I understand that RTIC may have needed a book also because it introduces so much non standard, I am wondering: could there be an embassy book, to introduce embassy and take n00bs like me 'by the hand' for helping learn embedded programming and how to think about it in an embassy context? :) .

We do have one: https://embassy.dev/book/dev/index.html


Val::frac for fractional value units

What problem does this solve or what need does it fill?
Val::Percent exists, but it's on a scale of 0.0 - 100.0 which feels unnatural considering fractional values are typically from 0.0 - 1.0. But this makes sense, since it's a percentage. This proposes a helper function on Val, pub fn frac(f: f32) -> Val, making a Val::Percent.
What solution would you like?
impl Val {
    pub fn frac(f: f32) -> Self {
        Self::Percent(f * 100.0)
    }
}

What alternative(s) have you considered?

Just using Percent but again this feels unnatural

Implementing that helper function myself:

pub trait ValFrac {
    fn frac(f: f32) -> Self;
}

impl ValFrac for Val {
    fn frac(f: f32) -> Self {
        Self::Percent(f * 100.0)
    }
}

Additional context

Another possibility would be to just make another variant of Val, which would be even nicer

The frac function feels a bit superfluous.
I don't like using percentages and if we were starting again I'd prefer Val::Frac over Val::Percent.
Adding a new variant seems easy but we already struggle to keep the helper functions and docs and tests updated and correct for all the enum variants that we have.

As noted in #9533 a new variant seems unpopular for maintainability reasons, due to this I propose a function instead.
#9538 talks about working on a new UI system, which may be an opportunity to rediscuss how Val or a Val-like type would work. That discussion notes https://github.com/vizia/morphorm which you can look at its Units type for another implementation of the same thing. It uses a Percentage variant. Even if morphorm is not used, the details of how to express spacing units should be looked at.


Make it possible to create interval from semitones

I was looking for a way to create an interval from number of semitones from a reference note, something similar with music.js's Interval.fromSemitones().
It's a bit verbose (please let me know if there's a better way), but it works as expected:
teoria.Interval().fromSemitones(0).name()  // => 'unison'
teoria.Interval().fromSemitones(1).name()  // => 'second'

I also added a helpful method to transpose a note without mutating it:
c = teoria.note('c')
c.name()  // => 'c'
c.transposeNew(teoria.Interval().fromSemitones(1)).name() // => 'd'
c.name()  // => 'c' (no changes on original note)

With the original Note.transpose() method we change the note itself:
c = teoria.note('c')
c.name()  // => 'c'
c.transpose(teoria.Interval().fromSemitones(1)).name() // => 'd'
c.name()  // => 'd' (note changed)

Please let me know if there's something I can improve on the PR. I'm actually using it on this live coding language where it's handy to create notes from intervals defined by semitones.

Im currently working on a music theory library as well, and had to tackle the same issue.
A number of semitones is not enough to define an interval, because of the existence of enharmonic intervals (ex augmented fourth and diminished 5th both have a distance of 6 semitones).
If you only care about the pitch class or the frequency of the resulting notes, this is not a problem, and will indeed work, but this can not be implemented naively in a musical theory library imo

AS @AtActionPark is saying, this isn't really possible to do "correctly", because an interval of 6 semitones can both be an #4 or b5, and 2 semitones can be M2 og d3. What I think is doable is to make a function that returns all possible intervals for a given semitone-defined interval. So:
Interval.fromSemitones(6) -> ['A4', 'd5']

ordered after what interval is most common. (M2 over d3, etc).

I think i've already implemented something like that in https://github.com/saebekassebil/piu - maybe you could rip that functionality out, make it its own module, and I'd be happy to include it in teoria :)


UI support for unmount cd-rom

support for unmount cd-rom disk
 spec:
      domain:
        cpu:
          cores: 4
        devices:
          disks:
          - bootOrder: 2
            cdrom:
              bus: sata
            name: isodisk1
          - bootOrder: 1
            disk:
              bus: virtio
            name: isodisk2
          - cdrom:
              bus: sata
              readonly: true
            name: cloudinitdisk

for CD-ROM type show a checkbox before it, and an eject button, after the user selects some CD-ROM and click the eject button, pop out a confirm dialogue.
eject CD-ROM

confirm dialogue:
Eject CD-ROM
Are you sure you want to eject CD-ROM name1, name2 ...., this action will restart the virtual machine

link to https://github.com/rancher/harvester/pull/99 , backend is merged.

Validated


Demo Wallet Banner

Overview
After you log in with a demo wallet, you can switch to a real wallet. However, the demo wallet banner stays at the top of the screen after logging in with a real wallet.
References and additional details

go to app.shapeshift.com
log in to a demo wallet
log in to another wallet on the demo wallet dashboard
once logged in with your wallet, noticed the demo wallet banner stays at the top of the screen

Acceptance Criteria
Demo wallet banner should disappear after logging in with a real wallet
Need By Date
No response
Screenshots/Mockups

Estimated effort
No response

Might have been introduced in https://github.com/shapeshift/web/pull/2410


When will WebSocketBundle be available for Symfony 4?

I thought it would already be possible to use WebSocketBundle with Symfony 4 ...
When will this be possible?
Or what am I doing wrong, if it is already possible?

I am running this bundle on Symfony 4.1 with no problems at all. What are your issues exactly?

I suppose he meant that there is no recipe and if you want to install it with composer require it will fail.
@JackFy before you try to install, create a new config file in the /config/packages/dev/web-socket.yaml and place the default config into it.
Then do a composer require and it will be installed.

@JackFy Did the solution proposed by @ghostika take an effect? Can we close this issue?

@vlachmilan
Hi, which version you use for Symfony 4.1? I can set it up, but it doesn't work. The client gets always disconnected. See #351

As @mbabker answer in the related issue, you have a message that said your firewall is not configured on gos config.
I continue the topic on dedicated thread, I clode this one to avoid multi discussion. See you there


Windows Phone 8.1: setOrientation not supported on device

Hello,
I freaked out when I saw, the plugin now supports Windows Phone 8. I thought, WP 8.1 might also be supported, but no luck :(
Is there any chance that you integrate also some code for the latest Windows Phone bild which is 8.1?
Or should it work?
Anyway: Thanks for your great plugin!
Bye Defcon

Hi @Defcon0 ,
I didn't write the WP8 side and don't have a development environment set up.  If you have one and can debug and resolve the issue please submit a pull request.
Unless anyone else can help??

I'm affraid, I'm not that experienced with cordova/WP 8.1 so that I could do that :( Maybe someone  more experienced could have a look.

I think I've worked this out, you can set the autoRotationPreferences straight from Javascript in 8.1 so if you change the setOrientation function in screenorientation.js to:
screenOrientation.setOrientation = function (orientation) {

    switch (orientation) {
        case 'portrait-primary':
            Windows.Graphics.Display.DisplayInformation.autoRotationPreferences = Windows.Graphics.Display.DisplayOrientations.portrait;
            break;
        case 'landscape-secondary':
            Windows.Graphics.Display.DisplayInformation.autoRotationPreferences = Windows.Graphics.Display.DisplayOrientations.landscapeFlipped;
            break;
        case 'portrait-secondary':
            Windows.Graphics.Display.DisplayInformation.autoRotationPreferences = Windows.Graphics.Display.DisplayOrientations.portraitFlipped;
            break;
        case 'landscape-primary':
            Windows.Graphics.Display.DisplayInformation.autoRotationPreferences = Windows.Graphics.Display.DisplayOrientations.landscape;
            break;
        default:
            Windows.Graphics.Display.DisplayInformation.autoRotationPreferences = Windows.Graphics.Display.DisplayOrientations.portrait;
    }

    // Get the current orientation
    //Windows.UI.ViewManagement.ApplicationView.getForCurrentView().orientation;
};

You'll get a really basic version working, the only issue is if you start the app with multiple orientations supported then you'll lose those the first time you set it to something new. You can set the autoRotationPrefences to allow for multiple rotations as follows:
Windows.Graphics.Display.DisplayInformation.autoRotationPreferences = [Windows.Graphics.Display.DisplayOrientations.portrait, Windows.Graphics.Display.DisplayOrientations.landscape];

But I haven't checked how it works on the other platforms, should it just save the default preferences when it starts?

Hi @DeadJoshWalking,
Each platform has it's own javascript file so you should be able to put this in the .wp8.js file.  You could possiblly check for the JS objects and then if found push to the native cordova exec call.
i.e.
if (Windows && Windows.Graphics && Windows.Graphics.Display) {
     // do switch here
} else {
    // exec()
}

If you have a windows environment setup, I'm happy to merge a tested PR.

resolved in #65


python chip-device-ctrl fails to connect with raspi on Mac

python chip-device-ctrl fails to connect with raspi on Mac. Lighting App used.
chip-device-ctrl > ble-scan-connect -t 120 1383
2021-01-12 18:54:55,094 ChipBLEMgr   INFO     adding to scan list:
2021-01-12 18:54:55,094 ChipBLEMgr   INFO
2021-01-12 18:54:55,094 ChipBLEMgr   INFO     Name            = ubuntuserver
2021-01-12 18:54:55,094 ChipBLEMgr   INFO     ID              = C17171C1-7421-4325-AB9C-3F40A8DF8169
2021-01-12 18:54:55,094 ChipBLEMgr   INFO     RSSI            = -59
2021-01-12 18:54:55,094 ChipBLEMgr   INFO     Pairing State   = 0
2021-01-12 18:54:55,094 ChipBLEMgr   INFO     Discriminator   = 1383
2021-01-12 18:54:55,094 ChipBLEMgr   INFO     Vendor Id       = 9050
2021-01-12 18:54:55,094 ChipBLEMgr   INFO     Product Id      = 65279
2021-01-12 18:54:55,094 ChipBLEMgr   INFO     ADV data: {
kCBAdvDataChannel = 37;
kCBAdvDataIsConnectable = 1;
kCBAdvDataLocalName = ubuntuserver;
kCBAdvDataServiceData =     {
FEAF = {length = 7, bytes = 0x0067055a23fffe};
};
kCBAdvDataServiceUUIDs =     (
FEAF
);
}
2021-01-12 18:54:55,094 ChipBLEMgr   INFO
2021-01-12 18:54:55,095 ChipBLEMgr   INFO     scanning stopped
2021-01-12 18:54:55,095 ChipBLEMgr   INFO     trying to connect to 1383
2021-01-12 18:55:10,095 ChipBLEMgr   INFO     connect fail
chip-device-ctrl > 2021-01-12 18:55:10,200 ChipBLEMgr   INFO     BLE disconnected, error = None

https://github.com/project-chip/connectedhomeip/issues/4399
Since there is another issue related to DevCtrl and BLE, I think these two issue can be fixed together

https://github.com/project-chip/connectedhomeip/issues/4399
Since there is another issue related to DevCtrl and BLE, I think these two issue can be fixed together

Cannot reproduce on my side (running macOS X 10.15.7), please consider restart linux chip-tool-server and chip-device-ctrl on mac.

Cannot reproduce on my side (running macOS X 10.15.7), please consider restart linux chip-tool-server and chip-device-ctrl on mac.

tried both i.e. restarting the chip-dev-ctrl and chip-tool-server (tried on linux, raspi and M5) , in either of the cases dev ctrl failed to connect. I am on Mac BigSur
chip-device-ctrl > ble-scan-connect -t 120 CHIP-3480
2021-01-22 16:18:04,419 ChipBLEMgr   INFO     BLE is ready!
2021-01-22 16:18:09,651 ChipBLEMgr   INFO     adding to scan list:
2021-01-22 16:18:09,651 ChipBLEMgr   INFO
2021-01-22 16:18:09,651 ChipBLEMgr   INFO     Name            = ubuntuserver
2021-01-22 16:18:09,651 ChipBLEMgr   INFO     ID              = C17171C1-7421-4325-AB9C-3F40A8DF8169
2021-01-22 16:18:09,651 ChipBLEMgr   INFO     RSSI            = -50
2021-01-22 16:18:09,651 ChipBLEMgr   INFO     Pairing State   = 0
2021-01-22 16:18:09,651 ChipBLEMgr   INFO     Discriminator   = 3840
2021-01-22 16:18:09,651 ChipBLEMgr   INFO     Vendor Id       = 9050
2021-01-22 16:18:09,652 ChipBLEMgr   INFO     Product Id      = 65279
2021-01-22 16:18:09,652 ChipBLEMgr   INFO     ADV data: {
kCBAdvDataChannel = 37;
kCBAdvDataIsConnectable = 1;
kCBAdvDataLocalName = ubuntuserver;
kCBAdvDataServiceData =     {
FEAF = {length = 7, bytes = 0x00000f5a23fffe};
};
kCBAdvDataServiceUUIDs =     (
FEAF
);
}
2021-01-22 16:18:09,652 ChipBLEMgr   INFO
2021-01-22 16:20:04,421 ChipBLEMgr   INFO     scanning stopped
2021-01-22 16:20:04,421 ChipBLEMgr   INFO     trying to connect to CHIP-3480
2021-01-22 16:20:04,421 ChipBLEMgr   INFO     connect fail
chip-device-ctrl > ble-scan-connect -t 120 CHIP-3480
2021-01-22 16:20:51,302 ChipBLEMgr   INFO     adding to scan list:
2021-01-22 16:20:51,302 ChipBLEMgr   INFO
2021-01-22 16:20:51,303 ChipBLEMgr   INFO     Name            = linuxboxathome
2021-01-22 16:20:51,303 ChipBLEMgr   INFO     ID              = B44C1B0B-3B1B-46E4-9699-87BBE6D53759
2021-01-22 16:20:51,303 ChipBLEMgr   INFO     RSSI            = -52
2021-01-22 16:20:51,303 ChipBLEMgr   INFO     Pairing State   = 0
2021-01-22 16:20:51,303 ChipBLEMgr   INFO     Discriminator   = 3840
2021-01-22 16:20:51,303 ChipBLEMgr   INFO     Vendor Id       = 9050
2021-01-22 16:20:51,303 ChipBLEMgr   INFO     Product Id      = 65279
2021-01-22 16:20:51,303 ChipBLEMgr   INFO     ADV data: {
kCBAdvDataChannel = 37;
kCBAdvDataIsConnectable = 1;
kCBAdvDataLocalName = linuxboxathome;
kCBAdvDataServiceData =     {
FEAF = {length = 7, bytes = 0x00000f5a23fffe};
};
kCBAdvDataServiceUUIDs =     (
FEAF
);
}
2021-01-22 16:20:51,303 ChipBLEMgr   INFO
2021-01-22 16:22:50,526 ChipBLEMgr   INFO     scanning stopped
2021-01-22 16:22:50,526 ChipBLEMgr   INFO     trying to connect to CHIP-3480
2021-01-22 16:22:50,527 ChipBLEMgr   INFO     connect fail
chip-device-ctrl > ble-scan-connect -t 120 CHIP-3480
2021-01-22 16:51:29,229 ChipBLEMgr   INFO     adding to scan list:
2021-01-22 16:51:29,229 ChipBLEMgr   INFO
2021-01-22 16:51:29,229 ChipBLEMgr   INFO     Name            = CHIP-3840
2021-01-22 16:51:29,230 ChipBLEMgr   INFO     ID              = E27D6633-2301-4061-A5C9-1401ED6F864F
2021-01-22 16:51:29,230 ChipBLEMgr   INFO     RSSI            = -44
2021-01-22 16:51:29,232 ChipBLEMgr   INFO     Pairing State   = 0
2021-01-22 16:51:29,232 ChipBLEMgr   INFO     Discriminator   = 3840
2021-01-22 16:51:29,232 ChipBLEMgr   INFO     Vendor Id       = 9050
2021-01-22 16:51:29,232 ChipBLEMgr   INFO     Product Id      = 65279
2021-01-22 16:51:29,233 ChipBLEMgr   INFO     ADV data: {
kCBAdvDataChannel = 37;
kCBAdvDataIsConnectable = 1;
kCBAdvDataLocalName = "CHIP-3840";
kCBAdvDataServiceData =     {
FEAF = {length = 7, bytes = 0x00000f5a23fffe};
};
}
2021-01-22 16:51:29,233 ChipBLEMgr   INFO
2021-01-22 16:53:27,228 ChipBLEMgr   INFO     scanning stopped
2021-01-22 16:53:27,230 ChipBLEMgr   INFO     trying to connect to CHIP-3480
2021-01-22 16:53:27,230 ChipBLEMgr   INFO     connect fail

tried both i.e. restarting the chip-dev-ctrl and chip-tool-server (tried on linux, raspi and M5) , in either of the cases dev ctrl failed to connect. I am on Mac BigSur
chip-device-ctrl > ble-scan-connect -t 120 CHIP-3480
2021-01-22 16:18:04,419 ChipBLEMgr   INFO     BLE is ready!
2021-01-22 16:18:09,651 ChipBLEMgr   INFO     adding to scan list:
2021-01-22 16:18:09,651 ChipBLEMgr   INFO
2021-01-22 16:18:09,651 ChipBLEMgr   INFO     Name            = ubuntuserver
2021-01-22 16:18:09,651 ChipBLEMgr   INFO     ID              = C17171C1-7421-4325-AB9C-3F40A8DF8169
2021-01-22 16:18:09,651 ChipBLEMgr   INFO     RSSI            = -50
2021-01-22 16:18:09,651 ChipBLEMgr   INFO     Pairing State   = 0
2021-01-22 16:18:09,651 ChipBLEMgr   INFO     Discriminator   = 3840
2021-01-22 16:18:09,651 ChipBLEMgr   INFO     Vendor Id       = 9050
2021-01-22 16:18:09,652 ChipBLEMgr   INFO     Product Id      = 65279
2021-01-22 16:18:09,652 ChipBLEMgr   INFO     ADV data: {
kCBAdvDataChannel = 37;
kCBAdvDataIsConnectable = 1;
kCBAdvDataLocalName = ubuntuserver;
kCBAdvDataServiceData =     {
FEAF = {length = 7, bytes = 0x00000f5a23fffe};
};
kCBAdvDataServiceUUIDs =     (
FEAF
);
}
2021-01-22 16:18:09,652 ChipBLEMgr   INFO
2021-01-22 16:20:04,421 ChipBLEMgr   INFO     scanning stopped
2021-01-22 16:20:04,421 ChipBLEMgr   INFO     trying to connect to CHIP-3480
2021-01-22 16:20:04,421 ChipBLEMgr   INFO     connect fail
chip-device-ctrl > ble-scan-connect -t 120 CHIP-3480
2021-01-22 16:20:51,302 ChipBLEMgr   INFO     adding to scan list:
2021-01-22 16:20:51,302 ChipBLEMgr   INFO
2021-01-22 16:20:51,303 ChipBLEMgr   INFO     Name            = linuxboxathome
2021-01-22 16:20:51,303 ChipBLEMgr   INFO     ID              = B44C1B0B-3B1B-46E4-9699-87BBE6D53759
2021-01-22 16:20:51,303 ChipBLEMgr   INFO     RSSI            = -52
2021-01-22 16:20:51,303 ChipBLEMgr   INFO     Pairing State   = 0
2021-01-22 16:20:51,303 ChipBLEMgr   INFO     Discriminator   = 3840
2021-01-22 16:20:51,303 ChipBLEMgr   INFO     Vendor Id       = 9050
2021-01-22 16:20:51,303 ChipBLEMgr   INFO     Product Id      = 65279
2021-01-22 16:20:51,303 ChipBLEMgr   INFO     ADV data: {
kCBAdvDataChannel = 37;
kCBAdvDataIsConnectable = 1;
kCBAdvDataLocalName = linuxboxathome;
kCBAdvDataServiceData =     {
FEAF = {length = 7, bytes = 0x00000f5a23fffe};
};
kCBAdvDataServiceUUIDs =     (
FEAF
);
}
2021-01-22 16:20:51,303 ChipBLEMgr   INFO
2021-01-22 16:22:50,526 ChipBLEMgr   INFO     scanning stopped
2021-01-22 16:22:50,526 ChipBLEMgr   INFO     trying to connect to CHIP-3480
2021-01-22 16:22:50,527 ChipBLEMgr   INFO     connect fail
chip-device-ctrl > ble-scan-connect -t 120 CHIP-3480
2021-01-22 16:51:29,229 ChipBLEMgr   INFO     adding to scan list:
2021-01-22 16:51:29,229 ChipBLEMgr   INFO
2021-01-22 16:51:29,229 ChipBLEMgr   INFO     Name            = CHIP-3840
2021-01-22 16:51:29,230 ChipBLEMgr   INFO     ID              = E27D6633-2301-4061-A5C9-1401ED6F864F
2021-01-22 16:51:29,230 ChipBLEMgr   INFO     RSSI            = -44
2021-01-22 16:51:29,232 ChipBLEMgr   INFO     Pairing State   = 0
2021-01-22 16:51:29,232 ChipBLEMgr   INFO     Discriminator   = 3840
2021-01-22 16:51:29,232 ChipBLEMgr   INFO     Vendor Id       = 9050
2021-01-22 16:51:29,232 ChipBLEMgr   INFO     Product Id      = 65279
2021-01-22 16:51:29,233 ChipBLEMgr   INFO     ADV data: {
kCBAdvDataChannel = 37;
kCBAdvDataIsConnectable = 1;
kCBAdvDataLocalName = "CHIP-3840";
kCBAdvDataServiceData =     {
FEAF = {length = 7, bytes = 0x00000f5a23fffe};
};
}
2021-01-22 16:51:29,233 ChipBLEMgr   INFO
2021-01-22 16:53:27,228 ChipBLEMgr   INFO     scanning stopped
2021-01-22 16:53:27,230 ChipBLEMgr   INFO     trying to connect to CHIP-3480
2021-01-22 16:53:27,230 ChipBLEMgr   INFO     connect fail

Hi, Chaitanya
I just reproduce your issue again with ble dongle
here is my observation and analysis
I attach external ble dongle, reboot, it shows
ubuntu@ubuntu:~/connectedhomeip$ hciconfig
hci0:	Type: Primary  Bus: USB
BD Address: 5C:F3:70:8A:0E:F0  ACL MTU: 1021:8  SCO MTU: 64:1
UP RUNNING PSCAN ISCAN
RX bytes:2096 acl:19 sco:0 events:108 errors:0
TX bytes:3942 acl:14 sco:0 commands:98 errors:0
which has hide the internal ble chip
python device controller in mac cannott make tthe ble cconnection to raspi4
then
i extract ble dongle, and reboot
it shows
ubuntu@ubuntu:~$ hciconfig hci0:	Type: Primary Bus: UART BD Address: DC:A6:32:8D:E0:57 ACL MTU: 1021:8 SCO MTU: 64:1 UP RUNNING PSCAN ISCAN RX bytes:1583 acl:0 sco:0 events:99 errors:0 TX bytes:2105 acl:0 sco:0 commands:99 errors:0
internal uart ble chip comes back
python device controller can make successful ble connection to raspi4 now
therefore
the root cause is that
once the external ble dongle is attached, ubuntu 20.04 masks the internal ble chip and show only usb bluetooth interface, which would confuse bluetooth-related connection from external

Hi, Chaitanya
I just reproduce your issue again with ble dongle
here is my observation and analysis
I attach external ble dongle, reboot, it shows
ubuntu@ubuntu:~/connectedhomeip$ hciconfig
hci0:	Type: Primary  Bus: USB
BD Address: 5C:F3:70:8A:0E:F0  ACL MTU: 1021:8  SCO MTU: 64:1
UP RUNNING PSCAN ISCAN
RX bytes:2096 acl:19 sco:0 events:108 errors:0
TX bytes:3942 acl:14 sco:0 commands:98 errors:0
which has hide the internal ble chip
python device controller in mac cannott make tthe ble cconnection to raspi4
then
i extract ble dongle, and reboot
it shows
ubuntu@ubuntu:~$ hciconfig hci0:	Type: Primary Bus: UART BD Address: DC:A6:32:8D:E0:57 ACL MTU: 1021:8 SCO MTU: 64:1 UP RUNNING PSCAN ISCAN RX bytes:1583 acl:0 sco:0 events:99 errors:0 TX bytes:2105 acl:0 sco:0 commands:99 errors:0
internal uart ble chip comes back
python device controller can make successful ble connection to raspi4 now
therefore
the root cause is that
once the external ble dongle is attached, ubuntu 20.04 masks the internal ble chip and show only usb bluetooth interface, which would confuse bluetooth-related connection from external


Add missing serialization

Serialization of some Enums still needs to be fixed: https://github.com/MagmaWorks/Taxonomy/pull/31/commits/54618a30157f2084cf411ab31609c640b200e9bc