Running npm install in a nodejs project may result in lost data or "Unknown system error -116"

Describe the bug
When running npm install in a NodeJS project that is on a SeaweedFS Fuse mount, unexpected behavior happens.
Either it fails with something like the following. One thing to note is that it fails on a different modules.
npm ERR! code Unknown system error -116
npm ERR! syscall rename
npm ERR! path<EMAIL_ADDRESS>npm ERR! dest /mnt/seaweedfs/tmp/my-app/node_modules/@types/uglify-js/node_modules/source-map
npm ERR! errno -116
npm ERR! Unknown system error -116: Unknown system error -116, rename<EMAIL_ADDRESS>-> '/mnt/seaweedfs/tmp/my-app/node_modules/@types/uglify-js/node_modules/source-map'

Or it results in the entire contents of the folder that it was working in being deleted.  In one case, it even lost the entire parent folder.
Reproduction Instructions
The following will download NodeJS, and try to create a react application, during that process it runs npm install
cd /mnt/seaweedfs
mkdir tmp
cd tmp
wget  https://nodejs.org/dist/v14.17.1/node-v14.17.1-linux-x64.tar.xz
tar xf  node-v14.17.1-linux-x64.tar.xz
export PATH=$PATH:`pwd`/node-v14.17.1-linux-x64/bin
npx create-react-app my-app

Alternative reproduction (with nodejs still being on the PATH):
cd /tmp
npx create-react-app my-app2
rm -rf my-app/node_modules
mv my-app2 /mnt/seaweedfs/tmp/
cd /mnt/seaweedfs/tmp/my-app2
npm install

System Setup
3 machines with single master and 3 volume servers.
version 30GB 2.56 a2979aa linux amd64
Master:

/srv/seaweedfs/weed master -mdir="/srv/seaweedfs/metadata"
/srv/seaweedfs/weed volume -max=30 -mserver=localhost:9333 -dir=/srv/seaweedfs/volumes
/srv/seaweedfs/weed filer -s3 -s3.config=/srv/seaweedfs/s3.config.json -master=localhost:9333
/srv/seaweedfs/weed mount -umask=000 -dir=/mnt/seaweedfs -collection=storage -filer=localhost:8888
filer.toml:

[leveldb2]
enabled = true
dir = "/srv/seaweedfs/filer"

Volume Servers:

/srv/seaweedfs/weed volume -max=30 -mserver=<IP_ADDRESS>:9333 -dir=/srv/seaweedfs/volumes
/srv/seaweedfs/weed mount -umask=000 -dir=/mnt/seaweedfs -collection=storage -filer=<IP_ADDRESS>:8888

Expected behavior
npm install should finish without any errors and the folder contents should stay where they are in any case.
Additional Context
The following lines appear in the error log on the master. This was collected during the reproduction run when the entire /mnt/seaweedfs/tmp folder was lost.
E0630 11:59:59 48653 dir.go:583] UpdateEntry dir /tmp/my-app2/node_modules/.staging/acorn-globals-146eb470: rpc error: code = Unknown desc = not found /tmp/my-app2/node_modules/.staging/acorn-globals-146eb470: filer: no entry is found in filer store
E0630 12:00:05 48653 dir.go:583] UpdateEntry dir /tmp/my-app2/node_modules/.staging/@types/webpack-sources-6888de33/lib: rpc error: code = Unknown desc = not found /tmp/my-app2/node_modules/.staging/@types/webpack-sources-6888de33/lib: filer: no entry is found in filer store
E0630 12:00:05 48653 meta_cache_subscribe.go:95] subscribing filer meta change: rpc error: code = Canceled desc = grpc: the client connection is closing
E0630 12:00:32 48653 dir.go:583] UpdateEntry dir<EMAIL_ADDRESS>rpc error: code = Unknown desc = not found<EMAIL_ADDRESS>filer: no entry is found in filer store

Seems working for me now after fixes:
nodejs$ npm i create-react-app
...
added 67 packages in 12s

4 packages are looking for funding
  run `npm fund` for details
npm notice
npm notice New minor version of npm available! 7.18.1 -> 7.19.1
npm notice Changelog: https://github.com/npm/cli/releases/tag/v7.19.1
npm notice Run npm install -g<EMAIL_ADDRESS>to update!
npm notice
nodejs$ ls
node_modules		package-lock.json	package.json
nodejs$ ls -al
total 85
drwxr-xr-x  1 chris  staff      0 Jul  1 01:29 .
drwxr-xr-x  1 chris  staff      0 Jul  2 23:52 ..
drwxr-xr-x  1 chris  staff      0 Jul  2 23:54 node_modules
-rw-r--r--  1 chris  staff  42651 Jul  2 23:54 package-lock.json
-rw-r--r--  1 chris  staff     61 Jul  2 23:54 package.json
nodejs$ pwd
/Users/chris/tmp/mm/nodejs
nodejs$ npx create-react-app my-app2

Creating a new React app in /Users/chris/tmp/mm/nodejs/my-app2.

Installing packages. This might take a couple of minutes.
Installing react, react-dom, and react-scripts with cra-template...

added 1923 packages in 4m

145 packages are looking for funding
  run `npm fund` for details

Initialized a git repository.

Installing template dependencies using npm...
npm http fetch GET 200
....................

added 43 packages in 10s

146 packages are looking for funding
  run `npm fund` for details
Removing template package using npm...

...

removed 1 package in 3s

146 packages are looking for funding
  run `npm fund` for details

Created git commit.

Success! Created my-app2 at /Users/chris/tmp/mm/nodejs/my-app2
Inside that directory, you can run several commands:

  npm start
    Starts the development server.

  npm run build
    Bundles the app into static files for production.

  npm test
    Starts the test runner.

  npm run eject
    Removes this tool and copies build dependencies, configuration files
    and scripts into the app directory. If you do this, you can’t go back!

We suggest that you begin by typing:

  cd my-app2
  npm start

Happy hacking!

During the finalization stage of it it still sometimes loses the entire directory here, but it isn't as consistent as it was before the fix.

Added one more fix related to this.

Unfortunately I still run into the same issue.
# /srv/seaweedfs/weed version
version 30GB 2.56 fa0dab60299e73acfb8bca72b50184722db8e3ab linux amd64

It hangs here:
/mnt/seaweedfs/tmp$ npx create-react-app my-app --template typescript
npx: installed 67 in 17.749s

Creating a new React app in /mnt/seaweedfs/tmp/my-app.

Installing packages. This might take a couple of minutes.
Installing react, react-dom, and react-scripts with cra-template-typescript...

><EMAIL_ADDRESS>postinstall /mnt/seaweedfs/tmp/my-app/node_modules/babel-runtime/node_modules/core-js
> node -e "try{require('./postinstall')}catch(e){}"

><EMAIL_ADDRESS>postinstall /mnt/seaweedfs/tmp/my-app/node_modules/core-js
> node -e "try{require('./postinstall')}catch(e){}"

><EMAIL_ADDRESS>postinstall /mnt/seaweedfs/tmp/my-app/node_modules/core-js-pure
> node -e "try{require('./postinstall')}catch(e){}"

><EMAIL_ADDRESS>postinstall /mnt/seaweedfs/tmp/my-app/node_modules/ejs
> node ./postinstall.js

+<EMAIL_ADDRESS>+<EMAIL_ADDRESS>+<EMAIL_ADDRESS>+<EMAIL_ADDRESS>added 1899 packages from 704 contributors and audited 1902 packages in 215.03s

145 packages are looking for funding
  run `npm fund` for details

found 5 vulnerabilities (3 moderate, 2 high)
  run `npm audit fix` to fix them, or `npm audit` for details

Initialized a git repository.

Installing template dependencies using npm...
[            ......] \ finalize:web-vitals: sill doSerial finalize 314

And at the same time:
/mnt/seaweedfs/tmp$ ls -lahs /mnt/seaweedfs/tmp/
total 0

was the weed binary updated on all servers and the mount?
seems the npx commands are stateful themselves. Is there any way to always execute the same steps?
It was fine on my local.
nodejs$ npx create-react-app my-app --template typescript

......

added 48 packages, and changed 1 package in 10s

146 packages are looking for funding
  run `npm fund` for details

We detected TypeScript in your project (src/App.test.tsx) and created a tsconfig.json file for you.

Your tsconfig.json has been populated with default values.

Removing template package using npm...

...

removed 1 package in 2s

146 packages are looking for funding
  run `npm fund` for details

Created git commit.

Success! Created my-app at /Users/chris/tmp/mm/nodejs/my-app
Inside that directory, you can run several commands:

  npm start
    Starts the development server.

  npm run build
    Bundles the app into static files for production.

  npm test
    Starts the test runner.

  npm run eject
    Removes this tool and copies build dependencies, configuration files
    and scripts into the app directory. If you do this, you can’t go back!

We suggest that you begin by typing:

  cd my-app
  npm start

Happy hacking!

was the weed binary updated on all servers and the mount?

Yes. I've stopped all services on all servers, updated it, started the master first and then on the other machines before mounting again.

seems the npx commands are stateful themselves. Is there any way to always execute the same steps?

You can also delete ~/.npm, but I didn't clean it up before reproducing.
One difference that I can see, is that I'm using a Linux client:
$ uname -a
Linux skymind-001 5.4.0-73-generic #82-Ubuntu SMP Wed Apr 14 17:39:42 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

$ cat /etc/issue
Ubuntu 20.04.2 LTS \n \l

And you seem to be on macos, so the underlying FUSE system is probably different.

Tried these with a docker linux instance:
cd docker/
make run_image
apk add tmux
tmux

Start Seaweed in one window
weed mount -dir=/mnt/seaweedfs -filer=<IP_ADDRESS>:8888

Start npm installation in another window
f4320db60165:~# cd /mnt/seaweedfs/
f4320db60165:/mnt/seaweedfs# ls
f4320db60165:/mnt/seaweedfs# mkdir nodejs
f4320db60165:/mnt/seaweedfs# cd nodejs/
f4320db60165:/mnt/seaweedfs/nodejs# apk add --update npm
fetch https://dl-cdn.alpinelinux.org/alpine/v3.14/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.14/community/x86_64/APKINDEX.tar.gz
(1/8) Installing ca-certificates (20191127-r5)
(2/8) Installing nghttp2-libs (1.43.0-r0)
(3/8) Installing brotli-libs (1.0.9-r5)
(4/8) Installing c-ares (1.17.1-r1)
(5/8) Installing libgcc (10.3.1_git20210424-r2)
(6/8) Installing libstdc++ (10.3.1_git20210424-r2)
(7/8) Installing nodejs (14.17.1-r0)
(8/8) Installing npm (7.17.0-r0)
Executing busybox-1.33.1-r2.trigger
Executing ca-certificates-20191127-r5.trigger
OK: 60 MiB in 28 packages
f4320db60165:/mnt/seaweedfs/nodejs# apk add git
(1/4) Installing libcurl (7.77.0-r1)
(2/4) Installing expat (2.4.1-r0)
(3/4) Installing pcre2 (10.36-r0)
(4/4) Installing git (2.32.0-r0)
Executing busybox-1.33.1-r2.trigger
OK: 72 MiB in 32 packages
f4320db60165:/mnt/seaweedfs/nodejs# npx create-react-app my-app

Creating a new React app in /mnt/seaweedfs/nodejs/my-app.

Installing packages. This might take a couple of minutes.
Installing react, react-dom, and react-scripts with cra-template...

added 1920 packages, and audited 1921 packages in 3m

145 packages are looking for funding
  run `npm fund` for details

19 vulnerabilities (9 moderate, 10 high)

To address all issues, run:
  npm audit fix

Run `npm audit` for details.

Initialized a git repository.

Installing template dependencies using npm...

added 43 packages, and audited 1964 packages in 11s

146 packages are looking for funding
  run `npm fund` for details

19 vulnerabilities (9 moderate, 10 high)

To address issues that do not require attention, run:
  npm audit fix

To address all issues (including breaking changes), run:
  npm audit fix --force

Run `npm audit` for details.
Removing template package using npm...

removed 1 package, and audited 1963 packages in 5s

146 packages are looking for funding
  run `npm fund` for details

19 vulnerabilities (9 moderate, 10 high)

To address issues that do not require attention, run:
  npm audit fix

To address all issues (including breaking changes), run:
  npm audit fix --force
Run `npm audit` for details.
Git commit not created Error: Command failed: git commit -m "Initialize project using Create React App"
    at checkExecSyncError (child_process.js:643:11)
    at execSync (child_process.js:679:15)
    at tryGitCommit (/mnt/seaweedfs/nodejs/my-app/node_modules/react-scripts/scripts/init.js:62:5)
    at module.exports (/mnt/seaweedfs/nodejs/my-app/node_modules/react-scripts/scripts/init.js:352:25)
    at [eval]:3:14
    at Script.runInThisContext (vm.js:134:12)
    at Object.runInThisContext (vm.js:311:38)
    at internal/process/execution.js:77:19
    at [eval]-wrapper:6:22
    at evalScript (internal/process/execution.js:76:60) {
  status: 128,
  signal: null,
  output: [ null, null, null ],
  pid: 326,
  stdout: null,
  stderr: null
}
Removing .git directory...

Success! Created my-app at /mnt/seaweedfs/nodejs/my-app
Inside that directory, you can run several commands:

  npm start
    Starts the development server.

  npm run build
    Bundles the app into static files for production.

  npm test
    Starts the test runner.

  npm run eject
    Removes this tool and copies build dependencies, configuration files
    and scripts into the app directory. If you do this, you can_t go back!

We suggest that you begin by typing:

  cd my-app
  npm start

Happy hacking!
f4320db60165:/mnt/seaweedfs/nodejs# npm install

up to date, audited 5 packages in 1s

found 0 vulnerabilities
f4320db60165:/mnt/seaweedfs/nodejs# npm audit
found 0 vulnerabilities

As I said, unfortunately it isn't exactly deterministic anymore after the last fixes you pushed, but it does still happen regularly here.
As I was testing some more, I noticed a new thing: I've been using that tmp folder over and over again during my testing, and now I can't even create a new file in it anymore:
/mnt/seaweedfs/tmp$ touch foo
touch: cannot touch 'foo': No such file or directory

And that continues to happen even after I've removed and recreated it - so that prefix seems to be effectively unusable now.
So I created a new folder and tried it again.
It again failed in a similar fashion as to the previous run:
npx create-react-app my-app2
npx: installed 67 in 6.261s

Creating a new React app in /mnt/seaweedfs/tmp2/my-app2.

Installing packages. This might take a couple of minutes.
Installing react, react-dom, and react-scripts with cra-template...

><EMAIL_ADDRESS>postinstall /mnt/seaweedfs/tmp2/my-app2/node_modules/babel-runtime/node_modules/core-js
> node -e "try{require('./postinstall')}catch(e){}"

><EMAIL_ADDRESS>postinstall /mnt/seaweedfs/tmp2/my-app2/node_modules/core-js
> node -e "try{require('./postinstall')}catch(e){}"

><EMAIL_ADDRESS>postinstall /mnt/seaweedfs/tmp2/my-app2/node_modules/core-js-pure
> node -e "try{require('./postinstall')}catch(e){}"

><EMAIL_ADDRESS>postinstall /mnt/seaweedfs/tmp2/my-app2/node_modules/ejs
> node ./postinstall.js

+<EMAIL_ADDRESS>+<EMAIL_ADDRESS>+<EMAIL_ADDRESS>+<EMAIL_ADDRESS>added 1899 packages from 704 contributors and audited 1902 packages in 151.512s

145 packages are looking for funding
  run `npm fund` for details

found 5 vulnerabilities (3 moderate, 2 high)
  run `npm audit fix` to fix them, or `npm audit` for details

Initialized a git repository.

Installing template dependencies using npm...
npm WARN<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>but none is installed. You must install peer dependencies yourself.
npm WARN<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>|| >= 3.2.0-dev || >= 3.3.0-dev || >= 3.4.0-dev || >= 3.5.0-dev || >= 3.6.0-dev || >= 3.6.0-beta || >= 3.7.0-dev || >= 3.7.0-beta but none is installed. You must install peer dependencies yourself.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY<EMAIL_ADDRESS>(node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for<EMAIL_ADDRESS>wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
npm WARN optional SKIPPING OPTIONAL DEPENDENCY<EMAIL_ADDRESS>(node_modules/webpack-dev-server/node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for<EMAIL_ADDRESS>wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
npm WARN optional SKIPPING OPTIONAL DEPENDENCY<EMAIL_ADDRESS>(node_modules/watchpack-chokidar2/node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for<EMAIL_ADDRESS>wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

+<EMAIL_ADDRESS>+<EMAIL_ADDRESS>+<EMAIL_ADDRESS>+<EMAIL_ADDRESS>added 29 packages from 78 contributors and audited 1931 packages in 12.822s

145 packages are looking for funding
  run `npm fund` for details

found 5 vulnerabilities (3 moderate, 2 high)
  run `npm audit fix` to fix them, or `npm audit` for details
Removing template package using npm...

[            ......] - finalize:fsevents: sill doSerial finalize 26

This time around I've got the following in the seaweedfs.logs in /tmp:
INFO:
I0704 08:34:36 56187 filer_grpc_server_sub_meta.go:204] => client filer:<IP_ADDRESS>:8888@<IP_ADDRESS>:47554: rpc error: code = Unavailable desc = transport is closing
E0704 08:34:36 56187 filer_grpc_server_sub_meta.go:127] processed to 2021-07-04 08:34:36.310570039 +0000 UTC: rpc error: code = Unavailable desc = transport is closing
I0704 08:34:36 56187 filer_grpc_server_sub_meta.go:204] => client filer:<IP_ADDRESS>:8888@<IP_ADDRESS>:47286: rpc error: code = Unavailable desc = transport is closing
E0704 08:34:36 56187 filer_grpc_server_sub_meta.go:127] processed to 2021-07-04 08:34:36.310570039 +0000 UTC: rpc error: code = Unavailable desc = transport is closing
I0704 08:34:36 56187 filer_grpc_server_sub_meta.go:204] => client mount@<IP_ADDRESS>:40204: rpc error: code = Unavailable desc = transport is closing
E0704 08:34:36 56187 filer_grpc_server_sub_meta.go:66] processed to 2021-07-04 08:34:36.310887609 +0000 UTC: rpc error: code = Unavailable desc = transport is closing
I0704 08:34:36 56187 filer_grpc_server_sub_meta.go:218] - listener mount@<IP_ADDRESS>:40204
I0704 08:34:37 56187 filer_grpc_server_sub_meta.go:218] - listener filer:<IP_ADDRESS>:8888@<IP_ADDRESS>:47286
I0704 08:34:37 56187 filer_grpc_server_sub_meta.go:218] - listener filer:<IP_ADDRESS>:8888@<IP_ADDRESS>:47554
I0704 08:34:47 56187 filer_grpc_server_sub_meta.go:204] => client mount@<IP_ADDRESS>:50078: rpc error: code = Unavailable desc = transport is closing
I0704 08:34:47 56187 filer_grpc_server_sub_meta.go:218] - listener mount@<IP_ADDRESS>:50078
I0704 08:34:48 56187 filer_grpc_server_sub_meta.go:204] => client mount@<IP_ADDRESS>:55706: rpc error: code = Unavailable desc = transport is closing
I0704 08:34:48 56187 filer_grpc_server_sub_meta.go:218] - listener mount@<IP_ADDRESS>:55706
I0704 08:34:48 56187 filer_grpc_server_sub_meta.go:204] => client mount@<IP_ADDRESS>:48828: rpc error: code = Unavailable desc = transport is closing
I0704 08:34:48 56187 filer_grpc_server_sub_meta.go:218] - listener mount@<IP_ADDRESS>:48828
I0704 08:34:48 56187 filer_grpc_server_sub_meta.go:204] => client mount@<IP_ADDRESS>:35448: rpc error: code = Unavailable desc = transport is closing
I0704 08:34:48 56187 filer_grpc_server_sub_meta.go:218] - listener mount@<IP_ADDRESS>:35448
I0704 08:39:56 56187 filer_grpc_server_sub_meta.go:204] => client mount@<IP_ADDRESS>:48918: rpc error: code = Unavailable desc = transport is closing
E0704 08:39:56 56187 filer_grpc_server_sub_meta.go:66] processed to 2021-07-04 08:39:56.535364 +0000 UTC: rpc error: code = Unavailable desc = transport is closing
I0704 08:39:56 56187 filer_grpc_server_sub_meta.go:218] - listener mount@<IP_ADDRESS>:48918
I0704 08:39:56 56187 filer_grpc_server_sub_meta.go:204] => client mount@<IP_ADDRESS>:43546: rpc error: code = Unavailable desc = transport is closing
E0704 08:39:56 56187 filer_grpc_server_sub_meta.go:66] processed to 2021-07-04 08:39:56.535364 +0000 UTC: rpc error: code = Unavailable desc = transport is closing
I0704 08:39:56 56187 filer_grpc_server_sub_meta.go:218] - listener mount@<IP_ADDRESS>:43546
I0704 08:40:47 56187 filer_grpc_server_sub_meta.go:100] read on disk filer:<IP_ADDRESS>:8888@<IP_ADDRESS>:51000 local subscribe / from 2021-07-04 08:40:46.712717104 +0000 UTC
I0704 08:40:49 56187 filer_grpc_server_sub_meta.go:115] read in memory filer:<IP_ADDRESS>:8888@<IP_ADDRESS>:51000 local subscribe / from 2021-07-04 08:40:47.308811913 +0000 UTC

ERROR:
E0704 08:34:36 56187 filer_grpc_server_sub_meta.go:127] processed to 2021-07-04 08:34:36.310570039 +0000 UTC: rpc error: code = Unavailable desc = transport is closing
E0704 08:34:36 56187 filer_grpc_server_sub_meta.go:127] processed to 2021-07-04 08:34:36.310570039 +0000 UTC: rpc error: code = Unavailable desc = transport is closing
E0704 08:34:36 56187 filer_grpc_server_sub_meta.go:66] processed to 2021-07-04 08:34:36.310887609 +0000 UTC: rpc error: code = Unavailable desc = transport is closing
E0704 08:39:56 56187 filer_grpc_server_sub_meta.go:66] processed to 2021-07-04 08:39:56.535364 +0000 UTC: rpc error: code = Unavailable desc = transport is closing
E0704 08:39:56 56187 filer_grpc_server_sub_meta.go:66] processed to 2021-07-04 08:39:56.535364 +0000 UTC: rpc error: code = Unavailable desc = transport is closing

WARNING: doesn't have anything related to that time frame.
So maybe it is something that only happens when multiple servers are involved? The connection between the servers is a rather simple though, they are all on the same network, and connected to the same network switch.

I tried again after https://github.com/chrislusf/seaweedfs/issues/2248
Everything seems running fine now.
Please help to verify if possible.

Closing for now.


fix python manage.py makemigrations

It fixes issue #12.

What exactly did you change? I don't see anything in your commit.

@eriktelepovsky I added a init.py.
The manage.py won't detect the migrations if there's not init.py in the folder.

Oh. I see now. Thanks.


Can you please host the pre-trained networks?

I do not have a GPU.
Training the network will take a long time, because I do not have a GPU.
I just want to be a cute girl.
Can you please upload the pretrained network weights?

Sorry, I don't think upload the pretrained model.
Because I used the dataset made from the VOICEROID product.
We can use Google Colaboratory that is the system usable GPU.
You can do it!


Bad URLS in responses

From Customer:
Please test the URLs embedded in your code. I got the following error message:

!!! It doesn't seem like DFU utilities are installed...

!!! For help with installing DFU Utilities, please see:
http://support.particle.io/hc/en-us/articles/203265730-Installing-the-Particle-CLI

Guess what: that URL doesn't work any more…

They have a point...

Does https://docs.particle.io/guide/tools-and-features/cli/#advanced-install look like a better url?


Fix: Base-json-serializer javascript

I confirm that this contribution is made under the terms of the license found in the root directory of this repository's source tree and that I have the authority necessary to make this contribution on behalf of its copyright owner.

I think @kevinhinterlong has this change in #1460


Jenkins build failures

Jenkins is failing on CrowdIn errors, due to no API key being found, and also due to syntax errors in messages.po.
It seems that CrowdIn have changed something in their integration to cause this. I'm investigating at the moment.

@tangentfairy could you point me to jenkins' output? thanks! :)

@dariocravero I do believe that Dean has someone who is going to look into this, but here it is anyway in case you have a bright idea:
[32m23 02 2016 14:25:18.355:INFO [PhantomJS 1.9.8 (Linux 0.0.0)]: Connected on socket /#VwkFyCv8sP9WqpmNAAAA with id 22076910
PhantomJS 1.9.8 (Linux 0.0.0) ERROR
  Error: Cannot find module '/var/lib/jenkins/jobs/cp-zen-platform-codedeploy/workspace/test/review-champion-application-spec.js'
  at /tmp/a5da887112fa69bbafec76e04a204426.browserify:1

PhantomJS 1.9.8 (Linux 0.0.0) ERROR
  Error: Cannot find module '/var/lib/jenkins/jobs/cp-zen-platform-codedeploy/workspace/test/stats-spec.js'
  at /tmp/a5da887112fa69bbafec76e04a204426.browserify:1

Unhandled rejection Error: Not running
    at Server.close (net.js:1237:11)
    at /var/lib/jenkins/jobs/cp-zen-platform-codedeploy/workspace/node_modules/karma/lib/server.js:342:17
    at tryCatcher (/var/lib/jenkins/jobs/cp-zen-platform-codedeploy/workspace/node_modules/karma/node_modules/bluebird/js/main/util.js:26:23)
    at Promise._settlePromiseFromHandler (/var/lib/jenkins/jobs/cp-zen-platform-codedeploy/workspace/node_modules/karma/node_modules/bluebird/js/main/promise.js:507:31)
    at Promise._settlePromiseAt (/var/lib/jenkins/jobs/cp-zen-platform-codedeploy/workspace/node_modules/karma/node_modules/bluebird/js/main/promise.js:581:18)
    at Promise._settlePromises (/var/lib/jenkins/jobs/cp-zen-platform-codedeploy/workspace/node_modules/karma/node_modules/bluebird/js/main/promise.js:697:14)
    at Async._drainQueue (/var/lib/jenkins/jobs/cp-zen-platform-codedeploy/workspace/node_modules/karma/node_modules/bluebird/js/main/async.js:123:16)
    at Async._drainQueues (/var/lib/jenkins/jobs/cp-zen-platform-codedeploy/workspace/node_modules/karma/node_modules/bluebird/js/main/async.js:133:10)
    at Async.drainQueues (/var/lib/jenkins/jobs/cp-zen-platform-codedeploy/workspace/node_modules/karma/node_modules/bluebird/js/main/async.js:15:14)
    at process._tickCallback (node.js:448:13)
npm ERR! Test failed.  See above for more details.
Build step 'Execute shell' marked build as failure
Finished: FAILURE

Yeah, I'd need more background on this I guess in order to even know where to start looking for that. At a first glance it seems like there might be some missing modules from the browserify bundle but your initial message says that it's related to CrowdIn so I wouldn't really know then...

Ah sorry, there are two issues! This one is about CrowdIn :) the issue is that it looks for an API key, and the solution is to add the crowdin.yaml file out of version control and update it on the server with the API key.
Someone internal needs to do this and it is assigned to me. Thanks anyway @dariocravero ! :)

Closing as we're gonna move to CircleCI and it's a non-blocking issue anyway ;)


Split ref_ops_test into multiple files

This sometimes takes forever to compile. We should split this across multiple files so it can compile in parallel.
EDIT:
The following ref tests under the test directory were moved to a separate ref directory:
ref_dot_op_test.cpp
ref_loop_test.cpp
ref_ops_nonstd_shape_test.cpp
ref_ops_test.cpp
ref_rnn_ops_test.cpp

Following changes were made:

Tests were stripped from ref_ops_test.cpp and ref_ops_nonstd_shape_test.cpp, and moved to separate files based on the type of operation.
Test cases having multiple tests were broken down to have a single test for each TEST_CASE.
CMakeLists.txt file was added to ref directory; all the tests are compiled to a single executable.

All of the new ref test source files have a false include dependency on
#include <migraphx/onnx.hpp>.  They should include #include <migraphx/program.hpp> instead.

All of the new ref test source files have a false include dependency on #include <migraphx/onnx.hpp>. They should include #include <migraphx/program.hpp> instead.

Yeah, you are right; onnx.hpp includes program.hpp which is what we actually need.
Thanks for catching this, I did not realize it because the original ref_ops_test.cpp file only included onnx.hpp (although no test seems to have needed this).
I have used this opportunity to remove unnecessary includes from some other files as well (loop.cpp, dot_op.cpp and rnn_ops.cpp).
If you find anything else, please do let me know. Thanks again!
PR is up for review with the changes: https://github.com/ROCmSoftwarePlatform/AMDMIGraphX/pull/2175


Use WindowedValue.withValue on hot paths #21250

This removed about half of the overhead for outputting a value in the common scenario where we are already using a valid timestamp (the input timestamp) and also that we can use the withValue hot path which is optimized for certain use cases (e.g. the globally windowed value case).
Before:
Benchmark                                Mode  Cnt     Score     Error  Units
ProcessBundleBenchmark.testLargeBundle  thrpt   15  3616.761 ± 157.844  ops/s

After:
Benchmark                                Mode  Cnt     Score     Error  Units
ProcessBundleBenchmark.testLargeBundle  thrpt   15  3666.889 ± 151.448  ops/s

This is for #21250.

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

R: @bhisevishal

retest this please

Run Java PreCommit

Precommits passed, github UI hasn't updated yet:
https://ci-beam.apache.org/job/beam_PreCommit_Java_Examples_Dataflow_Commit/22565/
https://ci-beam.apache.org/job/beam_PreCommit_Java_Examples_Dataflow_Java11_Commit/14601/


Add animations to all charts

Good work, Matt. However, expect pay to be deducted for your previously insolent behavior.


Request to Modify Message Sending Functionality in Claude-Telegram-Bot

Hello,
I have been using your Claude-Telegram-Bot and I am quite satisfied with its functionality. However, I have come across a requirement that I believe could be a valuable addition to the bot's capabilities.
Currently, when the bot is required to send a long message (more than 4000 characters) or to forward a code file, it attempts to send it as a text message. This can cause issues due to the maximum limit of characters in a single message imposed by Telegram.
I propose a modification to the bot's sending functionality. When a message exceeds 4000 characters or when a file needs to be forwarded, the bot could write the content to a file and send it as a document instead of a text message.
The modification could look something like this in the code:
def send_message_or_file(update, text):
if len(text) <= 4000:
bot.send_message(chat_id=update.effective_chat.id, text=text)
else:
with open('message.txt', 'w') as f:
f.write(text)
with open('message.txt', 'rb') as doc:
bot.send_document(chat_id=update.effective_chat.id, document=doc)
This function would replace all occurrences of bot.send_message.
I believe this would be a valuable enhancement to the bot's functionality, allowing it to handle longer texts and file forwarding more effectively. Could you consider implementing this change, or guide me through doing it myself if it aligns with the project's direction?
Thank you for considering my suggestion. I look forward to hearing your thoughts.
Best regards,
Ruslan
Projekt Xamelion AG

Thank you for your support and suggestions 😊. Implementing message segmentation involves a bit more complexity, considering the transitions between different responses and other functionalities (like with Google Bard), so it's not quite a simple fix 🤔. My schedule has been pretty packed lately, so it's tough to carve out time for updates . But hey, feel free to fork the repo and tweak it to your liking 👍!

Can it is possible to split the response when the character limit reached please tell me or guide me to implement this function


fix: throw error if ambiguous step definition implementation found

Proposed fix for issue #678.
I tried to use commitizen not sure if it went through correctly. Also not super experienced with JS unit tests so not sure what is needed there. Any advice much appreciated.
@badeball @lgandecki

Due to personal reasons, the previous maintainers of this package are stepping down and handing the reigns over to me, a long-time contributor to the project and a user of it myself. This is a responsibility I'm very excited about. Furthermore, I'd like to thank @lgandecki ++ for all the work that they've done so far.
Read more about the transfer of ownership here.
The repository has however moved and all outstanding issues are being closed. This is not a reflection of the perceived importance of your reported issue. However, if after upgrading to the new version, you still find there to be an issue, feel free to open up another ticket or comment below. Please make sure to read CONTRIBUTING.md before doing so.


[CI Problem][Flaky] test_meta_schedule_rpc_runner_exception

Branch/PR Failing
PR #9151, which was based on the main branch, and only had changes in an unrelated unit test.
Jenkins Link
https://ci.tlcpack.ai/blue/organizations/jenkins/tvm/detail/PR-9151/1/pipeline/#step-244-log-1263
Flakiness
Jenkins logs show tests/python/unittest/test_meta_schedule_runner.py::test_meta_schedule_rpc_runner_exception failing to establish a network connection with the tracker.  I've only seen this particular test fail the one time.  After restarting the CI with no changes, the test passed.

Should be fixed by #9172


Emulated hue id issue

Description:
Changed sequential id generated by emulated hue to use hash. Ids are reassigned to the same entities across restarts now. Google Home does not seem to evict devices from rooms any longer.
Related issue (if applicable): fixes #5048
Checklist:
If user exposed functionality or configuration variables are added/changed:

[ ] Documentation added/updated in home-assistant.github.io

If the code communicates with devices, web services, or third-party tools:

[x] Local tests with tox run successfully. Your PR cannot be merged unless tests pass
[ ] New dependencies have been added to the REQUIREMENTS variable (example).
[ ] New dependencies are only imported inside functions that use them (example).
[ ] New dependencies have been added to requirements_all.txt by running script/gen_requirements_all.py.
[ ] New files were added to .coveragerc.

If the code does not interact with devices:

[ ] Local tests with tox run successfully. Your PR cannot be merged unless tests pass
[ ] Tests have been added to verify that the new code works.

Pretty sure that when I tried this Google Home was parsing all IDs as integers and so anything that was a string ended up as 0.

If we know the maximum size it accepts we could always cast the first X bytes of the hash into an integer.

I tested this with my Google Home and all of the devices showed up. I assigned some to a room and restarted HA. Prior to this change, they'd jump all around or get removed from rooms but after this change they stay put and the JSON output from the API doesn't change.
I first tried just using the entity_id like Alexa does but my Home wouldn't find the devices even though the JSON output from the API was valid.

@cribbstechnologies just told us in the Gitter dev chat that a hash is still confusing Google Home which expects integers.
The correct solution would be to give each entity ID an integer ID and persist which entity id has been given across restarts (ie in a JSON file).


Performance cookbook

JAX has a lot of powerful building blocks for vectorizing and parallelizing code, but there are quite a few different APIs and using them well takes skill.
It would be nice to have a comprehensive "performance guide" that goes in depth into how to write performant code in JAX. Topics to include:

high level guidelines (e.g., "jit everything" and “watch out for excessive Python control flow” )
how to understand what’s being executed by JAX (e.g., use make_jaxpr)
how to profile code (e.g., using Tensorboard's profiler)
vectorizing code and using vmap/vectorize
performance/memory/compilation-time tradeoffs between vmap (batching)
lax.map (XLA loops) and map (Python looping).
parallelizing with pmap/soft_pmap (this probably needs a longer, separate introduction for collective operations)
backend specific guidance / gotchas:

low precision dtypes and matmul precision
how to use multiple cores on CPUs
speed of XLA loops (especially on GPUs)
padding on TPUs

custom call and dlpack for interfacing with external code

Writing up this documentation might be particularly helpful for understanding how these APIs might be improved (https://github.com/google/jax/issues/2939).

This is a priority for us.

Looking forward to reading this! Would be helpful to include practical advice for debugging slow compilation. Happened upon this, and a small variation led to an easy fix for my quadratic compile times. Here is the process...

Capture HLO with XLA_FLAGS="--xla_dump_hlo_as_text --xla_dump_to=xla_dump python run.py
Look for any outsized HLO with ls -Slh xla_dump/*before_optimizations.txt
Find unexpected repeated structures, especially reshapes on unexpectedly small arrays
Capture related stack traces within lax.py, in my case by adding here,  if new_shapes == repeated_shape: import inspect; print(inspect.stack())
Find the application-level function(s) creating unexpected repetition in HLO. In my case it was obvious and a simple fix.

It has been reassuring having even this inconvenient process at hand. Anything better would be icing on the cake.

@mattwescott glad that worked for you! Getting out lower level representations from JAX should definitely be part of this guide. Right now this looks rather painful for XLA!
One thing I’ll note is that often you can notice issues like quadratic compilation times just by looking at the size of JAX’s own IR, rather than going all the way to XLA. make_jaxpr is the utility function for pulling out this information.

Hi, chiming in on this. I'm wondering if there is any update? Unfortunately, it seems like not all existing tools for performance investigations (e.g. JAX_LOG_COMPILES) have been documented in the API reference until now.

+1, this would be great!

Sometimes using the profiler can be overwhelming and one is not sure exactly where to look to find the culprit of the slow runtime. It'd be lovely to have some guide on the cookbook on how to spot common mistakes specific to jax, e.g. unnecessary host-accelerator data transfer, recurrent jit compilations, etc.


Very Excellent instruction?Do you have any other documents?

Does EA have any documents for personal mod developers using python?
With you instruction, I can only type a hello command in game command window and I don't know any futher things...

And where we can find scripts.ini? Google search help me with winmm.dll only


Enable external storage shim (extstore).

Extstore info at: https://github.com/memcached/memcached/wiki/Extstore
Looks like it is maturing very quickly and would be worthwhile having access to it on Docker images.

Looks like we're running into a combination of https://github.com/memcached/memcached/issues/319 and later https://github.com/memcached/memcached/issues/374.

For reference, we support all of amd64, arm32v5, arm32v6, arm32v7, arm64v8, i386, ppc64le, and s390x, so I'd rather wait to merge this until the functionality is working on all supported platforms.

For the official builds it ought to be a release, but I'm happy to do some
tests on all our arches tomorrow from the next branch to help test! 🎊

Would be really useful, since I'd have to make another release if it didn't work :P

key:

:heavy_check_mark: success
:x: failure

arch
alpine
debian

amd64
:heavy_check_mark:
:heavy_check_mark:

arm32v5
N/A
:x:

arm32v6
:x:
N/A

arm32v7
N/A
:heavy_check_mark:

arm64v8
:x:??
:x:??

i386
:heavy_check_mark:
:heavy_check_mark:

ppc64le
:heavy_check_mark:
:heavy_check_mark:

s390x
:x:
:x:

https://gist.github.com/tianon/d3aa50500e22cf8686a08ae4fa2cbae1
(full build/test logs for ~every arch; see below -- git clone https://gist.github.com/d3aa50500e22cf8686a08ae4fa2cbae1.git xxx is probably the easiest way to consume/review given the length)
I'm confused about the arm64v8 failures, since https://github.com/memcached/memcached/commit/f939a8468cb4b9a219a1a34cdcef5338a8b84a77 should've covered that :confused: (but my assembly-foo is admittedly weak -- only ever really did z80 in any serious capacity :sweat_smile:).
also:
remote: error: File s390x-debian.log is 129.47 MB; this exceeds GitHub's file size limit of 100.00 MB
remote: error: File s390x-alpine.log is 129.44 MB; this exceeds GitHub's file size limit of 100.00 MB

There's one extstore-specific test that when it fails, it appears to spew some absolutely massive test data -- not sure how I'll share this massive log ATM or what to trim to get it down to a useful size without trimming useful data, and I think the offending test is t/chunked-extstore.t.  The arm32v6-alpine.log file looks pretty similar to the s390x files (same spewing output, but happened to be just barely under GitHub's 100MB limit so it's included in the gist).
Now, if your response to this is that we need to trim the list of architectures we support, that's fair too (we're not opposed to doing that), but the current image (which includes make test) does build successfully on all these combinations so IMO it'd be a shame to drop any (but again, totally willing to if that's your preference). :heart:
(Also, in the future, feel free to ping me directly on anything you'd like either multiarch or Docker tested before release -- always happy to help out where we can!! :heart: :heart:)

whaaat arm64v8 is that which doesn't have the crc32c instruction? Is there an march or cross compile issue?
the v5 failure doesn't seem to have anything to do with extstore but the v6 one looks crazy weird.
I get it though; the patch from qualcomm wasn't good enough for this. If the CPU instruction isn't known to the compiler it won't build, which is usally dealt with by replacing the symbol with the opcode. I can also add configure time tests which is a pain... but that arm64 really should have that instruction, my rpi3 does. Running tests on rpi2 now.
I bet chunked on the arm32's is something else... but repro'ing this might be a pain if my rpi2 won't do it (it's running tests now.. was too lazy to plug it in).
Is it possible to mask extstore for arm/390 but leave other options for the primary builds? Extstore doesn't make a ton of sense on small platforms, but it would on proper aarch64 server boards with lots of disk... those should build just fine with the crc instruction.

The arm64 box I was testing on is a HiSilicon Hi1616 chip (64 cores) graciously provided to us by the WorksOnArm project, so definitely a server chip, and we're compiling directly on that chip natively (no cross-compile, emulation, or anything like that), and thus even more odd that it didn't work.
I've also got access to an 8-core AppliedMicro XGene1, but it fails with a similar error:
/tmp/ccJlGhka.s: Assembler messages:
/tmp/ccJlGhka.s:156: Error: selected processor does not support `crc32cx w0,w0,x3'
/tmp/ccJlGhka.s:207: Error: selected processor does not support `crc32cb w0,w0,w2'
/tmp/ccJlGhka.s:217: Error: selected processor does not support `crc32ch w0,w0,w2'
/tmp/ccJlGhka.s:227: Error: selected processor does not support `crc32cw w0,w0,w1'
/tmp/ccJlGhka.s:241: Error: selected processor does not support `crc32cb w0,w0,w2'
/tmp/ccJlGhka.s:251: Error: selected processor does not support `crc32ch w0,w0,w1'
/tmp/ccJlGhka.s:264: Error: selected processor does not support `crc32cw w0,w0,w2'
/tmp/ccJlGhka.s:274: Error: selected processor does not support `crc32cb w0,w0,w1'

So maybe this instruction isn't well-supported across the breadth of arm64 chips?
I'm happy to re-test next on arm32v5 with extstore disabled again (the latest release works fine), and for the real builds we can definitely be choosy about which arches use this flag and which don't.
Regarding the IBM s390x mainframe stuff, I'll give a poke to some contacts I've got on that team and see if they've got any ideas (but for now, just planning to exclude extstore on s390x and arm32 is totally sane).

what platform is the arm32v5?
I haven't looked at the s390x failures, but I have no way of developing on one and am not sure of any users, certainly not extstore. I'd prefer to put my efforts to making sure arm works better. If it runs fine without extstore, please stick with that :)
All I have is a couple of RPI's though... one of the tests is flaky under 32bit mode, but they all do pass. I have an rpi2 in pure 32bit, and a rpi3 with a (painfully built) 64bit kernel running aarch64... also the patch I got which does this came from qualcomm, but unsure what they tested on exactly.
I'll do some blind googling and see if I can figure this out.

How much RAM do the armv5 and v6 platforms have?
Looks like some of my tests are flaky under 32bit mode, as it's trying to fill disk and expecting the page layout to be in specific forms. Staring hard at the failure for some of these that might just be the case.
armv5 failing normal chunked-items.t (not even extstore) might also be similar. unfortunately the test platform makes it hard to tell if the daemon just died during the test, or if it gave invalid output
re: v8, grep crc /proc/cpuinfo should say if that extension exists. I've been reading online that -march=armv8-a+crc is necessary to ensure the CRC instructions get built, but I don't see that being passed on my rpi3 and it's compiling fine. What version of GCC is on your v8 platforms?
I'll have to think about how to determine via configure if armv8-a+crc should be forced :/ might be as simple as "if I force this march and it works", but I have a deep fear of that accidentally compiling something that won't run if the target arch is actually v7 or something.

Alright, looks like I goofed my cross compile on the rpi3: it wasn't actually building with aarch64. You probably do need the -march to get the silly thing to compile.
For now, I've added a "--enable-arm-crc32" which gates the instructions for those who're familiar enough to build it properly. This is pushed to 'next' and you should try again with --enable-extstore but without --enable-arm-crc32.
The other platforms I can look at once you let me know how much RAM they have. I've been testing builds with 512mb. On the other hand, armv5 might be too old for me to want to really pursue.

what platform is the arm32v5?

We build them on that same HiSilicon chip (which has ~125Gb of RAM :sweat_smile:), although I've got a real one here at home (the "Pogoplug") that has 256mb of RAM.  Definitely 100% understand if you want to avoid wasting more cycles on this arch (it's very much on the way out -- generally has no hardware floating point chip, so it's about as speedy as you'd imagine).

I haven't looked at the s390x failures, but I have no way of developing on one and am not sure of any users, certainly not extstore. I'd prefer to put my efforts to making sure arm works better. If it runs fine without extstore, please stick with that :)

Yeah, I put out a feeler to IBM, but haven't seen a response so for now we'll just go extstore-less there. :+1:

... rpi3 with a (painfully built) 64bit kernel running aarch64 ...

Ouch.  I use https://wiki.debian.org/RaspberryPi3 on mine, which has worked pretty well (arm64 Debian).

(also, thanks so much for testing! It's really hard to get access to these exotic platforms...)

Amen -- so, so true. :disappointed:

How much RAM do the armv5 and v6 platforms have?

Ours is a special case (we build on the HiSilicon), but AFAIK, in the real-world v6 is really only the RPi 1 and RPi Zero -- even the RPi 2 is a v7 chip (and the 3 is a 64bit v8, as you know).

grep crc /proc/cpuinfo

HiSilicon machine:
$ grep crc /proc/cpuinfo | sort -u
Features	: fp asimd evtstrm aes pmull sha1 sha2 crc32 cpuid
$

XGene1:
$ grep crc /proc/cpuinfo | sort -u
$ grep '^Features' /proc/cpuinfo | sort -u
Features	: fp asimd evtstrm
$

:confused: :disappointed:
Is there any way to do runtime detection on arm64 instead? (I mean, you could scrape /proc/cpuinfo, but that seems a little hacky, and probably would eat some of the speed benefits of using the native instruction?)

What version of GCC is on your v8 platforms?

That'd be https://pkgs.alpinelinux.org/package/v3.7/main/aarch64/gcc and https://packages.debian.org/stretch/gcc, so 6.4.0 on Alpine and 6.3.0 on Debian.

This is pushed to 'next' and you should try again with --enable-extstore but without --enable-arm-crc32.

Sure thing, firing off some fresh builds!

Ok, arm64v8 is now green on both Debian and Alpine (specifying --enable-extstore and not --enable-arm-crc32).  arm32v5, arm32v6, and s390x are still no bueno, but that's OK (we can skip them for extstore for now). :+1:
Is it still useful for me to give you the arm32 failing logs, or are you good?
The arm32v5 failure seems really strange given that it passes on the current release, but not on next (and I've verified that it still fails on next when I disable extstore). :confused:  (Maybe we'll have to drop that arch completely at this point)

RE: Runtime detection, that's exactly what the code is doing, but if you have a compiler error you can't fix that at runtime :P I'll need to revisit by either stubbing the opcodes like I mentioned earlier, or some other trick. Most likely there'll just be a configure option to force the march build. :/
Is arm32v5 not failing on 1.5.7 presently? I'm confused, since nothing in next since 1.5.7 actually touches non-extstore code unless you're using seccomp? If you're using seccomp, does disabling it fix anything?
If 1.5.7 passes, does fac34333294ae6a05e822498e3295ec45ce16de4 fail? (or just bisect it, I have no idea which commit would've caused chunked items to fail in that way)

Ah, I've been building on a different host than the real builds -- they're currently all using a QEMU VM on my local machine instead.  Trying again now on the proper hardware...  (We've got a number of arm32 builds that fail on our arm64 hardware, even though it supports almost all arm32 instructions.)

Ok, building on the proper hardware, https://github.com/memcached/memcached/commit/cf3b8538cf22eb014ae557330ac8720b254c82b5 works fine with --enable-extstore on arm32v5.  Re-testing arm32v6 now. :+1:

that's nuts! was the QEMU VM set to low RAM or something? I guess it could also be a timing issue, since I still have some of those in the test suite.

No, I think it must be a timing issue -- the failures were from the beefy HiSilicon box (the one with over 100GB of RAM and 64 cores). :sweat_smile:
The QEMU emulated machine is a more modest 1GB of RAM with only 4 cores.

Sorry for the confusion!  To hopefully clear things up: most of our arm32vX builds use the beefy HiSilicon box I've been testing this on, but memcached is one exception where we instead build on my QEMU VM for the official builds.
arm32v6 with --enable-extstore still fails on my QEMU VM, so we might need to exclude that one as well as s390x (which fails in the same way).

Got it.. well, let me know which ones still fail on the real hardware. I can work through the timing issues in the tests over a longer period of time, which should eventually help with QEMU stuff.

The ones that fail on the real hardware are down to just arm32v6 and s390x. :+1:

Alright, lets focus on arm32v6. is it still that huge spew of data? If so you can probably cut like... the middle 50% of it and give me the start and the end bits.

For arm32v6, https://gist.githubusercontent.com/tianon/d3aa50500e22cf8686a08ae4fa2cbae1/raw/6c3c69dfc95273a37501df4259cf8918ea23659a/arm32v6-alpine.log is accurate. :+1:

It fails in a slightly different way when I run it on a RPi2 in arm32v6 mode:
+ make test
./sizes
Slab Stats	64
Thread stats	-6464
Global stats	184
Settings	528
Item (no cas)	32
Item (cas)	40
extstore header	12
Libevent thread	104
Connection	352
----------------------------------------
libevent thread cumulative	6440
Thread stats cumulative		6336
./testapp
1..52
ok 1 - cache_create
ok 2 - cache_constructor
ok 3 - cache_constructor_fail
ok 4 - cache_destructor
ok 5 - cache_reuse
ok 6 - cache_redzone
ok 7 - issue_161
ok 8 - strtol
ok 9 - strtoll
ok 10 - strtoul
ok 11 - strtoull
ok 12 - issue_44
ok 13 - vperror
ok 14 - issue_101
Signal handled: Terminated.
ok 15 - start_server
ok 16 - issue_92
ok 17 - issue_102
ok 18 - binary_noop
ok 19 - binary_quit
ok 20 - binary_quitq
ok 21 - binary_set
ok 22 - binary_setq
ok 23 - binary_add
ok 24 - binary_addq
ok 25 - binary_replace
ok 26 - binary_replaceq
ok 27 - binary_delete
ok 28 - binary_deleteq
ok 29 - binary_get
ok 30 - binary_getq
ok 31 - binary_getk
ok 32 - binary_getkq
ok 33 - binary_gat
ok 34 - binary_gatq
ok 35 - binary_gatk
ok 36 - binary_gatkq
ok 37 - binary_incr
ok 38 - binary_incrq
ok 39 - binary_decr
ok 40 - binary_decrq
ok 41 - binary_version
ok 42 - binary_flush
ok 43 - binary_flushq
ok 44 - binary_append
ok 45 - binary_appendq
ok 46 - binary_prepend
ok 47 - binary_prependq
ok 48 - binary_stat
ok 49 - binary_illegal
ok 50 - binary_pipeline_hickup
Signal handled: Interrupt.
ok 51 - shutdown
ok 52 - stop_server
getaddrinfo(): Name does not resolve
failed to listen on TCP port 37893: Invalid argument
slab class   1: chunk size        80 perslab   13107
slab class   2: chunk size       104 perslab   10082
slab class   3: chunk size       136 perslab    7710
slab class   4: chunk size       176 perslab    5957
slab class   5: chunk size       224 perslab    4681
slab class   6: chunk size       280 perslab    3744
slab class   7: chunk size       352 perslab    2978
slab class   8: chunk size       440 perslab    2383
slab class   9: chunk size       552 perslab    1899
slab class  10: chunk size       696 perslab    1506
slab class  11: chunk size       872 perslab    1202
slab class  12: chunk size      1096 perslab     956
slab class  13: chunk size      1376 perslab     762
slab class  14: chunk size      1720 perslab     609
slab class  15: chunk size      2152 perslab     487
slab class  16: chunk size      2696 perslab     388
slab class  17: chunk size      3376 perslab     310
slab class  18: chunk size      4224 perslab     248
slab class  19: chunk size      5280 perslab     198
slab class  20: chunk size      6600 perslab     158
slab class  21: chunk size      8256 perslab     127
slab class  22: chunk size     10320 perslab     101
slab class  23: chunk size     12904 perslab      81
slab class  24: chunk size     16136 perslab      64
slab class  25: chunk size     20176 perslab      51
slab class  26: chunk size     25224 perslab      41
slab class  27: chunk size     31536 perslab      33
slab class  28: chunk size     39424 perslab      26
slab class  29: chunk size     49280 perslab      21
slab class  30: chunk size     61600 perslab      17
slab class  31: chunk size     77000 perslab      13
slab class  32: chunk size     96256 perslab      10
slab class  33: chunk size    120320 perslab       8
slab class  34: chunk size    150400 perslab       6
slab class  35: chunk size    188000 perslab       5
slab class  36: chunk size    235000 perslab       4
slab class  37: chunk size    293752 perslab       3
slab class  38: chunk size    367192 perslab       2
slab class  39: chunk size    524288 perslab       2
<26 server listening (auto-negotiate)
<27 new auto-negotiating client connection
<27 connection closed.
slab class   1: chunk size        80 perslab   13107
slab class   2: chunk size       104 perslab   10082
slab class   3: chunk size       136 perslab    7710
slab class   4: chunk size       176 perslab    5957
slab class   5: chunk size       224 perslab    4681
slab class   6: chunk size       280 perslab    3744
slab class   7: chunk size       352 perslab    2978
slab class   8: chunk size       440 perslab    2383
slab class   9: chunk size       552 perslab    1899
slab class  10: chunk size       696 perslab    1506
slab class  11: chunk size       872 perslab    1202
slab class  12: chunk size      1096 perslab     956
slab class  13: chunk size      1376 perslab     762
slab class  14: chunk size      1720 perslab     609
slab class  15: chunk size      2152 perslab     487
slab class  16: chunk size      2696 perslab     388
slab class  17: chunk size      3376 perslab     310
slab class  18: chunk size      4224 perslab     248
slab class  19: chunk size      5280 perslab     198
slab class  20: chunk size      6600 perslab     158
slab class  21: chunk size      8256 perslab     127
slab class  22: chunk size     10320 perslab     101
slab class  23: chunk size     12904 perslab      81
slab class  24: chunk size     16136 perslab      64
slab class  25: chunk size     20176 perslab      51
slab class  26: chunk size     25224 perslab      41
slab class  27: chunk size     31536 perslab      33
slab class  28: chunk size     39424 perslab      26
slab class  29: chunk size     49280 perslab      21
slab class  30: chunk size     61600 perslab      17
slab class  31: chunk size     77000 perslab      13
slab class  32: chunk size     96256 perslab      10
slab class  33: chunk size    120320 perslab       8
slab class  34: chunk size    150400 perslab       6
slab class  35: chunk size    188000 perslab       5
slab class  36: chunk size    235000 perslab       4
slab class  37: chunk size    293752 perslab       3
slab class  38: chunk size    367192 perslab       2
slab class  39: chunk size    524288 perslab       2
<26 server listening (ascii)
<27 new ascii client connection.
<27 connection closed.
slab class   1: chunk size        80 perslab   13107
slab class   2: chunk size       104 perslab   10082
slab class   3: chunk size       136 perslab    7710
slab class   4: chunk size       176 perslab    5957
slab class   5: chunk size       224 perslab    4681
slab class   6: chunk size       280 perslab    3744
slab class   7: chunk size       352 perslab    2978
slab class   8: chunk size       440 perslab    2383
slab class   9: chunk size       552 perslab    1899
slab class  10: chunk size       696 perslab    1506
slab class  11: chunk size       872 perslab    1202
slab class  12: chunk size      1096 perslab     956
slab class  13: chunk size      1376 perslab     762
slab class  14: chunk size      1720 perslab     609
slab class  15: chunk size      2152 perslab     487
slab class  16: chunk size      2696 perslab     388
slab class  17: chunk size      3376 perslab     310
slab class  18: chunk size      4224 perslab     248
slab class  19: chunk size      5280 perslab     198
slab class  20: chunk size      6600 perslab     158
slab class  21: chunk size      8256 perslab     127
slab class  22: chunk size     10320 perslab     101
slab class  23: chunk size     12904 perslab      81
slab class  24: chunk size     16136 perslab      64
slab class  25: chunk size     20176 perslab      51
slab class  26: chunk size     25224 perslab      41
slab class  27: chunk size     31536 perslab      33
slab class  28: chunk size     39424 perslab      26
slab class  29: chunk size     49280 perslab      21
slab class  30: chunk size     61600 perslab      17
slab class  31: chunk size     77000 perslab      13
slab class  32: chunk size     96256 perslab      10
slab class  33: chunk size    120320 perslab       8
slab class  34: chunk size    150400 perslab       6
slab class  35: chunk size    188000 perslab       5
slab class  36: chunk size    235000 perslab       4
slab class  37: chunk size    293752 perslab       3
slab class  38: chunk size    367192 perslab       2
slab class  39: chunk size    524288 perslab       2
<26 server listening (binary)
<27 new binary client connection.
<27 connection closed.
Invalid value for binding protocol: http
 -- should be one of auto, binary, or ascii
Maximum connections must be greater than 0
Maximum connections must be greater than 0
Number of threads must be greater than 0
t/00-startup.t .............. ok
t/64bit.t ................... skipped: Skipping 64-bit tests on 32-bit build
t/binary-extstore.t ......... ok
t/binary-get.t .............. ok
t/binary-sasl.t ............. skipped: Skipping SASL tests
t/binary.t .................. ok
t/bogus-commands.t .......... ok
t/cas.t ..................... ok
t/chunked-extstore.t ........ ok
t/chunked-items.t ........... ok
t/daemonize.t ............... ok
t/dash-M.t .................. ok
t/dyn-maxbytes.t ............ ok
t/evictions.t ............... ok
t/expirations.t ............. ok
t/extstore-buckets.t ........ ok

#   Failed test '0 pages are free'
#   at t/extstore.t line 100.
#          got: '1'
#     expected: '0'
# Looks like you failed 1 test of 27.
t/extstore.t ................
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/27 subtests
t/flags.t ................... ok
t/flush-all.t ............... ok
t/getandtouch.t ............. ok
t/getset.t .................. ok
t/idle-timeout.t ............ ok
t/incrdecr.t ................ ok
t/inline_asciihdr.t ......... ok
t/issue_104.t ............... ok
t/issue_108.t ............... ok
t/issue_14.t ................ ok
t/issue_140.t ............... skipped: Fix for Issue 140 was only an illusion
t/issue_152.t ............... ok
t/issue_163.t ............... ok
t/issue_183.t ............... ok
t/issue_192.t ............... ok
t/issue_22.t ................ ok
t/issue_260.t ............... skipped: Only possible to test #260 under artificial conditions
t/issue_29.t ................ ok
t/issue_3.t ................. ok
t/issue_41.t ................ ok
t/issue_42.t ................ ok
t/issue_50.t ................ ok
t/issue_61.t ................ ok
t/issue_67.t ................ ok
t/issue_68.t ................ ok
t/issue_70.t ................ ok
Item max size cannot be less than 1024 bytes.
Cannot set item size limit higher than 1/2 of memory max.
t/item_size_max.t ........... ok
t/line-lengths.t ............ ok
t/lru-crawler.t ............. ok
t/lru-maintainer.t .......... ok
t/lru.t ..................... ok
t/malicious-commands.t ...... ok
t/maxconns.t ................ ok
t/misbehave.t ............... skipped: Privilege drop not supported
t/multiversioning.t ......... ok
t/noreply.t ................. ok
t/quit.t .................... ok
t/refhang.t ................. skipped: Test is flaky. Needs special hooks.
t/slabhang.t ................ skipped: Test is flaky. Needs special hooks.
t/slabs-reassign-chunked.t .. ok
t/slabs-reassign2.t ......... ok
t/slabs_reassign.t .......... ok
PORT: 46715
t/stats-conns.t ............. ok
t/stats-detail.t ............ ok
t/stats.t ................... ok
t/touch.t ................... ok
t/udp.t ..................... ok
t/unixsocket.t .............. ok
t/watcher.t ................. ok
t/whitespace.t .............. skipped: Skipping tests probably because you don't have git.

Test Summary Report
-------------------
t/extstore.t              (Wstat: 256 Tests: 27 Failed: 1)
  Failed test:  20
  Non-zero exit status: 1
Files=67, Tests=64765, 532 wallclock secs (122.04 usr  5.04 sys + 213.23 cusr 32.06 csys = 372.37 CPU)
Result: FAIL
make: *** [Makefile:1873: test] Error 1
The command '/bin/sh -c set -x 		&& apk add --no-cache --virtual .build-deps 		autoconf automake 		ca-certificates 		coreutils 		cyrus-sasl-dev 		dpkg-dev dpkg 		gcc 	libc-dev 		libevent-dev 		libressl 		linux-headers 		make 		perl 		perl-utils 		tar 	&& wget -O memcached.tar.gz "https://github.com/memcached/memcached/archive/$MEMCACHED_COMMIT.tar.gz" 	&& mkdir -p /usr/src/memcached 	&& tar -xzf memcached.tar.gz -C /usr/src/memcached --strip-components=1 	&& rm memcached.tar.gz && cd /usr/src/memcached 		&& ./autogen.sh 	&& ./configure --build="$(dpkg-architecture --query DEB_BUILD_GNU_TYPE)" 		--enable-extstore 		--enable-sasl 	&& make -j "$(nproc)" 		&& make test 	&& make install 		&& cd / && rm -rf /usr/src/memcached 	&& runDeps="$( 		scanelf --needed --nobanner --format '%n#p' --recursive /usr/local 			| tr ',' '\n' 			| sort -u 	| awk 'system("[ -e /usr/local/lib/" $1 " ]") == 0 { next } { print "so:" $1 }' )" 	&& apk add --virtual .memcached-rundeps $runDeps 	&& apk del .build-deps 		&& memcached -V' returned a non-zero code: 2

that failure I'm familiar with. my rpi2 did that once but on repeated runs works okay. test is flaky but is passable. I'll boot my pi2 back up and try to improve that now I guess

Anything more I can provide to help debug the failure in the longer log?  That's from the machine that'll actually be building this once it's released, so it's more concerning than just a flaky test (since we automatically re-attempt builds several times to account for flaky failures when we build the real thing).

Try 'next' branch again, please?
Can no longer repro the second failure on my rpi2, and the at least start of the log spew is likely the same issue in the other test. I changed the pacing of the inserts and ramped up the count a bit to make up for extra space available on 32bit systems.

broke it in a different way, hold on...

okay, pushed 'next' for chunked item changes again. re-tuned a lot of things, and shrunk the number of tests by a lot. if it spews again the file should at least be smaller.

Nice!  I've tested --enable-extstore on the 1.5.8 release across our arches, and we're down to just s390x and arm32v6 failing -- logs for all builds can be found in https://gist.github.com/tianon/d3aa50500e22cf8686a08ae4fa2cbae1. :+1:
The money shots:

arm32v6: https://gist.githubusercontent.com/tianon/d3aa50500e22cf8686a08ae4fa2cbae1/raw/dbd5d5f501b04b95fad8fe2fc1aa666b16a1f44c/arm32v6-alpine.log
s390x: https://gist.githubusercontent.com/tianon/d3aa50500e22cf8686a08ae4fa2cbae1/raw/dbd5d5f501b04b95fad8fe2fc1aa666b16a1f44c/s390x-debian.log

Think I need to task off of this for a while; does the arm32v6 always fail in that same spot, or does it pass sometimes? I think that's still a pacing issue.
the s390x... just mask it off for now. would you mind opening an issue on the memcached project and link back to both logs? There're some other user facing bugs and some bench work I'd like to prioritize for the next week.
Thanks a ton for all your help so far... sorry my tests are so flaky :| Feel dumb it took so long to figure out the compactor race. No idea what's wrong with the s390x though.

No problem -- definitely not complaining, just trying to be helpful in one of the only ways I can! :smile:
Will file issues to track these further and we'll just mask out on those two arches for now. :+1:

(Yes, I think both test failures are pretty consistent -- with s390x, we've got both Debian and Alpine failing in exactly the same way.)

  On arm32v6, I replaced make test with make test || make test || make test || make test and it eventually passed. :sweat_smile:

Do you still want another issue just for arm32v6, or just chalk it up to general flakiness?
(Filed s390x at https://github.com/memcached/memcached/issues/381)

Yeah please do. The tests shouldn't be flaky, and I beat most of that out of them already. Dunno howtf your platform is so sensitive to it though. The compaction algorithms changed a bunch since I first wrote the tests, so they were due up for some work.

Done deal!  https://github.com/memcached/memcached/issues/382
(I've just confirmed on s390x, and that one is definitely not a flakiness issue -- it's gotta be something deeper. :disappointed:)

oh shoot... actually in t/chunked-extstore.t, can you try changing:
    print $sock "extstore drop_under 3\r\n";
to... 1? that was a typo.

 I tried :)


Add the ability up upload or link to documents

Investigate using sharepoint/OneDrive to store documents
Should be able to link documents to Properties,Tenancies,Certificates, Journals
Attributes
FileName
Folder
Description

Property main image now selectable from Sharepoint


GNOME Connections

The name of the app (both Flatpak and native installation) is org.gnome.Connections.

Duplicate of #9


Azure Cosmos MongoDB API - .NET Core Console App Tutorial

Azure Cosmos MongoDB API - .NET Core Console App Tutorial - Proposal
Going through the Azure Cosmos DB documentation here - https://docs.microsoft.com/en-us/azure/cosmos-db/, under the MongoDB API > Tutorials > 1. Create & Manage Data, i notice we have a tutorial on Node.JS Console App.
I want to propose a tutorial with .NET Core Console App. I have already created the sample app in my repo here: https://github.com/lohithgn/netcore-cosmos-mongo-consoleapp. This is based on the same lines as that of Node.JS Console app.
If the proposal is good/accepted - i can create the markdown page for the documentation.
Thoughts ???

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: af9cbd0c-dcec-8316-208c-9e92b92107f1
Version Independent ID: f9dd684a-06de-5f14-f81a-4d1003deb477
Content: Use Azure Cosmos DB's API for MongoDB to build a Node.js app
Content Source: articles/cosmos-db/mongodb-samples.md
Service: cosmos-db
Sub-service: cosmosdb-mongo
GitHub Login: @sivethe
Microsoft Alias: sivethe

I also noticed that on the Node.JS console app, there is a selector for tech stack. When we select .NET Core - it redirects to SQL API and not MongoDB API.

@lohithgn Thanks a lot for all your efforts of going over and beyond by creating a repo and letting us know.
I am assigning it to the content author to review and take further call on this.

@AnuragSharma-MSFT any update on this one ?

Please assign to @gahl-levy

#reassign @gahl-levy

@lohithgn my apologies for the extreme delay in responding to this issue.
I appreciate your effort to create a .NET sample for MongoDB that matches our Family database example. Since your initially opened this issue another on our team took an existing Web API MongoDB sample from the .NET docs and updated it to use as our .NET sample for MongoDB.
Thanks again for your contribution. I'm sorry we can't use it at this time. Please feel free to suggest other areas where we might improve as well as even submit PR's on our docs if you like. We are always open for direct customer fixes.
thanks again.
#please-close

assign:markjbrown

#assign:markjbrown


Fix code style issues in python files in scripts dir and add files to codestyle checking

Fix code style issues in python files in scripts directory and add the fixed files to codestyle checking.

@stefanberger can you add those scripts to the make check command so that don't reintroduce those issues again? Otherwise LGTM.

@stefanberger can you add those scripts to the make check command so that don't reintroduce those issues again? Otherwise LGTM.

Already done via patches starting with title codestyle:.

Already done via patches starting with title codestyle:.

Yep, I somehow skipped over that sorry.


Calling .toArray() on the actions of the ItemStackRequest packet throws errors.

Describe the bug (Required)
With the 1.19.30 it appears that ContainerMenu a package for BDSX to use in plugins is broken, however after looking at it, it appears to be an issue with BDSX and not the package. Anyways, the code calls .toArray() and it appears the second time is where the error of invalid vector size is being thrown.

When
To Reproduce (Required)
Steps to reproduce the behavior:

Either make a test plugin with ContainerMenu or modify the code above.
Open the chest UI
Take an item out of the UI.
See error

Expected behavior
For the code in the screenshot to return the transaction actions as an array.
Error logs
The error thrown:

Environment

OS: W10
Version [Windows Sever 2019]

Additional context
No further context.

Sorry if this is not the proper place for this sort of issue, but I do think the error is on BDSX's side.

It would be better to use text instead of images for the code. It helps others more easily read and test your code.

Thanks for reporting it. it's fixed on the new commit.


Chokidar is missing

npm install -g @ angular/cli
not run because chokidar is missing, please help me
http fetch GET 404 https://registry.npmjs.org/chockidar-cli

Somebody misspelled "chokidar-cli"


[BUG] Delete kubernetes node did not remove node.longhorn.io

Describe the bug (🐛 if you encounter this issue)
Delete a kubernetest node which contains replica, did not remove node.longhorn.io. node.longhorn.io kept in reason: KubernetesNodeGone and status: "False" even though add back the kubernetes node.
And delete the not ready node.longhorn.io rejected by webhook.
status:
  autoEvicting: false
  conditions:
  - lastProbeTime: ""
    lastTransitionTime: "2023-12-28T08:49:48Z"
    message: 'Kubernetes node missing: node ip-172-31-45-203 has been removed from
      the cluster and there is no manager pod running on it'
    reason: KubernetesNodeGone
    status: "False"
    type: Ready

> k get nodes
NAME               STATUS   ROLES                  AGE     VERSION
ip-172-31-41-5     Ready    control-plane,master   13m     v1.28.2+k3s1
ip-172-31-45-203   Ready    <none>                 2m24s   v1.28.2+k3s1
ip-172-31-40-231   Ready    <none>                 13m     v1.28.2+k3s1
> k get nodes.longhorn.io -n longhorn-system
NAME               READY   ALLOWSCHEDULING   SCHEDULABLE   AGE
ip-172-31-45-203   False   true                            9m16s
ip-172-31-41-5     True    true              True          12m
ip-172-31-40-231   True    true              True          12m

Can reproduce in v1.5.3, but not in v1.5.2, in v1.5.2,  nodes.longhorn.io deleted immediately after kubernetes node deleted
To Reproduce

Install Longhorn on 3 worker node cluser
Create and attach a 3 replicas volume, wait volume healthy
Delete 1 of the kubernetes node kubectl delete node <node name>
Related nodes.longhorn.io not deleted and the Ready status: False
Add back the deleted kubernetes node  the nodes.longhorn.io still in not ready status
Reboot the added back node. nodes.longhorn.io still in not ready status

Expected behavior
nodes.longhorn.io should Ready after the deleted kubermetes node added back.
Support bundle for troubleshooting
supportbundle_e68e0970-6a59-4dc2-8b26-d50cb6a66e16_2023-12-28T09-01-45Z.zip
Environment

Longhorn version: v1.6.0-dev
Installation method (e.g. Rancher Catalog App/Helm/Kubectl): kubectl
Kubernetes distro (e.g. RKE/K3s/EKS/OpenShift) and version: v1.28.2+k3s1

Additional context
N/A

@ejweber Please help with this.

Sorry @innobead, I missed your comment this morning. I will look at this first thing tomorrow.

This issue has two main causes:

https://github.com/longhorn/longhorn-manager/pull/2138 introduced a potential nil pointer dereference. When the Kubernetes node is gone unexpectedly, this dereference causes the manager pod to crash during while checking whether replica eviction is necessary. As soon as the manager pod restarts, it recreates the Longhorn node object we previously cleaned up. It is this recreated Longhorn node object that we are seeing as not ready in the reproduce.
It is not possible to finish deleting a Longhorn node object (i.e. remove its finalizer) without meeting all of the conditions of the validating webhook. We immediately try again to delete the recreated Longhorn node, but we are not allowed to do so because of https://github.com/longhorn/longhorn-manager/blob/fe3ba614ab38260e4fa96f3278d4a5c77931266b/webhook/resources/node/validator.go#L110-L113.

@chriscchien, I tried a few times and was unable to reproduce this on v1.5.3. This makes sense, as the issue I identified (nil pointer dereference) wasn't introduced into v1.5.x code yet. Can you try to reproduce it again and provide the support bundle?

@chriscchien, I tried a few times and was unable to reproduce this on v1.5.3. This makes sense, as the issue I identified (nil pointer dereference) wasn't introduced into v1.5.x code yet. Can you try to reproduce it again and provide the support bundle? Maybe there is another rarer reason for the manager pod to crash, leading to the second cause still.

Hi @ejweber , I retested again on a fresh cluser with v1.5.3 and can not reproduce the issue this time, the v1.5.3 fail was a false alarm, I will be more carefully in the future and sorry for any inconvenience.

Since https://github.com/longhorn/longhorn-manager/pull/2138 will be backported as well, the fix to this issue needs to backport to 1.5 and 1.4.

Verified pass on longhorn master(longhorn-manager e53273)

Create a 3 replicas volume with workload, wirte data in mount point
Delete kubernetes node which longhorn engine located on(not located on tested as well), can see below behavior

Correspond nodes.longhorn.io gone
Add back the kubernetes node, nodes.longhorn.io came back and in ready state
Volume become healthy state, read/write data worked well.

I'm also seeing this issue on 1.5.3. Any recommendations of how to remove the stuck node? I can't uninstall the longhorn app as it is stuck in removing state.

I fixed it by running these commands
curl -s https://raw.githubusercontent.com/longhorn/longhorn-manager/master/hack/cleancrds.sh |bash -s v062
curl -s https://raw.githubusercontent.com/longhorn/longhorn-manager/master/hack/cleancrds.sh |bash -s v070
kubectl delete -f https://raw.githubusercontent.com/longhorn/longhorn/master/deploy/longhorn.yaml
kubectl delete -f https://raw.githubusercontent.com/longhorn/longhorn/master/uninstall/uninstall.yaml

then
kubectl patch crd/nodes.longhorn.io -p '{"metadata":{"finalizers":[]}}' --type=merge

really weird...


Support for 'const' keyword

To create non-writable labelled values (I have intentionally avoided using the word 'variables')

Done. Besides of Javascript-like behaviour, it supports tagging fields as constant in object literals. Example:
 var obj = {     a: "mutable",     const b: "not mutable" }


pyexasol encryption attribute

Hi,
I just got a friendly reminder from my admin that I should not connect to the database unencrypted.
If it's a quick thing for you to add it I'd be grateful.
Otherwise I'll try to add it as a profiles.yml attribute and write a PR (beginning of next week).
I guess it'll go somewhere around here:
C = connect(dsn=credentials.dsn, user=credentials.user,
                        password=credentials.password, **ecryption=credentials.encryption or False**, autocommit=True, protocol_version=protocol_version)

(I'm guessing a bit right now)
All the best
Hannes

Hello @johannes-becker-otto ,
Please feel free to submit a PR, we love to see it. Otherwise, we'll also get to it, but probably not so soon.
Kind regards!
Ilija

Hi,
just an FYI. I'm using my PR in tests and it works.
Best regards,
Hannes

Hi. Merged! Thank you for your contribution!

Pleasure :)


Add support for generating jsdoc typedef

It would be nice if the tool could optionally generate the JSDoc typedef comment blocks from the propTypes info - for the projects that aren't converted into TS yet.

Great idea, I'll try to work on that today

Fixed via commits e18f381a43f6d06c7eb10a91871dd90095845299 and 0bb03a67f3b7bcc8e07ef91019039432774302f7.  Thanks again for the idea!  This is a great addition to the tool!

Thanks for the quick response and glad that you found the suggestion useful! However, now that I tested it, the implementation is not exactly what I was looking for 😊  My feature request was to be able to generate JSDoc compatible type definitions for JavaScript files - not just copying existing ones. As an example:
Input:
MyComponent.propTypes = {
  bar: PropTypes.string.isRequired,
  foo: PropTypes.number,
}

Output:
/**
 * @typedef MyComponentProps
 * @property {string} bar
 * @property {number?} foo
 */

Do you think this could be in the scope of this project? If yes, should I re-open the ticket or create a new one?

What is the purpose of that?  Are you converting your prop types to JSDoc to continue using JavaScript and not switching to TypeScript files?  This project is focused on people converting their codebases from JavaScript to TypeScript and I don't really want to support JSDoc type checking like you described.

Yes, that is exactly the case: I'm working on a project that cannot be converted into TypeScript at this point (if ever 😬 ). However, by using JSDoc type hinting some editors (at least VS Code, IntelliJ IDEA, etc.) are able to guide and complete the code way better than just defining React.propTypes info. Therefore I suggested the feature to be added to this nifty tool.

Gotcha.  Yeah, that's not the focus or purpose of this tool, but if you are interested in pursing it, feel free to fork this repo and make the necessary modifications as this code is ISC licensed and free to use as you wish.


Linked list palindrome

Adding a program linked list palindrome

Hi @vksudin !
Can you please add a folder with the name of the question or concept, and create a small README.md, explaining the question or concept? You can add your solution file in that folder itself.
Follow this example to see how to add files to this repo.

Please check
On Fri, Oct 1, 2021 at 7:03 PM Abhinav Agrawal @.***>
wrote:

Hi @vksudin https://github.com/vksudin !
Can you please add a folder with the name of the question or concept, and
create a small README.md, explaining the question or concept? You can add
your solution file in that folder itself.
Follow this example
https://github.com/skully-coder/competitiveprogramming/tree/main/Linked Lists/Remove Nth Node From End of List
to see how to add files to this repo.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/skully-coder/competitiveprogramming/pull/16#issuecomment-932232922,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AQQ4ZPDFELFLWUQPAS7QNFDUEW2D7ANCNFSM5FEURBPQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

--
This message contains information that may be privileged or confidential
and is the property of the Academy of Technology. It is intended only for
the person to whom it is addressed. If you are not the intended recipient,
you are not authorized to read, print, retain copy, disseminate,
distribute, or use this message or any part thereof. If you receive this
message in error, please notify the sender immediately and delete all
copies of this message.
Powered by

@vksudin there are two extra code files in the commit but nevermind i will merge them.

Awesome, thanks!
On Fri, Oct 1, 2021 at 8:59 PM Abhinav Agrawal @.***>
wrote:

@vksudin https://github.com/vksudin there are two extra code files in
the commit but nevermind i will merge them.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/skully-coder/competitiveprogramming/pull/16#issuecomment-932330744,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AQQ4ZPD5HYQSALHBDD3IO6TUEXHWTANCNFSM5FEURBPQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

--
This message contains information that may be privileged or confidential
and is the property of the Academy of Technology. It is intended only for
the person to whom it is addressed. If you are not the intended recipient,
you are not authorized to read, print, retain copy, disseminate,
distribute, or use this message or any part thereof. If you receive this
message in error, please notify the sender immediately and delete all
copies of this message.
Powered by


chore: move go client code to numaflow repo

Moves the client code from the go-SDK to the numaflow repository.
Fixes #692

Can we have sink-client also inside sdkclient? Something like
sdkclient
  udf/{client,clienttest}
  sink/{client,clienttest}


chore(deps): upgrade dev dependencies

Upgrades project dependencies. See details in workflow run.

Automatically created by projen via the "upgrade-dev-dependencies-k8s-26-main" workflow

⚪ Backport skipped
The pull request was not backported as there were no branches to backport to. If this is a mistake, please apply the desired version labels or run the backport tool manually.
Manual backport
To create the backport manually run:
backport --pr 2729

Questions ?
Please refer to the Backport tool documentation


Adding unit tests

:tada: This PR is included in version 1.2.2 :tada:
The release is available on npm package (@latest dist-tag)
Your semantic-release bot :package::rocket:


Auto-emails fail because of empty email address in 'to:' field

some auto-emails fail to be sent (incl. introducing instructors to hosts) because the "to" field includes an empty address.
This was hot-fixed on the server on 2020-07-22 by Piotr.

This was fixed in https://github.com/carpentries/amy/pull/1684


[Feature Request] does Alova useRequest hook support mulitple middleware?

Please describe the bug in detail
when I try to use multi middleware, it failed. It seems it's a feature frequently used.
Reproduce link
No response
steps to reproduce
when I try to use multi middleware, it failed. It seems it's a feature frequently used.
expected performance
No response
alova version
2.17.0
device/browser/browser version
No response
additional content
No response

Now alova hasn't supported multi middleware. As I know, multi middleware is not a frequent using feature.
Could you tell me in detail, what scene you need to use the feature, so that I can consider based on your situation.

The fact is that:  I already used actionDelegationMiddleware  from @alova/scene-vue, then I want to add a custom middleware to intercept the API call when no permission, I found no place to write the custom middleware.
Now I have solved it in other ways. So it's just a suggestion~

yeah. congratulations to you.
my way is that define a wrapper of delegation middleware. sample code
useRequest({
  middleware: (ctx, next) => {
    // do your things
    return createActionDelegration(...)
  }
})


Fix small typo in readme example of "Estimating Pi"

I believe the README's sentence about referencing the API documentation is meant to say "guide," not "guild." 😄

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Safety problems with prattl clean

There is a chance that prattl clean can cause the wrong directory to be deleted. We should add a prompt to verify the path to be deleted

In commit e88078473d4f637d503e062498e6dc27df05eead I added a confirmation prompt to prattl clean


4.5 Beta: import assertion json requires moduleResolution + resolveJsonModule options

Bug Report
🔎 Search Terms
import assertions
🕗 Version & Regression Information
4.5 Beta
Repro:
index.ts
import some from "./some.json" assert { type: "json" }

some.json
[]

tsconfig.json
{
  "module": "ESNext",
  "target": "ESNext"
}

npx tsc

🙁 Actual behavior
// Cannot find module './some.json'. Did you mean to set the 'moduleResolution' option to 'node', or to add aliases to the 'paths' option?
🙂 Expected behavior
I would assume it should work without the moduleResolution option.

when `moduleResolution: node` is applied we get the following:
🙁 Actual behavior
// Cannot find module './some.json'. Consider using '--resolveJsonModule' to import module with '.json' extension.
🙂 Expected behavior
I would assume it should work without the resolveJsonModule option.

I would assume it should work without the resolveJsonModule option.

Why?

I would assume it should work without the resolveJsonModule option.

Why?

because it's now an official ecmascript language feature, as opposed to be only supported by node.js+commonjs.
https://www.typescriptlang.org/tsconfig#resolveJsonModule
Allows importing modules with a ‘.json’ extension, which is a common practice in node projects. This includes generating a type for the import based on the static JSON shape.

@andrewbranch what about moduleResolution ?


Giving wrong error "Data table should have at least 1 data row"

Expected behavior
Should run the spec successfully
Actual behavior
Data table should have at least 1 data row
Steps to reproduce

Table parameter hyphen not getting executed
===================================

     |something|
     |---------|
     |-        |
     |||
     ||

Should execute successfully
-------------------------------
tags: something

* Print <something>

With the implementation

    @Step("Print <something>")
    public void implementation12(Object arg0) {
        System.out.println(arg0);
    }

It gives an error saying

Data table should have at least 1 data row

Another test case

Table parameter hyphen not getting executed
===================================

     |something|
     ||

Should execute successfully
-------------------------------
tags: something

Gauge version
0.4.0

another test case updated

The first case mentioned here is working as expected.
If a table is given as below, it considers the hyphen as a separator between table header and its rows. But not as a data table row.
     |name|
     | -  |

If hyphen is the value that should be in first row of table, it should be given as:
     |name|
     |----|
     |-   |

The markdown table format should be as:
     |name |
     |-----|
     |gauge|
     |gocd |

The scenario
 |name|
     |----|
     |-   |

was fixed after this bug was raised and since it is being tested on the latest code now it is fixed.
However the updated scenario
Table parameter hyphen not getting executed
===================================

     |something|
     ||

Should execute successfully
-------------------------------
tags: something

is yet to be tested
considering that
Table parameter hyphen not getting executed
===================================

     |something|
     |aa|

Should execute successfully
-------------------------------
tags: something

executes successfully, either the above issue is to be fixed or "header separator should be made mandatory"

The specific case where table separator is not present and first row of table is empty is also fixed.
Table parameter hyphen not getting executed
===================================

     |something|
     ||

Should execute successfully
-------------------------------
tags: something

Fix should be available in nightly >= 17.1.2017


Upgrade docker version to docker-engine 1.10.

Fix #305
Change dokku-alt packaging in Docker to use docker-engine instead of lxc-docker.
Change docker version to integrate new docker version 1.10.0.

Looks useful, how to test this?


chinese will be supported ?

Due diligence

[X] I have done my due diligence in trying to find the answer myself.

Topic
Other / All
Question
could i speak chinese with the Moshi ?

عربي
في الاثنين، ٢٣ أيلول ٢٠٢٤, ٦:١٣ ص Lychee X @.***> كتب:

Due diligence

I have done my due diligence in trying to find the answer myself.

Topic
Other / All
Question
could i speak chinese with the Moshi ?
—
Reply to this email directly, view it on GitHub
https://github.com/kyutai-labs/moshi/issues/104, or unsubscribe
https://github.com/notifications/unsubscribe-auth/BHT35RCJMRVFDSL7OLZG7Q3ZX6BO7AVCNFSM6AAAAABOVFNBUWVHI2DSMVQWIX3LMV43ASLTON2WKOZSGU2DCNBYGE4DOOA
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

the same question
expectation...

I don't think we would have the dataset to train moshi on chinese. Our goal is to release some finetuning scripts at some point and we hope for the community to train moshi variants for different languages and use cases based on this.


Translations update from Hosted Weblate

Translations update from Hosted Weblate for Amarok-Hider/Amarok APP.
Current translation status:

@SecularSteve Thanks for the contribution :heart:


[Doc] docker quick start spark3

Purpose

Linked issue: --- #1151

Add spark3_docker_compose.yml in the docker/ path to start the spark3 cluster and introduce the paimon jar file
In the docs/content/engines/quick_start md file new start spark3 cluster and how to use the paimon create the catalog, table and data query and update the document
Tests
no

API and Format
no

Documentation
no

it looks like there is no enough requirement for docker quickstart now.