VSCode uses 18GB+ virtual address space

VSCode Version:
$code --version
1.52.1
ea3859d4ba2f3e577a159bc91e3074c5d85c0523
x64

OS Version:
CentOS Linux release 7.7.1908 (Core) 3.10.0-1062.el7.x86_64 #1 SMP

Steps to Reproduce:

invoke 'code' from command line
open folder
open a file in the editor
top shows that VIRTUAL metrics jumps up to dozens of Gb

   PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
113895 xxxx 20   0   14.8g 224412  81336 S   0.0  1.4   0:19.75 code

Does this issue occur when all extensions are disabled?: Yes
29460 xxxx   20   0   18.8g 205608  88452 S   0.0  1.3   0:03.39 code
Note when I disabled all the extensions including all the built-in extensions as well
(and have the same folder opened with the same single file in the editor) the mem usage is:
29963 xxxx 20   0 9165624 168560  69396 S   0.0  1.0   0:04.42 code
When I enable just the git SCM integration extension I get:
 29963 xxxx 20   0   18.7g 177348  71552 S   0.0  1.1   0:06.24 code
When I disable the git SCM integration extension (restart required):
30461 xxxx 20   0 9164932 188792  70388 S   0.7  1.2   0:02.21 code
When I enable the C/C++ Language Basics extension, the address space goes again from this
31389 xxxx 20   0 9150572 146860  68656 S   0.0  0.9   0:02.07 code
to this:
 31389 xxxx 20   0   18.7g 176080  68824 S   0.0  1.1   0:04.04 code
The side effect is that search or extensions installation or opening files in editor fail often with -ENOMEM.
I have checkd /proc/pid/maps and these are the biggest maps:
 92200960 5586437c1000-558648faf000 r-xp 01635000 fd:00 424266 /usr/share/code/code
106078208 7f8457dd1000-7f845e2fb000 r--p 00000000 fd:00 133047 /usr/lib/locale/locale-archive
132386816 ff0001ff000-ff008040000 ---p 00000000 00:00 0
2147483648 2cdd5a870000-2cddda870000 ---p 00000000 00:00 0
4081844224 ff00cb40000-ff100000000 ---p 00000000 00:00 0
8573157376 2cdddb870000-2cdfda870000 ---p 00000000 00:00 0

The last map is over 8GB in size.

I have tried other versions of vscode on the same host:
$code --version
1.50.1
d2e414d9e4239a252d1ab117bd7067f125afd80a
x64

as well as code-insiders-1.53.0-1611738706.el7.x86_64.rpm
All with the same effects (use of 18.7GB)
I have tried to go as far back as possible and 1.45.1 still somewhat bad but better
44926 xxxx 20   0   12.1g 182652  71224 S   0.0  1.1   0:05.02 code
Which in my case makes vscode about usable.
Finally 1.44.2 looks good :
 46015 xxxx 20   0 1298152 301568  89264 S   0.0  1.9   0:06.07 code

I have tried other versions of vscode on the same host:
$code --version
1.50.1
d2e414d9e4239a252d1ab117bd7067f125afd80a
x64

as well as code-insiders-1.53.0-1611738706.el7.x86_64.rpm
All with the same effects (use of 18.7GB)
I have tried to go as far back as possible and 1.45.1 still somewhat bad but better
44926 xxxx 20   0   12.1g 182652  71224 S   0.0  1.1   0:05.02 code
Which in my case makes vscode about usable.
Finally 1.44.2 looks good :
 46015 xxxx 20   0 1298152 301568  89264 S   0.0  1.9   0:06.07 code

Just to sum up. This looks like regression between 144.2 and 1.45.1, Where the majority 2/3 of virtual space usage increase occurred.

Just to sum up. This looks like regression between 144.2 and 1.45.1, Where the majority 2/3 of virtual space usage increase occurred.

I'm fairly certain something related to this is the root cause behind the lockups I am experiencing on a CentOS 7.7 VM with 8GB of memory with recent releases of VS Code.
Any release I've tried that's newer than 1.48.2 (July 2020) causes VS Code to immediately lock up when opening a new file/editor, with the following in the log (from code --verbose):
$ ./VSCode-linux-x64_1.52.1/bin/code --verbose
[...snip...]
[17110:0127/182147.152213:INFO:CONSOLE(1779)] "%cTRACE color: #888 CommandService#executeCommand workbench.action.files.newUntitledFile", source: file:///home/eswan/Downloads/VSCode-linux-x64_1.52.1/resources/app/out/vs/workbench/workbench.desktop.main.js (1779)
[17110:0127/182147.634814:INFO:CONSOLE(1779)] "%c  ERR color: #f33 WebAssembly.Memory(): could not allocate memory: RangeError: WebAssembly.Memory(): could not allocate memory
    at Object.default (/home/eswan/Downloads/VSCode-linux-x64_1.52.1/resources/app/node_modules.asar/vscode-oniguruma/release/main.js:1:8315)
    at /home/eswan/Downloads/VSCode-linux-x64_1.52.1/resources/app/node_modules.asar/vscode-oniguruma/release/main.js:1:5457
    at Object.t.loadWASM (/home/eswan/Downloads/VSCode-linux-x64_1.52.1/resources/app/node_modules.asar/vscode-oniguruma/release/main.js:1:5685)
    at E._doGetVSCodeOniguruma (file:///home/eswan/Downloads/VSCode-linux-x64_1.52.1/resources/app/out/vs/workbench/workbench.desktop.main.js:5988:17)
    at async Promise.all (index 1)
    at async E._getOrCreateGrammarFactory (file:///home/eswan/Downloads/VSCode-linux-x64_1.52.1/resources/app/out/vs/workbench/workbench.desktop.main.js:5983:921)
    at async file:///home/eswan/Downloads/VSCode-linux-x64_1.52.1/resources/app/out/vs/workbench/workbench.desktop.main.js:5984:841", source: file:///home/eswan/Downloads/VSCode-linux-x64_1.52.1/resources/app/out/vs/workbench/workbench.desktop.main.js (1779)
#
# Fatal error in , line 0
# Fatal process out of memory: Failed to reserve memory for new V8 Isolate
#
#
#
#FailureMessage Object: 0x2aebfd46f620#0 0x55844ded1d69
<unknown>

Versions 1.45.1 to 1.48.2 inclusive do not lock up, but syntax highlighting does not work (not sure if this is related or not). Versions 1.44.2 and prior work as expected (syntax highlighting works fine), so this seems to match up with you finding a regression in between 1.44.2 and 1.45.1.
However, I cannot reproduce the virtual address space issue on 1.45.1:
  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
14344 eswan     20   0<PHONE_NUMBER>80  58284 S   0.0  2.0   0:05.25 code

My environment:
$ cat /etc/centos-release
CentOS Linux release 7.7 (Core)

$ uname -r
3.10.0-1062.1.2.el7.x86_64

$ ulimit -v
8810313

I'm fairly certain something related to this is the root cause behind the lockups I am experiencing on a CentOS 7.7 VM with 8GB of memory with recent releases of VS Code.
Any release I've tried that's newer than 1.48.2 (July 2020) causes VS Code to immediately lock up when opening a new file/editor, with the following in the log (from code --verbose):
$ ./VSCode-linux-x64_1.52.1/bin/code --verbose
[...snip...]
[17110:0127/182147.152213:INFO:CONSOLE(1779)] "%cTRACE color: #888 CommandService#executeCommand workbench.action.files.newUntitledFile", source: file:///home/eswan/Downloads/VSCode-linux-x64_1.52.1/resources/app/out/vs/workbench/workbench.desktop.main.js (1779)
[17110:0127/182147.634814:INFO:CONSOLE(1779)] "%c  ERR color: #f33 WebAssembly.Memory(): could not allocate memory: RangeError: WebAssembly.Memory(): could not allocate memory
    at Object.default (/home/eswan/Downloads/VSCode-linux-x64_1.52.1/resources/app/node_modules.asar/vscode-oniguruma/release/main.js:1:8315)
    at /home/eswan/Downloads/VSCode-linux-x64_1.52.1/resources/app/node_modules.asar/vscode-oniguruma/release/main.js:1:5457
    at Object.t.loadWASM (/home/eswan/Downloads/VSCode-linux-x64_1.52.1/resources/app/node_modules.asar/vscode-oniguruma/release/main.js:1:5685)
    at E._doGetVSCodeOniguruma (file:///home/eswan/Downloads/VSCode-linux-x64_1.52.1/resources/app/out/vs/workbench/workbench.desktop.main.js:5988:17)
    at async Promise.all (index 1)
    at async E._getOrCreateGrammarFactory (file:///home/eswan/Downloads/VSCode-linux-x64_1.52.1/resources/app/out/vs/workbench/workbench.desktop.main.js:5983:921)
    at async file:///home/eswan/Downloads/VSCode-linux-x64_1.52.1/resources/app/out/vs/workbench/workbench.desktop.main.js:5984:841", source: file:///home/eswan/Downloads/VSCode-linux-x64_1.52.1/resources/app/out/vs/workbench/workbench.desktop.main.js (1779)
#
# Fatal error in , line 0
# Fatal process out of memory: Failed to reserve memory for new V8 Isolate
#
#
#
#FailureMessage Object: 0x2aebfd46f620#0 0x55844ded1d69
<unknown>

Versions 1.45.1 to 1.48.2 inclusive do not lock up, but syntax highlighting does not work (not sure if this is related or not). Versions 1.44.2 and prior work as expected (syntax highlighting works fine), so this seems to match up with you finding a regression in between 1.44.2 and 1.45.1.
However, I cannot reproduce the virtual address space issue on 1.45.1:
  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
14344 eswan     20   0<PHONE_NUMBER>80  58284 S   0.0  2.0   0:05.25 code

My environment:
$ cat /etc/centos-release
CentOS Linux release 7.7 (Core)

$ uname -r
3.10.0-1062.1.2.el7.x86_64

$ ulimit -v
8810313

I tried on a much larger CentOS 7 machine and I am able to reproduce the virtual address space issue on VS Code 1.52.1:
   PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
125468 eswan     20   0   18.5g 141760  58572 S   0.0  0.1   0:03.49 code

Environment is the same as above with the exception that on this bigger VM, ulimit -v is 290265558. VS Code does not lock up on this machine.

I tried on a much larger CentOS 7 machine and I am able to reproduce the virtual address space issue on VS Code 1.52.1:
   PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
125468 eswan     20   0   18.5g 141760  58572 S   0.0  0.1   0:03.49 code

Environment is the same as above with the exception that on this bigger VM, ulimit -v is 290265558. VS Code does not lock up on this machine.

I have figured out that virtual address space usage was correlated with Commit memory, which was a real culprit.
My Centos system comes with Commit Memmory restriction policy enabled by default.
/proc/sys/vm/overcommit_memory = 2, which with other parameters (actual physical memory, overcommit_ratio and swap) resulted in Commit Limit of some 16GB.
Opening my project folder and few windows bumps Commit AS metric (from /proc/meminfo) by 3GB alone.
Fortunately I was able to increase overcommit_ratio.

I have figured out that virtual address space usage was correlated with Commit memory, which was a real culprit.
My Centos system comes with Commit Memmory restriction policy enabled by default.
/proc/sys/vm/overcommit_memory = 2, which with other parameters (actual physical memory, overcommit_ratio and swap) resulted in Commit Limit of some 16GB.
Opening my project folder and few windows bumps Commit AS metric (from /proc/meminfo) by 3GB alone.
Fortunately I was able to increase overcommit_ratio.

@deepak1556 we should at least document that.

@deepak1556 we should at least document that.

I have been made aware that this is V8 javascript engine allocating whole 4GB 32-bit addressable blocks of virtual memory to avoid expensive boundary checking

I have been made aware that this is V8 javascript engine allocating whole 4GB 32-bit addressable blocks of virtual memory to avoid expensive boundary checking

I'd be curious to know if anyone can reproduce what I was experiencing with the current release of VS Code locking up on a Linux machine with only 8GB of physical or virtual memory.

Erik, With my 16GB VM I saw variety of problems: VS code stopping responding to menu actions and keywords, and searchbox not showing results but '-ENOMEM' instead.
Does it work for you now? Have you tried setting /proc/sys/vm/overcommit_memory to 0 or /proc/sys/vm/overcommit_ratio to 300?

Unfortunately I do not have root access on the VM or the ability to reconfigure the VM. Is this issue isolated to virtual machines? What happens on a machine with 8GB of physical memory?
It seems broken to me that a text editor would require >16GB of memory just to get basic functionality working correctly.

Is there anyway to avoid the Fatal process out of memory: Failed to reserve memory for new V8 Isolate error without making changes to the environment requiring root?
I am trying to support a customer and do not have control over their environment and they are experiencing this issue

@ErikSwan I can confirm this issue occurs independent of VsCode. If I simply package an electron.js app using electron version 12.0.9 I get this error in certain Centos7 environments (see fix below).
If I revert to a June 2020 version of electron, version 8.3.2, I do not get this error at all. Not sure if it matters, but the environment I'm running in seems to only work if I run with --no-sandbox --in-process-gpu commandline switches

@ErikSwan I can confirm this issue occurs independent of VsCode. If I simply package an electron.js app using electron version 12.0.9 I get your same Failed to reserve memory for new V8 Isolate error in certain Centos7 environments (see fix below).
If I revert to a < July 2020 version of electron.js like you say, version 8.3.2 (from June 15th), I do not get this error at all. Not sure if it matters, but the environment I'm running in seems to only work if I run with --no-sandbox --in-process-gpu commandline switches

Seems like it is an issue that would need to be fixed in the upstream Electron, then. Unfortunately I simply gave up on making it work and resigned myself to using a different editor in my CentOS 7 environment.

Closing as this is default allocation performed by the runtime and not something VSCode can manage.


feat: added istio resources

added istio resources
fixes #25

@swastik959 fix CI please

added istio resources

fixes #25

@eddycharly please review it now

@swastik959 now there's a conflict 🤣


Better padding to the "Add testimonial" Button

Description
I feel the button could use a bit of padding to the side.
Screenshots
Before

After adding horizontal padding

Additional information
No response

Can we have a common padding set for all buttons, if possible? That will make things consistent.

Where is this button?

Hey, I would like to work on this!

Where is this button?

My bad, let me update the question with the details

There are only two buttons like this. Not worth making a class

Hey @PerkinsDan , thank you for your input. Before making a PR we generally discuss with the maintainers. Also I saw someone had already asked to be assigned to the issue. Please consider this before creating a PR next time, Thank you!


Adding user sign in api to api design

Thank you @mohammadibrahemmustafa for this PR
I have a quick feedback to improve it:

The end point should be renamed from "/signup", to "/user", so we can have POST, GET, PATCH, PUT /user, as in petstore example, I suggest PATCH for user sign in
It would be easier for fronend developers if we used requestBody content as application/x-www-form-urlencoded instead of application/json, you can then test it using form inputs
please use camelCase, like countryId instead of country_id, as in petstore example
please add the main details about properties, like maxlength or pattern, depending on property type, in vs code you can use CTL + . to quick fix it, you need openApi extension to be installed first
it would be better to use 400 instead of "400" for response code
before submit your pull request, make an audit test, using the openApi extension (click on purple icon to the top right), keep the score over 50 out of 100 :) 😊

I hope doing this will move your code to the next level 👍

@haithamalnaeb Thank you for your review. I have adjusted the commit and I got 55 score on the vs code extenstion.

You did a great job 👍
almost done
the only issue is that we need to return token on success sign in, not otpToken
also if you add additionalProperties: false in both request and response schema and maxLength: 32 to response token will increase the security score to over 60
finally, for password pattern, you shoud set the minimum to 8, so use ^.{1,50}$ instead of ^.{8,50}$

Thank you for your review. I have adjusted the files again.

You did amazing contribution here @mohammadibrahemmustafa
I hope you gain good experience, keep it going


Failures of anisotropy correction of half-maps with tutorial data

The data and command are the same as in the tutorial, but when running Anisotropy Correction of half-maps, it reports the following problem:
spisonet.py reconstruct emd_8731_half_map_1.mrc emd_8731_half_map_2.mrc --aniso_file FSC3D.mrc --mask emd_8731_msk_1.mrc --limit_res 3.5 --epochs 30 --alpha 1 --beta 0.5 --output_dir isonet_maps --gpuID 0,1,2,3 --acc_batches 2
11-20 00:52:42, INFO     voxel_size 1.309999942779541
11-20 00:52:43, INFO     spIsoNet correction until resolution 3.5A!
Information beyond 3.5A remains unchanged
11-20 00:52:57, INFO     Start preparing subvolumes!
11-20 00:53:06, INFO     Done preparing subvolumes!
11-20 00:53:06, INFO     Start training!
11-20 00:53:09, INFO     Port number: 51405
learning rate 0.0003
['isonet_maps/emd_8731_half_map_1_data', 'isonet_maps/emd_8731_half_map_2_data']
0%|                                                                                | 0/125 [00:00<?, ?batch/s][rank1]:W1120 00:53:26.205000<PHONE_NUMBER>86848 torch/_logging/_internal.py:1034] [0/0] Profiler function <class 'torch.autograd.profiler.record_function'> will be ignored
[rank3]:W1120 00:53:26.225000<PHONE_NUMBER>45408 torch/_logging/_internal.py:1034] [0/0] Profiler function <class 'torch.autograd.profiler.record_function'> will be ignored
[rank0]:W1120 00:53:26.263000<PHONE_NUMBER>16128 torch/_logging/_internal.py:1034] [0/0] Profiler function <class 'torch.autograd.profiler.record_function'> will be ignored
[rank2]:W1120 00:53:26.357000<PHONE_NUMBER>12384 torch/logging/internal.py:1034] [0/0] Profiler function <class 'torch.autograd.profiler.record_function'> will be ignored
/tmp/tmpb9ffwjno/main.c:6:23: fatal error: stdatomic.h: No such file or directory
#include <stdatomic.h>
^
compilation terminated.
/tmp/tmpl5yntb4i/main.c:6:23: fatal error: stdatomic.h: No such file or directory
#include <stdatomic.h>
^
compilation terminated.
/tmp/tmph2p9sytq/main.c:6:23: fatal error: stdatomic.h: No such file or directory
#include <stdatomic.h>
^
compilation terminated.
/tmp/tmpbvg8egds/main.c:6:23: fatal error: stdatomic.h: No such file or directory
#include <stdatomic.h>
^
compilation terminated.
/tmp/tmph4ckum7q/main.c:6:23: fatal error: stdatomic.h: No such file or directory
#include <stdatomic.h>
^
compilation terminated.
/tmp/tmpmj2mj6b6/main.c:6:23: fatal error: stdatomic.h: No such file or directory
#include <stdatomic.h>
^
compilation terminated.
/tmp/tmpowqgpc9/main.c:6:23: fatal error: stdatomic.h: No such file or directory
#include <stdatomic.h>
^
compilation terminated.
/tmp/tmp_vj7apqe/main.c:6:23: fatal error: stdatomic.h: No such file or directory
#include <stdatomic.h>
^
compilation terminated.
/tmp/tmptthkhvx/main.c:6:23: fatal error: stdatomic.h: No such file or directory
#include <stdatomic.h>
^
compilation terminated.
/tmp/tmpjo5sie2e/main.c:6:23: fatal error: stdatomic.h: No such file or directory
#include <stdatomic.h>
^
compilation terminated.
/tmp/tmpqvdup2d8/main.c:6:23: fatal error: stdatomic.h: No such file or directory
#include <stdatomic.h>
^
compilation terminated.
/tmp/tmp8xzss5bc/main.c:6:23: fatal error: stdatomic.h: No such file or directory
#include <stdatomic.h>
^
compilation terminated.
0%|                                                                                | 0/125 [00:08<?, ?batch/s]
W1120 00:53:33.367000<PHONE_NUMBER>96480 torch/multiprocessing/spawn.py:146] Terminating process 40535 via signal SIGTERM
W1120 00:53:33.367000<PHONE_NUMBER>96480 torch/multiprocessing/spawn.py:146] Terminating process 47366 via signal SIGTERM
W1120 00:53:33.368000<PHONE_NUMBER>96480 torch/multiprocessing/spawn.py:146] Terminating process 47503 via signal SIGTERM
Traceback (most recent call last):
File "/spshared/apps/miniconda3/envs/spisonet/bin/spisonet.py", line 8, in
sys.exit(main())
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/spIsoNet/bin/spisonet.py", line 549, in main
fire.Fire(ISONET)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/fire/core.py", line 143, in Fire
component_trace = _Fire(component, args, parsed_flag_args, context, name)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/fire/core.py", line 477, in _Fire
component, remaining_args = _CallAndUpdateTrace(
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/fire/core.py", line 693, in _CallAndUpdateTrace
component = fn(*varargs, **kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/spIsoNet/bin/spisonet.py", line 182, in reconstruct
map_refine_n2n(halfmap1,halfmap2, mask_vol, fsc3d, alpha = alpha,beta=beta,  voxel_size=voxel_size, output_dir=output_dir,
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/spIsoNet/bin/map_refine.py", line 145, in map_refine_n2n
network.train([data_dir_1,data_dir_2], output_dir, alpha=alpha,beta=beta, output_base=output_base0, batch_size=batch_size, epochs = epochs, steps_per_epoch = 1000,
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/spIsoNet/models/network_n2n.py", line 265, in train
mp.spawn(ddp_train, args=(self.world_size, self.port_number, self.model,alpha,beta,
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/multiprocessing/spawn.py", line 282, in spawn
return start_processes(fn, args, nprocs, join, daemon, start_method="spawn")
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/multiprocessing/spawn.py", line 238, in start_processes
while not context.join():
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/multiprocessing/spawn.py", line 189, in join
raise ProcessRaisedException(msg, error_index, failed_process.pid)
torch.multiprocessing.spawn.ProcessRaisedException:
-- Process 1 terminated with the following error:
Traceback (most recent call last):
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/multiprocessing/spawn.py", line 76, in _wrap
fn(i, *args)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/spIsoNet/models/network_n2n.py", line 116, in ddp_train
preds = model(x1)# + noise.cuda())
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 433, in _fn
return fn(*args, **kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_dynamo/external_utils.py", line 38, in inner
return fn(*args, **kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/nn/parallel/distributed.py", line 1636, in forward
else self._run_ddp_forward(*inputs, **kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/nn/parallel/distributed.py", line 1454, in _run_ddp_forward
return self.module(*inputs, **kwargs)  # type: ignore[index]
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/spIsoNet/models/unet.py", line 97, in forward
x, down_sampling_features = self.encoder(x)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/spIsoNet/models/unet.py", line 98, in torch_dynamo_resume_in_forward_at_97
x = self.decoder(x, down_sampling_features)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 1110, in call
return hijacked_callback(
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 948, in call
result = self._inner_convert(
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 472, in call
return _compile(
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_utils_internal.py", line 84, in wrapper_function
return StrobelightCompileTimeProfiler.profile_compile_time(
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_strobelight/compile_time_profiler.py", line 129, in profile_compile_time
return func(*args, **kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/contextlib.py", line 79, in inner
return func(*args, **kwds)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 817, in _compile
guarded_code = compile_inner(code, one_graph, hooks, transform)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 231, in time_wrapper
r = func(*args, **kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 636, in compile_inner
out_code = transform_code_object(code, transform)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_dynamo/bytecode_transformation.py", line 1185, in transform_code_object
transformations(instructions, code_options)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 178, in _fn
return fn(*args, **kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 582, in transform
tracer.run()
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2451, in run
super().run()
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 893, in run
while self.step():
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 805, in step
self.dispatch_table[inst.opcode](self, inst)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2642, in RETURN_VALUE
self._return(inst)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2627, in _return
self.output.compile_subgraph(
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_dynamo/output_graph.py", line 1098, in compile_subgraph
self.compile_and_call_fx_graph(tx, list(reversed(stack_values)), root)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/contextlib.py", line 79, in inner
return func(*args, **kwds)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_dynamo/output_graph.py", line 1318, in compile_and_call_fx_graph
compiled_fn = self.call_user_compiler(gm)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 231, in time_wrapper
r = func(*args, **kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_dynamo/output_graph.py", line 1409, in call_user_compiler
raise BackendCompilerFailed(self.compiler_fn, e).with_traceback(
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_dynamo/output_graph.py", line 1390, in call_user_compiler
compiled_fn = compiler_fn(gm, self.example_inputs())
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/dynamo/backends/distributed.py", line 565, in compile_fn
return self.backend_compile_fn(gm, example_inputs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/dynamo/repro/after_dynamo.py", line 129, in call
compiled_gm = compiler_fn(gm, example_inputs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/init.py", line 1951, in call
return compile_fx(model, inputs, config_patches=self.config)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/contextlib.py", line 79, in inner
return func(*args, **kwds)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_inductor/compile_fx.py", line 1505, in compile_fx
return aot_autograd(
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_dynamo/backends/common.py", line 69, in call
cg = aot_module_simplified(gm, example_inputs, **self.kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_functorch/aot_autograd.py", line 954, in aot_module_simplified
compiled_fn, _ = create_aot_dispatcher_function(
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 231, in time_wrapper
r = func(*args, **kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_functorch/aot_autograd.py", line 687, in create_aot_dispatcher_function
compiled_fn, fw_metadata = compiler_fn(
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_functorch/_aot_autograd/jit_compile_runtime_wrappers.py", line 461, in aot_dispatch_autograd
compiled_fw_func = aot_config.fw_compiler(fw_module, adjusted_flat_args)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 231, in time_wrapper
r = func(*args, **kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_inductor/compile_fx.py", line 1410, in fw_compiler_base
return inner_compile(
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_dynamo/repro/after_aot.py", line 84, in debug_wrapper
inner_compiled_fn = compiler_fn(gm, example_inputs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_inductor/debug.py", line 304, in inner
return fn(*args, **kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/contextlib.py", line 79, in inner
return func(*args, **kwds)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/contextlib.py", line 79, in inner
return func(*args, **kwds)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 231, in time_wrapper
r = func(*args, **kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_inductor/compile_fx.py", line 527, in compile_fx_inner
compiled_graph = fx_codegen_and_compile(
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/contextlib.py", line 79, in inner
return func(*args, **kwds)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_inductor/compile_fx.py", line 831, in fx_codegen_and_compile
compiled_fn = graph.compile_to_fn()
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_inductor/graph.py", line 1751, in compile_to_fn
return self.compile_to_module().call
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 231, in time_wrapper
r = func(*args, **kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_inductor/graph.py", line 1680, in compile_to_module
self.codegen_with_cpp_wrapper() if self.cpp_wrapper else self.codegen()
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_inductor/graph.py", line 1640, in codegen
self.scheduler.codegen()
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 231, in time_wrapper
r = func(*args, **kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_inductor/scheduler.py", line 2741, in codegen
self.get_backend(device).codegen_node(node)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_inductor/codegen/cuda_combined_scheduling.py", line 69, in codegen_node
return self._triton_scheduling.codegen_node(node)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_inductor/codegen/simd.py", line 1148, in codegen_node
return self.codegen_node_schedule(node_schedule, buf_accesses, numel, rnumel)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_inductor/codegen/simd.py", line 1317, in codegen_node_schedule
src_code = kernel.codegen_kernel()
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_inductor/codegen/triton.py", line 2159, in codegen_kernel
**self.inductor_meta_common(),
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_inductor/codegen/triton.py", line 2047, in inductor_meta_common
"backend_hash": torch.utils._triton.triton_hash_with_backend(),
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/utils/_triton.py", line 63, in triton_hash_with_backend
backend = triton_backend()
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/utils/_triton.py", line 49, in triton_backend
target = driver.active.get_current_target()
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/triton/runtime/driver.py", line 23, in getattr
self._initialize_obj()
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/triton/runtime/driver.py", line 20, in _initialize_obj
self._obj = self._init_fn()
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/triton/runtime/driver.py", line 9, in _create_driver
return actives0
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/triton/backends/nvidia/driver.py", line 371, in init
self.utils = CudaUtils()  # TODO: make static
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/triton/backends/nvidia/driver.py", line 80, in init
mod = compile_module_from_src(Path(os.path.join(dirname, "driver.c")).read_text(), "cuda_utils")
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/triton/backends/nvidia/driver.py", line 57, in compile_module_from_src
so = build(name, src_path, tmpdir, library_dirs(), include_dir, libraries)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/triton/runtime/build.py", line 48, in build
ret = subprocess.check_call(cc_cmd)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/subprocess.py", line 369, in check_call
raise CalledProcessError(retcode, cmd)
torch.dynamo.exc.BackendCompilerFailed: backend='compile_fn' raised:
CalledProcessError: Command '['/usr/bin/gcc', '/tmp/tmptthkhvx/main.c', '-O3', '-shared', '-fPIC', '-o', '/tmp/tmptthkhvx/cuda_utils.cpython-310-x86_64-linux-gnu.so', '-lcuda', '-L/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/triton/backends/nvidia/lib', '-L/lib64', '-I/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/triton/backends/nvidia/include', '-I/tmp/tmptthkhvx', '-I/spshared/apps/miniconda3/envs/spisonet/include/python3.10']' returned non-zero exit status 1.
Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information
You can suppress this exception and fall back to eager by setting:
import torch._dynamo
torch._dynamo.config.suppress_errors = True
Thanks for your help!

Hi,
The error I believe comes from these lines in
spIsoNet/models/network_n2n.py:
if torch.version >= "2.0.0":
GPU_capability = torch.cuda.get_device_capability()
if GPU_capability[0] >= 7:
torch.set_float32_matmul_precision('high')
model = torch.compile(model)
Would you comment out these lines and try again?
On Wednesday, 20 November 2024, suqi-z @.***> wrote:

The data and command are the same as in the tutorial, but when running
Anisotropy Correction of half-maps, it reports the following problem:
spisonet.py reconstruct emd_8731_half_map_1.mrc emd_8731_half_map_2.mrc
--aniso_file FSC3D.mrc --mask emd_8731_msk_1.mrc --limit_res 3.5 --epochs
30 --alpha 1 --beta 0.5 --output_dir isonet_maps --gpuID 0,1,2,3
--acc_batches 2
11-20 00:52:42, INFO voxel_size 1.309999942779541
11-20 00:52:43, INFO spIsoNet correction until resolution 3.5A!
Information beyond 3.5A remains unchanged
11-20 00:52:57, INFO Start preparing subvolumes!
11-20 00:53:06, INFO Done preparing subvolumes!
11-20 00:53:06, INFO Start training!
11-20 00:53:09, INFO Port number: 51405
learning rate 0.0003
['isonet_maps/emd_8731_half_map_1_data', 'isonet_maps/emd_8731_half_
map_2_data']
0%| | 0/125 [00:00<?, ?batch/s][rank1]:W1120 00:53:26.205000
139648681686848 torch/_logging/_internal.py:1034] [0/0] Profiler function
<class 'torch.autograd.profiler.record_function'> will be ignored
[rank3]:W1120 00:53:26.225000<PHONE_NUMBER>45408 torch/_logging/_internal.py:1034]
[0/0] Profiler function <class 'torch.autograd.profiler.record_function'>
will be ignored
[rank0]:W1120 00:53:26.263000<PHONE_NUMBER>16128 torch/_logging/_internal.py:1034]
[0/0] Profiler function <class 'torch.autograd.profiler.record_function'>
will be ignored
[rank2]:W1120 00:53:26.357000<PHONE_NUMBER>12384 torch/
logging/internal.py:1034] [0/0] Profiler function <class
'torch.autograd.profiler.record_function'> will be ignored
/tmp/tmpb9ffwjno/main.c:6:23: fatal error: stdatomic.h: No such file or
directory #include <stdatomic.h> ^ compilation terminated.
/tmp/tmpl5yntb4i/main.c:6:23: fatal error: stdatomic.h: No such file or
directory #include <stdatomic.h> ^ compilation terminated.
/tmp/tmph2p9sytq/main.c:6:23: fatal error: stdatomic.h: No such file or
directory #include <stdatomic.h> ^ compilation terminated.
/tmp/tmpbvg8egds/main.c:6:23: fatal error: stdatomic.h: No such file or
directory #include <stdatomic.h> ^ compilation terminated.
/tmp/tmph4ckum7q/main.c:6:23: fatal error: stdatomic.h: No such file or
directory #include <stdatomic.h> ^ compilation terminated.
/tmp/tmpmj2mj6b6/main.c:6:23: fatal error: stdatomic.h: No such file or
directory #include <stdatomic.h> ^ compilation terminated.
/tmp/tmpowqgpc9/main.c:6:23: fatal error: stdatomic.h: No such file or
directory #include <stdatomic.h> ^ compilation terminated.
/tmp/tmp_vj7apqe/main.c:6:23: fatal error: stdatomic.h: No such file or
directory #include <stdatomic.h> ^ compilation terminated. /tmp/tmptthkhvx/main.c:6:23:
fatal error: stdatomic.h: No such file or directory
#include <stdatomic.h>
^
compilation terminated.
/tmp/tmpjo5sie2e/main.c:6:23: fatal error: stdatomic.h: No such file or
directory
#include <stdatomic.h>
^
compilation terminated.
/tmp/tmpqvdup2d8/main.c:6:23: fatal error: stdatomic.h: No such file or
directory
#include <stdatomic.h>
^
compilation terminated.
/tmp/tmp8xzss5bc/main.c:6:23: fatal error: stdatomic.h: No such file or
directory
#include <stdatomic.h>
^
compilation terminated.
0%| | 0/125 [00:08<?, ?batch/s]
W1120 00:53:33.367000<PHONE_NUMBER>96480 torch/multiprocessing/spawn.py:146]
Terminating process 40535 via signal SIGTERM
W1120 00:53:33.367000<PHONE_NUMBER>96480 torch/multiprocessing/spawn.py:146]
Terminating process 47366 via signal SIGTERM
W1120 00:53:33.368000<PHONE_NUMBER>96480 torch/multiprocessing/spawn.py:146]
Terminating process 47503 via signal SIGTERM
Traceback (most recent call last):
File "/spshared/apps/miniconda3/envs/spisonet/bin/spisonet.py", line 8,
in
sys.exit(main())
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/spIsoNet/bin/spisonet.py", line 549, in main
fire.Fire(ISONET)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/fire/core.py",
line 143, in Fire
component_trace = _Fire(component, args, parsed_flag_args, context, name)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/fire/core.py",
line 477, in _Fire
component, remaining_args = _CallAndUpdateTrace(
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/fire/core.py",
line 693, in _CallAndUpdateTrace
component = fn(*varargs, **kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/spIsoNet/bin/spisonet.py", line 182, in reconstruct
map_refine_n2n(halfmap1,halfmap2, mask_vol, fsc3d, alpha =
alpha,beta=beta, voxel_size=voxel_size, output_dir=output_dir,
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/spIsoNet/bin/map_refine.py", line 145, in map_refine_n2n
network.train([data_dir_1,data_dir_2], output_dir, alpha=alpha,beta=beta,
output_base=output_base0, batch_size=batch_size, epochs = epochs,
steps_per_epoch = 1000,
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/spIsoNet/models/network_n2n.py", line 265, in train
mp.spawn(ddp_train, args=(self.world_size, self.port_number,
self.model,alpha,beta,
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/multiprocessing/spawn.py", line 282, in spawn
return start_processes(fn, args, nprocs, join, daemon,
start_method="spawn")
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/multiprocessing/spawn.py", line 238, in
start_processes
while not context.join():
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/multiprocessing/spawn.py", line 189, in join
raise ProcessRaisedException(msg, error_index, failed_process.pid)
torch.multiprocessing.spawn.ProcessRaisedException:
-- Process 1 terminated with the following error:
Traceback (most recent call last):
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/multiprocessing/spawn.py", line 76, in _wrap
fn(i, *args)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/spIsoNet/models/network_n2n.py", line 116, in ddp_train
preds = model(x1)# + noise.cuda())
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/nn/modules/module.py", line 1553, in
_wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/_dynamo/eval_frame.py", line 433, in _fn
return fn(*args, **kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/_dynamo/external_utils.py", line 38, in inner
return fn(*args, **kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/nn/modules/module.py", line 1553, in
_wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/nn/parallel/distributed.py", line 1636, in forward
else self._run_ddp_forward(*inputs, **kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/nn/parallel/distributed.py", line 1454, in
_run_ddp_forward
return self.module(*inputs, **kwargs) # type: ignore[index]
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/nn/modules/module.py", line 1553, in
_wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/spIsoNet/models/unet.py", line 97, in forward
x, down_sampling_features = self.encoder(x)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/spIsoNet/models/unet.py", line 98, in
torch_dynamo_resume_in_forward_at_97
x = self.decoder(x, down_sampling_features)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/_dynamo/convert_frame.py", line 1110, in call
return hijacked_callback(
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/_dynamo/convert_frame.py", line 948, in call
result = self._inner_convert(
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/_dynamo/convert_frame.py", line 472, in call
return _compile(
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/_utils_internal.py", line 84, in wrapper_function
return StrobelightCompileTimeProfiler.profile_compile_time(
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/_strobelight/compile_time_profiler.py", line 129, in
profile_compile_time
return func(*args, **kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/contextlib.py",
line 79, in inner
return func(*args, **kwds)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/_dynamo/convert_frame.py", line 817, in _compile
guarded_code = compile_inner(code, one_graph, hooks, transform)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/_dynamo/utils.py", line 231, in time_wrapper
r = func(*args, **kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/_dynamo/convert_frame.py", line 636, in compile_inner
out_code = transform_code_object(code, transform)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/_dynamo/bytecode_transformation.py", line 1185, in
transform_code_object
transformations(instructions, code_options)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/_dynamo/convert_frame.py", line 178, in _fn
return fn(*args, **kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/_dynamo/convert_frame.py", line 582, in transform
tracer.run()
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/_dynamo/symbolic_convert.py", line 2451, in run
super().run()
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/_dynamo/symbolic_convert.py", line 893, in run
while self.step():
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/_dynamo/symbolic_convert.py", line 805, in step
self.dispatch_table[inst.opcode](self, inst)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/_dynamo/symbolic_convert.py", line 2642, in
RETURN_VALUE
self._return(inst)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/_dynamo/symbolic_convert.py", line 2627, in _return
self.output.compile_subgraph(
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/_dynamo/output_graph.py", line 1098, in
compile_subgraph
self.compile_and_call_fx_graph(tx, list(reversed(stack_values)), root)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/contextlib.py",
line 79, in inner
return func(*args, **kwds)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/_dynamo/output_graph.py", line 1318, in
compile_and_call_fx_graph
compiled_fn = self.call_user_compiler(gm)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/_dynamo/utils.py", line 231, in time_wrapper
r = func(*args, **kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/_dynamo/output_graph.py", line 1409, in
call_user_compiler
raise BackendCompilerFailed(self.compiler_fn, e).with_traceback(
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/_dynamo/output_graph.py", line 1390, in
call_user_compiler
compiled_fn = compiler_fn(gm, self.example_inputs())
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/
dynamo/backends/distributed.py", line 565, in compile_fn return
self.backend_compile_fn(gm, example_inputs) File
"/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/dynamo/repro/after_dynamo.py",
line 129, in call compiled_gm = compiler_fn(gm, example_inputs) File
"/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/init.py",
line 1951, in call return compile_fx(model, inputs,
config_patches=self.config)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/contextlib.py",
line 79, in inner
return func(*args, **kwds)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/_inductor/compile_fx.py", line 1505, in compile_fx
return aot_autograd(
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/_dynamo/backends/common.py", line 69, in call
cg = aot_module_simplified(gm, example_inputs, **self.kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/_functorch/aot_autograd.py", line 954, in
aot_module_simplified
compiled_fn, _ = create_aot_dispatcher_function(
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/_dynamo/utils.py", line 231, in time_wrapper
r = func(*args, **kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/_functorch/aot_autograd.py", line 687, in
create_aot_dispatcher_function
compiled_fn, fw_metadata = compiler_fn(
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/_functorch/_aot_autograd/jit_compile_runtime_wrappers.py",
line 461, in aot_dispatch_autograd
compiled_fw_func = aot_config.fw_compiler(fw_module, adjusted_flat_args)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/_dynamo/utils.py", line 231, in time_wrapper
r = func(*args, **kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/_inductor/compile_fx.py", line 1410, in
fw_compiler_base
return inner_compile(
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/_dynamo/repro/after_aot.py", line 84, in debug_wrapper
inner_compiled_fn = compiler_fn(gm, example_inputs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/_inductor/debug.py", line 304, in inner
return fn(*args, **kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/contextlib.py",
line 79, in inner
return func(*args, **kwds)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/contextlib.py",
line 79, in inner
return func(*args, **kwds)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/_dynamo/utils.py", line 231, in time_wrapper
r = func(*args, **kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/_inductor/compile_fx.py", line 527, in
compile_fx_inner
compiled_graph = fx_codegen_and_compile(
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/contextlib.py",
line 79, in inner
return func(*args, **kwds)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/_inductor/compile_fx.py", line 831, in
fx_codegen_and_compile
compiled_fn = graph.compile_to_fn()
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/_inductor/graph.py", line 1751, in compile_to_fn
return self.compile_to_module().call
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/_dynamo/utils.py", line 231, in time_wrapper
r = func(*args, **kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/_inductor/graph.py", line 1680, in compile_to_module
self.codegen_with_cpp_wrapper() if self.cpp_wrapper else self.codegen()
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/_inductor/graph.py", line 1640, in codegen
self.scheduler.codegen()
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/_dynamo/utils.py", line 231, in time_wrapper
r = func(*args, **kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/_inductor/scheduler.py", line 2741, in codegen
self.get_backend(device).codegen_node(node)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/_inductor/codegen/cuda_combined_scheduling.py", line
69, in codegen_node
return self._triton_scheduling.codegen_node(node)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/_inductor/codegen/simd.py", line 1148, in codegen_node
return self.codegen_node_schedule(node_schedule, buf_accesses, numel,
rnumel)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/_inductor/codegen/simd.py", line 1317, in
codegen_node_schedule
src_code = kernel.codegen_kernel()
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/_inductor/codegen/triton.py", line 2159, in
codegen_kernel
**self.inductor_meta_common(),
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/_inductor/codegen/triton.py", line 2047, in
inductor_meta_common
"backend_hash": torch.utils._triton.triton_hash_with_backend(),
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/utils/_triton.py", line 63, in
triton_hash_with_backend
backend = triton_backend()
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/torch/utils/_triton.py", line 49, in triton_backend
target = driver.active.get_current_target()
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/triton/runtime/driver.py", line 23, in getattr
self._initialize_obj()
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/triton/runtime/driver.py", line 20, in _initialize_obj
self._obj = self._init_fn()
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/triton/runtime/driver.py", line 9, in _create_driver
return actives0
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/triton/backends/nvidia/driver.py", line 371, in init
self.utils = CudaUtils() # TODO: make static
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/triton/backends/nvidia/driver.py", line 80, in init
mod = compile_module_from_src(Path(os.path.join(dirname,
"driver.c")).read_text(), "cuda_utils")
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/
site-packages/triton/backends/nvidia/driver.py", line 57, in
compile_module_from_src
so =
build(name, src_path, tmpdir, library_dirs(), include_dir, libraries)
File
"/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/triton/runtime/build.py",
line 48, in build ret = subprocess.check_call(cc_cmd) File
"/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/subprocess.py",
line 369, in check_call raise CalledProcessError(retcode, cmd)
torch.dynamo.exc.BackendCompilerFailed: backend='compile_fn' raised:
CalledProcessError: Command '['/usr/bin/gcc', '/tmp/tmptthkhvx/main.c',
'-O3', '-shared', '-fPIC', '-o',
'/tmp/tmptthkhvx/cuda_utils.cpython-310-x86_64-linux-gnu.so
http://cuda_utils.cpython-310-x86_64-linux-gnu.so', '-lcuda',
'-L/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/triton/backends/nvidia/lib',
'-L/lib64',
'-I/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/triton/backends/nvidia/include',
'-I/tmp/tmptthkhvx', '-I/spshared/apps/miniconda3/
envs/spisonet/include/python3.10']' returned non-zero exit status 1.
Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information
You can suppress this exception and fall back to eager by setting:
import torch._dynamo
torch._dynamo.config.suppress_errors = True
Thanks for your help!
—
Reply to this email directly, view it on GitHub
https://github.com/IsoNet-cryoET/spIsoNet/issues/17, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AEEHPLBBWFBLOWV4LX3ED232BR2TBAVCNFSM6AAAAABSENDI76VHI2DSMVQWIX3LMV43ASLTON2WKOZSGY3TKNRVGE2TGMA
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

--
Yuntao Liu,  Postdoc.
California NanoSystem Institute
University of California Los Angeles

Thanks for your reply!
After commenting out these lines, I rerun this step. It seems that there is no change.
The error information as followed:
spisonet.py reconstruct emd_8731_half_map_1.mrc emd_8731_half_map_2.mrc --aniso_file FSC3D.mrc --mask emd_8731_msk_1.mrc --limit_res 3.5 --epochs 30 --alpha 1 --beta 0.5 --output_dir isonet_maps --gpuID 0,1,2,3 --acc_batches 2
11-21 20:50:41, INFO     voxel_size 1.309999942779541
11-21 20:50:52, INFO     spIsoNet correction until resolution 3.5A!
Information beyond 3.5A remains unchanged
11-21 20:51:02, INFO     Start preparing subvolumes!
11-21 20:51:31, INFO     Done preparing subvolumes!
11-21 20:51:31, INFO     Start training!
11-21 20:51:34, INFO     Port number: 47458
learning rate 0.0003
['isonet_maps/emd_8731_half_map_1_data', 'isonet_maps/emd_8731_half_map_2_data']
0%|                                                                                | 0/125 [00:00<?, ?batch/s][rank3]:W1121 20:51:49.720000 27636 site-packages/torch/_logging/_internal.py:1081] [0/0] Profiler function <class 'torch.autograd.profiler.record_function'> will be ignored
[rank2]:W1121 20:51:49.887000 18250 site-packages/torch/_logging/_internal.py:1081] [0/0] Profiler function <class 'torch.autograd.profiler.record_function'> will be ignored
[rank1]:W1121 20:51:49.899000 10667 site-packages/torch/_logging/_internal.py:1081] [0/0] Profiler function <class 'torch.autograd.profiler.record_function'> will be ignored
[rank0]:W1121 20:51:49.933000 5556 site-packages/torch/_logging/internal.py:1081] [0/0] Profiler function <class 'torch.autograd.profiler.record_function'> will be ignored
/tmp/tmpbqb3z3m1/main.c:6:23: fatal error: stdatomic.h: No such file or directory
#include <stdatomic.h>
^
compilation terminated.
/tmp/tmpfmgniqil/main.c:6:23: fatal error: stdatomic.h: No such file or directory
#include <stdatomic.h>
^
compilation terminated.
/tmp/tmpwg638qpb/main.c:6:23: fatal error: stdatomic.h: No such file or directory
#include <stdatomic.h>
^
compilation terminated.
/tmp/tmp9no3ip6h/main.c:6:23: fatal error: stdatomic.h: No such file or directory
#include <stdatomic.h>
^
compilation terminated.
/tmp/tmp8083o31x/main.c:6:23: fatal error: stdatomic.h: No such file or directory
#include <stdatomic.h>
^
compilation terminated.
/tmp/tmphylv2zp/main.c:6:23: fatal error: stdatomic.h: No such file or directory
#include <stdatomic.h>
^
compilation terminated.
/tmp/tmp1hpb997j/main.c:6:23: fatal error: stdatomic.h: No such file or directory
#include <stdatomic.h>
^
compilation terminated.
/tmp/tmppir027c6/main.c:6:23: fatal error: stdatomic.h: No such file or directory
#include <stdatomic.h>
^
compilation terminated.
/tmp/tmp4y1k5_9o/main.c:6:23: fatal error: stdatomic.h: No such file or directory
#include <stdatomic.h>
^
compilation terminated.
/tmp/tmpbkgoo9g4/main.c:6:23: fatal error: stdatomic.h: No such file or directory
#include <stdatomic.h>
^
compilation terminated.
/tmp/tmpvdinu5f3/main.c:6:23: fatal error: stdatomic.h: No such file or directory
#include <stdatomic.h>
^
compilation terminated.
/tmp/tmp51ilakio/main.c:6:23: fatal error: stdatomic.h: No such file or directory
#include <stdatomic.h>
^
compilation terminated.
0%|                                                                                | 0/125 [00:06<?, ?batch/s]
[rank0]:[W1121 20:51:55.475865037 ProcessGroupNCCL.cpp:1250] Warning: WARNING: process group has NOT been destroyed before we destruct ProcessGroupNCCL. On normal program exit, the application should call destroy_process_group to ensure that any pending NCCL operations have finished in this process. In rare cases this process can exit before this point and block the progress of another member of the process group. This constraint has always been present,  but this warning has only been added since PyTorch 2.4 (function operator())
W1121 20:51:55.676000 30273 site-packages/torch/multiprocessing/spawn.py:160] Terminating process 5556 via signal SIGTERM
W1121 20:51:55.677000 30273 site-packages/torch/multiprocessing/spawn.py:160] Terminating process 10667 via signal SIGTERM
W1121 20:51:55.678000 30273 site-packages/torch/multiprocessing/spawn.py:160] Terminating process 18250 via signal SIGTERM
Traceback (most recent call last):
File "/spshared/apps/miniconda3/envs/spisonet/bin/spisonet.py", line 8, in
sys.exit(main())
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/spIsoNet/bin/spisonet.py", line 549, in main
fire.Fire(ISONET)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/fire/core.py", line 135, in Fire
component_trace = _Fire(component, args, parsed_flag_args, context, name)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/fire/core.py", line 468, in _Fire
component, remaining_args = _CallAndUpdateTrace(
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/fire/core.py", line 684, in _CallAndUpdateTrace
component = fn(*varargs, **kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/spIsoNet/bin/spisonet.py", line 182, in reconstruct
map_refine_n2n(halfmap1,halfmap2, mask_vol, fsc3d, alpha = alpha,beta=beta,  voxel_size=voxel_size, output_dir=output_dir,
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/spIsoNet/bin/map_refine.py", line 145, in map_refine_n2n
network.train([data_dir_1,data_dir_2], output_dir, alpha=alpha,beta=beta, output_base=output_base0, batch_size=batch_size, epochs = epochs, steps_per_epoch = 1000,
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/spIsoNet/models/network_n2n.py", line 265, in train
mp.spawn(ddp_train, args=(self.world_size, self.port_number, self.model,alpha,beta,
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/multiprocessing/spawn.py", line 328, in spawn
return start_processes(fn, args, nprocs, join, daemon, start_method="spawn")
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/multiprocessing/spawn.py", line 284, in start_processes
while not context.join():
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/multiprocessing/spawn.py", line 203, in join
raise ProcessRaisedException(msg, error_index, failed_process.pid)
torch.multiprocessing.spawn.ProcessRaisedException:
-- Process 3 terminated with the following error:
Traceback (most recent call last):
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_dynamo/output_graph.py", line 1446, in _call_user_compiler
compiled_fn = compiler_fn(gm, self.example_inputs())
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/dynamo/backends/distributed.py", line 506, in compile_fn
return self.backend_compile_fn(gm, example_inputs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/dynamo/repro/after_dynamo.py", line 129, in call
compiled_gm = compiler_fn(gm, example_inputs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/init.py", line 2234, in call
return compile_fx(model, inputs, config_patches=self.config)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_inductor/compile_fx.py", line 1521, in compile_fx
return aot_autograd(
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_dynamo/backends/common.py", line 72, in call
cg = aot_module_simplified(gm, example_inputs, **self.kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_functorch/aot_autograd.py", line 1071, in aot_module_simplified
compiled_fn = dispatch_and_compile()
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_functorch/aot_autograd.py", line 1056, in dispatch_and_compile
compiled_fn, _ = create_aot_dispatcher_function(
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_functorch/aot_autograd.py", line 522, in create_aot_dispatcher_function
return _create_aot_dispatcher_function(
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_functorch/aot_autograd.py", line 759, in _create_aot_dispatcher_function
compiled_fn, fw_metadata = compiler_fn(
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_functorch/_aot_autograd/jit_compile_runtime_wrappers.py", line 588, in aot_dispatch_autograd
compiled_fw_func = aot_config.fw_compiler(fw_module, adjusted_flat_args)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_inductor/compile_fx.py", line 1350, in fw_compiler_base
return _fw_compiler_base(model, example_inputs, is_inference)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_inductor/compile_fx.py", line 1421, in _fw_compiler_base
return inner_compile(
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_inductor/compile_fx.py", line 475, in compile_fx_inner
return wrap_compiler_debug(_compile_fx_inner, compiler_name="inductor")(
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_dynamo/repro/after_aot.py", line 85, in debug_wrapper
inner_compiled_fn = compiler_fn(gm, example_inputs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_inductor/compile_fx.py", line 661, in _compile_fx_inner
compiled_graph = FxGraphCache.load(
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_inductor/codecache.py", line 1334, in load
compiled_graph = compile_fx_fn(
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_inductor/compile_fx.py", line 570, in codegen_and_compile
compiled_graph = fx_codegen_and_compile(gm, example_inputs, **fx_kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_inductor/compile_fx.py", line 878, in fx_codegen_and_compile
compiled_fn = graph.compile_to_fn()
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_inductor/graph.py", line 1913, in compile_to_fn
return self.compile_to_module().call
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_inductor/graph.py", line 1839, in compile_to_module
return self._compile_to_module()
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_inductor/graph.py", line 1845, in _compile_to_module
self.codegen_with_cpp_wrapper() if self.cpp_wrapper else self.codegen()
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_inductor/graph.py", line 1784, in codegen
self.scheduler.codegen()
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_inductor/scheduler.py", line 3383, in codegen
return self._codegen()
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_inductor/scheduler.py", line 3461, in _codegen
self.get_backend(device).codegen_node(node)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_inductor/codegen/cuda_combined_scheduling.py", line 80, in codegen_node
return self._triton_scheduling.codegen_node(node)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_inductor/codegen/simd.py", line 1155, in codegen_node
return self.codegen_node_schedule(node_schedule, buf_accesses, numel, rnumel)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_inductor/codegen/simd.py", line 1364, in codegen_node_schedule
src_code = kernel.codegen_kernel()
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_inductor/codegen/triton.py", line 2661, in codegen_kernel
**self.inductor_meta_common(),
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_inductor/codegen/triton.py", line 2532, in inductor_meta_common
"backend_hash": torch.utils._triton.triton_hash_with_backend(),
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/utils/_triton.py", line 53, in triton_hash_with_backend
backend = triton_backend()
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/utils/_triton.py", line 45, in triton_backend
target = driver.active.get_current_target()
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/triton/runtime/driver.py", line 23, in getattr
self._initialize_obj()
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/triton/runtime/driver.py", line 20, in _initialize_obj
self._obj = self._init_fn()
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/triton/runtime/driver.py", line 9, in _create_driver
return actives0
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/triton/backends/nvidia/driver.py", line 371, in init
self.utils = CudaUtils()  # TODO: make static
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/triton/backends/nvidia/driver.py", line 80, in init
mod = compile_module_from_src(Path(os.path.join(dirname, "driver.c")).read_text(), "cuda_utils")
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/triton/backends/nvidia/driver.py", line 57, in compile_module_from_src
so = _build(name, src_path, tmpdir, library_dirs(), include_dir, libraries)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/triton/runtime/build.py", line 48, in _build
ret = subprocess.check_call(cc_cmd)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/subprocess.py", line 369, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/usr/bin/gcc', '/tmp/tmp4y1k5_9o/main.c', '-O3', '-shared', '-fPIC', '-o', '/tmp/tmp4y1k5_9o/cuda_utils.cpython-310-x86_64-linux-gnu.so', '-lcuda', '-L/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/triton/backends/nvidia/lib', '-L/lib64', '-I/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/triton/backends/nvidia/include', '-I/tmp/tmp4y1k5_9o', '-I/spshared/apps/miniconda3/envs/spisonet/include/python3.10']' returned non-zero exit status 1.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/multiprocessing/spawn.py", line 90, in _wrap
fn(i, *args)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/spIsoNet/models/network_n2n.py", line 116, in ddp_train
preds = model(x1)# + noise.cuda())
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
return forward_call(*args, **kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 465, in _fn
return fn(*args, **kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_dynamo/external_utils.py", line 40, in inner
return fn(*args, **kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
return forward_call(*args, **kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/nn/parallel/distributed.py", line 1643, in forward
else self._run_ddp_forward(*inputs, **kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/nn/parallel/distributed.py", line 1459, in _run_ddp_forward
return self.module(*inputs, **kwargs)  # type: ignore[index]
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
return forward_call(*args, **kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/spIsoNet/models/unet.py", line 97, in forward
x, down_sampling_features = self.encoder(x)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/spIsoNet/models/unet.py", line 98, in torch_dynamo_resume_in_forward_at_97
x = self.decoder(x, down_sampling_features)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 1263, in call
return hijacked_callback(
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 1064, in call
result = self._inner_convert(
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 526, in call
return _compile(
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 924, in _compile
guarded_code = compile_inner(code, one_graph, hooks, transform)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 666, in compile_inner
return _compile_inner(code, one_graph, hooks, transform)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_utils_internal.py", line 87, in wrapper_function
return function(*args, **kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 699, in _compile_inner
out_code = transform_code_object(code, transform)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_dynamo/bytecode_transformation.py", line 1322, in transform_code_object
transformations(instructions, code_options)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 219, in _fn
return fn(*args, **kwargs)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 634, in transform
tracer.run()
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2796, in run
super().run()
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 983, in run
while self.step():
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 895, in step
self.dispatch_table[inst.opcode](self, inst)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2987, in RETURN_VALUE
self._return(inst)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2972, in _return
self.output.compile_subgraph(
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_dynamo/output_graph.py", line 1117, in compile_subgraph
self.compile_and_call_fx_graph(tx, list(reversed(stack_values)), root)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_dynamo/output_graph.py", line 1369, in compile_and_call_fx_graph
compiled_fn = self.call_user_compiler(gm)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_dynamo/output_graph.py", line 1416, in call_user_compiler
return self._call_user_compiler(gm)
File "/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/torch/_dynamo/output_graph.py", line 1465, in _call_user_compiler
raise BackendCompilerFailed(self.compiler_fn, e) from e
torch._dynamo.exc.BackendCompilerFailed: backend='compile_fn' raised:
CalledProcessError: Command '['/usr/bin/gcc', '/tmp/tmp4y1k5_9o/main.c', '-O3', '-shared', '-fPIC', '-o', '/tmp/tmp4y1k5_9o/cuda_utils.cpython-310-x86_64-linux-gnu.so', '-lcuda', '-L/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/triton/backends/nvidia/lib', '-L/lib64', '-I/spshared/apps/miniconda3/envs/spisonet/lib/python3.10/site-packages/triton/backends/nvidia/include', '-I/tmp/tmp4y1k5_9o', '-I/spshared/apps/miniconda3/envs/spisonet/include/python3.10']' returned non-zero exit status 1.
Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information
You can suppress this exception and fall back to eager by setting:
import torch._dynamo
torch._dynamo.config.suppress_errors = True
Thanks for your help!

I have same issue, is there any update?


build: add a CMake based build for Windows

Since swift-package-manager doesn't work on Windows yet, add a CMake
based build system.

I had errors when trying to generate an Xcode project and build, so I'll trust you that this works on Windows.


Use non-validated HttpClient headers

We can save cycles by using non-validating enumeration when copying headers or accessing them from transforms.
https://github.com/dotnet/runtime/issues/35126#issuecomment-854040319
Related: #1083

Triage: Let's take it in 1.1 if it is not too large cost.


[Bug] Snapshot hub api breaks out

Describe the bug
Request https://hub.snapshot.page/api/voters?from=1608500000&to=1609500000&spaces=balancer,yam.eth, it redirects to https://hub.snapshot.page/api, and returns some data which makes no sense to me.
To Reproduce
Steps to reproduce the behavior:
Request https://hub.snapshot.page/api/voters?from=1608500000&to=1609500000&spaces=balancer,yam.eth
Expected behavior
Get the voter list
Screenshots

@zhouhanseng Hey this API endpoint was deprecated, you can use the GraphQL API instead have a look here: https://docs.snapshot.org/graphql-api#votes

Closing due to inactivity. If you're still having issues please open another issue.


Xamarin.Android.ManifestMerger causes crash on build

AndroidManifestMerger prevents build with Xamarin.Android 10.2 SDK version. This issue happens in the App Center. In VS it builds normally.

Update, Xamarin.Android.ManifestMerger prev-3 is not compatible with android SDK 10.2

@gvuksa
Forget Xamarin.Android.ManifestMerger nuget. The reason it never was published as stable is that manifest merging is now part of build (Xamarin.Android core) which is more logical.
Add this MsBuild property to your build (csproject):
  <PropertyGroup>
    <AndroidManifestMerger>manifestmerger.jar</AndroidManifestMerger>
  </PropertyGroup>

https://github.com/xamarin/GooglePlayServicesComponents/blob/master/source/GooglePlayServicesTargets.cshtml#L30-L32
If that fixes your problem, please close this issue yourself.

@moljac thanks for the info. The reason I was using Xamarin.Android.ManifestMerger was because of the issue with FirebaseAuth.Instance always being null. However, it seems that with Android SDK 10.2 this is no longer a case, the issue with FireabaseAuth is resolved and it does not require ManifestMerger or any other change in to the project. So, although ManifestMerger still crashes build I will close this issue.


fix (AE) When mode contains a hyphen, convert it to camelCase

This means browser-ext becomes browserExt

Closing because we changed stuff.


Italicize all taxon names for Kingdom Viruses

From https://forum.inaturalist.org/t/italicise-all-viral-taxa/16971
According to 3.30 of the ICTV code,

In formal taxonomic usage, the accepted names of virus, viroid and satellite realms, subrealms, kingdoms, subkingdoms, phyla, subphyla, classes, subclasses, orders, suborders, families, subfamilies, genera and subgenera are printed in italics and the first letters of the names are capitalized.

So we should italicize all virus names above species.

This is not trivial. Or rather the trivial ways to do this are all hacky, hard to maintain, and guaranteed to create more problems than they solve in the future. The right way to do this, in my opinion, is to make the Kingdom Viruses an iconic taxon, and to do that I need these things

[ ] An icon to represent all viruses
[ ] Some assurance that the Kingdom Viruses is actually named correctly
[ ] A color to represent viruses


[Bug]: Import Pipeline button is ready before pipeline server is ready to serve

Is there an existing issue for this?

[X] I have searched the existing issues

Deploy type
ODH Dashboard UI
Version
RHODS Operator 1.33.0
Environment
Any
Current Behavior
Error during the import.

Expected Behavior
Import pipeline button must be enabled once the pods are readyThe import
Steps To Reproduce
Create a workbench
Create a data connection
Create pipeline server
Once the Import pipeline button is ready, try to import the pipeline ( be fast )
Workaround (if any)
Refresh the page and import the pipeline again
Anything else
No response

We also have the following Error Message before the Pipeline Server becomes ready

Migrated to Jira: https://issues.redhat.com/browse/RHOAIENG-1666


Write docs for http-service

Write docs for http-service.

[x] inline docs
[ ] doc tests
[ ] readme
[ ] examples

👋 I happily take this one! Some guidance might be appreciated!

@gruberb yay awesome!
So most items already have docs, but we're missing doc tests + readme + examples. A starting point would be the recent Tide refactor to use http-service internally:

https://github.com/rustasync/tide/pull/139
https://github.com/rustasync/tide/pull/142

hopefully this can serve as a starting point, lmk if you have any questions! :sparkles:

So I started working on it and I am trying to figure out the smallest possible example to create a new http-service and to be able to http-service::serve().
What I came up so far is:
struct Server {
    message: String,
}

impl HttpService for Server {
    type Connection = ();
    type ConnectionFuture = future::Ready<Result<(), std::io::Error>>;
    type Fut = FutureObj<'static, Result<http_service::Response, std::io::Error>>;

    fn connect(&self) -> Self::ConnectionFuture {
        future::ok(())
    }

    fn respond(&self, _conn: &mut (), req: http_service::Request) -> Self::Fut {
        FutureObj::new(Box::new(
            Ok(self.message)
        ))
    }
}

#[cfg(feature = "hyper")]
fn main() {
    http_service_hyper::serve(Server, "<IP_ADDRESS>");
}

Am I on the right track or how is http-service suppose to be used?

@gruberb that seems about right. Possibly the mock backend could also be used so input / output can be inspected, but overall it seems pretty reasonable!
Also I'm assuming the cfg is just a placeholder, and not part of the final output right?

@yoshuawuyts Yeah, just a place holder!
I might need guidance on two places:

I want to create a new Server, where message is a Stream of bytes:

struct Server {
    message: StreamObj<'static, Result<Bytes, std::io::Error>>,
}

Now when I Impl the Server, I have actually no idea how to create a StreamObj, nor could I find any docs which would explain it:
impl Server {
    fn new(message: Bytes) {
        Server {
            message: StreamObj::new(Ok(
                message
            ))
        }
    }
}

How can I create a new StreamObj?

When I Impl HttpService for Server, I need to find a way to make this thread safe?

impl HttpService for Server {
    type Connection = ();
    type ConnectionFuture = future::Ready<Result<(), std::io::Error>>;
    type Fut = FutureObj<'static, Result<http_service::Response, std::io::Error>>;

    fn connect(&self) -> Self::ConnectionFuture {
        future::ok(())
    }

    fn respond(&self, _conn: &mut (), req: http_service::Request) -> Self::Fut {
        FutureObj::new(Box::new(
            async move {
                Ok(Response::new(http_service::Body {
                    stream: StreamObj::new(Box::new(self.message))
                }))
            }
        ))
    }
}

The message for FutureObj::new(... I am getting is:
FutureObj::new(Box::new(
             ^^^^^^^^^^^^^^ `*mut ()` cannot be shared between threads safely

It's kinda hard to find documentation, and I can't spend the full day coding lately, so some help would be good :)

@gruberb hmm, is it required for body to be a stream? If not just having it be a Vec<u8> of bytes might be ideal.
If not, could you push a branch somewhere that I could clone to try some stuff out? I don't know the answer either right now, but if I can poke at it I might be able to help.

@yoshuawuyts Nice, this helped! Got it to work! Will finish up the tests and write the docs. Publish a PR soon!

Issue related to StreamObj (it's getting removed) -- https://github.com/rust-lang-nursery/futures-rs/pull/1494

Closed via https://github.com/rustasync/http-service/pull/19


[SPARK-41658][SPARK-41656][DOCS][FOLLOW-UP] Update JIRAs in skipped tests' comments

What changes were proposed in this pull request?
This PR is a followup of https://github.com/apache/spark/pull/39347 and https://github.com/apache/spark/pull/39347, which updates the invalid JIRAs linked in the comments.
Why are the changes needed?
To track the issues properly, and reenable skipped tests in the future.
Does this PR introduce any user-facing change?
No, dev-only.
How was this patch tested?
Comment-only. Linter in CI should verify them. I also manually checked it in my local.

cc @zhengruifeng FYI

Merged to master.


Mob Pounder does not damage/kill Magma Cubes

Versions:

Minecraft: 1.15.2
Forge: 31.1.19
Flux Library: 3.2.2
Flux: 4.14.0

Steps to Reproduce:

place Mob Pounder next to magma cubes
power Mob Pounder

Description of issue:
Expected result: Magma Cube dies, or at least takes damage.
Actual result: they do not die, they do not even appear to take damage

thank you :)
nice mod, btw!


[add] branch symbol's Explanation

I add airline branch symbol's Explanation in doc.
What do you think? @chrisbra
Related PR

https://github.com/vim-airline/vim-airline-themes/issues/215

Isn't this already at the current documentation?
https://github.com/vim-airline/vim-airline/blob/16b57c7be4d13d21ffe945a31b20c2f6a990d5c7/doc/airline.txt#L486-L488

I wrote this patch because I thought it better to add it because the default value of the symbol of branch was not described.

well, can you move it up, right below to the existing part of the documentation? Thanks.

Thanks, Christian
I fixed.

thanks, looks good.


[Bug]: No Navigation for /community page.

What happened?
The Community Link is not present in the Navbar. We can add a Link in Navbar so that the users can Navigate to /community.
Add screenshots
Add screenshots
What browsers are you seeing the problem on?
No response
Code of Conduct

[X] I agree to follow this project's Code of Conduct

If needed, you can add it in the footer to avoid making the navbar lengthier.

Since you are currently working on two issues, I can only assign this issue to you after you complete the earlier two issues.

Since you are currently working on two issues, I can only assign this issue to you after you complete the earlier two issues.

I've already made a PR for all the assigned issues. You can assign other issues now


All server API documentation gone ?!

I was checking few classes and methods and I see that we've lost all the explanations on methods and classes with the migration from msdn to docs !
As we still have on-prem solutions, it is still useful for us to know what happens if we call this or that method on an object.
Would it be possible to recover at least the last version of the summaries, remarks and samples which were available on the previous platform (msdn)?
I can understand that you will spend effort to maintain the documentation on the client API, but the server API doc as it is now, is totally useless!

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 3d97ce3e-615b-24da-7c4f-3de70886aba7
Version Independent ID: 7de7634d-bb56-6f1d-7e72-f195d6bf408d
Content: Microsoft.SharePoint Namespace
Content Source: sharepoint-server/xml/ns-Microsoft.SharePoint.xml
Product: sharepoint
GitHub Login: @dotnet-bot
Microsoft Alias: o365devx

Lots of the CSOM API documentation has vanished as well. @VesaJuvonen mentioned that the migration from MSDN to docs.microsoft.com was taking a little longer than expected, so hopefully they will reappear in their new home before too long.

This is, unfortunately, work in progress due to migration challenges with the MSDN and docs systems. We are working on a solution for this as soon as possible and absolutely understand the urgency for this.

Thank you both for your comments.

So any updates on this? The missing CSOM documentation is getting really annoying.

Is there any progress?


Outdated cask: Medis

The latest release is 0.6.1

https://github.com/caskroom/homebrew-cask/pull/36382


Virtual scroll - bug on the official example

[x] bug report => search github for a similar issue or PR before submitting
[ ] feature request
[ ] support request => Please do not submit support request here, post on Stackoverflow or Gitter

Current behavior
Rows are not rendered when scrolling to the bottom of the table too fast with virtualization enabled.
Expected behavior
Rows should be rendered properly even when scrolled fast.
Reproduction of the problem

Open http://swimlane.github.io/ngx-datatable/#virtual-scroll
Grab the scrollbar with your mouse and scroll to the bottom of the table.

Please tell us about your environment:
Windows 10 64bit

Table version:

Angular version:

Browser: [all | Chrome XX | Firefox XX | IE XX | Safari XX | Mobile Chrome XX | Android X.X Web Browser | iOS XX Safari | iOS XX UIWebView | iOS XX WKWebView ]
Chrome 80, Firefox 73.

Language: [all | TypeScript X.X | ES6/7 | ES5]

http://swimlane.github.io/ngx-datatable/#virtual-scroll
Incorrect transform value when scrolling with the scrollbar

Last item Value when scrolling with the mouse wheel (the correct)

transform: translate3d(0px, 893510px, 0px);
Last item Value when scrolling with the scroll bar
transform: translate3d(0px, 888491px, 0px);

Tested on chrome and firefox

Ya.  I was having the exact same problem with my datatable implementation.
I went to go check the example on the demos page ("surely I must be doing something wrong!") and it had the exact same problem.

When scrolling with the mousewheel, rows are not rendered in quickly and you get a bunch of blank spaces.  On the next mousemove after the wheel stops (I think?) it then renders in.
When scrolling with the scrollbar, rows are not rendered in the same way but when you let go of the scrollbar thumb the entire list remains blank.

I was able to fix this by essentially attaching a (scroll) listener and forcing the whole table to refresh (by duplicating my items into a new list).
After doing this, the list correctly renders in after every scroll event but now my scrolling is at about 1fps.
I specifically chose this tech for my project because it was supposedly "good with virtual scrolling" but this is something of a UX disaster.

I see there's already a PR with a suggested fix for this: https://github.com/swimlane/ngx-datatable/pull/1798
Any chance of getting that PR looked at. This is a pretty bad user experience currently

Hello, any progress/workarounds on this? I'm having the same issue while scrolling a large set of data.

I fixed the issue this way. Create a rxjs Subject:
scrollEvent: Subject = new Subject();
then:
this.scrollEvent.pipe(
debounceTime(100),
map(_ => this.tableRef.offset),
distinctUntilChanged(),
).subscribe(() => this.tableRef.element.getElementsByTagName('datatable-body')[0].scrollTop++ );
and for the ngx-datatable scroll output, emit scrollEvent:
[html]
(scroll)="scrollEvent.next()"
Works pretty well with Angular 10.1.1 and ngx-datatable 18.0.0

tableRef

With what do you get the tableRef?

tableRef

With what do you get the tableRef?

@ViewChild('table') tableRef!: DatatableComponent;

@ViewChild('table') tableRef!: DatatableComponent;

Hi @Hypercubed, any chance to get this bug solved, soon?

I have a PR coming that should fix the demo and give some improved details on implementaion: https://github.com/swimlane/ngx-datatable/pull/1991

https://github.com/user-attachments/assets/bbe2f1eb-5f14-48bd-92e7-cd0cc9a74a2d
We are facing issues with scrolling while filtering data which has 100+ records in a data table
can any one help


Added exit code support

As per #82, we've wanted exit code support for a while and are finally adding it
We discovered the quirk that bash will persist exit codes until a new command is provided so we added a configuration to ignore that
In this PR:

Added non-zero exit code support
Added tests
Added PROMPT_SYMBOL_ERROR_COLOR environment variable support for customization
Added PROMPT_SHOW_ERROR_ONCE environment variable to allow opt-out of hiding persisted exit code
Added documentation
Depends on #88, don't land until that's landed (and then update this PR base to master)

Screenshots:

@rpdelaney I think this was missed while reviewing #88 as well =/

I saw it. It's looks a bit more complex so I'm waiting until the end of my work day so I can put some more time into looking at it.

Ah, gotcha. Sounds great. Thanks for your help as always @rpdelaney =)

Hmm, what's confusing about PROMPT_SHOW_ERROR_ONCE? The scenario it's solving for? Implementation? Something else?

Deciding to land for now, hopefully we can iterate on clarity in a future PR

Oh damn it, I forgot to update the base -_-;; Going to see if I can just cherry-pick to master anyway =/

I think I understand what the code is doing more than I understand why it's doing it :)

:+1: Alright, really verbose explanation of all the scenarios and how we fix them:
Blank terminal

Open a new terminal
Exit code is 0 by default
We display a white $
We press enter (no new command a few times)
Exit code is still 0 so we keep displaying a white $
Nothing unexpected here

Successful exit code

Open a new terminal
Execute a successful command (e.g. true)
Exit code is 0
We display a white $
We press enter (no new command a few times)
Exit code is still 0 so we keep displaying a white $
Nothing unexpected here

Non-zero exit code

Open a new terminal
Execute a successful command (e.g. false)
Exit code is non-zero (e.g. 1)
We display a red $
We press enter (no new command a few times)
Expected behavior (for me at least)

Blank command is interpreted as a new command
Exit code is set to 0
We display a white $

Actual behavior

Bash doesn't reset exit code on blank command
Exit code is still non-zero so we keep displaying a red $

Keyboard interrupt

Open a new terminal
Start typing a valid command but run keyboard interrupt instead (Ctrl+C)
Expected behavior (for me at least)

We can potentially identify this as a zero/non-zero exit code, I'm fine with either scenario
but the next lines matter...
We press enter (no new command a few times)
Blank command is interpreted as a new command
Exit code is set to 0
We display a white $

Actual behavior

Exit code is non-zero (i.e. 130)
We display a red $
We press enter (no new command a few times)
Bash doesn't reset exit code on blank command
Exit code is still non-zero (i.e. 130) so we keep displaying a red $

What our fix does

Updates behavior to detect when a blank command is entered

i.e. A valid command would have a new history entry
A blank command wouldn't be input in the history
As a result, if we see the same timestamp+command in history then we know it was a blank command

When a blank command is entered, then reset the exit code to our expected behavior

Does that clear things up? Is this good enough to link to in the code comments for a verbose explanation?

Want to add that I've realized my dotfiles has a HISTTIMEFORMAT setting in it which causes a timestamp in my history:
  499  2020-09-07T20:23:57-0700 false
  500  2020-09-07T20:23:57-0700 false
  501  2020-09-07T20:23:58-0700 history 5

The history check still works regardless though as we have the history number in each prompt (i.e. 500/501/etc):
  500  false
  501  history 5
  502  history 5
``


⚠ DDoS-Relay US-W-hil-2 has degraded performance

In 861caa8, DDoS-Relay US-W-hil-2 (http://<IP_ADDRESS>:42069/;csv) experienced degraded performance:

HTTP code: 200
Response time: 63 ms

Resolved: DDoS-Relay US-W-hil-2 performance has improved in 161e488 after 12 minutes.


ruff server: Support publish diagnostics as a fallback when pull diagnostics aren't supported

Summary
Fixes #11059
Several major editors don't support pull diagnostics, a method of sending diagnostics to the client that was introduced in version 0.3.17 of the specification. Until now, ruff server has only used pull diagnostics, which resulted in diagnostics not being available on Neovim and Helix, which don't support pull diagnostics yet (though Neovim 10.0 will have support for this).
ruff server will now utilize the older method of sending diagnostics, known as 'publish diagnostics', when pull diagnostics aren't supported by the client. This involves re-linting a document every time it is opened or modified, and then sending the diagnostics generated from that lint to the client via the textDocument/publishDiagnostics notification.
Test Plan
The easiest way to test that this PR works is to check if diagnostics show up on Neovim <=0.9.

I think our old extension also had an option to pull diagnostics only on save. But I don't mind pushing this out until we're asked to build this (especially because we can't really support it when using pull diagnostics)

@dhruvmanila I've opened an issue (https://github.com/astral-sh/ruff/issues/11114) to solve the problem you mentioned earlier.


Fix bug when flaw_bug_tracker is None (on attach_cve_flaws)

Python 3.6.15

>>> flaw_bug_tracker = None

>>> flaw_bug_tracker.type
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'NoneType' object has no attribute 'type'

>>> flaw_bug_tracker.type if hasattr(flaw_bug_tracker, 'type') else ''
''

Codecov Report

Merging #371 (860ee35) into master (7c70d79) will increase coverage by 6.24%.
The diff coverage is 52.24%.

@@            Coverage Diff             @@
##           master     #371      +/-   ##
==========================================
+ Coverage   30.65%   36.90%   +6.24%
==========================================
  Files          55       59       +4
  Lines        5349     5799     +450
  Branches     1179     1245      +66
==========================================
+ Hits         1640     2140     +500
+ Misses       3608     3499     -109
- Partials      101      160      +59

Impacted Files
Coverage Δ

elliottlib/assertion.py
100.00% <ø> (+21.05%)
:arrow_up:

elliottlib/cli/__main__.py
0.00% <0.00%> (ø)

elliottlib/cli/advisory_drop_cli.py
0.00% <0.00%> (ø)

elliottlib/cli/attach_bugs_cli.py
0.00% <0.00%> (ø)

elliottlib/cli/change_state_cli.py
0.00% <ø> (ø)

elliottlib/cli/create_cli.py
0.00% <0.00%> (ø)

elliottlib/cli/create_placeholder_cli.py
0.00% <0.00%> (ø)

elliottlib/cli/create_textonly_cli.py
0.00% <0.00%> (ø)

elliottlib/cli/rhcos_cli.py
0.00% <0.00%> (ø)

elliottlib/gitdata.py
19.41% <0.00%> (ø)

... and 25 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

/lgtm