Multi-gpu training gets stuck

Checklist

[x] I have searched related issues but cannot get the expected help.
[x] I have read the FAQ documentation but cannot get the expected help.
[x] The bug has not been fixed in the latest version.

Describe the bug
Single GPU training works fine, single node multi-GPU doesn't.
Relevant: #3823, #2193, #1979, #4535, maybe #3973
Rolling back intel-openmp doesn't help, and it uses only default configs.
I.e. running this is ok:
python tools/train.py configs/cityscapes/faster_rcnn_r50_fpn_1x_cityscapes.py

Outputs:
...
2021-11-17 17:33:20,582 - mmdet - INFO - initialize Bottleneck with init_cfg {'type': 'Constant', 'val': 0, 'override': {'name': 'norm3'}}
2021-11-17 17:33:20,583 - mmdet - INFO - initialize Bottleneck with init_cfg {'type': 'Constant', 'val': 0, 'override': {'name': 'norm3'}}
2021-11-17 17:33:20,584 - mmdet - INFO - initialize Bottleneck with init_cfg {'type': 'Constant', 'val': 0, 'override': {'name': 'norm3'}}
2021-11-17 17:33:20,586 - mmdet - INFO - initialize Bottleneck with init_cfg {'type': 'Constant', 'val': 0, 'override': {'name': 'norm3'}}
2021-11-17 17:33:20,588 - mmdet - INFO - initialize Bottleneck with init_cfg {'type': 'Constant', 'val': 0, 'override': {'name': 'norm3'}}
2021-11-17 17:33:20,589 - mmdet - INFO - initialize Bottleneck with init_cfg {'type': 'Constant', 'val': 0, 'override': {'name': 'norm3'}}
2021-11-17 17:33:20,598 - mmdet - INFO - initialize FPN with init_cfg {'type': 'Xavier', 'layer': 'Conv2d', 'distribution': 'uniform'}
2021-11-17 17:33:20,615 - mmdet - INFO - initialize RPNHead with init_cfg {'type': 'Normal', 'layer': 'Conv2d', 'std': 0.01}
2021-11-17 17:33:20,621 - mmdet - INFO - initialize Shared2FCBBoxHead with init_cfg [{'type': 'Normal', 'std': 0.01, 'override': {'name': 'fc_cls'}}, {'type': 'Normal', 'std': 0.001, 'override': {'name': 'fc_reg
'}}, {'type': 'Xavier', 'override': [{'name': 'shared_fcs'}, {'name': 'cls_fcs'}, {'name': 'reg_fcs'}]}]
loading annotations into memory...
Done (t=0.44s)
creating index...
index created!
loading annotations into memory...
Done (t=0.06s)
creating index...
index created!
2021-11-17 17:33:23,497 - mmdet - INFO - load checkpoint from http path: https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth
2021-11-17 17:33:23,597 - mmdet - WARNING - The model and loaded state dict do not match exactly

size mismatch for roi_head.bbox_head.fc_cls.weight: copying a param with shape torch.Size([81, 1024]) from checkpoint, the shape in current model is torch.Size([9, 1024]).
size mismatch for roi_head.bbox_head.fc_cls.bias: copying a param with shape torch.Size([81]) from checkpoint, the shape in current model is torch.Size([9]).
size mismatch for roi_head.bbox_head.fc_reg.weight: copying a param with shape torch.Size([320, 1024]) from checkpoint, the shape in current model is torch.Size([32, 1024]).
size mismatch for roi_head.bbox_head.fc_reg.bias: copying a param with shape torch.Size([320]) from checkpoint, the shape in current model is torch.Size([32]).
2021-11-17 17:33:23,600 - mmdet - INFO - Start running, host: vince@wombat, work_dir: /home/vince/workspace/mmdetection/work_dirs/faster_rcnn_r50_fpn_1x_cityscapes
2021-11-17 17:33:23,600 - mmdet - INFO - Hooks will be executed in the following order:
before_run:
(VERY_HIGH   ) StepLrUpdaterHook
(NORMAL      ) CheckpointHook
(LOW         ) EvalHook
(VERY_LOW    ) TextLoggerHook
...
 --------------------
after_val_epoch:
(VERY_LOW    ) TextLoggerHook
 --------------------
after_run:
(VERY_LOW    ) TextLoggerHook
 --------------------
2021-11-17 17:33:23,600 - mmdet - INFO - workflow: [('train', 1)], max: 8 epochs
2021-11-17 17:33:23,600 - mmdet - INFO - Checkpoints will be saved to /home/vince/workspace/mmdetection/work_dirs/faster_rcnn_r50_fpn_1x_cityscapes by HardDiskBackend.
2021-11-17 17:33:54,886 - mmdet - INFO - Epoch [1][100/23720]   lr: 1.988e-03, eta: 16:25:12, time: 0.312, data_time: 0.025, memory: 4024, loss_rpn_cls: 0.0439, loss_rpn_bbox: 0.0921, loss_cls: 0.7297, acc: 80.1914, loss_bbox: 0.3776, loss: 1.2433
2021-11-17 17:34:23,521 - mmdet - INFO - Epoch [1][200/23720]   lr: 3.986e-03, eta: 15:44:40, time: 0.286, data_time: 0.004, memory: 4073, loss_rpn_cls: 0.0450, loss_rpn_bbox: 0.1027, loss_cls: 0.3770, acc: 86.8848, loss_bbox: 0.2467, loss: 0.7713

Running this is not ok:
./tools/dist_train.sh configs/cityscapes/faster_rcnn_r50_fpn_1x_cityscapes.py 2

Outputs:
...
2021-11-17 17:31:12,229 - mmdet - INFO - initialize Bottleneck with init_cfg {'type': 'Constant', 'val': 0, 'override': {'name': 'norm3'}}
2021-11-17 17:31:12,230 - mmdet - INFO - initialize Bottleneck with init_cfg {'type': 'Constant', 'val': 0, 'override': {'name': 'norm3'}}
2021-11-17 17:31:12,231 - mmdet - INFO - initialize Bottleneck with init_cfg {'type': 'Constant', 'val': 0, 'override': {'name': 'norm3'}}
2021-11-17 17:31:12,232 - mmdet - INFO - initialize Bottleneck with init_cfg {'type': 'Constant', 'val': 0, 'override': {'name': 'norm3'}}
2021-11-17 17:31:12,233 - mmdet - INFO - initialize Bottleneck with init_cfg {'type': 'Constant', 'val': 0, 'override': {'name': 'norm3'}}
2021-11-17 17:31:12,233 - mmdet - INFO - initialize Bottleneck with init_cfg {'type': 'Constant', 'val': 0, 'override': {'name': 'norm3'}}
2021-11-17 17:31:12,234 - mmdet - INFO - initialize Bottleneck with init_cfg {'type': 'Constant', 'val': 0, 'override': {'name': 'norm3'}}
2021-11-17 17:31:12,236 - mmdet - INFO - initialize Bottleneck with init_cfg {'type': 'Constant', 'val': 0, 'override': {'name': 'norm3'}}
2021-11-17 17:31:12,237 - mmdet - INFO - initialize Bottleneck with init_cfg {'type': 'Constant', 'val': 0, 'override': {'name': 'norm3'}}
2021-11-17 17:31:12,238 - mmdet - INFO - initialize Bottleneck with init_cfg {'type': 'Constant', 'val': 0, 'override': {'name': 'norm3'}}
2021-11-17 17:31:12,239 - mmdet - INFO - initialize Bottleneck with init_cfg {'type': 'Constant', 'val': 0, 'override': {'name': 'norm3'}}
2021-11-17 17:31:12,241 - mmdet - INFO - initialize Bottleneck with init_cfg {'type': 'Constant', 'val': 0, 'override': {'name': 'norm3'}}
2021-11-17 17:31:12,242 - mmdet - INFO - initialize Bottleneck with init_cfg {'type': 'Constant', 'val': 0, 'override': {'name': 'norm3'}}
2021-11-17 17:31:12,246 - mmdet - INFO - initialize Bottleneck with init_cfg {'type': 'Constant', 'val': 0, 'override': {'name': 'norm3'}}
2021-11-17 17:31:12,250 - mmdet - INFO - initialize Bottleneck with init_cfg {'type': 'Constant', 'val': 0, 'override': {'name': 'norm3'}}
2021-11-17 17:31:12,253 - mmdet - INFO - initialize Bottleneck with init_cfg {'type': 'Constant', 'val': 0, 'override': {'name': 'norm3'}}
2021-11-17 17:31:12,270 - mmdet - INFO - initialize FPN with init_cfg {'type': 'Xavier', 'layer': 'Conv2d', 'distribution': 'uniform'}
2021-11-17 17:31:12,287 - mmdet - INFO - initialize RPNHead with init_cfg {'type': 'Normal', 'layer': 'Conv2d', 'std': 0.01}
2021-11-17 17:31:12,290 - mmdet - INFO - initialize Shared2FCBBoxHead with init_cfg [{'type': 'Normal', 'std': 0.01, 'override': {'name': 'fc_cls'}}, {'type': 'Normal', 'std': 0.001, 'override': {'name': 'fc_reg
'}}, {'type': 'Xavier', 'override': [{'name': 'shared_fcs'}, {'name': 'cls_fcs'}, {'name': 'reg_fcs'}]}]
loading annotations into memory...
Done (t=0.42s)
creating index...
index created!
loading annotations into memory...
Done (t=0.06s)
creating index...
index created!
2021-11-17 17:31:13,772 - mmdet - INFO - load checkpoint from http path: https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth

and then just waits.
Environment
sys.platform: linux
Python: 3.8.12 (default, Nov 17 2021, 08:17:37) [GCC 9.3.0]
CUDA available: True
GPU 0,1: NVIDIA GeForce GTX 1080 Ti
CUDA_HOME: /usr/local/cuda
NVCC: Build cuda_11.5.r11.5/compiler.30411180_0
GCC: gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
PyTorch: 1.10.0+cu113
PyTorch compiling details: PyTorch built with:
  - GCC 7.3
  - C++ Version: 201402
  - Intel(R) Math Kernel Library Version 2020.0.0 Product Build 20191122 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v2.2.3 (Git Hash 7336ca9f055cf1bfa13efb658fe15dc9b41f0740)
  - OpenMP 201511 (a.k.a. OpenMP 4.5)
  - LAPACK is enabled (usually provided by MKL)
  - NNPACK is enabled
  - CPU capability usage: AVX2
  - CUDA Runtime 11.3
  - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86
  - CuDNN 8.2
  - Magma 2.5.2
  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.3, CUDNN_VERSION=8.2.0, CXX_COMPILER=/opt/rh/devtoolset-7/root/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.10.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON,

TorchVision: 0.11.1+cu113
OpenCV: 4.5.4-dev
MMCV: 1.3.17
MMCV Compiler: GCC 7.3
MMCV CUDA Compiler: 11.3
MMDetection: 2.18.1+c76ab0e

Seems stuck at loading checkpoint? Could you interrupt the program and see where it really stuck?

@RangiLyu after CTRL-C it gives this:
2021-11-21 09:34:37,477 - mmdet - INFO - load checkpoint from http path: https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth
^CWARNING:torch.distributed.elastic.agent.server.api:Received 2 death signal, shutting down workers
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 4006362 closing signal SIGINT
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 4006363 closing signal SIGINT
^CWARNING:torch.distributed.elastic.multiprocessing.api:Sending process 4006362 closing signal SIGTERM
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 4006363 closing signal SIGTERM
^CTraceback (most recent call last):
  File "/home/vince/.pyenv/versions/cm2-p1/lib/python3.8/site-packages/torch/distributed/elastic/agent/server/api.py", line 709, in run
    result = self._invoke_run(role)
  File "/home/vince/.pyenv/versions/cm2-p1/lib/python3.8/site-packages/torch/distributed/elastic/agent/server/api.py", line 843, in _invoke_run
    time.sleep(monitor_interval)
  File "/home/vince/.pyenv/versions/cm2-p1/lib/python3.8/site-packages/torch/distributed/elastic/multiprocessing/api.py", line 60, in _terminate_process_handler
    raise SignalException(f"Process {os.getpid()} got signal: {sigval}", sigval=sigval)
torch.distributed.elastic.multiprocessing.api.SignalException: Process 4006288 got signal: 2

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/vince/.pyenv/versions/cm2-p1/lib/python3.8/site-packages/torch/distributed/elastic/agent/server/api.py", line 716, in run
    self._shutdown(e.sigval)
  File "/home/vince/.pyenv/versions/cm2-p1/lib/python3.8/site-packages/torch/distributed/elastic/agent/server/local_elastic_agent.py", line 190, in _shutdown
    self._pcontext.close(death_sig)
  File "/home/vince/.pyenv/versions/cm2-p1/lib/python3.8/site-packages/torch/distributed/elastic/multiprocessing/api.py", line 330, in close
    self._close(death_sig=death_sig, timeout=timeout)
  File "/home/vince/.pyenv/versions/cm2-p1/lib/python3.8/site-packages/torch/distributed/elastic/multiprocessing/api.py", line 720, in _close
    handler.proc.wait(time_to_wait)
  File "/home/vince/.pyenv/versions/3.8.12/lib/python3.8/subprocess.py", line 1083, in wait
    return self._wait(timeout=timeout)
  File "/home/vince/.pyenv/versions/3.8.12/lib/python3.8/subprocess.py", line 1800, in _wait
    time.sleep(delay)
  File "/home/vince/.pyenv/versions/cm2-p1/lib/python3.8/site-packages/torch/distributed/elastic/multiprocessing/api.py", line 60, in _terminate_process_handler
    raise SignalException(f"Process {os.getpid()} got signal: {sigval}", sigval=sigval)
torch.distributed.elastic.multiprocessing.api.SignalException: Process 4006288 got signal: 2

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/vince/.pyenv/versions/3.8.12/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/vince/.pyenv/versions/3.8.12/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/vince/.pyenv/versions/cm2-p1/lib/python3.8/site-packages/torch/distributed/launch.py", line 193, in <module>
    main()
  File "/home/vince/.pyenv/versions/cm2-p1/lib/python3.8/site-packages/torch/distributed/launch.py", line 189, in main
    launch(args)
  File "/home/vince/.pyenv/versions/cm2-p1/lib/python3.8/site-packages/torch/distributed/launch.py", line 174, in launch
    run(args)
  File "/home/vince/.pyenv/versions/cm2-p1/lib/python3.8/site-packages/torch/distributed/run.py", line 710, in run
    elastic_launch(
  File "/home/vince/.pyenv/versions/cm2-p1/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 131, in __call__
    return launch_agent(self._config, self._entrypoint, list(args))
  File "/home/vince/.pyenv/versions/cm2-p1/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 252, in launch_agent
    result = agent.run()
  File "/home/vince/.pyenv/versions/cm2-p1/lib/python3.8/site-packages/torch/distributed/elastic/metrics/api.py", line 125, in wrapper
    result = f(*args, **kwargs)
  File "/home/vince/.pyenv/versions/cm2-p1/lib/python3.8/site-packages/torch/distributed/elastic/agent/server/api.py", line 721, in run
    self._shutdown()
  File "/home/vince/.pyenv/versions/cm2-p1/lib/python3.8/site-packages/torch/distributed/elastic/agent/server/local_elastic_agent.py", line 190, in _shutdown
    self._pcontext.close(death_sig)
  File "/home/vince/.pyenv/versions/cm2-p1/lib/python3.8/site-packages/torch/distributed/elastic/multiprocessing/api.py", line 330, in close
    self._close(death_sig=death_sig, timeout=timeout)
  File "/home/vince/.pyenv/versions/cm2-p1/lib/python3.8/site-packages/torch/distributed/elastic/multiprocessing/api.py", line 720, in _close
    handler.proc.wait(time_to_wait)
  File "/home/vince/.pyenv/versions/3.8.12/lib/python3.8/subprocess.py", line 1083, in wait
    return self._wait(timeout=timeout)
  File "/home/vince/.pyenv/versions/3.8.12/lib/python3.8/subprocess.py", line 1800, in _wait
    time.sleep(delay)
  File "/home/vince/.pyenv/versions/cm2-p1/lib/python3.8/site-packages/torch/distributed/elastic/multiprocessing/api.py", line 60, in _terminate_process_handler
    raise SignalException(f"Process {os.getpid()} got signal: {sigval}", sigval=sigval)
torch.distributed.elastic.multiprocessing.api.SignalException: Process 4006288 got signal: 2

Does that help?

I did a bit of digging and switching to torchrun (as the deprecated warning says) makes it work. However, train.py needs to be changed slightly. I opened a PR (#6557) with the changes, let me know what you think.

@vakker Do you know why torch.distributed.launch will block？

Actually, the issue is not present because I didn't use the --launcher pytorch, so the distributed flag was not set at all.
If I set that (either for torchrun or torch.distributed.launch) then it hangs:

in seed = init_random_seed(args.seed) on the dist.broadcast(random_num, src=0) line
in train_detector on the model = MMDistributedDataParallel line

So, there might be multiple issues related to how the distributed training is initialized maybe.

@hhaAndroid do you have any input on moving this further?

@RangiLyu @hhaAndroid could you confirm that the issue can be reproduced on your end as well (just run the 2 commands in the issue description)? I'm happy to investigate this further, but it's unclear at this point if this is a broader issue or there's just something wrong with my setup.

@RangiLyu @hhaAndroid do you have any update on this issue?

We support three kinds of distributed launcher in MMCV
https://github.com/open-mmlab/mmcv/blob/86ed509a8bc783cdd0617efeba257375d5aa6658/mmcv/runner/dist_utils.py#L14
If you did not use the --launcher pytorch, the init_dist will not be triggered and some env variables will not be set. Would you like to check if this caused the problem?

Thanks for the response.
That's how I run it, please see my previous post:

If I set that (either for torchrun or torch.distributed.launch) then it hangs:
1. in `seed = init_random_seed(args.seed)` on the `dist.broadcast(random_num, src=0)` line

2. in `train_detector` on the `model = MMDistributedDataParallel` line

I.e. the issue is present when the pytorch launcher is used.
Could you please confirm at least the you're able to reproduce the issue?

@RangiLyu I just tried again today from the latest master and the issue still persists.
Just to reiterate the reproduction steps:
This works:
python tools/train.py configs/cityscapes/faster_rcnn_r50_fpn_1x_cityscapes.py

This gets stuck:
bash ./tools/dist_train.sh configs/cityscapes/faster_rcnn_r50_fpn_1x_cityscapes.py 2

Env info:
sys.platform: linux
Python: 3.8.12 (default, Nov 17 2021, 08:17:37) [GCC 9.3.0]
CUDA available: True
GPU 0,1: NVIDIA GeForce GTX 1080 Ti
CUDA_HOME: /usr/local/cuda
NVCC: Build cuda_11.5.r11.5/compiler.30411180_0
GCC: gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
PyTorch: 1.10.0+cu113
PyTorch compiling details: PyTorch built with:
  - GCC 7.3
  - C++ Version: 201402
  - Intel(R) Math Kernel Library Version 2020.0.0 Product Build 20191122 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v2.2.3 (Git Hash 7336ca9f055cf1bfa13efb658fe15dc9b41f0740)
  - OpenMP 201511 (a.k.a. OpenMP 4.5)
  - LAPACK is enabled (usually provided by MKL)
  - NNPACK is enabled
  - CPU capability usage: AVX2
  - CUDA Runtime 11.3
  - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86
  - CuDNN 8.2
  - Magma 2.5.2
  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.3, CUDNN_VERSION=8.2.0, CXX_COMPILER=/opt/rh/devtoolset-7/root/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.10.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON,

TorchVision: 0.11.1+cu113
OpenCV: 4.5.4-dev
MMCV: 1.3.17
MMCV Compiler: GCC 7.3
MMCV CUDA Compiler: 11.3
MMDetection: 2.18.1+a7a16af

Please let me know if you need any more information. Thanks!

Hi @vakker ,
It might be caused by limited memory. You may try workers_per_gpu=0 or 1.

@ZwwWayne Thanks for the suggestion, but that's not the issue. Just to be sure, I tried workers_per_gpu=0 (and also the default 2), and the issue persists, while there's more than 50GB of ram available on the machine.
If you run the reproduction steps above, do you experience the same or not? That way it would be easy to narrow down the possibilities. Thanks!

I tried but could not reproduce this problem.

@RangiLyu thanks, so just to be sure, you run bash ./tools/dist_train.sh configs/cityscapes/faster_rcnn_r50_fpn_1x_cityscapes.py 2 and it didn't get stuck, right?
Then could you provide your environment info, so I can compare and see what causes this? Thanks

@RangiLyu thanks, so just to be sure, you run bash ./tools/dist_train.sh configs/cityscapes/faster_rcnn_r50_fpn_1x_cityscapes.py 2 and it didn't get stuck, right? Then could you provide your environment info, so I can compare and see what causes this? Thanks

Yes, I run bash ./tools/dist_train.sh configs/cityscapes/faster_rcnn_r50_fpn_1x_cityscapes.py 2 the same as you provided. But the difference is that I used srun to start the dist_train.sh because our cluster uses slurm, but I still used the pytorch launcher.
And here is my environment:
sys.platform: linux
Python: 3.6.13 | packaged by conda-forge | (default, Sep 23 2021, 07:56:31) [GCC 9.4.0]
CUDA available: True
GPU 0,1: TITAN Xp
CUDA_HOME: /mnt/lustre/share/polaris/dep/cuda-9.0-cudnn7.6.5
NVCC: Cuda compilation tools, release 9.0, V9.0.176
GCC: gcc (GCC) 5.4.0
PyTorch: 1.8.1+cuda90.cudnn7.6.5
PyTorch compiling details: PyTorch built with:
  - GCC 5.4
  - C++ Version: 201402
  - Intel(R) Math Kernel Library Version 2020.0.0 Product Build 20191122 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v1.7.0 (Git Hash 7aed236906b1f7a05c0917e5257a1af05e9ff683)
  - OpenMP 201307 (a.k.a. OpenMP 4.0)
  - NNPACK is enabled
  - CPU capability usage: AVX2
  - CUDA Runtime 9.0
  - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_52,code=sm_52;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70
  - CuDNN 7.6.5
  - Magma 2.5.0
  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=9.0, CUDNN_VERSION=7.6.5, CXX_COMPILER=/usr/local/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=0 -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp
-DNDEBUG -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare
 -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-psabi -Wno-error=pedantic -Wno-error=redund
ant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512
=1, TORCH_VERSION=1.8.1, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON,

TorchVision: 0.8.0a0+2f40a48
OpenCV: 4.5.4
MMCV: 1.4.0
MMCV Compiler: GCC 5.4
MMCV CUDA Compiler: 9.0
MMDetection: 2.20.0+ff9bc39

@RangiLyu thanks for the details. I'll try to match your setup in terms of library versions, as you're using different Python, Pytorch, Torchvision, Cuda, CuDNN versions, and see if I can reproduce the issue.

@RangiLyu @hhaAndroid  I was able to reproduce this issue with Docker, I tried a broad range of settings:

Pytorch
Cuda
CuDNN
Works

1.6.0
10.1
7
:heavy_check_mark:

1.7.0
11.0
8
:x:

1.8.0
11.1
8
:x:

1.9.0
10.2
7
:x:

1.9.0
11.1
8
:x:

1.10.0
11.3
8
:x:

Dockerfile
Base on the official Dockerfile:
ARG PYTORCH="1.6.0"
ARG CUDA="10.1"
ARG CUDNN="7"

FROM pytorch/pytorch:${PYTORCH}-cuda${CUDA}-cudnn${CUDNN}-devel

ARG MMCV="1.4.6"

ARG PYTORCH
ARG CUDA

ENV TORCH_CUDA_ARCH_LIST="6.0 6.1 7.0+PTX"
ENV TORCH_NVCC_FLAGS="-Xfatbin -compress-all"
ENV CMAKE_PREFIX_PATH="$(dirname $(which conda))/../"

RUN apt-get update && apt-get install -y ffmpeg libsm6 libxext6 git ninja-build libglib2.0-0 libsm6 libxrender-dev libxext6 \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists/*

# Install MMCV
RUN pip install --no-cache-dir --upgrade pip wheel setuptools
RUN ["/bin/bash", "-c", "pip install mmcv-full==${MMCV} -f https://download.openmmlab.com/mmcv/dist/cu${CUDA//./}/torch${PYTORCH}/index.html"]

# Install MMDetection
RUN conda clean --all
RUN git clone https://github.com/open-mmlab/mmdetection.git /mmdetection
WORKDIR /mmdetection
ENV FORCE_CUDA="1"
RUN pip install --no-cache-dir -r requirements/build.txt
RUN pip install --no-cache-dir -e .

Build:
docker build \
    --build-arg PYTORCH=<pytorch> \
    --build-arg CUDA=<cuda> \
    --build-arg CUDNN=<cudnn> \
    -t mmdet:<pytorch>-<cuda>-cudnn> .

After some digging, I found this discussion, so I tried using the gloo backend instead of nccl, i.e. in configs/_base_/default_runtime.py I changed dist_params = dict(backend='nccl') to dist_params = dict(backend='gloo').
That does make it work on all the test scenarios:

Pytorch
Cuda
CuDNN
Works

1.6.0
10.1
7
:heavy_check_mark:

1.7.0
11.0
8
:heavy_check_mark:

1.8.0
11.1
8
:heavy_check_mark:

1.9.0
10.2
7
:heavy_check_mark:

1.9.0
11.1
8
:heavy_check_mark:

1.10.0
11.3
8
:heavy_check_mark:

Please note, that the Cuda version that you used (9.0) to reproduce this issue is really old, it was released in 2017, I couldn't even find a Pytorch Docker image for that. It might make sense to test the code using versions that are more commonly used recently (10.2, 11.3). I'm not sure Pytorch even supports anything less than 10.2, see here.
Please, verify if this is an actual bug that you can reproduce.

Hi @vakker @RangiLyu
I experienced the same problem, what should be the recommended fix here?
Using torchrun or down-grade pytorch version?

@RangiLyu @hhaAndroid I was able to reproduce this issue with Docker, I tried a broad range of settings:
Pytorch	Cuda	CuDNN	Works
1.6.0	10.1	7	✔
1.7.0	11.0	8	❌
1.8.0	11.1	8	❌
1.9.0	10.2	7	❌
1.9.0	11.1	8	❌
1.10.0	11.3	8	❌
Dockerfile
Base on the official Dockerfile:
ARG PYTORCH="1.6.0"
ARG CUDA="10.1"
ARG CUDNN="7"

FROM pytorch/pytorch:${PYTORCH}-cuda${CUDA}-cudnn${CUDNN}-devel

ARG MMCV="1.4.6"

ARG PYTORCH
ARG CUDA

ENV TORCH_CUDA_ARCH_LIST="6.0 6.1 7.0+PTX"
ENV TORCH_NVCC_FLAGS="-Xfatbin -compress-all"
ENV CMAKE_PREFIX_PATH="$(dirname $(which conda))/../"

RUN apt-get update && apt-get install -y ffmpeg libsm6 libxext6 git ninja-build libglib2.0-0 libsm6 libxrender-dev libxext6 \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists/*

# Install MMCV
RUN pip install --no-cache-dir --upgrade pip wheel setuptools
RUN ["/bin/bash", "-c", "pip install mmcv-full==${MMCV} -f https://download.openmmlab.com/mmcv/dist/cu${CUDA//./}/torch${PYTORCH}/index.html"]

# Install MMDetection
RUN conda clean --all
RUN git clone https://github.com/open-mmlab/mmdetection.git /mmdetection
WORKDIR /mmdetection
ENV FORCE_CUDA="1"
RUN pip install --no-cache-dir -r requirements/build.txt
RUN pip install --no-cache-dir -e .

Build:
docker build \
    --build-arg PYTORCH=<pytorch> \
    --build-arg CUDA=<cuda> \
    --build-arg CUDNN=<cudnn> \
    -t mmdet:<pytorch>-<cuda>-cudnn> .

After some digging, I found this discussion, so I tried using the gloo backend instead of nccl, i.e. in configs/_base_/default_runtime.py I changed dist_params = dict(backend='nccl') to dist_params = dict(backend='gloo').
That does make it work on all the test scenarios:
Pytorch	Cuda	CuDNN	Works
1.6.0	10.1	7	✔
1.7.0	11.0	8	✔
1.8.0	11.1	8	✔
1.9.0	10.2	7	✔
1.9.0	11.1	8	✔
1.10.0	11.3	8	✔
Please note, that the Cuda version that you used (9.0) to reproduce this issue is really old, it was released in 2017, I couldn't even find a Pytorch Docker image for that. It might make sense to test the code using versions that are more commonly used recently (10.2, 11.3). I'm not sure Pytorch even supports anything less than 10.2, see here.
Please, verify if this is an actual bug that you can reproduce.

I experienced a similar problem. It works for me. The outputs are as follows.
2022-04-01 12:15:36,770 - mmseg - INFO - Set random seed to 0, deterministic: False
/code/mmsegmentation/mmseg/models/backbones/resnet.py:431: UserWarning: DeprecationWarning: pretrained is a deprecated, please use "init_cfg" instead
  warnings.warn('DeprecationWarning: pretrained is a deprecated, '
/code/mmsegmentation/mmseg/models/backbones/resnet.py:431: UserWarning: DeprecationWarning: pretrained is a deprecated, please use "init_cfg" instead
  warnings.warn('DeprecationWarning: pretrained is a deprecated, '
/code/mmsegmentation/mmseg/models/backbones/resnet.py:431: UserWarning: DeprecationWarning: pretrained is a deprecated, please use "init_cfg" instead
  warnings.warn('DeprecationWarning: pretrained is a deprecated, '
/code/mmsegmentation/mmseg/models/backbones/resnet.py:431: UserWarning: DeprecationWarning: pretrained is a deprecated, please use "init_cfg" instead
  warnings.warn('DeprecationWarning: pretrained is a deprecated, '
/code/mmsegmentation/mmseg/models/backbones/resnet.py:431: UserWarning: DeprecationWarning: pretrained is a deprecated, please use "init_cfg" instead
  warnings.warn('DeprecationWarning: pretrained is a deprecated, '
/code/mmsegmentation/mmseg/models/backbones/resnet.py:431: UserWarning: DeprecationWarning: pretrained is a deprecated, please use "init_cfg" instead
  warnings.warn('DeprecationWarning: pretrained is a deprecated, '
/code/mmsegmentation/mmseg/models/backbones/resnet.py:431: UserWarning: DeprecationWarning: pretrained is a deprecated, please use "init_cfg" instead
  warnings.warn('DeprecationWarning: pretrained is a deprecated, '
2022-04-01 12:15:37,557 - mmseg - INFO - initialize ResNetV1c with init_cfg {'type': 'Pretrained', 'checkpoint': 'open-mmlab://resnet101_v1c'}
2022-04-01 12:15:37,558 - mmcv - INFO - load model from: open-mmlab://resnet101_v1c
2022-04-01 12:15:37,558 - mmcv - INFO - load checkpoint from openmmlab path: open-mmlab://resnet101_v1c
[E ProcessGroupNCCL.cpp:587] [Rank 7] Watchdog caught collective operation timeout: WorkNCCL(OpType=ALLREDUCE, Timeout(ms)=1800000) ran for 1800337 milliseconds before timing out.
[E ProcessGroupNCCL.cpp:341] Some NCCL operations have failed or timed out. Due to the asynchronous nature of CUDA kernels, subsequent GPU operations might run on corrupted/incomplete data. To avoid this inconsistency, we are taking the entire process down.
terminate called after throwing an instance of 'std::runtime_error'
  what():  [Rank 7] Watchdog caught collective operation timeout: WorkNCCL(OpType=ALLREDUCE, Timeout(ms)=1800000) ran for 1800337 milliseconds before timing out.
[E ProcessGroupNCCL.cpp:587] [Rank 3] Watchdog caught collective operation timeout: WorkNCCL(OpType=ALLREDUCE, Timeout(ms)=1800000) ran for 1800795 milliseconds before timing out.
[E ProcessGroupNCCL.cpp:587] [Rank 5] Watchdog caught collective operation timeout: WorkNCCL(OpType=ALLREDUCE, Timeout(ms)=1800000) ran for 1800809 milliseconds before timing out.
[E ProcessGroupNCCL.cpp:587] [Rank 1] Watchdog caught collective operation timeout: WorkNCCL(OpType=ALLREDUCE, Timeout(ms)=1800000) ran for 1800832 milliseconds before timing out.
[E ProcessGroupNCCL.cpp:341] Some NCCL operations have failed or timed out. Due to the asynchronous nature of CUDA kernels, subsequent GPU operations might run on corrupted/incomplete data. To avoid this inconsistency, we are taking the entire process down.
terminate called after throwing an instance of 'std::runtime_error'
  what():  [Rank 3] Watchdog caught collective operation timeout: WorkNCCL(OpType=ALLREDUCE, Timeout(ms)=1800000) ran for 1800795 milliseconds before timing out.
[E ProcessGroupNCCL.cpp:341] Some NCCL operations have failed or timed out. Due to the asynchronous nature of CUDA kernels, subsequent GPU operations might run on corrupted/incomplete data. To avoid this inconsistency, we are taking the entire process down.
terminate called after throwing an instance of 'std::runtime_error'
  what():  [Rank 1] Watchdog caught collective operation timeout: WorkNCCL(OpType=ALLREDUCE, Timeout(ms)=1800000) ran for 1800832 milliseconds before timing out.
[E ProcessGroupNCCL.cpp:587] [Rank 2] Watchdog caught collective operation timeout: WorkNCCL(OpType=ALLREDUCE, Timeout(ms)=1800000) ran for 1801043 milliseconds before timing out.
[E ProcessGroupNCCL.cpp:341] Some NCCL operations have failed or timed out. Due to the asynchronous nature of CUDA kernels, subsequent GPU operations might run on corrupted/incomplete data. To avoid this inconsistency, we are taking the entire process down.
terminate called after throwing an instance of 'std::runtime_error'
  what():  [Rank 5] Watchdog caught collective operation timeout: WorkNCCL(OpType=ALLREDUCE, Timeout(ms)=1800000) ran for 1800809 milliseconds before timing out.
[E ProcessGroupNCCL.cpp:341] Some NCCL operations have failed or timed out. Due to the asynchronous nature of CUDA kernels, subsequent GPU operations might run on corrupted/incomplete data. To avoid this inconsistency, we are taking the entire process down.
terminate called after throwing an instance of 'std::runtime_error'
  what():  [Rank 2] Watchdog caught collective operation timeout: WorkNCCL(OpType=ALLREDUCE, Timeout(ms)=1800000) ran for 1801043 milliseconds before timing out.
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 2000 closing signal SIGTERM
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 2001 closing signal SIGTERM
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 2002 closing signal SIGTERM
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 2004 closing signal SIGTERM
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 2005 closing signal SIGTERM
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 2006 closing signal SIGTERM
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: -6) local_rank: 3 (pid: 2003) of binary: /opt/conda/bin/python
Traceback (most recent call last):
  File "/opt/conda/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/opt/conda/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/opt/conda/lib/python3.7/site-packages/torch/distributed/launch.py", line 193, in <module>
    main()
  File "/opt/conda/lib/python3.7/site-packages/torch/distributed/launch.py", line 189, in main
    launch(args)
  File "/opt/conda/lib/python3.7/site-packages/torch/distributed/launch.py", line 174, in launch
    run(args)
  File "/opt/conda/lib/python3.7/site-packages/torch/distributed/run.py", line 713, in run
    )(*cmd_args)
  File "/opt/conda/lib/python3.7/site-packages/torch/distributed/launcher/api.py", line 131, in __call__
    return launch_agent(self._config, self._entrypoint, list(args))
  File "/opt/conda/lib/python3.7/site-packages/torch/distributed/launcher/api.py", line 261, in launch_agent
    failures=result.failures,
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:
=====================================================
tools/train.py FAILED
-----------------------------------------------------
Failures:
[1]:
  time      : 2022-04-01_12:45:45
  host      : d7537e2e1710
  rank      : 7 (local_rank: 7)
  exitcode  : -6 (pid: 2007)
  error_file: <N/A>
  traceback : Signal 6 (SIGABRT) received by PID 2007
-----------------------------------------------------
Root Cause (first observed failure):
[0]:
  time      : 2022-04-01_12:45:45
  host      : d7537e2e1710
  rank      : 3 (local_rank: 3)
  exitcode  : -6 (pid: 2003)
  error_file: <N/A>
  traceback : Signal 6 (SIGABRT) received by PID 2003
=====================================================

It works for me. Thanks!

My solution is below:
Add following commands in ~/.bashrc
export NCCL_P2P_DISABLE="1"
export NCCL_IB_DISABLE="1"
Then, source ~/.bashrc.   It works for me. No need to modify nccl -> gool.

Thank you but none of the above works for me : (
After days of searching and trying, my workaround is to set the system run level to 3:
sudo init 3

The GPUs are happy for now and I will report if anything goes wrong again.
Ref:
https://nvidia.custhelp.com/app/answers/detail/a_id/3029/~/using-cuda-and-x

My solution is below: Add following commands in ~/.bashrc
export NCCL_P2P_DISABLE="1" export NCCL_IB_DISABLE="1"
Then, source ~/.bashrc. It works for me. No need to modify nccl -> gool.

This will slow down the training speed?

I have been completely unable to get distributed training working with the Docker container even after following all of the suggestions in this thread. Has anybody else had better luck resolving this issue? I'm at my wits end here trying to get this to work.

I was trying to run distribute training on a VertexAI Workbench notebook. After 10000000 tries, I finally got it working in this way:

Get a machine with Pytorch 2.2 installed (CUDA 12.1), and how many GPUs you want (4 in my case):
DO NOT CREATE A CONDA ENV and install this in the base env:

pip install -U openmim
mim install mmengine
mim install "mmcv>=2.0.0"

git clone https://github.com/open-mmlab/mmdetection.git
cd mmdetection
pip install -v -e .

Change mmcv_maximum_version = '2.2.0' to mmcv_maximum_version = '2.2.1' in mmdetection/mmdet/__init__.py for avoiding any versions mismatch.
Make sure that the LD_LIBRARY_PATH=/usr/local/cuda/lib64:/usr/local/nccl2/lib:/usr/local/cuda/extras/CUPTI/lib64:/usr/lib/x86_64-linux-gnu/
and NOT:
LD_LIBRARY_PATH=/usr/local/cuda/lib64:/usr/local/nccl2/lib:/usr/local/cuda/extras/CUPTI/lib64:/usr/lib/x86_64-linux-gnu/:/usr/local/cuda/lib64:/usr/local/nccl2/lib:/usr/local/cuda/extras/CUPTI/lib64:/usr/lib/x86_64-linux-gnu/ (this happened when I was getting a machine with CUDA only, no PyTorch)
export NCCL_P2P_DISABLE='1' and  export NCCL_IB_DISABLE='1' in the base env
run bash mmdetection/tools/dist_train.sh <CONFIG_PATH> 4

Thank you all the others for your helpful guidance!


Backport of the Slider from the FiveM repo

This is a work in progress for the Slider item added on b5977ffe4a0c9de4daf42a5fd3f32e3833c1adcd at the FiveM repo.

@Guad can you test this and let me know what changes it needs.

Requested changes have been completed.

Looks good. I'll test it later tonight ingame


GPL 2.0 licensed dependency

Addresser uses cfonts package that are licensed under GPL 2.0 that makes it incompatible with addressr's Apache-2.0 license.

Thanks for the heads up. I’ll check this and fix ASAP

@dominikwilkowski, we inadvertently included you cfonts library this Apache-2.0 licensed program. We have removed, but I wanted to reach to let you know and to find out if there is anything else you wanted us to do.

Hey @tompahoward
Appreciate the diligence and care here. No need to do anything else.


Problems with gestures in version 1.2.7

I had a problem with the last update (1.2.7), the gestures for the next screen stopped working and I had to go back to version 1.2.6. The problem happens when I try to drag to the next page, instead of making a smooth transition, the page I pulled back to the starting position and then it goes to the landing page.
The page is this image, when I drag from right to left instead of making the smooth transition as I did in the previous section, the page returns to the starting position and then the second page appears without any animation.

@luizalexandrew Are you on Polymer Slack? If not join us, it will be easier to find a solution! The last issue was probably introduced with the Carrousel implementation that has been PRed but not tested on my side...


Remove cruft code, dependencies

Removes unused code, commands from dodola.
More specifically, this removes dodola commands:

dodola biascorrect
dodola downscale
dodola buildweight

Internally, this removes:

dodola.core.apply_bias_correction
dodola.core.apply_downscaling
dodola.core.build_xesmf_weights_file
dodola.services.bias_correct
dodola.services.downscale
dodola.services.build_weights

Relevant tests and external package dependencies (scikit-downscale) are also removed.
close #20
close #82

For reviews, this is largely a heads-up. Please let me know if I'm removing something critical.


Automatic rush update --full

This is an automatic PR generated weekly with changes from running the command rush update --full

Its the release week, lets hold off on merging this until end of the week


[RDK-35868] - Add unit tests to BundleGen

As part of this pull request, added the workflow changes for L1 unit testing

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Looks OK, but seems GitHub actions aren't enabled in this repo so workflow isn't running. Will try to get actions enabled in this repo

@KaurSup - workflow ran after merge, but failed with error

Traceback (most recent call last):
  File "/home/runner/work/BundleGen/BundleGen/Bundlegen/unit_tests/L1_testing/test_files/test_Bundleprocessor_ut.py", line 26, in <module>
    from bundlegen.core.bundle_processor import BundleProcessor
  File "/home/runner/work/BundleGen/BundleGen/Bundlegen/bundlegen/core/bundle_processor.py", line 22, in <module>
    from jsonschema import validate
ModuleNotFoundError: No module named 'jsonschema'
Error: Process completed with exit code 1.

Please submit a new PR to fix


Getting some issue in Loom

After running 'ng serve'. Following error is displayed.

Thanks for looking into this, I assume this issue is closed now.


[Bug] Fix some bugs

Proposed changes

Fix a potential BE coredump of sending batch when loading data. (Fix #6656)
Fix a potential BE coredump when doing schema change. (Fix #6657)
Optimize the metric of base_compaction_request_failed.
Add Order column in show tablet result. (Fix #6658)
Fix bug that tablet repair slot not being released. (Fix #6659)
Fix bug that REPLICA_MISSING error can not be handled. (Fix #6660)
Modify column name of SHOW PROC "/cluster_balance/cluster_load_stat"
Optimize the result of SHOW PROC "/statistic" to show COLOCATE_MISMATCH tablets (Fix #6663)
Fix bug that show load where state='pending' can not be executed. (Fix #6664)

Types of changes
What types of changes does your code introduce to Doris?
Put an x in the boxes that apply

[x] Bugfix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Documentation Update (if none of the other choices apply)
[ ] Code refactor (Modify the code structure, format the code, etc...)
[x] Optimization. Including functional usability improvements and performance improvements.
[ ] Dependency. Such as changes related to third-party components.
[ ] Other.

Checklist
Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code.

[ ] I have created an issue on (Fix #ISSUE) and described the bug/feature there in detail
[ ] Compiling and unit tests pass locally with my changes
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] If these changes need document changes, I have updated the document
[ ] Any dependent changes have been merged

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

cc @caiconghui #6656

LGTM


Check simd type for scann index

We need to check simd type for all operations(like deser/search...) of scann index. Before this pr, we miss the check when we deserialize index.
/kind improvement

/lgtm


Change delimiter ';' on ' ' in QT file dialog filters

With ';' wildcards of extensions are not recognized on some systems.
For example with filter equals (.png;.exr) not png not exr files are shown.
On the other hand, ' ' delimiter works well on any system and also advised in documentation.

Thanks!


Add MultiMap branch. Add mutable.MultiMap and immutable.MultiMap leaves.

This is another idea to provide a MultiMap collection type. Here, MultiMap[K, V] is a new “branch” of the hierarchy, it inherits from Iterable[(K, Set[V])]. The PR provides two implementations, a mutable one and an immutable one.
Cons:

the fact that MultiMap has its own branch makes it incomparable with Map collections.
if we want to make it work with an underlying SortedMap we have to create a new branch for that in the hierarchy.

Pros:

map, flatMap and concat return a MultiMap[K, V] instead of a Map[K, Set[V]
maybe cleaner hierarchy?

Also, when we manipulate a MultiMap[K, V], sometimes we want to see the entries as (K, Set[V]) pairs, but sometimes we want to see them as (K, V) pairs… I don’t see a reason to not support both, actually. So, what I’ve done is that several methods have two variants: map shows the entries as (K, Set[V]) pairs, and mapByValue shows the entries as (K, V) pairs ; iterator() returns an Iterator[(K, Set[V])] and iteratorByValue() returns an Iterator[(K, V)]. Basically I used the suffix “byValue” when an operation sees its entries as (K, V) pairs.

Superseded by #314


Update wallet live demo

📝 Changed routes:

/dashboard/wallet

(view full size)

Commit 7c3071272502ae5dd16b66872f4ae3e71633b8d0 (https://thirdweb-ntdidai1a.thirdweb-preview.com).


Update OpenTabletDriver package to 0.5.3

Short Changelog
Full changelog

Fixed Wacom touch reports being handled as auxiliary buttons (now reports touch data through ITouchReport)

Touch implementation is subject to change once we find more touch capable tablets and parse their data structures.

Added 26 new tablet configurations (Wacom, Huion, XP-Pen)
Fixed and/or renamed 33 existing tablet configurations (Gaomon, Huion, Wacom, XP-Pen)
Improved common exception messages to be more descriptive and link to OpenTabletDriver FAQ
Workaround bad/partial reports resulting in device detection failing

Just going to mention that I still get a visible log on startup
[runtime] 2021-04-13 03:15:35 [debug]: Device: Set tablet feature: 02-02
[runtime] 2021-04-13 03:15:35 [verbose]: Detect: Found tablet 'Wacom CTH-470'
[runtime] 2021-04-13 03:15:35 [important]: Detect: Failed to find auxiliary device, express keys may be unavailable.

Not sure if there are actually important enough to appear to the end user, and I can already see the 470 wasn't updated in terms of support, but thought it may be worth mentioning off hand.


Rename all occurrences of the library name

We got a notice that the library name is being protected by trademark and have until the beginning of December 2022 to change it to something else.
Upon internal discussions we concluded that strato will become venin (and hedera-venin-js respectively).
This will also affect the NPM package which we will have to change it from @buidlerlabs/hedera-strato-js to @buidlerlabs/hedera-venin-js with version 0.8.1 being the first one in the repo.
Doc references will also have to be updated as well as the doc CNAME from strato.buidlerlabs.com to venin.buidlerlabs.com. Of course, we will maintain a redirect between the old address and the new one. I don't know how things will behave on GitHub, though, if we update the repo name.
I guess we will find out soon.

To speed up the release, we will keep all non-public references (such as class names and functions) the same.


Edit and re-arrange order of sample value labels

When setting up a new project, can we add the ability to edit and rearrange the order of the sample value labels?
Thanks!
Karis

The user interface around this has changed drastically since this post.  If this is still a request.  Please resubmit referencing the new design.


/aviser/export/pages -JPEGS of entire pages

It would be great for Transkribus projects to have an endpoint, which delivers a .zip file with images of entire pages of newspaper in the jpeg format.
So similar to /aviser/export/images, but with entire pages instead.

@maxodsbjerg & @johanheinsen Just letting you know that the first version is available at labs now: http://labs.statsbiblioteket.dk/labsapi/api//api-docs?url=/labsapi/api/openapi.json#/labsapi/exportImages


chore: import repository https://github.com/iMckify/react-spring.git

this commit will trigger a pipeline to generate the CI/CD configuration which will create a second commit on this Pull Request before it auto merges

[jx-info] Hi, we've detected that the pipelines in this repository are using a syntax that will soon be deprecated.
We'll continue to update you through PRs as we progress. Please check #8589 for further information.


Add new source & category

Description
- [ ] Add [whatismybrowser.com](https://developers.whatismybrowser.com/useragents/explore/) as source
     - [ ]  Scrape User Agents based on their categories
     - [ ] Add relevant categories
- [ ]  Add `-latest` flag to fetch latest User Agents.
- [ ]  Add common/popular flags ( -chrome etc )
- [ ] Add using as library example / build test

UserAgent data on whatismybrowser.com is most recent & structured implementing this will prevent bad data

chrome-legacy
chrome-latest
chrome-mobile
firefox-legacy
...

Closing as not planned - Unfortunately I think this is not allowed (Ref: https://developers.whatismybrowser.com/useragents/database/legal/)


Automated cherry pick of #2672: Convert NodeName obtained from env var to lowercase on

Cherry pick of #2672 on release-1.3.
#2672: Convert NodeName obtained from env var to lowercase on
For details on the cherry pick process, see the cherry pick requests page.

Codecov Report

Merging #2690 (71204ef) into release-1.3 (8c2abcd) will increase coverage by 0.00%.
The diff coverage is 0.00%.

@@             Coverage Diff              @@
##           release-1.3    #2690   +/-   ##
============================================
  Coverage        41.05%   41.06%
============================================
  Files              158      158
  Lines            19243    19245    +2
============================================
+ Hits              7900     7902    +2
+ Misses           10596    10591    -5
- Partials           747      752    +5

Flag
Coverage Δ

unit-tests
41.06% <0.00%> (+<0.01%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

pkg/util/env/env.go
26.82% <0.00%> (-1.38%)
:arrow_down:

pkg/apiserver/handlers/endpoint/handler.go
58.82% <0.00%> (-11.77%)
:arrow_down:

...gent/controller/noderoute/node_route_controller.go
45.22% <0.00%> (+1.12%)
:arrow_up:

/test-all

/skip-all


Keepass import tool

Here is a importer-tool for keepass.
I tested it with a few different keepass-databases and it seems to be working all right.

I added the changes you requested :)

Thanks!


SwitchStore in Preview results in multiple screens

Description
Hi!
So I was working with the SwitchStore and wanted to showcase various cases using the XCode's Live Preview feature
There are no crash or important bug per se, but in the Preview's Canvas, I see N variations for each use case I have in the preview. With N being the number of cases in my enum State
I tried with a Vanilla SwiftUI switch statement in my View, but the behaviour was, as expected, to see only three previews in the canvas
I believe that this behaviour is a direct result of CaseLet view using the IfLetStore under the hood, which results in multiple if-else statements in place of a switch statement
Checklist

[X] I have determined whether this bug is also reproducible in a vanilla SwiftUI project.
[ ] If possible, I've reproduced the issue using the main branch of this package.
[X] This issue hasn't been addressed in an existing GitHub issue or discussion.

Expected behavior
Seeing only 3 preview contents (3 because I have explicitly added 3 previews to my preview provider)

Actual behavior
I see N preview contents (N being 3 times the number CaseLet/Default views I have in my SwitchStore)

Steps to reproduce
Using the following sample, just check the XCode's live preview:
import ComposableArchitecture
import SwiftUI

struct TestReducer: ReducerProtocol {
    enum State: Equatable {
        case content(String)
        case other(String)
        case nothing
    }

    enum Action: Equatable {}

    var body: some ReducerProtocolOf<Self> {
        EmptyReducer()
    }
}

struct TestView: View {
    let store: StoreOf<TestReducer>

    var body: some View {
        SwitchStore(store) { _ in
            CaseLet(
                state: /TestReducer.State.content,
                action: { (action: TestReducer.Action) in action }
            ) { store in
                Text("Content")
            }

            CaseLet(
                state: /TestReducer.State.other,
                action: { (action: TestReducer.Action) in action }
            ) { store in
                Text("Other")
            }

            CaseLet(
                state: /TestReducer.State.nothing,
                action: { (action: TestReducer.Action) in action }
            ) { store in
                Text("Nothing")
            }
        }
    }
}

struct TestView_Previews: PreviewProvider {
    static var previews: some View {
        TestView(store: .init(
            initialState: .content("Hello, world!"),
            reducer: TestReducer()
        ))
        .previewDisplayName("Content")

        TestView(store: .init(
            initialState: .other("Hello, other world!"),
            reducer: TestReducer()
        ))
        .previewDisplayName("Other")

        TestView(store: .init(
            initialState: .nothing,
            reducer: TestReducer()
        ))
        .previewDisplayName("Loading")
    }
}

The Composable Architecture version information
0.54.0
Destination operating system
iOS 15 and later
Xcode version information
Version 14.3.1 (14E300c)
Swift Compiler version information
swift-driver version: 1.75.2 Apple Swift version 5.8.1 (swiftlang-<IP_ADDRESS>.5 clang-14<IP_ADDRESS>.100)
Target: arm64-apple-macosx13.0

SwitchStore expects you to explicitly switch over the initial state that is passed along:
-SwitchStore(store) { _ in
+SwitchStore(store) {
+  switch $0 {
+  case .content:
     CaseLet(
       state: /TestReducer.State.content,
       action: { (action: TestReducer.Action) in action }
     ) { store in
       Text("Content")
     }

+  case .other:
     CaseLet(
       state: /TestReducer.State.other,
       action: { (action: TestReducer.Action) in action }
     ) { store in
       Text("Other")
     }

+  case .nothing:
     CaseLet(
       state: /TestReducer.State.nothing,
       action: { (action: TestReducer.Action) in action }
     ) { store in
       Text("Nothing")
     }
+  }
 }

I believe this will get previews to work more as you expect.
Also note that CaseLet does not require the state: named argument. You can simply do:
CaseLet(/TestReducer.State.content, /* ... */)

Because this isn't a bug in the library I'm going to convert this to a discussion.


Buggy after 2024.5.3

Locomotives stutter badly after today's updates, also when you select a loco now your player gets stuck in the seat or ejected into the tender. movement in locos is very odd now. Disabling this mod and restarting the game fixed the issues.

fixed


Design a new creative footer

It will be better if you refer to some footer design of the Indian government website.

Can i work on this. Please assign it to me.

I'll assign you this issue when the contribution period starts from 10th June, then you can start working on this issue.

Sir plz can i work on this issue.

@Naman-Joshi-14 okk,
but keep in mind since many contributors are willing to resolve this issue only the best design/solution will be accepted.

@QAZIMAAZARSHAD  Thank you sir

Hii,  I would like to work on this issue as LGM SOC'21 participant from June 10. Do assign me if possible. Thank You!

@QAZIMAAZARSHAD Sir i want to add the social media logos in the image folder but I am unable to do so.

@QAZIMAAZARSHAD Sir i want to add the social media logos in the image folder but I am unable to do so.

don't use images try font awesome icons

@QAZIMAAZARSHAD  ok sir

Sir can I use Icon and logo for footer design.

okk

I would like to work on this issue
.

I would like to work on this issue
.

Already assigned

May I work on this issue?

I resolved this issue, please review the code.

@RiyaGupta89 this issue is to completely redesign the footer.

Oh okay sure. I'll do it. Do you want something else like some icon in the footer section too? Or you just want to keep this much content only and just to redesign the footer?

Oh okay sure. I'll do it. Do you want something else like some icon in the footer section too? Or you just want to keep this much content only and just to redesign the footer?

No, you can add many things if you think it will be better.
It will be better if you refer to some footer designs of some Indian government websites.

Please have a look if it seems okay?

If you want anymore changes you can let me know else can I request for the PR?


Feature Request: Custom IDs

Would it be feasible to provide an API that allows us to specify the IDs rather than having your library generate them as sequental ints? Since the "thing" we want to schedule probably already has an identifier of sorts (I couldn't think of a use case in my code where that wouldn't be true), having to store a returned ID requires us to write caching and mapping code. An overload that would take the ID would make all that go away, e.g.
    public void SetReminder(Appointment app)
    {
        //cancel a potentially scheduled appointment
        CrossNotifications.Current.Cancel(app.Id);

        //schedule appointment
        CrossNotifications.Current.Send("Foo", "Bar", app.Timestamp, id: app.Id);
    }

Thanks for the consideration!
Philipp

Thanks for the FR.  I have actually been considering this idea for a long time and I do agree with it.  I just wanted it to be set automatically or random generation.  I don't want to deal with both scenarios.

Regarding the scenarios: That would be an easy overload, not? If specified use that ID, otherwise just generate it (either using your int scheme, or by just generating a GUID? (in case you went for ints rather than strings due to a platform limitation, you probably could still use strings in the API and then hash those with a super low risk of collisions).

Taking the options directly isn't hard.  It is dealing with the platforms that can be painful.
I have to watch for "collisions" of IDs as some OS's will silently take the ID and overwrite a previous notification schedule.  The other problem is that in order to cancel on android, I need to keep IDs around somewhere and I have a limited storage basin.

I don't know the specifics, so I'm sure you're better to judge. My gut feeling was that collisions shouldn't be much of a problem (or responsibility could be delegated to the caller, if solving it generically is a problem), but I might be totally wrong there.

Some time ago I made some quick changes to add reoccurring notifications and add id references.
Pushed the changes to my fork. Might work for you :)

This is part of v4 which is due any day


window.devicePixelRatio fails in Firefox and Chrome 57

Reading the value of window.devicePixelRatio throws an error in Firefox:

TypeError: 'get devicePixelRatio' called on an object that does not implement interface Window.

It seems this might be caused by the same issue as was reported in #38
    export class Visual implements IVisual {
        constructor(options: VisualConstructorOptions) {
            console.log("devicePixelRatio test");
            try {
                console.log("devicePixelRatio ok", window.devicePixelRatio);
            } catch (e) {
                console.error("devicePixelRatio err", e);
            }
        }
    }

The same code works correctly on Chrome.

This also happens now on Chrome 57.

Same issue here chrome 57 not working. This is having a significant impact.

Any updates on this? It's a huge issue for us at the moment, breaking critical visualizations

The current workaround is to use window.window.devicePixelRatio.
Please note we've created the common issue to track the current status.


How to run TDVT over a custom connector made by dev team?

About You:
Name: Moemen
Company:
Your question:
Dev team developed a new connector after testing it manually over tableau, I wanted to run TDVT suite , however I keep hitting signature issue, isn't there a way to run custom un-signatured connector?

Hi Moemen,
Try adding -DDisableVerifyConnectorPluginSignature=True to the CommandLineOveride in your ini file. Should do the trick.
-Tucker

2020-08-27 12:05:09,652 Thread-[7]  calling C:\Program Files\Tableau\Tableau 2020.2\bin\tabquerytool.exe --expression-file-list C:\Users\ahmed\AppData\Local\Temp\eba92a1e5cf0e6a9d3cfd1121b9ff4e93f925b7b4c63aeaff45f6f08rz7x9h6i\CastCalcsConnectionTestsqli_tdvt\tests.txt -d C:\Users\ahmed\Desktop\TDVT_TCC\tds\cast_calcs.sqli_tdvt.tds --combined --password-file C:\Users\ahmed\Desktop\TDVT_TCC\tds\sqli_tdvt.password --output-dir C:\Users\ahmed\AppData\Local\Temp\eba92a1e5cf0e6a9d3cfd1121b9ff4e93f925b7b4c63aeaff45f6f08rz7x9h6i -DLogDir=C:\Users\ahmed\AppData\Local\Temp\eba92a1e5cf0e6a9d3cfd1121b9ff4e93f925b7b4c63aeaff45f6f08rz7x9h6i\CastCalcsConnectionTestsqli_tdvt -DOverride=ProtocolServerNewLog -DDisableVerifyConnectorPluginSignature=True -DLogicalQueryRewriteDisable=Funcall:RewriteConstantFuncall -DInMemoryLogicalCacheDisable
2020-08-27 12:05:16,862 Thread-[7] CalledProcessError: Return code: 16 LoadDatasource DataSourceException: Package signature verification failed during connection creation.
I keep getting this message, despite as you can see the disable signature is added and true.

Hi @MoemenEb ! Are you still facing this issue. We had another case where this error occurred once but it was intermittent and did not occur later. You could also use the latest version 2020.3 of tableau and check if it happens in that case too

tfs 1197795

Closing. Please reopen if the issue still occurs.


New crime data

@longhotsummer

Are we showing per 10,000k population, or youth population?

It's showing Total population.

Cool.


Enlarge map size

It should be 250x250 at least

Coupled with #197

I could ask a ss13 mapper to map a station, if that's good.

I doubt that it will be helpful on this stage of the development. I will do it myself.

bool settings map editor support because anchored_ is needed

More signs
Mode medical analyzers in left part of the map, and more atmos tools in right part of the map
More various floors
More professions (assistant, security, doctor, barman, clown)
Change starting texts

Better atmos tool output

Random human body types (black, white, etc)

Seems like done overall


Include help

Hey! Cool little utility.
Perhaps include a --help or -h or help or man ansiweather feature? :)
/Storm

There is a man page, check ansiweather.1.

λ brew info ansiweather
ansiweather: stable 1.16.0, HEAD
Weather in your terminal, with ANSI colors and Unicode symbols
https://github.com/fcambus/ansiweather
/usr/local/Cellar/ansiweather/1.16.0 (7 files, 23.5KB) *
  Built from source on 2020-06-02 at 17:48:52
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/ansiweather.rb
==> Dependencies
Required: jq ✔
==> Options
--HEAD
	Install HEAD version
==> Analytics
install: 184 (30 days), 609 (90 days), 4,154 (365 days)
install-on-request: 179 (30 days), 592 (90 days), 3,777 (365 days)
build-error: 0 (30 days)
λ ansiweather
 Weather in Rzeszów: 23 °C - UVI: 7.15 - Wind: 2.6 m/s W - Humidity: 60% - Pressure: 1010 hPa
λ ansiweather --help
/usr/local/bin/ansiweather: illegal option -- -
λ ansiweather -help
 Weather in Rzeszów: 23 °C - UVI: 7.15 - Wind: 2.6 m/s W - Pressure: 1010 hPa
λ ansiweather -h
/usr/local/bin/ansiweather: option requires an argument -- h
λ man ansiweather
No manual entry for ansiweather
λ man -d ansiweather
Reading config file /private/etc/man.conf
Looked whether there exists a message catalog man, but there is none
(and for English messages none is needed)

found man directory /usr/share/man
found man directory /usr/local/share/man
found man directory /usr/X11/man
found man directory /Library/Apple/usr/share/man
found manpath map /bin --> /usr/share/man
found manpath map /sbin --> /usr/share/man
found manpath map /usr/bin --> /usr/share/man
found manpath map /usr/sbin --> /usr/share/man
found manpath map /usr/local/bin --> /usr/local/share/man
found manpath map /usr/local/sbin --> /usr/local/share/man
found manpath map /usr/X11/bin --> /usr/X11/man
found manpath map /usr/bin/X11 --> /usr/X11/man
found manpath map /usr/bin/mh --> /usr/share/man

using less as pager

using /usr/bin/less -is as browser

using /bin/cat to dump HTML pages as text
adding /usr/local/man to manpath
adding /usr/share/man to manpath
adding /usr/local/share/man to manpath
adding /opt/X11/share/man to manpath
adding /Applications/Wireshark.app/Contents/Resources/share/man to manpath
No manual entry for ansiweather
λ fd ansi /usr/local/share/man/man1/
/usr/local/share/man/man1/ansifilter.1.gz

This is a packaging issue. Please contact the ansiweather package maintainer in Homebrew and ask them to add the man page to the package. Thanks.


Add UI filter to filter on Healthy/ Unhealthy shares at all places where the shares are filtered.

Is your idea related to a problem? Please describe.
At dataset->shares page, the UI currently provides some filter options to filter shares on dataset.
Describe the solution you'd like
Add another additional filter for checking what shares are healthy/ unhealthy in the Shares page as well as the Dataset-> Shares tab
Along with that add a UI column in the list of shares to show the status of shares. Also show what the last time those shares were verified.
P.S. Don't attach files. Please, prefer add code snippets directly in the message body.

Nice enhancement! Thanks @TejasRGitHub for opening an issue. Will you be working on it?

Hi @dlpzx , Not planned to work on this soon. I am currently working on share expiration task https://github.com/data-dot-all/dataall/issues/1083


A function for getting the current load arguments

Describe the problem
fetch is great, but it's common to need a more powerful abstraction when loading data — a GraphQL client, or a custom api.get('endpoint') helper, or even just something that adds some logging around the fetch.
Right now that's somewhat cumbersome, because in order to make credentialled fetches during SSR, you need the instance of fetch that was passed into load. So you have to pick between these two monstrosities:
<script context="module">
  import * as api from '$lib/api';

  export async function load({ fetch }) {
    const data = api.get(fetch, 'endpoint');
    // ...
  }
</script>

<script context="module">
  export async function load({ stuff }) {
    const data = stuff.api.get('endpoint');
    // ...
  }
</script>

It gets worse if you need to pass page or session around for logging purposes.
The reason we have to do this is that the global fetch (supplied by node-fetch, in environments that don't natively support it) has no concept of credentials. A request for the /account page receives cookies; those cookies need to be forwarded when the page's load makes a request for the /account.json data that powers it, and the load-specific fetch argument makes that possible.
It would be nice if using an abstraction around fetch was less cumbersome.
Describe the proposed solution
Just like onMount registers a callback with the component that is currently initialising, we could introduce a function that makes the arguments to the current load invocation available to a function that gets called during that invocation. For example:
// src/lib/api.js
import { getLoadArguments } from '$app/load';

const base = import.meta.env.VITE_API_BASE_URL;

export async function get(endpoint) {
  const { fetch, page, session } = getLoadArguments();

  const start = Date.now();
  const url = `${base}/${endpoint}`;
  const res = await fetch(url);
  const name = session.user ? session.user.name : 'anonymous user';
  console.log(`fetched ${url} from ${page.path} for ${name} in ${Date.now() - start}ms`);

  return res;
}

<script context="module">
  import * as api from '$lib/api';

  export async function load() {
    const data = api.get('endpoint');
    // ...
  }
</script>

This function (hopefully we can come up with a better name than getLoadArguments) would only work during the invocation of a load; if called at any other time, it would throw an error.
Alternatives considered
No response
Importance
would make my life easier
Additional Information
No response

A thought occurs to me. When we designed load in https://github.com/sveltejs/kit/issues/268, we added context, since renamed stuff. As I recall it (and judging by the evidence in that issue) the primary motivation for this was to enable things like this:
<script context="module">
  export async function load({ stuff }) {
    const data = stuff.api.get('endpoint');
    // ...
  }
</script>

One concern raised by @jonatansberg was that this would harm performance in some cases by forcing sequential execution of load functions (a page's load cannot run until parent __layout components have loaded, in case they inherit stuff).
As a compromise, we added the ability to return a Promise for props instead of an object. But the ergonomics of this aren't great.
If we had an API like getLoadArguments (please, someone come up with a better name) presumably stuff would be unnecessary. Is there any other use case for it? Or would this enable us to do away with one of the wartier (in both API and performance terms) parts of SvelteKit?

In a not-yet-production SvelteKit rewrite of a production Sapper app, I have some data I want to be able to load once and then have available throughout the whole app. In the Sapper version of the app, I was loading this data in middleware and then exposing it in $session - which was gross. In the SvelteKit rewrite, I was using stuff to fetch this data in the layout component and then pass it down to all other pages - which is a lot nicer.
I have mixed feelings about eliminating the stuff feature. I'll have to see how much more work it is (and how confusing it is) to use something like Svelte's own context feature to pass this data down from the layout component.

I imagine there being cases where someone wants to have things passed to stuff available inside children already. Weak example: Run an API call in layout which returns something you need in subsequent API calls in loads below.
And what about people deciding to redirect in layout and rely on loads not being called before that? If they would run in parallel and then would all crash (401 for example), how to handle that?
What if there was loadBlocking which is the same as load but you can fill stuff in there and loads below it wait on it - like the current load behavior? Then the user could decide what to use, whether he needs blocking loads or parallel loads or blocking up until a point but then in parallel. (exporting both load and loadBlocking would be an error)

Hey all,
Just chiming in here as someone who has been trying to integrate urql with svelte-kit, first and foremost, awesome work already!
To give a little detail on how we work in urql up-to-date, there are probably several ways to tweak this to svelte-kit, we have an ssr plugin which hooks into our request-flow and saves all data that happens during a server-side render.
Example: urql.query(x) will be called on a page which will then be written to the ssr-plugin, when the ssr render completes we call ssrPlugin.extractData() and pass it on in a script tag that goes to the client and rehydrate from there.
In Next we've had to do this a bit more linear to this here, we have a load-function where we manually call the functions needed for the page we are rendering and then call ssrPlugin.extractData()

Attempt in svelte-kit
What we advise in Next

Just throwing these out here so we can have a wide variety of use-cases, thank you for starting this @Rich-Harris! 🙌

And what about people deciding to redirect in layout and rely on loads not being called before that?

Ah yeah, I guess error and redirect are reason enough by themselves for load to run sequentially. In which case there's much less reason to want to get rid of stuff.

If the fetch succeeds at caching POST requests for GraphQL it should be possible, generally we do a lot of this due to us stringifying the query + variables and transforming it into a unique key for that result.

It's designed to work that way — during SSR it will hash the request body...
https://github.com/sveltejs/kit/blob/ae7f7904c4f3c111dbff1ad1209ad92fb6f057cf/packages/kit/src/runtime/server/page/render.js#L179-L188
...then if a fetch is attempted client-side with the same body it will use the serialized data (if no match, it will fall back to the network, so it's resilient):
https://github.com/sveltejs/kit/blob/ae7f7904c4f3c111dbff1ad1209ad92fb6f057cf/packages/kit/src/runtime/client/renderer.js#L47-L63

This is not well thought-out, but is there any reasonable way to combine this feature with making the $app/stores exports magically work inside the load function?
I have various helper functions in my app that I need to sometimes call within a load function and sometimes within the main body of a component, and it currently takes a little bit of fiddling to allow them to either be passed in the session from load or to get the $session store from $app/stores.
With the changes proposed in this issue, I couldn't quite get rid of these extra hoops, because these functions still won't know whether to use the new $app/load or to use $app/stores. Is there anything that makes sense here?

I was just thinking about this again and ... how is this feature actually going to work on the server? Doesn't this basically require some sort of continuation-local storage system?
Are we going to say that getLoadArguments needs to be called synchronously after the start of load? Does the JS spec even say that any code that's run in an async function before the first await will be synchronously run right when you call the function?

yes, the async-ness of a function has no bearing on the matter — this will always log 'before', 'fooing', 'after':
async function foo() {
  console.log('fooing');
}

function test() {
  console.log('before');
  foo();
  console.log('after');
}

test();

Which means that figuring out which load function is running involves no magic:
export function getLoadArguments() {
  return current_load_arguments;
}

function navigate() {
  current_load_arguments = { url, params, session, stuff, fetch };
  const promise = load.call(null, current_load_arguments);
  current_load_arguments = null;
}

Gotcha, okay, so we'll just need to document that getLoadArguments will need to be called synchronously after the start of load. The main place I see this causing a problem is when someone wants to run two network requests in series during load - the second one won't have access to the load arguments.

Would there be benefit in making the arguments available during the whole load method, resetting it only after the promise has resolved. Or does this not work because the load arguments could become stale/outdated or sth?

It doesn't work because there might be multiple requests being served concurrently by the server - the same reason that they are arguments to load to begin with rather than being some sort of globals.

Hi there,
I would really like this proposal to work but I think the fact that getLoadArguments can only be called at the start of the load function is a constraint that makes it pretty useless in may real world use cases as it only works with single or multiple independent requests. It's no longer safe to use getLoadArguments as soon as requests depend on each other because api requests outside of the load function may be called during that time.
I would like to propose an other solution that introduces some kind of hook that makes it possible to customize the way load is called.
function loadHook(node: BranchNode, load_input: LoadInput): LoadOutput {
  const api = API.createWithLoadArguments(load_input);

  return node.module.load(load_input, api);
}

The load function now receives a custom api argument.
async function load({page}, api) {
  const group = await api.getGroup(page.params.groupid);
  const members = await api.followLink(group, 'members');

  return {
    props: { group, members }
  }
}

The src/runtime/client/renderer.js#L676 would have to be changed as follows.
- const loaded = await module.load.call(null, load_input);
+ const loaded = await (loadHook ? loadHook(node, load_input) : module.load.call(null, load_input));

@Conduitry, load & stores are already magic.
1/ In SSR, load fetch the data
2/ load is executed in the client and reuse the result of the SSR (not a second fetch, thx to SvelteKit).
3/ At this time you can fill stores with your data
=> You can check it out here: KitQL - Demo 1
Not that nice to pass the fetch, but minimal amount of code 👍

I'll be very interested in adding getLoadArguments to the generating part of KitQL and be able to write:
import { KQL_AllContinents } from '$lib/graphql/_kitql/graphqlStores';

export async function load() {
	await KQL_AllContinents.queryLoad(); // Work in SSR & Client mode
	// await KQL_another-operation-filling-another-store.queryLoad();
	return {};
}

And getting rid of the fetch part.

Hi. I've been hitting this problem with my app that I'm building, and I'm currently going with the first snippet that Rich posted:
<script context="module">
  import * as api from '$lib/api';

  export async function load({ fetch }) {
    const data = api.get(fetch, 'endpoint');
    // ...
  }
</script>

And yeah, passing fetch like that is really messy.
I read the part about "credentials", but I don't fully understand the reasoning. Couldn't things like cookies be "split out" from the request on the server-side code? Why do they need to be tied to a load-specific fetch ?

I've just posted this discussion which is along the same lines of the thinking here - how to reduce the cumbersome boilerplate in using SvelteKit's fetch implementation when you need to add headers, handle responses, add logging, etc. Ideally I'd like a solution which works in load() (both in SSR and in the browser) and also in the component lifecycle as well (as @Conduitry mentioned above).
I did have one question in response to this from @Rich-Harris:

This function (hopefully we can come up with a better name than getLoadArguments) would only work during the invocation of a load; if called at any other time, it would throw an error.

Can I confirm that, outside of a load() invocation (i.e. in a standard component, perhaps when a form is submitted or a button is pressed), there is no advantage to using the 'special' SvelteKit load() implementation, and we should use window.fetch instead? If so, it'd be great to document that clearly.

Someone else can feel free to correct me, but afaik the only thing sveltekit’s provided fetch does is hydrate the response fetched during load on the server, so there would be no reason to use it outside of that context (if it even works outside of that context)

#4625 is one thing the client-side load fetch doesn't do today but which we'd like it to do.

(hopefully we can come up with a better name than getLoadArguments)

maybe getLoadInput()?
Is the work on this goal advanced? Because I don't know whether to build access to the external API with tricks or wait for the completion of this track?
By the way, I wanted to ask about this:

<script context="module">
  export async function load({ stuff }) {
    const data = stuff.api.get('endpoint');
    // ...
  }
</script>

Rich in the initiating post gave two examples that he called "monstrosities", the first I understand, but the one above does not. Where in stuff does the api come from?

@jaclas I'm currently using stuff in the above-mentioned discussion: https://github.com/sveltejs/kit/discussions/5173. Rich's proposal would make this much cleaner.

We realised last night that this feature is unnecessary — you can do everything with a higher order function:
export const load = extra_fancy_load(({ client }) => {
  return client.get_stuff();
});

This is much less magical, and doesn't come with any caveats around timing.
The one major caveat is that extra_fancy_load would most likely want to destructure the event...
return fn.call(null, { ...event, client });

...which would have the effect of triggering getters for url and params that would mark them as dependencies, incorrectly. It's easy enough to work around, but it'd be nice to have some better patterns for that.

We realised last night that this feature is unnecessary — you can do everything with a higher order function:
export const load = extra_fancy_load(({ client }) => {
  return client.get_stuff();
});

Could you please elaborate a bit more on this technique (like how to define and use extra_fancy_load)? I couldn't find anything on the docs.

@gterras a simple implementation of extra_fancy_load could look something like this.
const extra_fancy_load = (loadFunction) => {
  return async function load(args) {
    const client = API.createClientWithLoadArguments(args);
    return await loadFunction({...args, client});
  }
}

The beauty of this (compared to the initial examples) is that the load arguments don't need to be passed around in the background it has no side effects.
And it is much more composeable.
Example:
const logging_load = (loadFunction) => {
  return async function load(args) {
    const start = Date.now();
    const result = await loadFunction(args);
    console.log(`Loaded ${args.url.href} in ${Date.now() - start}ms`);
    return result;
  }
}

Simple usage:
export const load = logging_load(async ({fetch}) => {
  const res = await fetch('/some/data');
  const json = await res.json();
  return json;
})

Or compose it with extra_fancy_load:
export const load = logging_load(extra_fancy_load(({ client }) => {
  return client.get_stuff();
}))

@gterras a simple implementation of extra_fancy_load could look something like this.

Thanks for this, very interesting this should be mentioned in the docs.
What would be the proper way to TS type extra_fancy_load?


[kr-ko] Fix configmap to configMap in configmap.md

Why
'configmap' is an invalid field, correct to 'configMap'
Changes

Fix configmap field in example to configMap

Syntax error is fixed now.
Preview page

Thanks @SegmenFault
LGTM

@SegmenFault Thank you!
/approve


add initial flyway migration script for empty referral

What does this pull request do?
adds a simply initial migration script for the existing referral entity which is super simple, and adds flyway config to run migrations on startup
What is the intent behind these changes?
prepare for running against a real database in our dev environment

Out of curiosity, do we know how will we kick off the migration locally, on docker-compose, and during deployment?

Out of curiosity, do we know how will we kick off the migration locally, on docker-compose, and during deployment?

the migrations run on application start in all three scenarios

the migrations run on application start in all three scenarios

Is that safe in case more than one pod boots up at the same time?

We might want to pull the migration script out in a separate init container: https://kubernetes.io/docs/concepts/workloads/pods/init-containers/#init-containers-in-use

yeh i'd be up for something like this. i'll open a ticket to look into it later.


Add Support For SoftDescriptor

The SoftDescriptor is what appears on billing statements. I have been asked many times how to change this. The SoftDescriptor is set during the "Authorize" API call.
https://payments.amazon.com/documentation/apireference/201751630#201752010
By default it is set to "AMZ*<SELLER_NAME> amzn.com/pmts WA" and can only be 16 characters long. I'm not sure what the verbiage would be as the Pay with Amazon option, however.

Proposed functionality:
Use Store view name as default SoftDescriptor in Authorize call. Document that store name will be truncated at 16 characters and provide instructions for manually changing the SoftDescriptor in the code, which should amount to a single line/string change.

behavior will be new config for Soft Descriptor, with note describing how Store Name will be used if blank

Added "Soft Descriptor" config field under "Transaction." Field is limited to 16 chars.
Help text: "Billing descriptor to display on credit card statements. Limited to 16 characters. If left blank, current Magento store name will be used."


IE 11 error

In IE 11 error

SCRIPT5009: "Promise" is not defined
fetch.js (328,1)

I'm sorry I used the old version. After the upgrade, everything works

@Shperung Thanks for updating. Just so you know: you can close your own issue when it turns out it wasn't a bug after all. 😉


Configure release-please to add the built zip as a release artefact

Refer to krystal/vale-package


Data classification: The generated Console App and Web API projects are running failed.

System Information (please complete the following information):
Windows OS: Windows-11-Enterprise-22H2
ML.Net Model Builder 2022: <IP_ADDRESS>1401 (Main Build)
Microsoft Visual Studio Enterprise: 2022(17.4.1)
.Net: 6.0
Describe the bug

On which step of the process did you run into an issue:
The generated Console App is running failed.

Data Source
https://testpass.blob.core.windows.net/test-pass-data/wikipedia-detox-250-line-data.tsv
SQL Password
To Reproduce
Steps to reproduce the behavior:

Select Create a new project from the Visual Studio 2022 start window.
Choose the C# Console App (.NET Core6.0) project template.
Add model builder by right click on the project.
Select Data classification scenario.
On Data page, select the data source and choose Sentiment column as label.
On the Train page, click "Start training" button to complete the training.
Click "Add to solution" button on the Consume page to generate Console App project.
Run the generated Console App project.

Expected behavior
The generated Console App project should be running successfully.
Screenshots
File:

SQL Password:

Additional context:

The bug is also repro for Text classification scenario: The generated Console App project is running failed; the generated Web API project is running successfully.

Thanks for finding this @v-Hailishi!
Dev team - possible root cause is that this is binary classification?

Verified on the latest main build <IP_ADDRESS>2002, the bug is repro on Data classification, Value predication, Recommendation, Forecasting, Text classification scenarios.

For  this

binary is not an issue , it happens for text classification types such as data-classification(binary and multi) and Text- classification but it is not consistence

It is not reproduceable from the latest main

Code behind generation missed  installation of  System.Data.SqlClient

@v-Hailishi  FYI: to validate code in the generated project we should use the newly mbconfig file, if you use old mbconfig file you might get some errors

We're going to change our code to convert to strings for now in AutoML pipelines and CodeGen. We'll also look into supporting other types in ML.NET.
@michaelgsharp has something that we've briefly tested and it works.

For AutoML we can try the following change:
https://github.com/dotnet/machinelearning/blob/a758217121c85cf5af9a2ea3f759feae2020d7b5/src/Microsoft.ML.AutoML/SweepableEstimator/Estimators/MapValueToKey.cs#L11
Change to something like this:
.Append(mlContext.Transforms.Conversion.MapValueToKey(outputColumnName:@"Label",inputColumnName:@"Label", addKeyValueAnnotationsAsText: true))

Verified on the latest main build <IP_ADDRESS>6001, the generated Console App and Web API projects are still running failed on scenarios: Data classification, Value predication, Recommendation, Forecasting, Text classification scenarios.
The following are the details:

Data classification

1.1 File1 (https://testpass.blob.core.windows.net/test-pass-data/wikipedia-detox-250-line-data.tsv)
Console App:

Web API:

1.2 File2 (https://testpass.blob.core.windows.net/test-pass-data/issues.tsv.txt)
Console App: Run successful

Web API:

1.3 Microsoft SQL Server: Binary (SQL Password)
Console App:

Web API:

1.4 Microsoft SQL Server Database File (https://github.com/dotnet/machinelearning-samples/raw/main/samples/modelbuilder/MulticlassClassification_RestaurantViolations/RestaurantScores.zip)
Console App: Run successful

Web API:

Value predication:
The generated Console App project is running successfully; the generated Web API project is running fail.

Console App: Run successful

Web API:
File:

SQL:

Recommendation:
The generated Console App project is running successfully; the generated Web API project is running fail.

Console App: Run successful

Web API:
File:

SQL:

Forecasting:
Both of the generated Console App and Web API projects are still running failed

File:

SQL:

Text classification:

Console App:

Web API:
File:

SQL:

Verified on the latest main build <IP_ADDRESS>6101, the generated Console App and Web API projects are still running failed on scenarios: Data classification, Value predication, Recommendation, Forecasting, Image Classification-Local, Text classification scenarios.
The following are the details:

Data classification

1.1 File1 (https://testpass.blob.core.windows.net/test-pass-data/wikipedia-detox-250-line-data.tsv)
Console App: Run successful

Web API:

1.2 File2 (https://testpass.blob.core.windows.net/test-pass-data/issues.tsv.txt)
Console App:

Web API:

1.3 Microsoft SQL Server: Binary (SQL Password)
Console App:

Web API:

1.4 Microsoft SQL Server Database File (https://github.com/dotnet/machinelearning-samples/raw/main/samples/modelbuilder/MulticlassClassification_RestaurantViolations/RestaurantScores.zip)
Console App:

Web API:

Value predication & Recommendation:
The generated Console App project is running successfully; the generated Web API project is running fail.

Web API:
File:

SQL:

Forecasting:
Both of the generated Console App and Web API projects are still running failed.

File:

SQL:

Image Classification-Local:
The generated Console App project is running fail; the generated Web API project is running successfully.

Console App:

Text classification:
The generated Console App project is running successfully; the generated Web API project is running fail.

Console App: Run successful

Web API:
File:

SQL:

Verified on the latest main build <IP_ADDRESS>6701, the generated Console App and Web API projects are still running failed on scenarios: Data classification, Value predication, Recommendation, Forecasting, Image Classification-Local, Object detection, Text classification scenarios.
The following are the details:

Data classification

1.1 File (https://testpass.blob.core.windows.net/test-pass-data/wikipedia-detox-250-line-data.tsv)
Console App: Run successful
Web API: Failed (After removing the unnecessary using directive, can run successful)

1.2 Microsoft SQL Server: Binary (SQL Password)
Console App: Failed

Web API: Failed

After removing the unnecessary using directive, still run failed.

1.3 Microsoft SQL Server Database File (https://github.com/dotnet/machinelearning-samples/raw/main/samples/modelbuilder/MulticlassClassification_RestaurantViolations/RestaurantScores.zip)
Console App: Run successful
Web API: Failed

After removing the unnecessary using directive, still run failed.

Value predication & Recommendation:
The generated Console App project is running successfully; the generated Web API project is running fail.

Web API:
File: (After removing the unnecessary using directive, can run successful)

SQL:

After removing the unnecessary using directive, still run failed.

Forecasting:
Both of the generated Console App and Web API projects are still running failed.

File:
Console App:

Web API:

SQL:
Console App:

Web API:

Image Classification-Local & Object detection:
The generated Console App project is running successfully; the generated Web API project is running failed.

Web API: (After removing the unnecessary using directive, can run successful)

Text classification:
The generated Console App project is running successfully; the generated Web API project is running fail.

Web API:
File: Failed (After removing the unnecessary using directive, can run successful)

SQL:

After removing the unnecessary using directive, still run failed.

Verified on the latest main build <IP_ADDRESS>6903, the generated Console App and Web API projects are still running failed on scenarios: Data classification, Value predication, Recommendation, Forecasting, Text classification scenarios.
The following are the details:

Data classification

1.1 File (https://testpass.blob.core.windows.net/test-pass-data/wikipedia-detox-250-line-data.tsv)
Console App: Run successful
Web API: Run successful
1.2 Microsoft SQL Server: Binary (SQL Password)
Console App: Failed

Web API: Failed

1.3 Microsoft SQL Server Database File (https://github.com/dotnet/machinelearning-samples/raw/main/samples/modelbuilder/MulticlassClassification_RestaurantViolations/RestaurantScores.zip)
Console App: Run successful
Web API: Failed (After choosing "Use local version '4.8.3'" can run successful.)

Value predication & Recommendation & Text classification:
The generated Console App project is running successfully; the generated Web API project is running fail.

Web API:
File: Run successful
SQL: Failed (After choosing "Use local version '4.8.3'" can run successful.)

Forecasting:
Both of the generated Console App and Web API projects are still running failed.

File:
Console App: Failed (After choosing "using System.IO;" can run successful.)

Web API: Failed

SQL:
Console App: Failed (After choosing "using Microsoft.ML.Data;" and "using System.IO;" and "Use local version '4.8.3'" can run successful.)

Web API: Failed

Verified on the latest main build <IP_ADDRESS>7701, the generated Console App and Web API projects are still running failed on scenarios: Data classification.

Data classification
Only failed for Microsoft SQL Server: Binary (SQL Password)
Console App: Failed

Web API: Failed

@v-Hailishi   what do you mean SQL password? is that a dataset,  are you able to share your dataset? thanks

@zewditu "SQL password" just means I use the SQL Server Authentication to log on to the server.

Besides, in the database, I use the dataset "https://testpass.blob.core.windows.net/test-pass-data/wikipedia-detox-250-line-data.tsv".

@zewditu This isn't regarding SQL. You can repro this by using a boolean type. I used the yelp dataset and set the prediction column to type boolean

The bug has been fixed on the latest main build <IP_ADDRESS>0605.