RuntimeError: CUDA error: invalid device ordinal

是否已有关于该错误的issue或讨论？ | Is there an existing issue / discussion for this?

[X] 我已经搜索过已有的issues和讨论 | I have searched the existing issues / discussions

该问题是否在FAQ中有解答？ | Is there an existing answer for this in FAQ?

[X] 我已经搜索过FAQ | I have searched FAQ

当前行为 | Current Behavior
Trying to fine tune a model using sample code provided finetune_lora.sh, finetune.py, dataset.py, trainer.py provided in your github repository.
I have set export CUDA_VISIBLE_DEVICES=0,1
root@584a1774aaad:/# nvidia-smi
Sat Jul 20 21:07:53 2024
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.129.03             Driver Version: 535.129.03   CUDA Version: 12.2     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA A100-SXM4-80GB          On  | 00000000:0A:00.0 Off |                    0 |
| N/A   32C    P0              61W / 400W |      4MiB / 81920MiB |      0%      Default |
|                                         |                      |             Disabled |
+-----------------------------------------+----------------------+----------------------+
|   1  NVIDIA A100-SXM4-80GB          On  | 00000000:44:00.0 Off |                    0 |
| N/A   33C    P0              61W / 400W |      4MiB / 81920MiB |      0%      Default |
|                                         |                      |             Disabled |
+-----------------------------------------+----------------------+----------------------+
+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
|  No running processes found                                                           |
+---------------------------------------------------------------------------------------+
root@584a1774aaad:/# nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Mon_Apr__3_17:16:06_PDT_2023
Cuda compilation tools, release 12.1, V12.1.105
Build cuda_12.1.r12.1/compiler.32688072_0
root@584a1774aaad:/#
root@584a1774aaad:/workspace/minicpm/finetune# ./finetune_lora.sh
W0720 21:08:27.097000<PHONE_NUMBER>92768 torch/distributed/run.py:757]
W0720 21:08:27.097000<PHONE_NUMBER>92768 torch/distributed/run.py:757] *****************************************
W0720 21:08:27.097000<PHONE_NUMBER>92768 torch/distributed/run.py:757] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
W0720 21:08:27.097000<PHONE_NUMBER>92768 torch/distributed/run.py:757] *****************************************
[2024-07-20 21:08:35,215] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-07-20 21:08:35,216] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-07-20 21:08:35,233] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-07-20 21:08:35,233] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-07-20 21:08:35,234] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-07-20 21:08:35,239] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-07-20 21:08:35,270] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-07-20 21:08:35,270] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect)
df: /root/.triton/autotune: No such file or directory
[WARNING]  async_io requires the dev libaio .so object and headers but these were not found.
[WARNING]  async_io: please install the libaio-dev package with apt
[WARNING]  If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
[WARNING]  Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
[WARNING]  async_io requires the dev libaio .so object and headers but these were not found.
[WARNING]  async_io: please install the libaio-dev package with apt
[WARNING]  If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
[WARNING]  Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
[WARNING]  async_io requires the dev libaio .so object and headers but these were not found.
[WARNING]  async_io: please install the libaio-dev package with apt
[WARNING]  If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
[WARNING]  Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
[WARNING]  async_io requires the dev libaio .so object and headers but these were not found.
[WARNING]  async_io: please install the libaio-dev package with apt
[WARNING]  If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
[WARNING]  Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
[WARNING]  async_io requires the dev libaio .so object and headers but these were not found.
[WARNING]  async_io: please install the libaio-dev package with apt
[WARNING]  If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
[WARNING]  Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
[WARNING]  async_io requires the dev libaio .so object and headers but these were not found.
[WARNING]  async_io requires the dev libaio .so object and headers but these were not found.
[WARNING]  async_io: please install the libaio-dev package with apt
[WARNING]  If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
[WARNING]  Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
[WARNING]  async_io: please install the libaio-dev package with apt
[WARNING]  If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
[WARNING]  Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
[WARNING]  async_io requires the dev libaio .so object and headers but these were not found.
[WARNING]  async_io: please install the libaio-dev package with apt
[WARNING]  If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
[WARNING]  Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
[WARNING]  sparse_attn requires a torch version >= 1.5 and < 2.0 but detected 2.3
[WARNING]  using untested triton version (2.3.1), only 1.0.0 is known to be compatible
[WARNING]  sparse_attn requires a torch version >= 1.5 and < 2.0 but detected 2.3
[WARNING]  using untested triton version (2.3.1), only 1.0.0 is known to be compatible
[WARNING]  sparse_attn requires a torch version >= 1.5 and < 2.0 but detected 2.3
[WARNING]  using untested triton version (2.3.1), only 1.0.0 is known to be compatible
[WARNING]  sparse_attn requires a torch version >= 1.5 and < 2.0 but detected 2.3
[WARNING]  using untested triton version (2.3.1), only 1.0.0 is known to be compatible
[WARNING]  sparse_attn requires a torch version >= 1.5 and < 2.0 but detected 2.3
[WARNING]  using untested triton version (2.3.1), only 1.0.0 is known to be compatible
[WARNING]  sparse_attn requires a torch version >= 1.5 and < 2.0 but detected 2.3
[WARNING]  using untested triton version (2.3.1), only 1.0.0 is known to be compatible
[WARNING]  sparse_attn requires a torch version >= 1.5 and < 2.0 but detected 2.3
[WARNING]  using untested triton version (2.3.1), only 1.0.0 is known to be compatible
[WARNING]  sparse_attn requires a torch version >= 1.5 and < 2.0 but detected 2.3
[WARNING]  using untested triton version (2.3.1), only 1.0.0 is known to be compatible
/usr/local/lib/python3.10/dist-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: '/usr/local/lib/python3.10/dist-packages/torchvision/image.so: undefined symbol: _ZN3c1017RegisterOperatorsD1Ev'If you don't plan on using image functionality from torchvision.io, you can ignore this warning. Otherwise, there might be something wrong with your environment. Did you have libjpeg or libpng installed before building torchvision from source?
warn(
/usr/local/lib/python3.10/dist-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: '/usr/local/lib/python3.10/dist-packages/torchvision/image.so: undefined symbol: _ZN3c1017RegisterOperatorsD1Ev'If you don't plan on using image functionality from torchvision.io, you can ignore this warning. Otherwise, there might be something wrong with your environment. Did you have libjpeg or libpng installed before building torchvision from source?
warn(
/usr/local/lib/python3.10/dist-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: '/usr/local/lib/python3.10/dist-packages/torchvision/image.so: undefined symbol: _ZN3c1017RegisterOperatorsD1Ev'If you don't plan on using image functionality from torchvision.io, you can ignore this warning. Otherwise, there might be something wrong with your environment. Did you have libjpeg or libpng installed before building torchvision from source?
warn(
/usr/local/lib/python3.10/dist-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: '/usr/local/lib/python3.10/dist-packages/torchvision/image.so: undefined symbol: _ZN3c1017RegisterOperatorsD1Ev'If you don't plan on using image functionality from torchvision.io, you can ignore this warning. Otherwise, there might be something wrong with your environment. Did you have libjpeg or libpng installed before building torchvision from source?
warn(
/usr/local/lib/python3.10/dist-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: '/usr/local/lib/python3.10/dist-packages/torchvision/image.so: undefined symbol: _ZN3c1017RegisterOperatorsD1Ev'If you don't plan on using image functionality from torchvision.io, you can ignore this warning. Otherwise, there might be something wrong with your environment. Did you have libjpeg or libpng installed before building torchvision from source?
warn(
/usr/local/lib/python3.10/dist-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: '/usr/local/lib/python3.10/dist-packages/torchvision/image.so: undefined symbol: _ZN3c1017RegisterOperatorsD1Ev'If you don't plan on using image functionality from torchvision.io, you can ignore this warning. Otherwise, there might be something wrong with your environment. Did you have libjpeg or libpng installed before building torchvision from source?
warn(
/usr/local/lib/python3.10/dist-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: '/usr/local/lib/python3.10/dist-packages/torchvision/image.so: undefined symbol: _ZN3c1017RegisterOperatorsD1Ev'If you don't plan on using image functionality from torchvision.io, you can ignore this warning. Otherwise, there might be something wrong with your environment. Did you have libjpeg or libpng installed before building torchvision from source?
warn(
/usr/local/lib/python3.10/dist-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: '/usr/local/lib/python3.10/dist-packages/torchvision/image.so: undefined symbol: _ZN3c1017RegisterOperatorsD1Ev'If you don't plan on using image functionality from torchvision.io, you can ignore this warning. Otherwise, there might be something wrong with your environment. Did you have libjpeg or libpng installed before building torchvision from source?
warn(
/workspace/pypacks/transformers/training_args.py:1494: FutureWarning: evaluation_strategy is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use eval_strategy instead
warnings.warn(
/workspace/pypacks/transformers/training_args.py:1494: FutureWarning: evaluation_strategy is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use eval_strategy instead
warnings.warn(
/workspace/pypacks/transformers/training_args.py:1494: FutureWarning: evaluation_strategy is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use eval_strategy instead
warnings.warn(
/workspace/pypacks/transformers/training_args.py:1494: FutureWarning: evaluation_strategy is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use eval_strategy instead
warnings.warn(
/workspace/pypacks/transformers/training_args.py:1494: FutureWarning: evaluation_strategy is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use eval_strategy instead
warnings.warn(
/workspace/pypacks/transformers/training_args.py:1494: FutureWarning: evaluation_strategy is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use eval_strategy instead
warnings.warn(
/workspace/pypacks/transformers/training_args.py:1494: FutureWarning: evaluation_strategy is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use eval_strategy instead
warnings.warn(
/workspace/pypacks/transformers/training_args.py:1494: FutureWarning: evaluation_strategy is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use eval_strategy instead
warnings.warn(
[2024-07-20 21:08:49,356] [INFO] [comm.py:637:init_distributed] cdb=None
[2024-07-20 21:08:49,356] [INFO] [comm.py:637:init_distributed] cdb=None
[2024-07-20 21:08:49,357] [INFO] [comm.py:637:init_distributed] cdb=None
[2024-07-20 21:08:49,357] [INFO] [comm.py:637:init_distributed] cdb=None
[2024-07-20 21:08:49,358] [INFO] [comm.py:637:init_distributed] cdb=None
[2024-07-20 21:08:49,358] [INFO] [comm.py:637:init_distributed] cdb=None
[2024-07-20 21:08:49,358] [INFO] [comm.py:637:init_distributed] cdb=None
[2024-07-20 21:08:49,358] [INFO] [comm.py:668:init_distributed] Initializing TorchBackend in DeepSpeed with backend nccl
[2024-07-20 21:08:49,358] [INFO] [comm.py:637:init_distributed] cdb=None
[rank5]: Traceback (most recent call last):
[rank5]:   File "/workspace/minicpm/finetune/finetune.py", line 281, in
[rank5]:     train()
[rank5]:   File "/workspace/minicpm/finetune/finetune.py", line 162, in train
[rank5]:     ) = parser.parse_args_into_dataclasses()
[rank5]:   File "/workspace/pypacks/transformers/hf_argparser.py", line 339, in parse_args_into_dataclasses
[rank5]:     obj = dtype(**inputs)
[rank5]:   File "", line 136, in init
[rank5]:   File "/workspace/pypacks/transformers/training_args.py", line 1693, in post_init
[rank5]:     self.device
[rank5]:   File "/workspace/pypacks/transformers/training_args.py", line 2171, in device
[rank5]:     return self._setup_devices
[rank5]:   File "/workspace/pypacks/transformers/utils/generic.py", line 60, in get
[rank5]:     cached = self.fget(obj)
[rank5]:   File "/workspace/pypacks/transformers/training_args.py", line 2108, in _setup_devices
[rank5]:     self.distributed_state = PartialState(**accelerator_state_kwargs)
[rank5]:   File "/workspace/pypacks/accelerate/state.py", line 280, in init
[rank5]:     self.set_device()
[rank5]:   File "/workspace/pypacks/accelerate/state.py", line 790, in set_device
[rank5]:     torch.cuda.set_device(self.device)
[rank5]:   File "/workspace/pypacks/torch/cuda/init.py", line 399, in set_device
[rank5]:     torch._C._cuda_setDevice(device)
[rank5]: RuntimeError: CUDA error: invalid device ordinal
[rank5]: CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
[rank5]: For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
[rank5]: Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.
[rank3]: Traceback (most recent call last):
[rank3]:   File "/workspace/minicpm/finetune/finetune.py", line 281, in
[rank3]:     train()
[rank3]:   File "/workspace/minicpm/finetune/finetune.py", line 162, in train
[rank3]:     ) = parser.parse_args_into_dataclasses()
[rank3]:   File "/workspace/pypacks/transformers/hf_argparser.py", line 339, in parse_args_into_dataclasses
[rank3]:     obj = dtype(**inputs)
[rank3]:   File "", line 136, in init
[rank3]:   File "/workspace/pypacks/transformers/training_args.py", line 1693, in post_init
[rank3]:     self.device
[rank3]:   File "/workspace/pypacks/transformers/training_args.py", line 2171, in device
[rank3]:     return self._setup_devices
[rank3]:   File "/workspace/pypacks/transformers/utils/generic.py", line 60, in get
[rank3]:     cached = self.fget(obj)
[rank3]:   File "/workspace/pypacks/transformers/training_args.py", line 2108, in _setup_devices
[rank3]:     self.distributed_state = PartialState(**accelerator_state_kwargs)
[rank3]:   File "/workspace/pypacks/accelerate/state.py", line 280, in init
[rank3]:     self.set_device()
[rank3]:   File "/workspace/pypacks/accelerate/state.py", line 790, in set_device
[rank3]:     torch.cuda.set_device(self.device)
[rank3]:   File "/workspace/pypacks/torch/cuda/init.py", line 399, in set_device
[rank3]:     torch._C._cuda_setDevice(device)
[rank3]: RuntimeError: CUDA error: invalid device ordinal
[rank3]: CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
[rank3]: For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
[rank3]: Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.
[rank6]: Traceback (most recent call last):
[rank6]:   File "/workspace/minicpm/finetune/finetune.py", line 281, in
[rank6]:     train()
[rank6]:   File "/workspace/minicpm/finetune/finetune.py", line 162, in train
[rank6]:     ) = parser.parse_args_into_dataclasses()
[rank6]:   File "/workspace/pypacks/transformers/hf_argparser.py", line 339, in parse_args_into_dataclasses
[rank6]:     obj = dtype(**inputs)
[rank6]:   File "", line 136, in init
[rank6]:   File "/workspace/pypacks/transformers/training_args.py", line 1693, in post_init
[rank6]:     self.device
[rank6]:   File "/workspace/pypacks/transformers/training_args.py", line 2171, in device
[rank6]:     return self._setup_devices
[rank6]:   File "/workspace/pypacks/transformers/utils/generic.py", line 60, in get
[rank6]:     cached = self.fget(obj)
[rank6]:   File "/workspace/pypacks/transformers/training_args.py", line 2108, in _setup_devices
[rank6]:     self.distributed_state = PartialState(**accelerator_state_kwargs)
[rank6]:   File "/workspace/pypacks/accelerate/state.py", line 280, in init
[rank6]:     self.set_device()
[rank6]:   File "/workspace/pypacks/accelerate/state.py", line 790, in set_device
[rank6]:     torch.cuda.set_device(self.device)
[rank6]:   File "/workspace/pypacks/torch/cuda/init.py", line 399, in set_device
[rank6]:     torch._C._cuda_setDevice(device)
[rank6]: RuntimeError: CUDA error: invalid device ordinal
[rank6]: CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
[rank6]: For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
[rank6]: Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.
[rank2]: Traceback (most recent call last):
[rank2]:   File "/workspace/minicpm/finetune/finetune.py", line 281, in
[rank2]:     train()
[rank2]:   File "/workspace/minicpm/finetune/finetune.py", line 162, in train
[rank2]:     ) = parser.parse_args_into_dataclasses()
[rank2]:   File "/workspace/pypacks/transformers/hf_argparser.py", line 339, in parse_args_into_dataclasses
[rank2]:     obj = dtype(**inputs)
[rank2]:   File "", line 136, in init
[rank2]:   File "/workspace/pypacks/transformers/training_args.py", line 1693, in post_init
[rank2]:     self.device
[rank2]:   File "/workspace/pypacks/transformers/training_args.py", line 2171, in device
[rank2]:     return self._setup_devices
[rank2]:   File "/workspace/pypacks/transformers/utils/generic.py", line 60, in get
[rank2]:     cached = self.fget(obj)
[rank2]:   File "/workspace/pypacks/transformers/training_args.py", line 2108, in _setup_devices
[rank2]:     self.distributed_state = PartialState(**accelerator_state_kwargs)
[rank2]:   File "/workspace/pypacks/accelerate/state.py", line 280, in init
[rank2]:     self.set_device()
[rank2]:   File "/workspace/pypacks/accelerate/state.py", line 790, in set_device
[rank2]:     torch.cuda.set_device(self.device)
[rank2]:   File "/workspace/pypacks/torch/cuda/init.py", line 399, in set_device
[rank2]:     torch._C._cuda_setDevice(device)
[rank2]: RuntimeError: CUDA error: invalid device ordinal
[rank2]: CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
[rank2]: For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
[rank2]: Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.
[rank4]: Traceback (most recent call last):
[rank4]:   File "/workspace/minicpm/finetune/finetune.py", line 281, in
[rank4]:     train()
[rank4]:   File "/workspace/minicpm/finetune/finetune.py", line 162, in train
[rank4]:     ) = parser.parse_args_into_dataclasses()
[rank4]:   File "/workspace/pypacks/transformers/hf_argparser.py", line 339, in parse_args_into_dataclasses
[rank4]:     obj = dtype(**inputs)
[rank4]:   File "", line 136, in init
[rank4]:   File "/workspace/pypacks/transformers/training_args.py", line 1693, in post_init
[rank4]:     self.device
[rank4]:   File "/workspace/pypacks/transformers/training_args.py", line 2171, in device
[rank4]:     return self._setup_devices
[rank4]:   File "/workspace/pypacks/transformers/utils/generic.py", line 60, in get
[rank4]:     cached = self.fget(obj)
[rank4]:   File "/workspace/pypacks/transformers/training_args.py", line 2108, in _setup_devices
[rank4]:     self.distributed_state = PartialState(**accelerator_state_kwargs)
[rank4]:   File "/workspace/pypacks/accelerate/state.py", line 280, in init
[rank4]:     self.set_device()
[rank4]:   File "/workspace/pypacks/accelerate/state.py", line 790, in set_device
[rank4]:     torch.cuda.set_device(self.device)
[rank4]:   File "/workspace/pypacks/torch/cuda/init.py", line 399, in set_device
[rank4]:     torch._C._cuda_setDevice(device)
[rank4]: RuntimeError: CUDA error: invalid device ordinal
[rank4]: CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
[rank4]: For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
[rank4]: Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.
[rank7]: Traceback (most recent call last):
[rank7]:   File "/workspace/minicpm/finetune/finetune.py", line 281, in
[rank7]:     train()
[rank7]:   File "/workspace/minicpm/finetune/finetune.py", line 162, in train
[rank7]:     ) = parser.parse_args_into_dataclasses()
[rank7]:   File "/workspace/pypacks/transformers/hf_argparser.py", line 339, in parse_args_into_dataclasses
[rank7]:     obj = dtype(**inputs)
[rank7]:   File "", line 136, in init
[rank7]:   File "/workspace/pypacks/transformers/training_args.py", line 1693, in post_init
[rank7]:     self.device
[rank7]:   File "/workspace/pypacks/transformers/training_args.py", line 2171, in device
[rank7]:     return self._setup_devices
[rank7]:   File "/workspace/pypacks/transformers/utils/generic.py", line 60, in get
[rank7]:     cached = self.fget(obj)
[rank7]:   File "/workspace/pypacks/transformers/training_args.py", line 2108, in _setup_devices
[rank7]:     self.distributed_state = PartialState(**accelerator_state_kwargs)
[rank7]:   File "/workspace/pypacks/accelerate/state.py", line 280, in init
[rank7]:     self.set_device()
[rank7]:   File "/workspace/pypacks/accelerate/state.py", line 790, in set_device
[rank7]:     torch.cuda.set_device(self.device)
[rank7]:   File "/workspace/pypacks/torch/cuda/init.py", line 399, in set_device
[rank7]:     torch._C._cuda_setDevice(device)
[rank7]: RuntimeError: CUDA error: invalid device ordinal
[rank7]: CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
[rank7]: For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
[rank7]: Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.
config.json: 100%|████████████████████████████████████████████████████████| 1.37k/1.37k [00:00<00:00, 6.74MB/s]
configuration_minicpm.py: 100%|███████████████████████████████████████████| 4.06k/4.06k [00:00<00:00, 20.8MB/s]
A new version of the following files was downloaded from https://huggingface.co/openbmb/MiniCPM-Llama3-V-2_5:

configuration_minicpm.py
. Make sure to double-check they do not contain any added malicious code. To avoid downloading new versions of the code file, you can pin a revision.
A new version of the following files was downloaded from https://huggingface.co/openbmb/MiniCPM-Llama3-V-2_5:
configuration_minicpm.py
. Make sure to double-check they do not contain any added malicious code. To avoid downloading new versions of the code file, you can pin a revision.
modeling_minicpmv.py: 100%|███████████████████████████████████████████████| 25.0k/25.0k [00:00<00:00, 58.2MB/s]
resampler.py: 100%|███████████████████████████████████████████████████████| 35.8k/35.8k [00:00<00:00, 80.6MB/s]
A new version of the following files was downloaded from https://huggingface.co/openbmb/MiniCPM-Llama3-V-2_5:
resampler.py
. Make sure to double-check they do not contain any added malicious code. To avoid downloading new versions of the code file, you can pin a revision.
A new version of the following files was downloaded from https://huggingface.co/openbmb/MiniCPM-Llama3-V-2_5:
modeling_minicpmv.py
resampler.py
. Make sure to double-check they do not contain any added malicious code. To avoid downloading new versions of the code file, you can pin a revision.
A new version of the following files was downloaded from https://huggingface.co/openbmb/MiniCPM-Llama3-V-2_5:
resampler.py
. Make sure to double-check they do not contain any added malicious code. To avoid downloading new versions of the code file, you can pin a revision.
A new version of the following files was downloaded from https://huggingface.co/openbmb/MiniCPM-Llama3-V-2_5:
modeling_minicpmv.py
resampler.py
. Make sure to double-check they do not contain any added malicious code. To avoid downloading new versions of the code file, you can pin a revision.
W0720 21:08:52.136000<PHONE_NUMBER>92768 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1751 closing signal SIGTERM
W0720 21:08:52.137000<PHONE_NUMBER>92768 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1752 closing signal SIGTERM
E0720 21:08:52.466000<PHONE_NUMBER>92768 torch/distributed/elastic/multiprocessing/api.py:826] failed (exitcode: 1) local_rank: 2 (pid: 1753) of binary: /usr/bin/python
Traceback (most recent call last):
File "/usr/local/bin/torchrun", line 8, in
sys.exit(main())
File "/workspace/pypacks/torch/distributed/elastic/multiprocessing/errors/init.py", line 347, in wrapper
return f(*args, **kwargs)
File "/workspace/pypacks/torch/distributed/run.py", line 879, in main
run(args)
File "/workspace/pypacks/torch/distributed/run.py", line 870, in run
elastic_launch(
File "/workspace/pypacks/torch/distributed/launcher/api.py", line 132, in call
return launch_agent(self._config, self._entrypoint, list(args))
File "/workspace/pypacks/torch/distributed/launcher/api.py", line 263, in launch_agent
raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:
============================================================
finetune.py FAILED

Failures:
[1]:
time      : 2024-07-20_21:08:52
host      : 584a1774aaad
rank      : 3 (local_rank: 3)
exitcode  : 1 (pid: 1754)
error_file: <N/A>
traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
[2]:
time      : 2024-07-20_21:08:52
host      : 584a1774aaad
rank      : 4 (local_rank: 4)
exitcode  : 1 (pid: 1755)
error_file: <N/A>
traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
[3]:
time      : 2024-07-20_21:08:52
host      : 584a1774aaad
rank      : 5 (local_rank: 5)
exitcode  : 1 (pid: 1756)
error_file: <N/A>
traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
[4]:
time      : 2024-07-20_21:08:52
host      : 584a1774aaad
rank      : 6 (local_rank: 6)
exitcode  : 1 (pid: 1757)
error_file: <N/A>
traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
[5]:
time      : 2024-07-20_21:08:52
host      : 584a1774aaad
rank      : 7 (local_rank: 7)
exitcode  : 1 (pid: 1758)
error_file: <N/A>
traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
Root Cause (first observed failure):
[0]:
time      : 2024-07-20_21:08:52
host      : 584a1774aaad
rank      : 2 (local_rank: 2)
exitcode  : 1 (pid: 1753)
error_file: <N/A>
traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
root@584a1774aaad:/workspace/minicpm/finetune#
期望行为 | Expected Behavior
Finetuning  to not have CUDA errors
复现方法 | Steps To Reproduce
Take a few rows and images per the  vl_finetune_data.json and try to run ./finetune_lora.sh and run on unbuntu
运行环境 | Environment
- OS:ubuntu 22.04
- Python:3.10.12
- Transformers:4.42.4
- PyTorch:2.3.1
- CUDA (`python -c 'import torch; print(torch.version.cuda)'`):12.1

备注 | Anything else?
No response

Because I see that your machine only has two graphics cards, is this how GPUS_PER_NODE=2 is written in your fine-tuned bash script?


Bring back dev branch, re-enable storage emulator tests

cc @robdmoore
P.s. I see I have editor tab/space issues here, I'll normalise them separately in dev branch.

OK this seems to be ready for merge into dev. We might have to live with some instability in the CI, as I noticed the first build failed with a strange error related to local storage emulator.

This looks good. Will the emulator tests run on app veyor or is that still a todo?

@robdmoore they work OOTB on the vs2015 image, surprisingly :+1:

thanks @nblumhardt - they should be fixed now

Merge me please :smiley_cat:


Array multiplication or matrix multiplication?

In line of 108 and 110 of histwords/representations/embedding.py it seems that you use array multiplication instead of matrix multiplication. Is this intended? Because usually s and u are combined by using matrix multiplication, I think.

I guess I found the reason: randomized_svd() doesn't output a m x m matrix but a flattened version of it. Array multiplication of the flattened version and matrix multiplication of its diagonalization are equivalent.


Migrated to Spring 3.0

What does this PR do ?

I've migrated the application from Spring 2.0 to Spring 3.0 using Open Rewrite
Ran all the test cases to make sure everything is working as expected

Fixes #44

(I've attached a single screenshot only)

@Shiroo77 looks good so far! Could we unify all the changes under a single commit?

More info about migrating to Spring Boot 3.0 with Open Rewrite: https://docs.openrewrite.org/recipes/java/spring/boot3/upgradespringboot_3_0


Package Update Umbrella Issue

[ ] keep field ordering when updating #240
[ ] sophisticated resource merging during updates

raise error on conflicts between updated, common ancestor, and local
allow users to specify per-field merge strategies -- always updated, always local

1 was due to the format step after update which was fixed a while ago.
resource-merge already takes care of 3-way merge and other requests are not relevant in the current context.


Fix installing on OSX

I did some more reading and found a similar issue in SDL itself
http://lists.libsdl.org/pipermail/commits-libsdl.org/2011-September/012828.html
https://twitter.com/elmindreda/status/398479855324372992
It seems as if apple have defined both buffer objects as long
typedef long GLintptrARB;
typedef long GLsizeiptrARB;

Setting as x64 only resulted in the same error being generated - this does raise a question if this ever worked on osx?

I'm pretty sure this has worked on osx before since I know people wood used it there, but I haven't used it there myself (I've tested it on windows with 32 bit and Ubuntu with 64 bit).

Think I've got this sorted in my own fix. Tell me if you still have problems.
Thanks for your efforts!


Help wanted with packages unikraft images

I was attempting to package some of my unikraft applications with Bima to run with urunc, but I am struggling to understand how to write my ContainerFile to package them with Bima.
Could you please guide me on some examples of ContainerFiles for some of the examples in the Unikraft Catalog, such as

HTTP Application in C++
Python 3.10 Flask HTTP Application

Thanks in Advance!!

Hello @alanpjohn , thank you for trying out our tool.
If you simply want to get Unikraft catalog images and run them in urunc we would recommend to use pun. We built this tool for simply obtaining unikraft images from unikraft's registry and adding the necessary annotations for urunc.
If you want to build your own unikernel image with unikraft and then package it for urunc you can still use pun or check bima's README.
In both cases you will need to specify the annotations for urunc. Therefore let me go over the available annotations:
Required:

com.urunc.unikernel.unikernelType: The type of the unikernel (can be rumprun, unikraft, etc)
com.urunc.unikernel.hypervisor: The desired hypervisor to run the unikernel (e.g. qemu, firecracker, hvt)
com.urunc.unikernel.binary: The path to the unikernel binary inside the container image
com.urunc.unikernel.cmdline: The cmdline of the application that will run inside the unikernel.

Except of the above, urunc accepts the following optional annotations:

com.urunc.unikernel.version: The version of the framework used to build the unikernel (e.g. 0.17.0). If you use newer unikraft versions you should definitely use this annotation.
com.urunc.unikernel.initrd: The path to the initrd of the unikernel inside the container image.
com.urunc.unikernel.block: The path to a block image, in order to attach to the unikernel
com.urunc.unikernel.blkMntPoint: The mount point of the block image inside the unikernel

Therefore for the C++ HTTP application the annotations could be:
LABEL com.urunc.unikernel.binary="path/to/unikernel/binary"
LABEL "com.urunc.unikernel.cmdline"="run /http_server" ## obtained from the Kraftfile,
LABEL "com.urunc.unikernel.unikernelType"="unikraft"
LABEL "com.urunc.unikernel.hypervisor"="qemu"

For the cmdline you may omit run as soon as this PR get merged in urunc.
Depending on either you build from scratch the unikraft image or not you might need to also use the COPY command as specified in bima's README.

Hi @cmainas, thank you for responding.
I built the native/http-c example for the unikraft demo without using pun by following some instructions on https://github.com/nubificus/urunc/issues/52 but most of the applications that I have built on unikraft run similarly to how examples/http-cpp examples/http-go examples/http-python-3.12-flask, I am struggling a bit in packaging these with pun. For examples/http-cpp, after building the initrd, I try to package with pun using the following ContainerFile based on instructions here
FROM unikraft.org/base:latest

COPY ./.unikraft/build/initramfs-x86_64.cpio /unikraft/initrd
LABEL "com.urunc.unikernel.binary"="/unikraft/bin/kernel"
LABEL "com.urunc.unikernel.cmdline"="run http_server"
LABEL "com.urunc.unikernel.unikernelType"="unikraft"
LABEL "com.urunc.unikernel.hypervisor"="qemu"

LABEL "com.urunc.unikernel.initrd"="/unikraft/initrd"

And then run pun to get the following error
pun -f Containerfile | sudo buildctl build --local context=${PWD} --output "type=oci" > output.tar
error: failed to parse input: proto: illegal wireType 6

I have tried different buildkit outputs but all of them have the same line of error. I am quite new to buildkit and buildctl so I am not sure if I am using them wrong. Also not sure if this is question better asked in the pun issues board.
Here are the details of my environment:

Application
Version Details

buildkitd
buildkitd github.com/moby/buildkit v0.17.0 fd61877fa73693dcd4ef64c538f894ec216409a3

buildctl
buildctl github.com/moby/buildkit v0.17.0 fd61877fa73693dcd4ef64c538f894ec216409a3

pun
https://github.com/nubificus/pun/tree/e661806a7b2fc35a7e9117a991728fc6acda4bef

docker
27.3.1

Hello @alanpjohn
it seems the README in pun was not up-to-date. We have changed the default behavior to be a buildkit frontend and not printing a LLB. As a result, the buildctl command in the README was not correct. It is still possible to print the LLB with pun but we need to use the --LLB argument. We have updated the README of the repo.
As such, I was able to build the http-cpp example following your steps. In particular:

Cloned unikraft's catalog repository git clone https://github.com/unikraft/catalog.git
Go to http-cpp example cd catalog/examples/http-cpp/
Build it with kraft kraft build --no-cache --no-update --plat qemu --arch x86_64
Create the following Containerfile

#syntax=harbor.nbfc.io/nubificus/urunc/pun/llb:latest
FROM unikraft.org/base:latest

COPY ./.unikraft/build/initramfs-x86_64.cpio /unikernel/initramfs-x86_64.cpio

LABEL com.urunc.unikernel.binary="/unikraft/bin/kernel"
LABEL "com.urunc.unikernel.cmdline"="/http_server"
LABEL "com.urunc.unikernel.initrd"="/unikernel/initramfs-x86_64.cpio"
LABEL "com.urunc.unikernel.unikernelType"="unikraft"
LABEL "com.urunc.unikernel.unikernelVersion"="0.17.0"
LABEL "com.urunc.unikernel.hypervisor"="qemu"

Build it with docker build -f Containerfile_http_cpp -t harbor.nbfc.io/nubificus/urunc/http-cpp-unikraft-qemu:build . or

./pun -LLB -f Containerfile_http_cpp | sudo buildctl --addr docker-container://buildkitd build ... --local context=${PWD} --output type=docker,name=harbor.nbfc.io/nubificus/urunc/http-cpp-unikraft-qemu:buildctl | sudo docker load

Run it with sudo docker run --rm -it --runtime io.containerd.urunc.v2 harbor.nbfc.io/nubificus/urunc/http-cpp-unikraft-qemu:build

Output:
SeaBIOS (version 1.13.0-1ubuntu1.1)

iPXE (http://ipxe.org) 00:02.0 C000 PCI2.10 PnP PMM+0FF8C820+0FECC820 C000

Booting from ROM..1: Set IPv4 address <IP_ADDRESS> mask <IP_ADDRESS> gw <IP_ADDRESS>
en1: Added
en1: Interface is up
Powered by Unikraft Calypso (0.17.0~62132e5)
Listening on port 8080

Some notes:

In the cmdline annotation, you need to use /, otherwise unikraft's elfloader was not able to find the binary
Since https://github.com/nubificus/urunc/pull/72 is merged, you can directly use /http_server as cmdline.
I was running the buildkit in a docker container, hence I had to pass the address in the builctl command. I run buildkit with docker run -d --name buildkitd --privileged moby/buildkit:latest

Hi @cmainas
This was helpful; thank you so much. I was able to fix, build and deploy on urunc. You can go ahead and close this issue.

Hello @alanpjohn ,
we are glad that you were able to fix any issues and even deploy them in urunc. I will close the issue as you suggest. Feel free to open it again if a similar issue arises.


VS 2017 CMAKE Build Issues

Hi I try to get CMAKE gui to create build. But encounter library not found.

Build the source will encounter this error below.

Two questions:

Did you set up OpenSSL in the right place (or at all)?
Can you paste the full text of the VS 2017 compile errors, as right now I don't see what file+line the error concerns

Also try with my newest commits to the stable-ish branch

@vutle, if NASTY_OPENSSL_WORKAROUND is ON, then (as described in Wiki article), should openssl's root dir should be in .. relatively to Irrlicht's root dir. Since this is Windows and you never know how things are, currently cmake script assumes that openssl dir looks like this:
irrlicht_root/../openssl/
--- lib/
------ libeay32.lib
--- include/
------ openssl header files...

The other way is to uncheck this NASTY_OPENSSL_WORKAROUND option and manually provide paths to eay lib to SSL_EAY_DEBUG, SSL_EAY_RELEASE, LIB_EAY_DEBUG and LIB_EAY_RELEASE cmake vars and path to include dir to OPENSSL_INCLUDE_DIR.
As for errors you experience: apparently MSVC is a little buggy and doesn't inherit default constructor. Already fixed, so you can build from my fork, branch fix or wait wait until @devshgraphicsprogramming merge my PR and build from his stable-ish branch.

Once I verify it builds on my windows laptop I close the issue

@Crisspl I have a massive error with my cmake-gui, it generates win32 visual studio targets not x86!
Why is it trying to make a 32bit build target instead of 64bit?

I think, you have to choose other compiler/IDE option in cmake-gui if i remember correctly (there are 2 distinct options for x86 and x86_64). I had same issue once and it was the problem. Apprently i forgot to place it into the wiki, sorry


per-tag and per-series user access control

It would be great to have a hierarchical user access control, and for that I think tags can act like Unix user groups. If for example I make sure all records have a tag called group_id. Then will be useful to give user foo access to all records that their associated group_id is e.g. 1 but give user manager access to groups of [1,3,10].
Lets assume there is a forward-looking electricity company that wants to engage the users by giving them more data. If for example the sensor is sensing apartment electricity consumption (series be electricity.watts and tags be group_id=1), each tenant should be able to query her own electricity consumption (perhaps do average in month, join with electricity.bills, ...) but the electricity company may need to do higher level data mining and should be able to access all homes and apartments and commercial places. A renting apartment complex manager should be able to only see apartments in his complex.
One way to control this hierarchy would be to have separate databases, but question becomes what is the granularity? Do we need one database per apartment/user?
Having many databases makes the job of the electricity company harder because they will need to join from multiple databases (not even different tables), which bring up per-series user access control: per-series authentication is better than per-database because series are like tables.
There is nothing set in stone and there may be better ways to handle this, any idea is appreciated.

This level of access control is out of scope of the database for us. We'll be putting out a product later this year that will give you this level of control, but it's something that will sit outside the database itself and act as a proxy. Like ElasticSearch's Shield product.
In the meantime, the best thing to do is to create an authorization proxy on your own that gives you the level of control you need.

+1


Installation added to readme file

As an API v2 user, I encountered with this problem, I needed to install it but since there wasn't clear information about if it is available in pip I tried and saw that it is available, but with adding these tutorial lines it would be easier for the new users. It also includes the downloading and installing it via setup.py version of the installation.
Need to be added:

[ ] Features
[ ] Dependencies

Waiting for review 👍

Thank you for the contribution!


[Bug] Radon Crash when the Host Turtle is using an Inventory Directly Adjacent

If you have an Inventory directly near the Host Turtle then radon correctly shows the amounts available, tho if you try buying one of those items radon crashes like follows:

The inventory and turtle need to be networked together, can't be a directly wrapped peripheral.

It would be nice to have this scenario handled with a proper descriptive error message.


There are multiple Python2 files in the source code

After noticing some python2 syntax in the source code, I ran this script
import ast
import pathlib

for path in pathlib.Path(".").glob("**/*.py"):
    try:
        ast.parse(path.read_text())
    except:
        print(path)

to identify these 3 files:

Code/GraphMol/Descriptors/test3D.py
Code/GraphMol/Descriptors/test3D_old.py
External/Eigen/eigen/scripts/relicense.py

These all seem related to 3D descriptors and haven't seen much action since around 2017.

I don't think we're going to fix the Eigen file, but closing this case since @mcs07 removed the other two files!


Big output is slow output.

So, I've experienced this, but not quite traced it through yet.
Reactotron gets really slow and pegs the CPU sometimes.  I'm not sure if its related to the code colourizing, web sockets, or something tragic I've done.

On the CLI, lets detect a maximum size we're cool with and render larger ones with a warning instead of spitting everything out.

Planning on deprecating CLI shortly.


Change target page depending on server/client rendered?

Hi! I'm really struggling to wrap my head around how I can conditionally config my routes depending on whether they are rendered on the server or client.
Given these two routes:
routes.add('knowledge', '/knowledge', 'knowledge')
routes.add('article', '/knowledge/:id-:slug', 'article')

On the client side, I want the article to open in an overlay but have the URL updated appropriately, so I'd like the target page to be 'knowledge', same as the index of articles. If the URL is accessed directly, I'd like the server to render the knowledge page.
Any ideas of how I can achieve this?

There is an example app nextgram that does something similar, maybe that helps?

@fridays ah! Perfect, many thanks

@fridays can you clarify how to do this with next-routes? Particularly this line:
https://github.com/now-examples/nextgram/blob/master/pages/index.js#L40
Seems like the first string is saying "let's stay on the same route and add in this query param", and the second string is saying "let's act like we're going to navigate to this URL, but don't actually navigate. let's just change the URL to look like this."
Not sure how to mimic this with Router.pushRoute(...)

@howells were you by chance able to figure this one out?

@joncursi afraid not!


[DOCFIX]Update docs/cn/Running-Alluxio-On-Docker.md

[DOCFIX]Update docs/cn/Running-Alluxio-On-Docker.md

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/18501/
Test PASSed.


Bad result code: 0 alert in file manager

Samsung Galaxy S5 Android 6.0.1
Huaway Y6 Pro  5.1

When I open file manager with DocumentPicker.show(), then go into some folder and then click system back button, system show me alert: Bad result code: 0. I cannot fix it(((

Oh have to look into that!

resultCode 0 is the result code for user cancellation of the picker, RNDP doesn't handle it.
I've added a proper cancellation error for both Android and iOS and a simple way of checking for them: https://github.com/Elyx0/react-native-document-picker/pull/77/commits/8648b12b12cac2ae1f27e89d45319382cc09937d

Several users are also seeing this resultCode 0 in my application, even when they did not cancel: https://github.com/laurent22/joplin/issues/177
Not sure what the issue is, maybe the module is incorrectly interpreting the response from the Android file manager. Unfortunately I cannot replicate this in the simulator.

I also receive "Bad result code: 0" when invoking DocumentPicker.show() on Android 4.2.2 ASUS tablet. However, on Android 7.0 Samsung Galaxy J7 everything is ok with the same code.

It seems it can happen on any device, but I've never managed to replicate it on an emulator. Some users have reported it happens on relatively recent devices like Pixel 2 running Oreo 8.1.


Re-PID Tuning Subsystem

Implementing PIDF tuning to the Arm, Wrist, and Shooter and retuning said subsystems.

[x] Arm PIDF
[x] Arm tuned
[x] Wrist PIDF
[x] Wrist tuned
[x] Shooter PIDF
[x] Shooter tuned

Merged into dev


Is possible to run your module with azure Identity?

Hello.
I need to run project on VM in azure cloud with identity.
I defined environment variables export ARM_USE_MSI="true" and export ARM_SKIP_PROVIDER_REGISTRATION="true"
Did:
az login --identity

And when tried to run apply I got the error:
╷
│ Error: Error in function call
│
│   on .terraform/modules/caf/locals.tf line 224, in locals:
│  224:   object_id = coalesce(var.logged_user_objectId, var.logged_aad_app_objectId, try(data.azurerm_client_config.current.object_id, null), try(data.azuread_service_principal.logged_in_app.0.object_id, null))
│     ├────────────────
│     │ data.azuread_service_principal.logged_in_app is empty tuple
│     │ data.azurerm_client_config.current.object_id is ""
│     │ var.logged_aad_app_objectId is null
│     │ var.logged_user_objectId is null
│
│ Call to function "coalesce" failed: no non-null, non-empty-string
│ arguments.
╵

Or maybe do I have to configure some additional rights? I tried to create resource group. Without caf module, I can do it.
Terraform code:
variable "tags" {
  type = map(string)
}

variable "subscription_id" {
  type = string
}

variable "caf_global_settings" {
  type = object({})
}

variable "resource_groups" {
  type = object({})
}

locals {
  caf_p = tomap({"prefixes" = [var.tags["Environment"]]})
}

data "azurerm_subscription" "subs" {
  subscription_id = var.subscription_id
}

module "caf" {
  source  = "aztfmod/caf/azurerm"
  #version = "5.4.1"

  global_settings = merge(var.caf_global_settings, local.caf_p)
  resource_groups = var.resource_groups
  tags = merge(data.azurerm_subscription.subs.tags, var.tags)
}

Try to add:
export logged_user_objectId="object_id of the MSI"

Closing as solved.


Fix login with amazon and integrate paypal

@jitendra Vinsol's fork is behind spree-contrib repo. So, you will be seeing commits from other authors also.

@nitin007 - Can you close this one and create a new PR.
As this fork is not updated earlier and has too many not required commits.

Sure


Concern regarding possible resource leak

I have a concern regarding the following example:
var options = new DbContextOptionsBuilder<BloggingContext>()
    .UseSqlServer(new SqlConnection(connectionString))
    .Options;

We can see that a new connection object is passed the the DbContextOptionsBuilder - should this connection be disposed at some point? I have examined the ef core code but I couldn't find a confirmation that this provided connection is actually disposed at some point. So the example should be changed to mention that the connection should be disposed of at some point. If a factory is used, then this factory should track the connections in a weak reference and dispose them when itself is disposed.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 1073c238-9389-d826-a696-f1108a4f2ce0
Version Independent ID: cf56babd-def0-d76e-8eee-e33341b6a009
Content: Transactions - EF Core
Content Source: entity-framework/core/saving/transactions.md
Product: entity-framework
Technology: entity-framework-core
GitHub Login: @roji
Microsoft Alias: avickers

@akovac35 Yes, this is a leak. If the application creates the DbConnection, then the application is responsible for disposing it.
/cc @roji

Was this me? :fearful:
Pushed a fix into #2882


Add constant time compare to xsrf

This change is

Going to merge this now.  Please take a look when you can.

:+1:


Not working at all

Not scanning or fuzzing but only displaying wordlist

Hi @krivadna, the releases are working properly, if you are having issues running ffuf, please provide full screenshots, the command you are running, and some details about your environment.
Feel free to re-open this same issue when you have it.


RTSP protocol

hello, could this project support RTSP protocol?
I tried some RTSP uris but  it can't work.

RTSP is a supported network protocol so it should work. Are you sure the stream uses supported codecs? Can you provide a test URI?

I tried to test my onvif device but it didn't work. URI is like rtsp://[username]:[password]@192.168.x.x:554/... , would you tell me what's happened?

I'm sorry but that is obviously not an URI I can test.

Hello, I think I found a solution which I tried to do Basic Authentication with header but I don't know how to add header in your code. Would you tell me how to do this?
http://cruxintw.blogspot.tw/2015/03/android-mediaplayer-rtsp-streaming.html

You can use one of these methods and supply the headers as a map:

MediaPlayer#setDataSource(new UriSource(Context context, Uri uri, Map<String, String> headers))
MediaPlayer#setDataSource(Context context, Uri uri, Map<String, String> headers)
VideoView#setVideoSource(new UriSource(Context context, Uri uri, Map<String, String> headers))
VideoView#setVideoURI(Uri uri, Map<String, String> headers)


IdentityServer is not returning "Email" as part of id_token.

I have added Email as part of GetIdentityResources() in Config.cs as well as AllowedScopes  but id_token doesn't contain user email at all.
decoded id_token : it only contains id not email
{
"nbf":<PHONE_NUMBER>,
"exp":<PHONE_NUMBER>,
"iss": "https://******com",
"aud": "",
"nonce": "*******-574a-477c-af85-510b5e5f7fd",
"iat": **7847177,
"c_hash": "****WgKjTnCqOwf4TQ",
"sid": "***be84668c81e19a21ae56824c6",
"sub": "a0e5850c-8c9a-43e4-9b39-da6293291f77",
"auth_time":<PHONE_NUMBER>,
"idp": "local",
"amr": [
"pwd"
]
}
How can I make it return Email in id_token

For some reason Identity Server doesn't return user details in id token when both id token and access token are requested in the same implicit flow request. If you just request id token, then user detail claims will be issued. If you request both tokens, then afterwards you need to call userinfo endpoint to get user details.

For some reason Identity Server doesn't return user details in id token when both id token and access token are requested in the same implicit flow request. If you just request id token, then user detail claims will be issued. If you request both tokens, then afterwards you need to call userinfo endpoint to get user details.

That's per-spec.

I have added Email as part of GetIdentityResources() in Config.cs as well as AllowedScopes but id_token doesn't contain user email at all.

You need to configure the user claims you want in the corresponding tokens via the IdentityResource or ApiResources. That's covered in our docs.

This seems to be a general question about IdentityServer - not a bug report or an issue.
Please use one of the our free or commercial support options
See here for more details.
Thanks!


Discussion: UseDefaultHostingConfiguration has been removed

Discussion for https://github.com/aspnet/Announcements/issues/174

Does this mean, that the environment is no longer set, unless you explicitly add the environment variables? or is there an exception for this one case? (I think I remember an issue about that?!)

Oh good catch, let me update the announcement

I've just seen that's this is one of the referenced issues - but having this in the announcement would be good IMO, yes.

In addition ASPNETCORE_ENVIRONMENT is included as a fallback if you don't add any environment variables yourself.

Still a bit confusing in my opinion. Do you really check, if we've called AddEnvironmentVariables or is the fallback only used, when we don't call UseConfiguration at all?
Or in other words: if I use just var config = new ConfigurationBuilder().AddCommandLine(args); and pass this to UseConfiguration, will you still read the hosting-environment from the ASPNETCORE_ENVIRONMENT environment variable?

@cwe1ss :+1:
My situation is to use .UseEnvironment(...). Testing seems to indicate that doing it this way, the env var is set without having .UseConfiguration(...) at all. Yes, indeed, a little more detail on the processing model would be helpful: In addition to what @cwe1ss is asking, how does .UseEnvironment(...) play into this both with and without .UseConfiguration(...)? ... I have the gut feeling you are going to say "last one called wins."

@BrennanConroy
This caused hosting.json and "dotnet run /server.urls http://xxxx:5002" not to work any more.

Souldn't the example code should load variables backwards? First JSON, then Environment and finally CommandLine. I know it's an example but it's more consistent with previous behavior.
var config = new ConfigurationBuilder()
    .AddJsonFile("hosting.json", optional: true)
    .AddEnvironmentVariables(prefix: "ASPNETCORE_")
    .AddCommandLine(args)
    .Build();

@mcliment Changed for consistency
@xatabhk That's because they are no longer default and you need to follow the example
@cwe1ss We check if you've configured any environment variables (i.e. via AddEnvironmentVariables), if you haven't configured any then ASPNETCORE_ENVIRONMENT is read

This is really asinine.
We want sane defaults. Convention over configuration.

The defaults are sane. The most relevant configuration setting is environment and we kept it. The others can be configured in your application. You don't need it to be as complicated as the above code snippet, that adds no value. If you care about configuring some of the supported values from a configuration file, then make the configuration from that file and call use configuration.

@davidfowl
c# .UseDefaultHostingConfiguration(args)
vs
`c#
var config = new ConfigurationBuilder()
.AddJsonFile("hosting.json", optional: true)
.AddEnvironmentVariables(prefix: "ASPNETCORE_")
.AddCommandLine(args)
.Build();
var host = new WebHostBuilder()
.XYZ()
.UseConfiguration(config)
.ZYX()
.Build()
`
6 lines vs 1 line. How is this an improvement? Is the args that are omitted in UseDefaultHostingConfiguration(args) extremely cumbersome?

@dotnetchris most apps we've seen do not need or want all three config sources. They also want to change the name/location of the json file. UseDefaultHostingConfiguration was convenient, but did not offer enough granular control.

On OS X - using current RC2-  I have to incorporate SetBasePath in order to load hosting.json:
https://github.com/aspnet/Hosting/issues/761

@Tratcher if the information you provided was included in the original message a month~ ago this might be more obvious. instead the communication was "take 1 line, make it 6, ENJOY"
Also useful would have been better illustrations of real world usage, in addition to only communicating how to mirror what it previously provided.

Hi there. Hope this is a good thread to ask for the following:
Is this change from server.urls to just urls documented somewhere?
Thanks

@davidfowl

We are closing this issue because no further action is planned for this issue. If you still have any issues or questions, please log a new issue with any additional details that you have.


Reduce Size: Replace Zod with Valibot

Zod makes up 30% of the size of better-commits. Zod is crucial to better-commits, so it shouldn't simply be removed. But, better-commits is a very small project and uses a minimal amount of the Zod API.
Thus, replacing Zod with Valibot seems like a pretty good choice for a project of this size.
Action Items

[ ] Replace Zod with Valibot
[ ] Replace Zod-Validation-Error
[ ] Remove Zod

Caveat: Better-commits makes use of advanced utilities like refine and transform. See this documentation for replacements. Namely, custom and transform
Caveat: Unsure of effort needed to replace Zod-Validation-Error

Hi @Everduin94 ,
I would like to do this task. I will come up with my findings. And then, can you please assign this issue to me?

@ravvi-kumar - Sure thing, just a heads up. Based on how the project is written, this is a deceptively large task.

Hi @Everduin94,
I'm hoping to test the recent changes locally. Is there a recommended workflow for this?
Additionally, I'm currently encountering a blocker related to returning clear custom error messages .
while I've been exploring the Valibot documentation on custom validation (custom validation documentation: https://valibot.dev/api/custom/), I'm still unsure about how to return clear error message like you are doing with zod's refine method.
Could you provide some guidance on how to achieve this?

@ravvi-kumar - Hey this looks like a great start. Give me a few days to get to helping on this.
This might help you on the custom error: https://github.com/fabian-hiller/valibot/discussions/361 - If not, I would post a question there
To test locally: npm run start. - I would either comment out the part that does git commit or just cancel every commit.

@Everduin94 -
hope this works 🤞,
Btw, the test was passed https://github.com/fabian-hiller/valibot/pull/494

Hi @Everduin94 ,
can you please review this PR https://github.com/Everduin94/better-commits/pull/89

@ravvi-kumar - Sorry for the delay. I'll get to this when I have the chance. I've been busy at work + I have a 5 month old baby  + I have a bjj tournament in a week. I bit off a bit more than I can chew at the moment 😅

@Everduin94 - congrats man and wish you good luck for you match. no issue, we can continue whenever you get a chance. 😊


CLOUDP-66613: mongocli datalake delete

Proposed changes
Jira ticket: datalake delete
Closes #[issue number]
Checklist

[] I have signed the MongoDB CLA
[] I have added tests that prove my fix is effective or that my feature works
[] I have added any necessary documentation (if appropriate)
[] I have run make fmt and formatted my code

Further comments

FYI once we have create and delete that's the point where it would be nice to have some e2e testing going


Can not be fixed support version

All com.android.support libraries must use the exact same version specification (mixing versions can lead to runtime crashes). Found versions 27.0.1, 26.1.0. Examples include com.android.support:animated-vector-drawable:27.0.1 and com.android.support:cardview-v7:26.1.0 less... (Ctrl+F1)
There are some combinations of libraries, or tools and libraries, that are incompatible, or can lead to bugs. One such incompatibility is compiling with a version of the Android support libraries that is not the latest version (or in particular, a version lower than your targetSdkVersion.)

@gedoor add the following dependencies to your app:
    implementation "com.android.support:appcompat-v7:27.0.1"
    implementation "com.android.support:support-compat:27.0.1"
    implementation "com.android.support:support-fragment:27.0.1"

Please add this in the setup instructions as well, because without these dependencies, the layout of the app is disturbed and text in the appbar has not correct style.


No way to find the 'purge' command

[Enter feedback here]
I followed the doc and using the latest version of Azure CLI, even tried it in Cloud Shell, but still the same error like this:
2021/08/31 04:04:04 Executing step ID: acb_step_0. Timeout(sec): 600, Working directory: '', Network: 'acb_default_network'
2021/08/31 04:04:04 Launching container with name: acb_step_0
ERROR: 'purge' is misspelled or not recognized by the system.
TRY THIS:
https://aka.ms/cli_ref
Read more about the command in reference docs
2021/08/31 04:04:09 Container failed during run: acb_step_0. No retries remaining.
failed to run step ID: acb_step_0: exit status 2
Run ID: cb4 failed after 9s. Error: failed during run, err: exit status 1
Run failed
/ # az version
{
"azure-cli": "2.27.2",
"azure-cli-core": "2.27.2",
"azure-cli-telemetry": "1.0.6",
"extensions": {}
}
I was running this in a Azure-CLI container.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 40e52f8e-a63e-3e85-6c84-f881db5d3b6c
Version Independent ID: 1290c892-ea9d-d12c-be84-7ed75f28fca0
Content: Purge tags and manifests - Azure Container Registry
Content Source: articles/container-registry/container-registry-auto-purge.md
Service: container-registry
GitHub Login: @dlepow
Microsoft Alias: danlep

@suuuuuper, thank you for sharing your concern here. We are internally investigating this and will get back to you with further updates.

@suuuuuper can you please share the value of the PURGE_CMD that you have set and the exact command you ran before hitting this error?

@suuuuuper any update on this?

Hi @suuuuuper , the purge command is supported in a separate CLI called acr-cli instead of Azure CLI.

Looks like it was my mistake. Thank you guys, this issue had been solved.


How to setup?

Hey, I found your custom integration, but Iam not sure how to set it up. I did install it as a custom repository in HACS and downloaded it. It didnt appear in my sidebar (after restarting and clearing the cache).
The readme says "After the install via HACS, you must then set it up via a config flow that does nothing." => I have no clue, what that means... I did google "config flow" in context of Home Assisstant, but I really dont know what to do...^^
Could you add some extra explanation, please?
Thanks!

I've just updated the readme. That should hopefully be sufficient to get you up and running


JIRA: allow different templates for jira description rendering

Depending on your JIRA instance or JIRA project, you may want to use different templates for rendering the issue description. Sometimes you want to provide all the details and want them to be visible in JIRA.
Sometimes you want to provide only a link back into defect dojo and not too much details.
This PR allows you to chose between these two templates, or add even more of your own templates.
Any templates added to (or modified in) dojo/templates/issue-trackers automatically become options in the jira config dropdowns.
The PR is always falling back to the default full description template already in use. So when people to do nothing after upgrading, it just keeps work as it was. It's all optional.
The PR also adds the component_name, component_version and all the SAST_xxxx fields to the "full/default" jira template, fixing #3931

BTW The first attempt at this PR was using a new model JIRAIssueTemplate that could be edited via the Django Admin portal at /admin. But this opened up risks that people could use the template language to retrieve all kinds of sensitive details. In the current world I think a lot of setups still have "everyone" as staff/superuser, so there was too much risk.
With a better permission model we could still switch to using a model instead of having to use templates on the filesystem and having to rebuild the images (or use complex mounting nobody understands ;-))
BTW2 In v2 of this PR I used the FilePathField on the model to let django populate the dropdown for us. But this resulted in the full path being stored in the issue_template field, where the template loader expects a relative path. So this required adding/removing of the prefix in some places, which broke the model validation, etc.
BTW3 So then I moved to just a CharField with choices. This worked, but if a user adds a template on disk, the choices change and django expects you to make a migration. Our initializer automatically makes those resulting in instances getting our of sync and in conflict with upstream releases.
So I fell back to just using a CharField and populating the dropdown on the forms.

fyi @yilmi


beast::http::body model for boost::asio::buffer

What is the correct body model to implement this?
request.body = boost::asio::buffer("Hello, world!");

http://vinniefalco.github.io/beast/beast/ref/beast__http__string_view_body.html

void blub1()
{
    beast::http::request<beast::http::string_view_body> request;
    request.body = boost::asio::buffer("Hello, world!");
}
void blub2()
{
    beast::http::request<beast::http::buffer_body> request;
    request.body = boost::asio::buffer("Hello, world!");
}

==>
gcc.compile.c++ bin/gcc-7/release/example/foo.o

    "g++"  -ftemplate-depth-128 -march=native -ftemplate-depth=256 -std=c++17 -O3 -finline-functions -Wno-inline -Wall -fPIC  -DBOOST_ALL_DYN_LINK -DNDEBUG  -I"/home/mike/workspace/Beast/include" -I"include" -c -o "bin/gcc-7/release/example/foo.o" "example/foo.cpp"

example/foo.cpp: In function ‘void blub1()’:
example/foo.cpp:168:55: error: no match for ‘operator=’ (operand types are ‘beast::http::string_view_body::value_type {aka boost::basic_string_ref<char, std::char_traits<char> >}’ and ‘boost::asio::const_buffers_1’)
     request.body = boost::asio::buffer("Hello, world!");
                                                       ^
In file included from /home/mike/workspace/Beast/include/beast/core/string.hpp:11:0,
                 from /home/mike/workspace/Beast/include/beast/core/static_string.hpp:12,
                 from /home/mike/workspace/Beast/include/beast/core.hpp:29,
                 from /home/mike/workspace/Beast/include/beast.hpp:13,
                 from example/foo.cpp:8:
/usr/include/boost/utility/string_ref.hpp:77:27: note: candidate: constexpr boost::basic_string_ref<charT, traits>& boost::basic_string_ref<charT, traits>::operator=(const boost::basic_string_ref<charT, traits>&) [with charT = char; traits = std::char_traits<char>]
         basic_string_ref& operator=(const basic_string_ref &rhs) BOOST_NOEXCEPT
                           ^~~~~~~~
/usr/include/boost/utility/string_ref.hpp:77:27: note:   no known conversion for argument 1 from ‘boost::asio::const_buffers_1’ to ‘const boost::basic_string_ref<char, std::char_traits<char> >&’
example/foo.cpp: In function ‘void blub2()’:
example/foo.cpp:173:55: error: no match for ‘operator=’ (operand types are ‘beast::http::buffer_body::value_type’ and ‘boost::asio::const_buffers_1’)
     request.body = boost::asio::buffer("Hello, world!");
                                                       ^
In file included from /home/mike/workspace/Beast/include/beast/http.hpp:14:0,
                 from /home/mike/workspace/Beast/include/beast.hpp:14,
                 from example/foo.cpp:8:
/home/mike/workspace/Beast/include/beast/http/buffer_body.hpp:30:12: note: candidate: constexpr beast::http::buffer_body::value_type& beast::http::buffer_body::value_type::operator=(const beast::http::buffer_body::value_type&)
     struct value_type
            ^~~~~~~~~~
/home/mike/workspace/Beast/include/beast/http/buffer_body.hpp:30:12: note:   no known conversion for argument 1 from ‘boost::asio::const_buffers_1’ to ‘const beast::http::buffer_body::value_type&’
/home/mike/workspace/Beast/include/beast/http/buffer_body.hpp:30:12: note: candidate: constexpr beast::http::buffer_body::value_type& beast::http::buffer_body::value_type::operator=(beast::http::buffer_body::value_type&&)
/home/mike/workspace/Beast/include/beast/http/buffer_body.hpp:30:12: note:   no known conversion for argument 1 from ‘boost::asio::const_buffers_1’ to ‘beast::http::buffer_body::value_type&&’
...failed gcc.compile.c++ bin/gcc-7/release/example/foo.o...

Oh, you literally want to assign a boost::asio::const_buffers_1. There's no way to do that. I thought you were asking the more general question "which body type represents a non-owning buffer".

Is this still an issue?

Not really -> close :-)

I could add these to buffer_body::value_type
        /// Assign a `boost::asio::const_buffers` to `*this`
        value_type&
        operator=(boost::asio::const_buffer const& b)
        {
            data = boost::asio::buffer_cast<void*>(b);
            size = boost::asio::buffer_size(b);
            return *this;
        }

        /// Assign a `boost::asio::const_buffers_1` to `*this`
        value_type&
        operator=(boost::asio::const_buffers_1 const& b)
        {
            data = boost::asio::buffer_cast<void*>(b);
            size = boost::asio::buffer_size(b);
            return *this;
        }

So this would compile?
void blub2()
{
    beast::http::request<beast::http::buffer_body> request;
    request.body = boost::asio::buffer("Hello, world!");
}

Yes, it should. I'm adding a test for it.

 I might have to break this up into two classes... ugh

Unfortunately no, this doesn't work. Because value_type::data is void* not void const*. I would have to cast away the const-ness. I am thinking about making two classes
buffer_body which would be const, and mutable_buffer_body which would be mutable.

Sounds like the pattern i suggested for strings too:
Strings:

mutable_sting_body for write and read
immutable_sting_body for write only

Buffers:

mutable_buffer_body for write and read
immutable_buffer_body for write only

const not immutable

But the basic idea is the same - only different names :-)

I don't like how this turned out, it makes a small issue with the HTTP relay example worse. I am going to have to leave it alone for now, and revisit it later. I will open an issue.

Closing this


Allow using custom JWT token classes

Use the full path to the token class in jwt_token_class_name so that customized implementations (which are fully supported by djangorestframework-simplejwt) can be used without needing to override the serializer get_token_instance() method.

Hey @vainu-arto
Thank you for the interest in the project!
Your changes are merged in this PR:
https://github.com/st4lk/django-rest-social-auth/pull/167

Available in pypi rest-social-auth v8.0.0


Tooltip disappears when it receives focus

When tabbing to an item that has a tooltip, we focus the item and show the tooltip. If the user presses tab again, we focus the tooltip (which allows a screen reader to read it out) but the tooltip automatically disappears.
https://github.com/EightfoldAI/octuple/assets/119535119/3bb57324-ad2f-4748-8ea5-60a6e202f506

The element that triggers the tooltip references the tooltip element with aria-describedby.

Tooltips should not be included in the tab loop.

Tracking with ENG-74737


@typedef function params do not get shown when using @type

When setting a type for a function using @type, I would expect the @typedof information for the description, params, return statement to show inline.

This is the typedef
/**
 *
Redux is a predictable state container for JavaScript apps.

 *
 *
It helps you write applications that behave consistently, run in
 * different environments (client, server, and native), and are easy to test.
 * On top of that, it provides a great developer experience, such as live code
 * editing combined with a time traveling debugger.

 *
 * @typedef Redux
 * @namespace Redux
 * @see {@link https://redux.js.org/}
 */

/**
 * Reducers specify how the application's state changes in response to
 * actions sent to the store. Remember that actions only describe what
 * happened, but don't describe how the application's state changes.
 * @memberof Redux
 * @typedef {Function} Redux.Reducer
 * @param {Redux.Store} previousState - The current state
 * @param {Redux.Action} action - An action to execute to update the store
 * @return {Redux.Store} The next state
 * @see {@link https://redux.js.org/basics/reducers}
 */

This is the usage
/**
   * Application reducer.
   * @type {Redux.Reducer}
   * @module Application/reducer
   */
  export default function reducer(state = defaultState, action) {

Absolutely nothing gets printed for this module unless I manually start putting in @params etc statements.

I guess my general expectation is that the tags would automatically be copied and displayed inline.  It's probably possible to do this with a plugin....

Worked around this with custom tags so I guess I'll just close this...


Simple compilation (of complex code) broken by "-inft-suffix .ml"

Hello,
Abstract: passing -intf-suffix .ml to ocamlopt breaks compilation in some cases.
Details:
The code in test.ml doesn't compile to native with jbuilder. Here is the output of jbuilder build test.exe:
    ocamldep test.depends.ocamldep-output
      ocamlc test.{cmi,cmo,cmt}
    ocamlopt test.{cmx,o} (exit 2)
(cd _build/default && /home/vincent/.opam/rdas-4.05.0+flambda/bin/ocamlopt.opt -w -40 -g -intf-suffix .ml -no-alias-deps -I . -o test.cmx -c -impl test.ml)
File "test.ml", line 1:
Error: The implementation test.ml does not match the interface test.cmi:
       Values do not match:
         val g : ([< `Poly of 'b M'.t & 'a M'.t ] as 'a) -> int
       is not included in
         val g : ([< `Poly of 'b M'.t & 'a M'.t ] as 'a) -> int
       File "test.ml", line 13, characters 8-9: Actual declaration

Note that the error is quite weird: the expected and actual types for val g are identical.
If I remove the -intf-suffix .ml flags and run (cd _build/default && /home/vincent/.opam/rdas-4.05.0+flambda/bin/ocamlopt.opt -w -40 -g -no-alias-deps -I . -o test.cmx -c -impl test.ml) manually, it compiles OK.
The code itself is rather complex: it contains a module alias, a recursive function, and uses a dynamic variant to make a recursive type from a parametric type, and I think all are needed to exhibit the issue.
But compiling it is simple: it is self-contained and can actually be compiled by ocamlopt test.ml.
The runme.sh script and the jbuild file in the same gists can be used to reproduce the issue.
Questions:

what is the purpose of -intf-suffix .ml?
could it be added only when it's required?
could it be removed entirely?

Thanks!

After further investigation, it appears that it is not a jbuilder issue but
a compiler one. Do you want to report the issue, or do you prefer me
to do so?
As a workaround, @lpw25 suggested to add an annotation to g:
let rec g : 'a . ([< `Poly of 'a M'.t ] as 'a) -> int = fun expr ->

Wow, I've found a bug in the OCaml compiler! Thanks for the investigation, I'll repackage the gist to demo the bug outside of jbuilder and submit a new ticket in the OCaml Mantis.
For my curiosity, and as this seems to trigger the compiler bug, can you please tell me a bit about using -intf--suffix .ml? It feels weird to tell the compiler to use .ml files as interface files. Or maybe I'm misunderstanding the doc in ocamlc --help: -intf-suffix <string>  Suffix for interface files (default: .mli).

Also, answers to your questions:

-intf-suffix .ml is used to tell the compiler that the file extension for interfaces is .ml (it is .mli by default);
it is added by jbuilder only when required: it is a trick used when there is no .mli file ; the module is compiled by ocamlc to get a cmi and then the switch tricks ocamlopt into thinking that there is actually an interface file that produced the cmi file (if you don't do that, ocamlopt will try to produce a cmi file, just like ocamlc, and it can result in a race condition during parallel builds);
jbuilder will generate a compilation command with no -intf-suffix switch if you have an mli file.

Hope this helps.

Awesome, thanks! I'll report to the OCaml Mantis today or tomorrow.

I've just opened https://caml.inria.fr/mantis/view.php?id=7629

Great; thanks!


Possible fix for notifications bug in #998

Should address #998 .

BREAKING CHANGE: Please move celeryconfig.py to security_monkey/celeryconfig.py.


NewToolchain 003: Define a user to own directories we create in make

Merge Checklist

All boxes should be checked before merging the PR (just tick any boxes which don't apply to this PR)

[x] The toolchain has been rebuilt successfully (or no changes were made to it)
[x] The toolchain/worker package manifests are up-to-date
[x] Any updated packages successfully build (or no packages were changed)
[x] Packages depending on static components modified in this PR (Golang, *-static subpackages, etc.) have had their Release tag incremented.
[x] Package tests (%check section) have been verified with RUN_CHECK=y for existing SPEC files, or added to new SPEC files
[x] All package sources are available
[x] cgmanifest files are up-to-date and sorted (./cgmanifest.json, ./toolkit/scripts/toolchain/cgmanifest.json, .github/workflows/cgmanifest.json)
[x] LICENSE-MAP files are up-to-date (./SPECS/LICENSES-AND-NOTICES/data/licenses.json, ./SPECS/LICENSES-AND-NOTICES/LICENSES-MAP.md, ./SPECS/LICENSES-AND-NOTICES/LICENSE-EXCEPTIONS.PHOTON)
[x] All source files have up-to-date hashes in the *.signatures.json files
[x] sudo make go-tidy-all and sudo make go-test-coverage pass
[x] Documentation has been updated to match any changes to the build system
[ ] Ready to merge

Summary

Part of https://github.com/microsoft/CBL-Mariner/pull/6892, related to https://github.com/microsoft/CBL-Mariner/issues/6788
This is part of a series of PRs for the new toolchain prototype. Some PRs are fundamental bug fixes for 2.0, some are new features better targeted at 3.0, and some are early prototypes/RFCs.
Destination
2.0 - Maybe
3.0 - Yes
RFC -
Specific PR summary:
The calling user should probably own at least some of the files the tools generate. Lets start by defining the ideal owner, and making the directories we create be owned by that user.
Change Log

Environment variable MARINER_BUILDER_USER will be set to either $SUDO_USER if defined, or the current $USER otherwise.
Create makedirs.sh to replace the existing create_folder define in Make. The script will be called instead and may now report an error back if it fails.

Does this affect the toolchain?

YES/NO
Associated issues

https://github.com/microsoft/CBL-Mariner/issues/6788

Test Methodology

Local builds
Full tests TBD

Had to add a bit more handling for non-sudo calls. runuser can only be used as sudo, even if you are just asking to run as yourself.


Use semaphores for ratelimit handling

Summary
This PR attempts to make use semaphores for ratelimit handling
Checklist

[ ] If code changes were made then they have been tested.

[ ] I have updated the documentation to reflect the changes.

[ ] This PR fixes an issue.
[ ] This PR adds something new (e.g. new method or parameters).
[ ] This PR is a breaking change (e.g. methods or parameters removed/renamed)
[ ] This PR is not a code change (e.g. documentation, README, ...)

Since MangaDex currently aren't going to expose ratelimit headers on every endpoint except special-cased ones, I believe this PR should be closed, both due to the length of time it has been open with no updates (on their side, since I know this was queried), and due to the current implementation being acceptable for the current requirements.
If in future you still wish to implement this feel free. Thank you very much for your time.


Downloading

[Enter feedback here]
How can I download fonts?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: d3c2a6fb-3005-b23b-bc83-f8046b852851
Version Independent ID: f5d0fbf0-990b-7b3d-0c4a-bd4b51c8532e
Content: Segoe UI font family - Typography
Content Source: typographydocs/font-list/segoe-ui.md
Product: typography
GitHub Login: @nihar
Microsoft Alias: alib

The Segoe UI fonts ship with Windows and are available as part of Windows app design toolkits, but aren't otherwise offered for free download. To use in other contexts, however, separate licenses can be obtained: see https://www.fonts.com/content/microsoft-typography.

Just
C:\Windows\Fonts


task list component spacing breakpoints not aligned with grid

Noticed this while testing our prototype, we're using this page as the start of the journey.
Seen on https://www.gov.uk/view-driving-licence
I think maybe the margins added need to have the same breakpoints as the grid

This seems to be a problem for pages in frontend, but works fine on pages in government-frontend. Are our breakpoints different across applications @nickcolley? Please say no..

Well, alphagov/frontend does use the old 'wrapper' layout which doesnt use the new grid. So it's possible.

Looks like the step by step is no longer on these pages so we can't see if this is still the case. I'm assuming this will be picked up once new step by steps are created.


IntelliSense shows error for past code version

Issue Type: Bug
STR: I have been typing out "string" for a definition.
AR: After I finished, VS Code underlines "str" and tells me that that it cannot it. I had to delete the last character and re-type it so it catches up.
ER: Don't lag behind my typing input and show errors for outdated code
https://user-images.githubusercontent.com/8599/128807918-fa11d5d6-31c2-42be-b57f-2e9576243cfa.mov
I could reproduce this in the same file/line repeatedly. I have to type the g slowly when fixing, otherwise it complains about strin.
VS Code version: Code - Insiders 1.60.0-insider (1ee61f368ee0570feeb220605578a8768d99e762, 2021-08-09T05:59:19.046Z)
OS version: Darwin x64 20.6.0
Restricted Mode: No

System Info

Item
Value

CPUs
Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz (16 x 2400)

GPU Status
2d_canvas: enabledgpu_compositing: enabledmetal: disabled_offmultiple_raster_threads: enabled_onoop_rasterization: enabledopengl: enabled_onrasterization: enabledskia_renderer: disabled_off_okvideo_decode: enabledwebgl: enabledwebgl2: enabled

Load (avg)
2, 2, 2

Memory (System)
64.00GB (18.78GB free)

Process Argv
--enable-proposed-api donjayamanne.kusto --log trace --crash-reporter-id 4cf0a6f0-aaf8-4da3-9b2b-81f6c639c12b

Screen Reader
no

VM
0%

Extensions (39)

Extension
Author (truncated)
Version

Bookmarks
ale
13.1.0

npm-intellisense
chr
1.4.0

bracket-pair-colorizer
Coe
1.0.61

vscode-github-actions
csc
0.20.3

vscode-eslint
dba
2.1.23

kusto
don
0.3.5

typescript-notebook
don
1.4.0

gitlens-insiders
eam
2021.7.1905

tsl-problem-matcher
eam
0.4.0

EditorConfig
Edi
0.16.4

vscode-npm-script
eg2
0.3.22

prettier-vscode
esb
8.1.0

auto-close-tag
for
0.5.12

auto-rename-tag
for
0.1.8

codespaces
Git
0.11.3

copilot
Git
1.3.2283

remotehub-insiders
Git
2021.8.91328

vscode-pull-request-github
Git
0.29.0

restore-git-branch-tabs
gko
0.4.1

gzipdecompressor
Hyu
1.2.0

vscode-styled-components
jpo
1.6.6

theme-monokai-pro-vscode
mon
1.1.19

vscode-docker
ms-
1.15.0

vscode-edge-devtools
ms-
1.2.1

python
ms-
2021.9.1106798209-dev

vscode-pylance
ms-
2021.8.1-pre.1

jupyter
ms-
2021.8.2022228455

remote-containers
ms-
0.191.0

azure-account
ms-
0.9.8

cpptools
ms-
1.5.1

live-server
ms-
0.2.7

debugger-for-edge
msj
1.0.15

java
red
0.80.0

vscode-commons
red
0.0.6

vscode-xml
red
0.17.0

smart-css-autocomplete
sai
1.4.2

msft-todo-unofficial
tyl
0.0.14

markdown-all-in-one
yzh
3.4.0

vscode-proto3
zxh
0.5.4

A/B Experiments
vsliv695:30137379
vsins829:30139715
vsliv368cf:30146710
vsreu685:30147344
python383cf:30185419
pythonvspyt602:30291494
vspor879:30202332
vspor708:30202333
vspor363:30204092
pythonvspyt639:30291487
pythontb:30258533
pythonvspyt551cf:30291413
pythonptprofiler:30281269
vshan820:30294714
pythondataviewer:30285072
pythonvsuse255:30319630
00j7a122:30294756
vscod805:30301674
pythonvspyt200:30323110
vsccppwtct:30312693
pythonvssor306:30340298
bridge0708:30335490
pygetstartedt2:30349639
dockerwalkthrucf:30348714

Maybe related, I also see issues that highlight the wrong part of the code:

what language and what extension is this?

@jrieken the first screenshot is a TypeScript React file, the latter is TypeScript.
@mjbvz pinged me to re-test in latest Insiders and I can't reproduce the original issue anymore. Closing for now, until it happens again.


Reloadium randomly breaks itself on PyCharm Manjaro

Describe the bug
Reloadium nuked itself and now throws a stack trace whenever launching PyCharm.
To Reproduce
Steps to reproduce the behavior:

Launch IDE
Reloadium Fails to load. Worked earlier on this exact system.

Expected behavior
Reloadium to work.
Desktop (please complete the following information):

OS: Manjaro
OS version: 21.3.1 Ruah
Reloadium package version: Can't tell, Reloadium doesn't work.
PyCharm plugin version: 0.8.3
Editor: PyCharm
Run mode: No Reloadium Option

Additional context
https://pastebin.com/aDtdGsXw

Interesting.
Not sure if it's the plugin's or PyCharm's fault .
Have your tried uninstalling and installing the plugin again?

Yeah. Latest version of PyCharm too. I am running the Arch User Repository version.


Refactor: removed duplicated words in comments

Summary
Found and removed duplicates of the word "the" in comments.
Changelog
[Internal] [Removed] – Removed duplicates of the word "the" in comments.
Test Plan
Not applicable.

test_windows seems to be fail on other prs as well in a similar way. For example: https://app.circleci.com/pipelines/github/facebook/react-native/16051/workflows/b2c09dc2-584b-4e75-acdc-92f234690ba7/jobs/302101
Suspect issue with the main or test?


Hit migration and improved duplicate removal steps

Removed Track extension flag so that TCE is always done
Ensure hits indicies are set for objects up to TC
Changed duplicate removal steps (T5,pT5,pT3) to used saved hit indices instead of navigating the chain of objects to get the hits
Changed first pass of T5 duplicate removal. Compares fewer T5s but overall recovers good efficiency with similar (slightly better) final fake and duplicate rates. Makes slightly more pT5s.

Validation plots : https://www.classe.cornell.edu/~mgr85/www/SegmentLinking_keep/PR112/

timing


Fix list struct print bug

Part of: https://github.com/elixir-explorer/explorer/issues/1014
Fixes an issue where we can't print columns with a dtype like {:list, {:struct, ...}} where the root of the tree isn't a :struct but it contains a :struct. The cause is that we're not accounting for nested structs in our print function:
https://github.com/elixir-explorer/explorer/blob/07f80fb7af0bdf3fbbe3dcbb189ea43c229527fb/lib/explorer/data_frame.ex#L5982-L5990
The solution is a recursive function.
I've also made some additional choices:

Contents of structs now look the same inside and outside a struct.
Before we were calling inspect/1. That resulted in e.g. strings getting "" delimiters. Strings don't usually get "" delimiters when printed in a dataframe.
Lists now get [] delimiters.
This is especially helpful in identifying empty lists and singleton lists. Before these cases were a bit confusing.
Structs now get {} delimiters instead of %{}.
Keys of structs go from "x" => to x:.
nil is now represented as the literal nil instead of an empty string.
We now intent one space when nesting.

Overall I found these choices looked much better. Here's just one example of the new format:
+--------------------------------------------------------+
|       Explorer DataFrame: [rows: 3, columns: 2]        |
+--------------------------------+-----------------------+
|             col_d              |         col_j         |
|          <struct[2]>           |   <list[list[f32]]>   |
+================================+=======================+
| {                              | [                     |
|  q: 2901-09-24 22:53:38.923913 |  [                    |
|  w:<PHONE_NUMBER>432017527        |   1.100000023841858   |
| }                              |   2.200000047683716   |
|                                |  ]                    |
|                                | ]                     |
+--------------------------------+-----------------------+
| {                              | [[3.299999952316284]] |
|  q: nil                        |                       |
|  w: nil                        |                       |
| }                              |                       |
+--------------------------------+-----------------------+
| {                              | [                     |
|  q: nil                        |  nil                  |
|  w:<PHONE_NUMBER>191965499        |  [nil]                |
| }                              |  []                   |
|                                | ]                     |
+--------------------------------+-----------------------+

I decided on x: over "x" => because 1) horizontal space is limited and 2) we don't use literals elsewhere. E.g. strings are my string not "my string"
If later anyone feels strongly about these choices though I'm happy to walk them back!


fix(build): include root attribute on path for source maps

Currently, the build task statically uses the 'src/' folder to output source maps. We have added an additional parameter to the paths object in the gulp task, called root, which is used for generating source maps and for prefixing the source and html attributes.
closes 26

@davismj Can you amend the commit message to remove the greater than and less than symbols. It should just be: fix(build): include root attribute on path for source maps

done

Excellent. I'll get it in shortly. Thanks!

@davismj You just changed the title in github.  I'll do it for you this time, it's no problem.


Unable to flash self-compiled source

Hi, I am attempting to test some modifications to the code (will PR when ready).  However, I have been unable to get any image that I compile to run successfully.  When I compile even master (unchanged) from this repo, the resulting flash attempt results in a "bricked" RFID Tool.  I can use esptool.py to flash the last release bin and it works without issue.  I assume the settings in my Arduino IDE are not accurate, but in spite of tweaking these settings a bunch of times, I can't seem to get something that works.  Was hoping you could provide more information on what settings you are using to compile the official releases.  I'm currently using Arduino IDE 1.8.15.  I am using the board definition "NodeMCU 1.0 (ESP-12E Module)" from https://arduino.esp8266.com/stable/package_esp8266com_index.json.
Leaving all other settings as default (though I've also tried tweaking the FS/OTA settings).
One thing I've noticed, the last official bin from this repo looks like this when I look at the image with esptool.py:
python esptool.py image_info esprfidtool.bin
esptool.py v3.0
Image version: 1
Entry point: 4010f29c
1 segments

Segment 1: len 0x00568 load 0x4010f000 file_offs 0x00000008 []
Checksum: 2d (valid)

Whenever I generate an image it has two segments:
python esptool.py image_info esprfidtool.ino.nodemcu.bin
esptool.py v3.0
Image version: 1
Entry point: 4010f494
2 segments

Segment 1: len 0x00d84 load 0x4010f000 file_offs 0x00000008 []
Segment 2: len 0x00028 load 0x3fff20b8 file_offs 0x00000d94 [DRAM]
Checksum: c9 (valid)

Issue is the need to annotate interrupt handlers with ICACHE_RAM_ATTR these days.  I'll add to my PR


monado: Add serviceSupport option

Motivation for this change

rebases PR https://github.com/NixOS/nixpkgs/pull/107001 by @expipiplus1 on top of current master

monado: Add serviceSupport option
xrgears: init at unstable-2020-04-15
libsurvive: add expipiplus1 to maintainers

Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

@GrahamcOfBorg build xrgears monado

@GrahamcOfBorg build xrgears monado

This is a semi-automatic executed nixpkgs-review which does not build all packages (e.g. lumo, tensorflow or pytorch)
If you find some bugs or got suggestions for further things to search or run please reach out to SuperSandro2000 on IRC.
Result of nixpkgs-review pr 108820 run on x86_64-linux 1

2 packages built:

monado
xrgears

This is a semi-automatic executed nixpkgs-review which does not build all packages (e.g. lumo, tensorflow or pytorch)
If you find some bugs or got suggestions for further things to search or run please reach out to SuperSandro2000 on IRC.
Result of nixpkgs-review pr 108820 run on x86_64-linux 1

2 packages built:

monado
xrgears

OK, merged. Thanks @expipiplus1 for the work and sorry for causing a mess :-)

OK, merged. Thanks @expipiplus1 for the work and sorry for causing a mess :-)

And thanks to you too!
On Sun, Jan 10, 2021, 12:59 AM Pavol Rusnak<EMAIL_ADDRESS>wrote:

OK, merged. Thanks @expipiplus1 https://github.com/expipiplus1 for the
work and sorry for causing the mess :-)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/NixOS/nixpkgs/pull/108820#issuecomment-757335171, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAGRJXAK4ECKP32M7IID27LSZCDNBANCNFSM4V3DKKPA
.

And thanks to you too!
On Sun, Jan 10, 2021, 12:59 AM Pavol Rusnak<EMAIL_ADDRESS>wrote:

OK, merged. Thanks @expipiplus1 https://github.com/expipiplus1 for the
work and sorry for causing the mess :-)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/NixOS/nixpkgs/pull/108820#issuecomment-757335171, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAGRJXAK4ECKP32M7IID27LSZCDNBANCNFSM4V3DKKPA
.


Top Hits - Improper strain-level interpretations

Description of the bug

E Coli really struggles in kraken2, and ultimately, it is a mess of 100s of strains being classified. Increasing how conservative the LCA mapping is is one solution OR we can compare, across an entire species like E.Coli, if all the summations of children (s1,s2,etc) are LESS than a given species, then omit them from top hits and only do species

Flu labeling is full of "reference genomes" in the metadata at the species level and false positives from k2 output leads to S1 or below being missed in the top hits passing. With that, we need to work in the opposite direction, where for a given top 10 species we travel down the tree to the lowest "child" within a species and use that IF the clade assigned is higher for that than at the species level

Essentially, false positives are causing a mess in what references we pull for alignment, and filtering out a lot of true top hits for heavily populated assemblies like flu or e. coli (edited)
Command used and terminal output
No response
Relevant files
No response
System information
No response

Fixed in Release 1.6.0


implement reading time functionality

Currently reading time is hard-coded. Use reading-time and potentially gatsby-remark-reading-time to implement a functional solution

If this requires a custom implementation, here are some additional notes: Gatsby GraphQL schema