A100/Cuda11.4 tensorflow/stream_executor/cuda/cuda_blas.cc:429] failed to run cuBLAS routine: CUBLAS_STATUS_EXECUTION_FAILED

Below are details of docker image and Nvidia smi
Docker image - nvcr.io/nvidia/tlt-streamanalytics:v2.0_py3
Server - Dell R740
OS - CentOS Linux release 8.1.1911 (Core)
Docker - [root@mlperf1 ~]# docker --version
Docker version 20.10.7, build f0df350
Cuda - 11.4
GPU - Nvidia A100s
[root@mlperf1 ~]# nvidia-smi
Wed Oct 27 11:05:11 2021
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.57.02    Driver Version: 470.57.02    CUDA Version: 11.4     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA A100-PCI...  Off  | 00000000:AF:00.0 Off |                    0 |
| N/A   38C    P0    36W / 250W |      0MiB / 40536MiB |      0%      Default |
|                               |                      |             Disabled |
+-------------------------------+----------------------+----------------------+
|   1  NVIDIA A100-PCI...  Off  | 00000000:D8:00.0 Off |                    0 |
| N/A   42C    P0    33W / 250W |      0MiB / 40536MiB |      0%      Default |
|                               |                      |             Disabled |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+
Below are the logs when started training :--
root@d62860ede997:/workspace/face-mask-detection# tlt-train detectnet_v2 -e $SPECS_DIR/detectnet_v2_train_resnet18_kitti.txt \

                    -r $USER_EXPERIMENT_DIR/experiment_dir_unpruned \
                    -k $KEY \
                    -n resnet18_detector \
                    --gpus $NUM_GPUS

2021-10-26 10:45:21.645365: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2021-10-26 10:45:21.662062: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2021-10-26 10:45:24.668378: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2021-10-26 10:45:24.669238: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2021-10-26 10:45:24.731904: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties:
name: NVIDIA A100-PCIE-40GB major: 8 minor: 0 memoryClockRate(GHz): 1.41
pciBusID: 0000:d8:00.0
2021-10-26 10:45:24.731957: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2021-10-26 10:45:24.732028: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
2021-10-26 10:45:24.733421: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10.0
2021-10-26 10:45:24.734200: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10.0
2021-10-26 10:45:24.735998: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10.0
2021-10-26 10:45:24.736317: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties:
name: NVIDIA A100-PCIE-40GB major: 8 minor: 0 memoryClockRate(GHz): 1.41
pciBusID: 0000:af:00.0
2021-10-26 10:45:24.736399: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2021-10-26 10:45:24.736515: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
2021-10-26 10:45:24.737361: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10.0
2021-10-26 10:45:24.737455: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2021-10-26 10:45:24.739121: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10.0
2021-10-26 10:45:24.739726: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 1
2021-10-26 10:45:24.739765: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2021-10-26 10:45:24.740054: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10.0
2021-10-26 10:45:24.742686: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10.0
2021-10-26 10:45:24.744696: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10.0
2021-10-26 10:45:24.744790: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2021-10-26 10:45:24.779874: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0
2021-10-26 10:45:24.779979: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2021-10-26 10:51:06.808215: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix:
2021-10-26 10:51:06.808281: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165]      0
2021-10-26 10:51:06.808295: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 0:   N
2021-10-26 10:51:06.813715: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1304] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 37457 MB memory) -> physical GPU (device: 0, name: NVIDIA A100-PCIE-40GB, pci bus id: 0000:af:00.0, compute capability: 8.0)
Using TensorFlow backend.
2021-10-26 10:51:06,828 [INFO] iva.detectnet_v2.scripts.train: Loading experiment spec at /workspace/face-mask-detection/tlt_specs/detectnet_v2_train_resnet18_kitti.txt.
2021-10-26 10:51:06,834 [INFO] iva.detectnet_v2.spec_handler.spec_loader: Merging specification from /workspace/face-mask-detection/tlt_specs/detectnet_v2_train_resnet18_kitti.txt
2021-10-26 10:51:07,208 [INFO] iva.detectnet_v2.scripts.train: Cannot iterate over exactly 3899 samples with a batch size of 24; each epoch will therefore take one extra step.
2021-10-26 10:51:07,208 [INFO] iva.detectnet_v2.scripts.train: Cannot iterate over exactly 81 steps per epoch with 24 processors; each processor will therefore take one extra step per epoch.
2021-10-26 10:51:10.812551: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix:
2021-10-26 10:51:10.812602: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165]      1
2021-10-26 10:51:10.812611: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 1:   N
2021-10-26 10:51:10.817200: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1304] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 37457 MB memory) -> physical GPU (device: 1, name: NVIDIA A100-PCIE-40GB, pci bus id: 0000:d8:00.0, compute capability: 8.0)
Using TensorFlow backend.
2021-10-26 10:51:10,833 [INFO] iva.detectnet_v2.scripts.train: Loading experiment spec at /workspace/face-mask-detection/tlt_specs/detectnet_v2_train_resnet18_kitti.txt.
2021-10-26 10:51:10,837 [INFO] iva.detectnet_v2.spec_handler.spec_loader: Merging specification from /workspace/face-mask-detection/tlt_specs/detectnet_v2_train_resnet18_kitti.txt
2021-10-26 10:51:11,236 [INFO] iva.detectnet_v2.scripts.train: Cannot iterate over exactly 3899 samples with a batch size of 24; each epoch will therefore take one extra step.
2021-10-26 10:51:11,236 [INFO] iva.detectnet_v2.scripts.train: Cannot iterate over exactly 81 steps per epoch with 24 processors; each processor will therefore take one extra step per epoch.

 Bboxes will be automatically converted to polygon coordinates.
2021-10-26 10:51:18.793585: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties:
name: NVIDIA A100-PCIE-40GB major: 8 minor: 0 memoryClockRate(GHz): 1.41
pciBusID: 0000:af:00.0
2021-10-26 10:51:18.794755: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 1 with properties:
name: NVIDIA A100-PCIE-40GB major: 8 minor: 0 memoryClockRate(GHz): 1.41
pciBusID: 0000:d8:00.0
2021-10-26 10:51:18.794786: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2021-10-26 10:51:18.794818: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
2021-10-26 10:51:18.794853: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10.0
2021-10-26 10:51:18.794874: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10.0
2021-10-26 10:51:18.794894: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10.0
2021-10-26 10:51:18.794914: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10.0
2021-10-26 10:51:18.794932: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2021-10-26 10:51:18.799183: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0, 1
2021-10-26 10:51:19,039 [INFO] modulus.blocks.data_loaders.multi_source_loader.data_loader: shuffle: True - shard 0 of 2
2021-10-26 10:51:19,046 [INFO] modulus.blocks.data_loaders.multi_source_loader.data_loader: sampling 1 datasets with weights:
2021-10-26 10:51:19,046 [INFO] modulus.blocks.data_loaders.multi_source_loader.data_loader: source: 0 weight: 1.000000
2021-10-26 10:51:19,592 [INFO] iva.detectnet_v2.scripts.train: Found 3899 samples in training set

Layer (type)                    Output Shape         Param #     Connected to
input_1 (InputLayer)            (None, 3, 544, 960)  0

conv1 (Conv2D)                  (None, 64, 272, 480) 9472        input_1[0][0]

bn_conv1 (BatchNormalization)   (None, 64, 272, 480) 256         conv1[0][0]

activation_1 (Activation)       (None, 64, 272, 480) 0           bn_conv1[0][0]

block_1a_conv_1 (Conv2D)        (None, 64, 136, 240) 36928       activation_1[0][0]

block_1a_bn_1 (BatchNormalizati (None, 64, 136, 240) 256         block_1a_conv_1[0][0]

block_1a_relu_1 (Activation)    (None, 64, 136, 240) 0           block_1a_bn_1[0][0]

block_1a_conv_2 (Conv2D)        (None, 64, 136, 240) 36928       block_1a_relu_1[0][0]

block_1a_conv_shortcut (Conv2D) (None, 64, 136, 240) 4160        activation_1[0][0]

block_1a_bn_2 (BatchNormalizati (None, 64, 136, 240) 256         block_1a_conv_2[0][0]

block_1a_bn_shortcut (BatchNorm (None, 64, 136, 240) 256         block_1a_conv_shortcut[0][0]

add_1 (Add)                     (None, 64, 136, 240) 0           block_1a_bn_2[0][0]
block_1a_bn_shortcut[0][0]

block_1a_relu (Activation)      (None, 64, 136, 240) 0           add_1[0][0]

block_1b_conv_1 (Conv2D)        (None, 64, 136, 240) 36928       block_1a_relu[0][0]

block_1b_bn_1 (BatchNormalizati (None, 64, 136, 240) 256         block_1b_conv_1[0][0]

block_1b_relu_1 (Activation)    (None, 64, 136, 240) 0           block_1b_bn_1[0][0]

block_1b_conv_2 (Conv2D)        (None, 64, 136, 240) 36928       block_1b_relu_1[0][0]

block_1b_bn_2 (BatchNormalizati (None, 64, 136, 240) 256         block_1b_conv_2[0][0]

add_2 (Add)                     (None, 64, 136, 240) 0           block_1b_bn_2[0][0]
block_1a_relu[0][0]

block_1b_relu (Activation)      (None, 64, 136, 240) 0           add_2[0][0]

block_2a_conv_1 (Conv2D)        (None, 128, 68, 120) 73856       block_1b_relu[0][0]

block_2a_bn_1 (BatchNormalizati (None, 128, 68, 120) 512         block_2a_conv_1[0][0]

block_2a_relu_1 (Activation)    (None, 128, 68, 120) 0           block_2a_bn_1[0][0]

block_2a_conv_2 (Conv2D)        (None, 128, 68, 120) 147584      block_2a_relu_1[0][0]

block_2a_conv_shortcut (Conv2D) (None, 128, 68, 120) 8320        block_1b_relu[0][0]

block_2a_bn_2 (BatchNormalizati (None, 128, 68, 120) 512         block_2a_conv_2[0][0]

block_2a_bn_shortcut (BatchNorm (None, 128, 68, 120) 512         block_2a_conv_shortcut[0][0]

add_3 (Add)                     (None, 128, 68, 120) 0           block_2a_bn_2[0][0]
block_2a_bn_shortcut[0][0]

block_2a_relu (Activation)      (None, 128, 68, 120) 0           add_3[0][0]

block_2b_conv_1 (Conv2D)        (None, 128, 68, 120) 147584      block_2a_relu[0][0]

block_2b_bn_1 (BatchNormalizati (None, 128, 68, 120) 512         block_2b_conv_1[0][0]

block_2b_relu_1 (Activation)    (None, 128, 68, 120) 0           block_2b_bn_1[0][0]

block_2b_conv_2 (Conv2D)        (None, 128, 68, 120) 147584      block_2b_relu_1[0][0]

block_2b_bn_2 (BatchNormalizati (None, 128, 68, 120) 512         block_2b_conv_2[0][0]

add_4 (Add)                     (None, 128, 68, 120) 0           block_2b_bn_2[0][0]
block_2a_relu[0][0]

block_2b_relu (Activation)      (None, 128, 68, 120) 0           add_4[0][0]

block_3a_conv_1 (Conv2D)        (None, 256, 34, 60)  295168      block_2b_relu[0][0]

block_3a_bn_1 (BatchNormalizati (None, 256, 34, 60)  1024        block_3a_conv_1[0][0]

block_3a_relu_1 (Activation)    (None, 256, 34, 60)  0           block_3a_bn_1[0][0]

block_3a_conv_2 (Conv2D)        (None, 256, 34, 60)  590080      block_3a_relu_1[0][0]

block_3a_conv_shortcut (Conv2D) (None, 256, 34, 60)  33024       block_2b_relu[0][0]

block_3a_bn_2 (BatchNormalizati (None, 256, 34, 60)  1024        block_3a_conv_2[0][0]

block_3a_bn_shortcut (BatchNorm (None, 256, 34, 60)  1024        block_3a_conv_shortcut[0][0]

add_5 (Add)                     (None, 256, 34, 60)  0           block_3a_bn_2[0][0]
block_3a_bn_shortcut[0][0]

block_3a_relu (Activation)      (None, 256, 34, 60)  0           add_5[0][0]

block_3b_conv_1 (Conv2D)        (None, 256, 34, 60)  590080      block_3a_relu[0][0]

block_3b_bn_1 (BatchNormalizati (None, 256, 34, 60)  1024        block_3b_conv_1[0][0]

block_3b_relu_1 (Activation)    (None, 256, 34, 60)  0           block_3b_bn_1[0][0]

block_3b_conv_2 (Conv2D)        (None, 256, 34, 60)  590080      block_3b_relu_1[0][0]

block_3b_bn_2 (BatchNormalizati (None, 256, 34, 60)  1024        block_3b_conv_2[0][0]

add_6 (Add)                     (None, 256, 34, 60)  0           block_3b_bn_2[0][0]
block_3a_relu[0][0]

block_3b_relu (Activation)      (None, 256, 34, 60)  0           add_6[0][0]

block_4a_conv_1 (Conv2D)        (None, 512, 34, 60)  1180160     block_3b_relu[0][0]

block_4a_bn_1 (BatchNormalizati (None, 512, 34, 60)  2048        block_4a_conv_1[0][0]

block_4a_relu_1 (Activation)    (None, 512, 34, 60)  0           block_4a_bn_1[0][0]

block_4a_conv_2 (Conv2D)        (None, 512, 34, 60)  2359808     block_4a_relu_1[0][0]

block_4a_conv_shortcut (Conv2D) (None, 512, 34, 60)  131584      block_3b_relu[0][0]

block_4a_bn_2 (BatchNormalizati (None, 512, 34, 60)  2048        block_4a_conv_2[0][0]

block_4a_bn_shortcut (BatchNorm (None, 512, 34, 60)  2048        block_4a_conv_shortcut[0][0]

add_7 (Add)                     (None, 512, 34, 60)  0           block_4a_bn_2[0][0]
block_4a_bn_shortcut[0][0]

block_4a_relu (Activation)      (None, 512, 34, 60)  0           add_7[0][0]

block_4b_conv_1 (Conv2D)        (None, 512, 34, 60)  2359808     block_4a_relu[0][0]

block_4b_bn_1 (BatchNormalizati (None, 512, 34, 60)  2048        block_4b_conv_1[0][0]

block_4b_relu_1 (Activation)    (None, 512, 34, 60)  0           block_4b_bn_1[0][0]

block_4b_conv_2 (Conv2D)        (None, 512, 34, 60)  2359808     block_4b_relu_1[0][0]

block_4b_bn_2 (BatchNormalizati (None, 512, 34, 60)  2048        block_4b_conv_2[0][0]

add_8 (Add)                     (None, 512, 34, 60)  0           block_4b_bn_2[0][0]
block_4a_relu[0][0]

block_4b_relu (Activation)      (None, 512, 34, 60)  0           add_8[0][0]

output_bbox (Conv2D)            (None, 8, 34, 60)    4104        block_4b_relu[0][0]

output_cov (Conv2D)             (None, 2, 34, 60)    1026        block_4b_relu[0][0]
Total params: 11,200,458
Trainable params: 11,190,730
Non-trainable params: 9,728

2021-10-26 10:51:21,662 [INFO] modulus.blocks.data_loaders.multi_source_loader.data_loader: Serial augmentation enabled = False
2021-10-26 10:51:21,662 [INFO] modulus.blocks.data_loaders.multi_source_loader.data_loader: Pseudo sharding enabled = False
2021-10-26 10:51:21,662 [INFO] modulus.blocks.data_loaders.multi_source_loader.data_loader: Max Image Dimensions (all sources): (0, 0)
2021-10-26 10:51:21,662 [INFO] modulus.blocks.data_loaders.multi_source_loader.data_loader: number of cpus: 88, io threads: 88, compute threads: 44, buffered batches: 4
2021-10-26 10:51:21,663 [INFO] modulus.blocks.data_loaders.multi_source_loader.data_loader: total dataset size 3899, number of sources: 1, batch size per gpu: 24, steps: 82
2021-10-26 10:51:21,785 [INFO] iva.detectnet_v2.dataloader.default_dataloader: Bounding box coordinates were detected in the input specification! Bboxes will be automatically converted to polygon coordinates.
2021-10-26 10:51:21.819131: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties:
name: NVIDIA A100-PCIE-40GB major: 8 minor: 0 memoryClockRate(GHz): 1.41
pciBusID: 0000:af:00.0
2021-10-26 10:51:21.820356: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 1 with properties:
name: NVIDIA A100-PCIE-40GB major: 8 minor: 0 memoryClockRate(GHz): 1.41
pciBusID: 0000:d8:00.0
2021-10-26 10:51:21.820392: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2021-10-26 10:51:21.820428: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
2021-10-26 10:51:21.820452: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10.0
2021-10-26 10:51:21.820467: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10.0
2021-10-26 10:51:21.820482: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10.0
2021-10-26 10:51:21.820496: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10.0
2021-10-26 10:51:21.820509: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2021-10-26 10:51:21.824645: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0, 1
2021-10-26 10:51:22,059 [INFO] modulus.blocks.data_loaders.multi_source_loader.data_loader: shuffle: True - shard 1 of 2
2021-10-26 10:51:22,066 [INFO] modulus.blocks.data_loaders.multi_source_loader.data_loader: sampling 1 datasets with weights:
2021-10-26 10:51:22,066 [INFO] modulus.blocks.data_loaders.multi_source_loader.data_loader: source: 0 weight: 1.000000
2021-10-26 10:51:22,346 [INFO] modulus.blocks.data_loaders.multi_source_loader.data_loader: Serial augmentation enabled = False
2021-10-26 10:51:22,346 [INFO] modulus.blocks.data_loaders.multi_source_loader.data_loader: Pseudo sharding enabled = False
2021-10-26 10:51:22,346 [INFO] modulus.blocks.data_loaders.multi_source_loader.data_loader: Max Image Dimensions (all sources): (0, 0)
2021-10-26 10:51:22,346 [INFO] modulus.blocks.data_loaders.multi_source_loader.data_loader: number of cpus: 88, io threads: 176, compute threads: 88, buffered batches: 4
2021-10-26 10:51:22,346 [INFO] modulus.blocks.data_loaders.multi_source_loader.data_loader: total dataset size 974, number of sources: 1, batch size per gpu: 24, steps: 41
2021-10-26 10:51:22,382 [INFO] iva.detectnet_v2.dataloader.default_dataloader: Bounding box coordinates were detected in the input specification! Bboxes will be automatically converted to polygon coordinates.
2021-10-26 10:51:22,596 [INFO] iva.detectnet_v2.scripts.train: Found 3899 samples in training set
2021-10-26 10:51:22,644 [INFO] modulus.blocks.data_loaders.multi_source_loader.data_loader: shuffle: False - shard 0 of 1
2021-10-26 10:51:22,650 [INFO] modulus.blocks.data_loaders.multi_source_loader.data_loader: sampling 1 datasets with weights:
2021-10-26 10:51:22,650 [INFO] modulus.blocks.data_loaders.multi_source_loader.data_loader: source: 0 weight: 1.000000
2021-10-26 10:51:23,017 [INFO] iva.detectnet_v2.scripts.train: Found 974 samples in validation set
2021-10-26 10:51:26.742907: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties:
name: NVIDIA A100-PCIE-40GB major: 8 minor: 0 memoryClockRate(GHz): 1.41
pciBusID: 0000:af:00.0
2021-10-26 10:51:26.742994: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2021-10-26 10:51:26.743041: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
2021-10-26 10:51:26.743074: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10.0
2021-10-26 10:51:26.743090: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10.0
2021-10-26 10:51:26.743105: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10.0
2021-10-26 10:51:26.743119: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10.0
2021-10-26 10:51:26.743133: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2021-10-26 10:51:26.745478: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0
2021-10-26 10:51:26.968410: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties:
name: NVIDIA A100-PCIE-40GB major: 8 minor: 0 memoryClockRate(GHz): 1.41
pciBusID: 0000:d8:00.0
2021-10-26 10:51:26.968524: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2021-10-26 10:51:26.968669: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
2021-10-26 10:51:26.968734: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10.0
2021-10-26 10:51:26.968761: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10.0
2021-10-26 10:51:26.968787: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10.0
2021-10-26 10:51:26.968812: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10.0
2021-10-26 10:51:26.968830: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2021-10-26 10:51:26.971026: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 1
2021-10-26 10:51:27.078355: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix:
2021-10-26 10:51:27.078385: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165]      1
2021-10-26 10:51:27.078392: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 1:   N
2021-10-26 10:51:27.166997: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix:
2021-10-26 10:51:27.167036: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165]      0
2021-10-26 10:51:27.167045: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 0:   N
2021-10-26 10:51:27.285792: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1304] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 37457 MB memory) -> physical GPU (device: 1, name: NVIDIA A100-PCIE-40GB, pci bus id: 0000:d8:00.0, compute capability: 8.0)
2021-10-26 10:51:27.288712: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1304] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 37457 MB memory) -> physical GPU (device: 0, name: NVIDIA A100-PCIE-40GB, pci bus id: 0000:af:00.0, compute capability: 8.0)
2021-10-26 10:51:35.503289: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
2021-10-26 10:52:00.791520: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
2021-10-26 10:52:41.008907: E tensorflow/stream_executor/cuda/cuda_blas.cc:429] failed to run cuBLAS routine: CUBLAS_STATUS_EXECUTION_FAILED
2021-10-26 10:52:41.008970: E tensorflow/stream_executor/cuda/cuda_blas.cc:2437] Internal: failed BLAS call, see log for details
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/client/session.py", line 1365, in _do_call
return fn(*args)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/client/session.py", line 1350, in _run_fn
target_list, run_metadata)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/client/session.py", line 1443, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.InternalError: 2 root error(s) found.
(0) Internal: Blas xGEMMBatched launch failed : a.shape=[24,3,3], b.shape=[24,3,3], m=3, n=3, k=3, batch_size=24
[[{{node CompositeTransform_6/CompositeTransform_5/CompositeTransform_4/CompositeTransform_3/CompositeTransform_2/CompositeTransform_1/CompositeTransform/RandomFlip/MatMul}}]]
[[gradients/resnet18_nopool_bn_detectnet_v2/block_1b_bn_1/FusedBatchNormV3_grad/FusedBatchNormGradV3/_5859]]
(1) Internal: Blas xGEMMBatched launch failed : a.shape=[24,3,3], b.shape=[24,3,3], m=3, n=3, k=3, batch_size=24
[[{{node CompositeTransform_6/CompositeTransform_5/CompositeTransform_4/CompositeTransform_3/CompositeTransform_2/CompositeTransform_1/CompositeTransform/RandomFlip/MatMul}}]]
0 successful operations.
0 derived errors ignored.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/bin/tlt-train-g1", line 8, in
sys.exit(main())
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/common/magnet_train.py", line 55, in main
File "", line 2, in main
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/detectnet_v2/utilities/timer.py", line 46, in wrapped_fn
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/detectnet_v2/scripts/train.py", line 773, in main
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/detectnet_v2/scripts/train.py", line 691, in run_experiment
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/detectnet_v2/scripts/train.py", line 624, in train_gridbox
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/detectnet_v2/scripts/train.py", line 149, in run_training_loop
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/training/monitored_session.py", line 754, in run
run_metadata=run_metadata)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/training/monitored_session.py", line 1360, in run
raise six.reraise(*original_exc_info)
File "/usr/local/lib/python3.6/dist-packages/six.py", line 693, in reraise
raise value
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/training/monitored_session.py", line 1345, in run
return self._sess.run(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/training/monitored_session.py", line 1418, in run
run_metadata=run_metadata)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/training/monitored_session.py", line 1176, in run
return self._sess.run(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/client/session.py", line 956, in run
run_metadata_ptr)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/client/session.py", line 1180, in _run
feed_dict_tensor, options, run_metadata)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/client/session.py", line 1359, in _do_run
run_metadata)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/client/session.py", line 1384, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InternalError: 2 root error(s) found.
(0) Internal: Blas xGEMMBatched launch failed : a.shape=[24,3,3], b.shape=[24,3,3], m=3, n=3, k=3, batch_size=24
[[node CompositeTransform_6/CompositeTransform_5/CompositeTransform_4/CompositeTransform_3/CompositeTransform_2/CompositeTransform_1/CompositeTransform/RandomFlip/MatMul (defined at /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/ops.py:1748) ]]
[[gradients/resnet18_nopool_bn_detectnet_v2/block_1b_bn_1/FusedBatchNormV3_grad/FusedBatchNormGradV3/_5859]]
(1) Internal: Blas xGEMMBatched launch failed : a.shape=[24,3,3], b.shape=[24,3,3], m=3, n=3, k=3, batch_size=24
[[node CompositeTransform_6/CompositeTransform_5/CompositeTransform_4/CompositeTransform_3/CompositeTransform_2/CompositeTransform_1/CompositeTransform/RandomFlip/MatMul (defined at /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/ops.py:1748) ]]
0 successful operations.
0 derived errors ignored.
Original stack trace for 'CompositeTransform_6/CompositeTransform_5/CompositeTransform_4/CompositeTransform_3/CompositeTransform_2/CompositeTransform_1/CompositeTransform/RandomFlip/MatMul':
File "/usr/local/bin/tlt-train-g1", line 8, in
sys.exit(main())
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/common/magnet_train.py", line 55, in main
File "", line 2, in main
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/detectnet_v2/utilities/timer.py", line 46, in wrapped_fn
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/detectnet_v2/scripts/train.py", line 773, in main
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/detectnet_v2/scripts/train.py", line 691, in run_experiment
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/detectnet_v2/scripts/train.py", line 599, in train_gridbox
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/detectnet_v2/scripts/train.py", line 430, in build_training_graph
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/detectnet_v2/dataloader/drivenet_dataloader.py", line 579, in get_dataset_tensors
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/core/build_wheel.runfiles/ai_infra/moduluspy/modulus/blocks/data_loaders/multi_source_loader/processors/pipeline.py", line 231, in call
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/core/build_wheel.runfiles/ai_infra/moduluspy/modulus/blocks/data_loaders/multi_source_loader/processors/transform_processor.py", line 146, in process
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/core/build_wheel.runfiles/ai_infra/moduluspy/modulus/processors/processors.py", line 240, in call
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/core/build_wheel.runfiles/ai_infra/moduluspy/modulus/blocks/data_loaders/multi_source_loader/processors/transform_processor.py", line 275, in call
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/core/build_wheel.runfiles/ai_infra/moduluspy/modulus/processors/processors.py", line 240, in call
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/core/build_wheel.runfiles/ai_infra/moduluspy/modulus/blocks/data_loaders/multi_source_loader/processors/transform_processor.py", line 275, in call
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/core/build_wheel.runfiles/ai_infra/moduluspy/modulus/processors/processors.py", line 240, in call
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/core/build_wheel.runfiles/ai_infra/moduluspy/modulus/blocks/data_loaders/multi_source_loader/processors/transform_processor.py", line 275, in call
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/core/build_wheel.runfiles/ai_infra/moduluspy/modulus/processors/processors.py", line 240, in call
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/core/build_wheel.runfiles/ai_infra/moduluspy/modulus/blocks/data_loaders/multi_source_loader/processors/transform_processor.py", line 275, in call
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/core/build_wheel.runfiles/ai_infra/moduluspy/modulus/processors/processors.py", line 240, in call
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/core/build_wheel.runfiles/ai_infra/moduluspy/modulus/blocks/data_loaders/multi_source_loader/processors/transform_processor.py", line 275, in call
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/core/build_wheel.runfiles/ai_infra/moduluspy/modulus/processors/processors.py", line 240, in call
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/core/build_wheel.runfiles/ai_infra/moduluspy/modulus/blocks/data_loaders/multi_source_loader/processors/transform_processor.py", line 275, in call
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/core/build_wheel.runfiles/ai_infra/moduluspy/modulus/processors/processors.py", line 240, in call
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/core/build_wheel.runfiles/ai_infra/moduluspy/modulus/blocks/data_loaders/multi_source_loader/processors/transform_processor.py", line 275, in call
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/core/build_wheel.runfiles/ai_infra/moduluspy/modulus/processors/processors.py", line 240, in call
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/core/build_wheel.runfiles/ai_infra/moduluspy/modulus/processors/augment/random_flip.py", line 78, in call
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/util/dispatch.py", line 180, in wrapper
return target(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/math_ops.py", line 2716, in matmul
return batch_mat_mul_fn(a, b, adj_x=adjoint_a, adj_y=adjoint_b, name=name)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/gen_math_ops.py", line 1712, in batch_mat_mul_v2
"BatchMatMulV2", x=x, y=y, adj_x=adj_x, adj_y=adj_y, name=name)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/op_def_library.py", line 794, in _apply_op_helper
op_def=op_def)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/util/deprecation.py", line 507, in new_func
return func(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/ops.py", line 3357, in create_op
attrs, op_def, compute_device)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/ops.py", line 3426, in _create_op_internal
op_def=op_def)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/ops.py", line 1748, in init
self._traceback = tf_stack.extract_stack()
2021-10-26 10:52:42.314258: E tensorflow/stream_executor/cuda/cuda_blas.cc:429] failed to run cuBLAS routine: CUBLAS_STATUS_EXECUTION_FAILED
2021-10-26 10:52:42.314306: E tensorflow/stream_executor/cuda/cuda_blas.cc:2437] Internal: failed BLAS call, see log for details
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/client/session.py", line 1365, in _do_call
return fn(*args)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/client/session.py", line 1350, in _run_fn
target_list, run_metadata)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/client/session.py", line 1443, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.InternalError: 2 root error(s) found.
(0) Internal: Blas xGEMMBatched launch failed : a.shape=[24,3,3], b.shape=[24,3,3], m=3, n=3, k=3, batch_size=24
[[{{node CompositeTransform_6/CompositeTransform_5/CompositeTransform_4/CompositeTransform_3/CompositeTransform_2/CompositeTransform_1/CompositeTransform/RandomFlip/MatMul}}]]
[[gradients/AddN_57/_4723]]
(1) Internal: Blas xGEMMBatched launch failed : a.shape=[24,3,3], b.shape=[24,3,3], m=3, n=3, k=3, batch_size=24
[[{{node CompositeTransform_6/CompositeTransform_5/CompositeTransform_4/CompositeTransform_3/CompositeTransform_2/CompositeTransform_1/CompositeTransform/RandomFlip/MatMul}}]]
0 successful operations.
0 derived errors ignored.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/bin/tlt-train-g1", line 8, in
sys.exit(main())
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/common/magnet_train.py", line 55, in main
File "", line 2, in main
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/detectnet_v2/utilities/timer.py", line 46, in wrapped_fn
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/detectnet_v2/scripts/train.py", line 773, in main
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/detectnet_v2/scripts/train.py", line 691, in run_experiment
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/detectnet_v2/scripts/train.py", line 624, in train_gridbox
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/detectnet_v2/scripts/train.py", line 149, in run_training_loop
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/training/monitored_session.py", line 754, in run
run_metadata=run_metadata)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/training/monitored_session.py", line 1360, in run
raise six.reraise(*original_exc_info)
File "/usr/local/lib/python3.6/dist-packages/six.py", line 693, in reraise
raise value
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/training/monitored_session.py", line 1345, in run
return self._sess.run(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/training/monitored_session.py", line 1418, in run
run_metadata=run_metadata)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/training/monitored_session.py", line 1176, in run
return self._sess.run(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/client/session.py", line 956, in run
run_metadata_ptr)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/client/session.py", line 1180, in _run
feed_dict_tensor, options, run_metadata)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/client/session.py", line 1359, in _do_run
run_metadata)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/client/session.py", line 1384, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InternalError: 2 root error(s) found.
(0) Internal: Blas xGEMMBatched launch failed : a.shape=[24,3,3], b.shape=[24,3,3], m=3, n=3, k=3, batch_size=24
[[node CompositeTransform_6/CompositeTransform_5/CompositeTransform_4/CompositeTransform_3/CompositeTransform_2/CompositeTransform_1/CompositeTransform/RandomFlip/MatMul (defined at /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/ops.py:1748) ]]
[[gradients/AddN_57/_4723]]
(1) Internal: Blas xGEMMBatched launch failed : a.shape=[24,3,3], b.shape=[24,3,3], m=3, n=3, k=3, batch_size=24
[[node CompositeTransform_6/CompositeTransform_5/CompositeTransform_4/CompositeTransform_3/CompositeTransform_2/CompositeTransform_1/CompositeTransform/RandomFlip/MatMul (defined at /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/ops.py:1748) ]]
0 successful operations.
0 derived errors ignored.
Original stack trace for 'CompositeTransform_6/CompositeTransform_5/CompositeTransform_4/CompositeTransform_3/CompositeTransform_2/CompositeTransform_1/CompositeTransform/RandomFlip/MatMul':
File "/usr/local/bin/tlt-train-g1", line 8, in
sys.exit(main())
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/common/magnet_train.py", line 55, in main
File "", line 2, in main
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/detectnet_v2/utilities/timer.py", line 46, in wrapped_fn
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/detectnet_v2/scripts/train.py", line 773, in main
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/detectnet_v2/scripts/train.py", line 691, in run_experiment
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/detectnet_v2/scripts/train.py", line 599, in train_gridbox
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/detectnet_v2/scripts/train.py", line 430, in build_training_graph
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/detectnet_v2/dataloader/drivenet_dataloader.py", line 579, in get_dataset_tensors
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/core/build_wheel.runfiles/ai_infra/moduluspy/modulus/blocks/data_loaders/multi_source_loader/processors/pipeline.py", line 231, in call
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/core/build_wheel.runfiles/ai_infra/moduluspy/modulus/blocks/data_loaders/multi_source_loader/processors/transform_processor.py", line 146, in process
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/core/build_wheel.runfiles/ai_infra/moduluspy/modulus/processors/processors.py", line 240, in call
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/core/build_wheel.runfiles/ai_infra/moduluspy/modulus/blocks/data_loaders/multi_source_loader/processors/transform_processor.py", line 275, in call
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/core/build_wheel.runfiles/ai_infra/moduluspy/modulus/processors/processors.py", line 240, in call
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/core/build_wheel.runfiles/ai_infra/moduluspy/modulus/blocks/data_loaders/multi_source_loader/processors/transform_processor.py", line 275, in call
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/core/build_wheel.runfiles/ai_infra/moduluspy/modulus/processors/processors.py", line 240, in call
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/core/build_wheel.runfiles/ai_infra/moduluspy/modulus/blocks/data_loaders/multi_source_loader/processors/transform_processor.py", line 275, in call
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/core/build_wheel.runfiles/ai_infra/moduluspy/modulus/processors/processors.py", line 240, in call
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/core/build_wheel.runfiles/ai_infra/moduluspy/modulus/blocks/data_loaders/multi_source_loader/processors/transform_processor.py", line 275, in call
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/core/build_wheel.runfiles/ai_infra/moduluspy/modulus/processors/processors.py", line 240, in call
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/core/build_wheel.runfiles/ai_infra/moduluspy/modulus/blocks/data_loaders/multi_source_loader/processors/transform_processor.py", line 275, in call
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/core/build_wheel.runfiles/ai_infra/moduluspy/modulus/processors/processors.py", line 240, in call
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/core/build_wheel.runfiles/ai_infra/moduluspy/modulus/blocks/data_loaders/multi_source_loader/processors/transform_processor.py", line 275, in call
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/core/build_wheel.runfiles/ai_infra/moduluspy/modulus/processors/processors.py", line 240, in call
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/core/build_wheel.runfiles/ai_infra/moduluspy/modulus/blocks/data_loaders/multi_source_loader/processors/transform_processor.py", line 275, in call
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/core/build_wheel.runfiles/ai_infra/moduluspy/modulus/processors/processors.py", line 240, in call
File "/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/core/build_wheel.runfiles/ai_infra/moduluspy/modulus/processors/augment/random_flip.py", line 78, in call
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/util/dispatch.py", line 180, in wrapper
return target(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/math_ops.py", line 2716, in matmul
return batch_mat_mul_fn(a, b, adj_x=adjoint_a, adj_y=adjoint_b, name=name)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/gen_math_ops.py", line 1712, in batch_mat_mul_v2
"BatchMatMulV2", x=x, y=y, adj_x=adj_x, adj_y=adj_y, name=name)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/op_def_library.py", line 794, in _apply_op_helper
op_def=op_def)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/util/deprecation.py", line 507, in new_func
return func(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/ops.py", line 3357, in create_op
attrs, op_def, compute_device)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/ops.py", line 3426, in _create_op_internal
op_def=op_def)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/ops.py", line 1748, in init
self._traceback = tf_stack.extract_stack()

Primary job  terminated normally, but 1 process returned
a non-zero exit code. Per user-direction, the job has been aborted.

mpirun.real detected that one or more processes exited with non-zero status, thus causing
the job to be terminated. The first process to do so was:
Process name: [[47981,1],0]
Exit code:    1

Hi Karan,
Were you able to resolve this issue. I am getting exactly the same error. Any suggestions if you have resolved it.
Thanks.


oh-my-zsh: 2016-04-20 -> 2016-06-18

Motivation for this change
Update to latest version
Things done

[X] Tested using sandboxing
(nix.useSandbox on NixOS,
or option build-use-sandbox in nix.conf
on non-NixOS)
Built on platform(s)

[X] NixOS
[ ] OS X
[ ] Linux

[X] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[X] Tested execution of all binary files (usually in ./result/bin/)
[X] Fits CONTRIBUTING.md.

Looks fine to me, thanks for keeping the package up-to-date!


Run waitpid on childs to ensure no zombies are left behind

What type of PR is this?
/kind bug
What this PR does / why we need it:
We can call waitpid on the process to reparent and ensure no zombies are left behind.
Which issue(s) this PR fixes:
Fixes https://github.com/containers/crun/issues/1482
Fixes https://github.com/containers/conmon-rs/issues/2242
Special notes for your reviewer:
None
Does this PR introduce a user-facing change?

Fixed issue that containers cannot be removed if init command was not found.

Fixes https://github.com/containers/conmon-rs/issues/2242

:warning: Please install the  to ensure uploads and comments are reliably processed by Codecov.
Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 63.33%. Comparing base (4e0f474) to head (9e29d8a).
Report is 722 commits behind head on main.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@             Coverage Diff             @@
##             main    #2415       +/-   ##
===========================================
+ Coverage   37.53%   63.33%   +25.79%
===========================================
  Files          15      121      +106
  Lines        1268     4007     +2739
  Branches      414      830      +416
===========================================
+ Hits          476     2538     +2062
- Misses        526     1469      +943
+ Partials      266        0      -266

/approve
/lgtm


PHPC-430: Fix corruption of Query arguments

https://jira.mongodb.org/browse/PHPC-430

On a related note: https://github.com/mongodb-labs/mongo-php-driver-prototype/commit/187892e89dc87dade37c3c536d0c0131761fd36a#diff-64369bd4c6bdc829a0453db408eeaa75R74

lgtm. Will the PHP7 branch "automatically" get 12265c8 when it gets merged into master, or do we need to specifically cherrypick it?

@bjori: It should make it into the PHP7 branch the next time you rebase on master.


[Feature] v1.2如何训练firefly_gan_vq 有大佬尝试过吗，训练脚本如何配置，直接这样不太行 python fish_speech/train.py --config-name firefly_gan_vq

Feel free to ask any kind of questions in the issues page, but please use English since other users may find your questions valuable.
Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Describe the solution you'd like
A clear and concise description of what you want to happen.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.

有大佬尝试过吗，训练脚本如何配置，直接这样不太行 python fish_speech/train.py --config-name firefly_gan_vq

不推荐自训VQGAN


Sporadically returns 500 error code and doesn't like that I'm not uuid 1000

Sometimes the vbox power api get route will return an http error code.
My uid is 1002, and I'm not seeing my virtualbox machines, but those of another user.

It may occur because that user is not joined to the virtualbox group.


When multiple properties change templates are evaluated in an unexpected order

 Issue by johnmccutchan
Originally opened as dart-lang/sdk#13386

See issue and discussion here: https://github.com/Polymer/polymer/issues/265
John Messerly on web-ui list:
yeah, this is interesting. I dunno if it's a problem in TemplateBinding (formerly MDV) or our port of it. Observers are supposed to run in their birth order, but that doesn't necessarily help when there's all kinds of async delays built into the system.
in this case:

"json" changed -> enqueue the bindProperty observer and the "json.keys" observer
run bindProperty observer, it enqueues change to "isMap"
run the "json.keys" observer (it was already queued)
run the "isMap" observer

Maybe we're getting the order of #3 and #4 wrong... certainly, the "isMap" observer should have priority over the "json.keys" observer, because it was registered first.
Tentatively, I think this is a package:observe bug.

 Comment by johnmccutchan

Added Area-Polymer label.

 Comment by jmesserly

most likely in the observe library, or it is a bug/design issue in JS too.
I'm curious to see if it reproduces in JS.

Added Library-Observe label.

 Comment by sigmundch

Removed this from the Later milestone.
Added this to the M9 milestone.

 Comment by sigmundch

Added this to the M8 milestone.

 Comment by clayberg

Removed this from the M9 milestone.
Added this to the 1.1 milestone.

 Comment by jmesserly

chatting with Polymer.js team, the general feeling is one can't rely on <template if> to be a guard. I would still like to see if we can do anything on the Dart side but I don't think we have a design to land in time for M8

Removed this from the M8 milestone.
Added this to the Later milestone.

 Comment by jmesserly

Removed the owner.

 Comment by sigmundch

Removed this from the 1.1 milestone.
Added this to the 1.2 milestone.

 Comment by jmesserly

Removed Area-Polymer label.
Added Area-Pkg label.

 Comment by sigmundch

Removed Priority-Unassigned label.
Added Priority-Medium label.

 Comment by anders-sandholm

Removed Library-Observe label.
Added Pkg-Observe label.

 Comment by kasperl

Removed this from the 1.2 milestone.
Added this to the 1.6 milestone.

 Comment by sigmundch

Removed this from the 1.6 milestone.
Added Polymer-P-1 label.

 Comment by sigmundch

Removed Polymer-P-1 label.
Added Polymer-Milestone-Next label.

 Comment by sigmundch

Issue dart-lang/sdk#19943 has been merged into this issue.

cc @jmesserly.
cc @justinfagnani.

 Comment by jmesserly

chasing down the thread, this was fixed in: https://code.google.com/p/dart/issues/detail?id=13393
the general async nature of things is by design though (at least in polymer.js 0.5.x):
https://github.com/Polymer/polymer/issues/265#issuecomment-25116261
polymer Polymer.js 0.8-preview does not bind properties in the same way, so it shouldn't have the same bug.

Added NotPlanned label.

 Comment by sigmundch

Removed Polymer-Milestone-Next label.
Added PolymerMilestone-Next label.


feature request: lora.tx_only configuration option

This commit suggests a new config option in the lora section: tx_only (-> assurance, that TX never happens)
At most locations, we operate in listen-only mode.
-> digi.active = false
digi.beacon = false
Sometimes, we are behind an rx rf-amplifier, or use the rx-antenna together with
an other rx.
Then it's very important inhibit unexpected or TX, i.e. after config loss. Else it
would seriously damage our hardware.
If there are bugs, or if new features will come in (i.E. initial beacon on boot up),
I suppose a variable
tx_enabled in the 'lora' section.
which defaults to false.
This variable assures that digi.active and digi.beacon is set to false, txpower is
set to 0dBm, and that the tx-routine is never entered.
=> The new configuration variable in is-cfg.json in the
"lora": {
}
section is
tx_enabled = false
I.e.:
..
"lora": {
"frequency_rx": 433775000,
"gain_rx": 0,
"frequency_tx": 433775000,
"power": 20,
"spreading_factor": 12,
"signal_bandwidth": 125000,
"coding_rate4": 5,
"tx_enabled" = false
},
..
Set tx_enabled to true if you are sure you safely can tx on 70cm lora.
Signed-off-by: Thomas Osterried<EMAIL_ADDRESS>

Es feht noch ein "bool tx_enabled;"  in der project_configuration.h Zeile 79, also bei mir.

closing with feature implemented in https://github.com/lora-aprs/LoRa_APRS_iGate/pull/164. But thank you for your work! 👍


FlutterEngineTest.CanOverrideBackgroundColor is flakey

I've seen FlutterEngineTest.CanOverrideBackgroundColor flake a fair bit the past couple of weeks.
Example instance
See 2nd failure compared to first failure: https://ci.chromium.org/ui/p/flutter/builders/try/Mac Engine Drone/788840/overview
related issues

https://github.com/flutter/flutter/issues/124677

cc @cbracken

Thanks; re-disabling it. There are a variety of flakes in the macOS tests that I've been fixing related to re-use of resources. Thought that perhaps the memory leaks (there was no cleanup/autorelease pool wrapping the tests, would do it, but there's a separate class of resource-related flakiness in some tests where NSWindows are created, which is the case for this test. I'll see if working around that will work. Re-disabling it for now though.

Fixed in flutter/engine#51669


aggregation statistics

Something is not right in the aggregation statistics in some cases.

Compare toy problem evidence with pychord and dynesty

Apparently it was not aggregation statistics, but rather in the sampling from multi-ellipsoids constrained to the unit-cube.
In the logical branch where we constrain to the unit-cube we had forgotten to exponentiate the RV by 1/dim, and as a consequence the samples were non-uniformly distributed in the elliposid.


Move paging to common module

@msfjarvis can you review the code (needs some cleanup like removing the reload method and spotless)

Did some initial work on this https://github.com/msfjarvis/multiplatform-paging/tree/hs/invalidate-support

Rebased

Pursuing #313 instead


AHK V2 Throws an error on unassigned variables

Hi, I'm trying to use the v2 of this class but it seems to throw an error pointing to the ImageArray method:
	217: }
	...
	235: {
▶	236: If (!c := this.ImageArray(image,a,variance,centerResults))
	237: Return 0
	238: min := maxRadius

The code that I'm using for testing:
...

scan := new ShinsImageScanClass(defaultApp)
scanImageArray := []

MsgBox scan.ImageArray("base.png", scanImageArray)

It seems that same thing happens on any class method I test.

Ahk v2 has different syntax, I don't personally use v2 so I can't say exactly, but I think it should look like this
scan := ShinsImageScanClass("window title")

MsgBox scan.ImageArray("base.png", &scanImageArray)

Thanks for the response, I did try that yesterday but the error points on the same line. I was able to fix it by initializing the variable. so that fixed the issue as well. I'll create a pull request if you don't mind.
In V2, creating an instance of the class by using "new" throws an error as well:
scan := new ShinsImageScanClass(defaultApp) ; throws an error
scan := ShinsImageScanClass(defaultApp) ; doesn't throw an error

Ah ok I see the issue, there was a missing ampersand, the issue has been fixed
ahk v2 does not use new when instantiating a new class
thanks for the feedback, let me know if anything else comes up


Patch: Remove deprecated CaseTemplate query key

Fix #183

This is needed for TheHive 3, but useless in fact for TheHive 4.

Hello @milesflo we are still using the status field in TheHive 3, so I would keep it for TheHive 3, but will remove it from TheHive 4.
I'll close this PR and fix the bug.

Thanks for the PR anyways ;)


Clone WIndows VM (winrm) with Bootstrap Option ?

It seems like the --bootstrap option will not work when cloning a Windows VM that has winrm setup.
Waiting for sshd...DEBUG: value for config item ssh_port: 22
.DEBUG: value for config item ssh_port: 22
.DEBUG: value for config item ssh_port: 22
.DEBUG: value for config item ssh_port: 22

i am getting the same problem however mine just hangs on "Waiting for sshd..." any ideas?

Hey folks, can you have a look at the latest version of this plugin? winrm support was recently added.
Closing this issue. If you end up with any problems do let us know by opening a new issue.


Pls support Android Studio

I do at least half my programming in AndroidStudio and would appreciate this as well.

We now support Android Studio! https://tabnine.com/install


tonumber does not work as expected

[x] tonumber(" ") gives 0 instead of nil.
[] Assertion failure: assert(tonumber'+0.01' == 1/100)
[] Assertion failure: assert(not tonumber'1.0e+')
[] Hex strings like 0xdeadbeef are not recognized

This is a bug. I think we should fix it.

These two issues are not important:
[] Assertion failure: assert(tonumber'+0.01' == 1/100)
[] Assertion failure: assert(not tonumber'1.0e+')
The "0xdeadbeef" issue should be fixed.


Make this anonymous inner class a lambda

To improve the code quality, sonsider make this anonymous inner class a lambda.
Before Java 8, the only way to partially support closures in Java was by using anonymous inner classes. But the syntax of anonymous classes may seem unwieldy and unclear.
https://rules.sonarsource.com/java/type/Code%20Smell/RSPEC-1604

@xuejiangtao Thanks for contributing!

thanks for merging!


Update FAQs.md

EU R&D projects based on CORTX

Will request all parters to do the PR against their names (which is incomplete now)  once this edit is merged

Will request all parters to do the PR against their names (which is incomplete now)  once this edit is merged

Not sure how the page looks like after my latest commit. I would request to still merge it as I will do another "clean up" PR once all partner inputs are available, if thats Ok.

Not sure how the page looks like after my latest commit. I would request to still merge it as I will do another "clean up" PR once all partner inputs are available, if thats Ok.


can not detect model type of : unet/iclight_sd15_fc.safetensors

i can not fix this bug, please !

Read https://github.com/huchenlei/ComfyUI-IC-Light-Native#download-models.

@zhenhua22
ComfyUI-IC-Light-Native 's checkpoint is different from IC-Light-SD-WebUI's, so, need to download right model that @huchenlei has mentioned

Read https://github.com/huchenlei/ComfyUI-IC-Light-Native#download-models.

I followed the instructions on the web page above, and I still get the error in the picture

Do you fix it? I have same problem.


Why not Platformio?

Hi! Arduino IDE is really outdated environment. One of it's problems is dependency management - you need to copy them manually. It's not possible to reliably update and set versions.
Platformio does have library management and env setup is only click compile. It will download all necessary and needed versions of libs and arduino framework.
Also, with platformio its easy to integrate CI service (like Travis CI or Github Actions) to automate code checks (linters, build checks and so on) as well as automatic releases.
For example, marlin 2.0 firmware migrated to platformio, so its not something "special".
Is there are any plans to migrate the development to platformio?

Version 6.2.1 is output of PlatformIO.


Decryption capability

I may be misunderstanding the code, but can this tool decrypt the aktsk AssetBundles or is it just information at this time?
If so then how exactly?, as I have not been able to determine the process.

Where and how does the tool expect the bundle files?
From what I can see from main.go, it expects the bundle files to be in resleriana_tools-main\main\dat.
But this does not change what looks to be a file not found error.
goroutine 1 [running]: main.main()         C:/Users/adamw/Downloads/resleriana_tools-main/main/main.go:62 +0x4ed exit status 2
The catalog.json file implies that bundle files should be in an Embed folder and with the extension .bundle, but this seems to have no effect.


using proxy in VR like v2ray or clash, MFP can't  connect deovr

Describe the bug
when using proxy in vr

when close proxy in vr

I think this problem is kind of important, cuz some aisa country like china can't access to google or SLR

I don't know what could be done in MFP to "fix" this. There is nothing to be done if it can't access the IP on local network. This is more of a firewall/networking issue. I'm assuming you are running the proxy on your quest, it probably blocks all lan traffic and routes everything thru a tunnel. You need to make it to allow access to lan.

Closing, if you think this is actually a bug comment here and I'll reopen.


Autocomplete doesn't show columns of a table in first shot

if u type the following
SELECT actor. than the Popup doesnt show columns
Autocomplete dosn't show columns in first shot

If u remove . after typed it and wrote again a fullstop, than columns are shown

This is already fixed in recent SQL completion improvements.
Now you also can use aliases and DBeaver will show column lists after a dot char.
Thanks for the report anyway


Extension causes high cpu load

Issue Type: Performance
Extension Name: bracket-pair-colorizer-2
Extension Version: 0.0.28
OS Version: Darwin x64 18.5.0
VSCode version: 1.31.1

:warning: Make sure to attach this file from your home-directory: /Users/thirstytraveler/CoenraadS.bracket-pair-colorizer-2-unresponsive.cpuprofile.txt :warning:
Find more details here: https://github.com/Microsoft/vscode/wiki/Explain:-extension-causes-high-cpu-load

Duplicate of #55. Please close this and post there instead.
@MasaruKitajima


Minimized version for production use?

Would be nice to have a version of the file that is already minimized for production use.

This is now available.


[RHCLOUD-19663] Implement Simple HPA for clowder

This patch adds simple HPA (k8s included HPA) to Clowder. Configuration is simple, requiring only an autoScaler mode of simple on a ClowdEnvironment as well as new simpleAutoScaler options for Deployments. Scaling can be performed on CPU and RAM resources reaching thresholds defined as either average utilization or an average value (or both!)
Examples:
  - name: highcpu
    podSpec:
      image: quay.io/rh_ee_addrew/highload:latest
    simpleAutoScaler:
      replicas:
        min: 1
        max: 4
      cpu:
        scaleAtUtilization: 80

The above deployment will scale to a maximum of 4 replicas as each reaches an average CPU utilization of 80%
  - name: highram
    podSpec:
      image: quay.io/psav/clowder-hello
    simpleAutoScaler:
      replicas:
        min: 1
        max: 4
      ram:
        scaleAtValue: 1Mi

The above deployment will scale to a maximum of 4 replicas when RAM utlization reaches an average value of 1Mi.
A kuttl test that validates the functionality is included that observes replica counts as assertions.
This is a fairly large change so I'm sending the code up now even though I intend to amend the PR with further changes to add unit tests.
The HPA functionality requires metrics server to work. I've tried to update docs, scripts, etc that run minikube to include metrics-server wherever possible, but I could have missed something - we'll see. I also have no idea if this works on open shift yet because of that metrics-server dependency.

/retest

/retest

/retest

 --- FAIL: kuttl/harness/test-ephemeral-gateway (403.89s)

/retest

15:12:13         --- FAIL: kuttl/harness/test-kafka-strimzi-topic-basic (404.56s)
15:12:13         --- FAIL: kuttl/harness/test-kafka-strimzi-ephemeral (405.26s)
15:12:13         --- FAIL: kuttl/harness/test-kafka-strimzi-pvc (405.62s)

:shrug: :question:
We had a kafka / ephemeral outage yesterday so maybe it was that? Let's /retest again!

All my clowder PRs are failign with the same tests (listed above) failing. Trouble is afoot at the cirlcle k.

We should also ensure that there is a validating webhook configuration that prevents trying to set simple and the pass through or keda at the same time

We should also ensure that there is a validating webhook configuration that prevents trying to set simple and the pass through or keda at the same time

Now that's spicy - I don't know how to do that. I'll poke around and see if I can figure it out.

Wait, the way its implemented you can't set multiple HPA types on a given ClowdEnvironment :
apiVersion: cloud.redhat.com/v1alpha1
kind: ClowdEnvironment
metadata:
  name: test-autoscaler-simple
spec:
  targetNamespace: test-autoscaler-simple
  providers:
    autoScaler:
      mode: simple

You set the ClowdEnvironment autoScaler mode to simple and then you set simpleAutoScaler on any deployments you want in that env's apps. So, based on that you shouldn't be able to ever have 2 auto scaler modes active in a given environment.
Is that good enough? Or do we need to worry about one ClowdEnv using keda and another using simple. I would think that would be OK.

Found an old option in clowdapp_types.go under AutoScaler that was left in from earlier in development but doesn't actually function. Also, pre-push.

Alright mate, this turned out to be a slightly larger change than I think we expected in review. But, I think the end result is significantly better in basically every way. Waiting on tests, will mop up any problems and should be g2g tomorrow.

Ugh, I'm going to change it again. I changed AutoScaler.Mode of type string to AutoScaler.Enabled of type boolean because I thought it was cleaner and more accurate. But I notice a couple of things:

Most ClowdEnvironment options are set with a mode flag that is set to none if we don't want it active
As I have it now this is a breaking change which is probably not a good thing

So, at least one more commit incoming. Grr.

/retest

Today did not go well. Ran into problem after problem. I think we're going to need to team up on this. Stuff I hit:

HPA doesn't want to be owned by ClowdApp; it complains about not being able to find the target deployments
wrapping errors in ProvideSimpleAutoScaler causes the HPAs to never get created

OK I believe I hit all your input. Pretty happy with this last set of changes: much refine, very quality. Passing tests locally so we'll see if Jenkins is happy. If he's cool then I'll ping you.


Fix inheritance for actions and mutations (#20)

Improve tests to expose unexpected behavior in actions and mutations when multiple modules inherit from the same base class.
Implement the minimal fix.

All the changes look fine!
However I don't like the naming of the mutations/actions in the inheritance tests, as it is impossible to read what the actions/mutations does by its name. This is not your fault, as they were already named this way before your changes (the files somehow got merged without me noticing it), but if you don't mind, feel free to change them to more meaningful names (e.g. action3 -> updateToDoubleFoo or likewise). I can also do it myself after merging the pull request which I'll do after you've decided whether you want to commit your other improvements or not. :)

As I mentioned in the issue, for me it would be the easiest, if you merge the PR first. For additional improvements I would create separate PRs. Because this one was clearly a bug it should get fixed before any refactoring, even if I needed some refactoring before to wrap my head arround the code and identify the problem.


Tech Debt (Placeholder)

Things we need to be thinking about/expanding on:

Unifying the signup/confirmation process (we can always elevate to admin with claims through some other process)
Getting a templating process in place for emails/SMS such that templates and related language are defined external to the source code (but preferably under source control)
Ensuring users can manage notification preferences (SMS, email, digest, DND)
Pushing the code base to use a unified messaging service/command in such a way that the caller doesn't need to know how the user prefers to be contacted.

big fan of numbers 3 and 4.

Adding one item to the list for discussion and planning:

Domain driven design and single responsibility. We're starting to get some quite complex logic on our EF model classes and sometimes repeated in the ViewModels as well. Ideally it feels as if we're at the point where some of the more complex business logic may be better suited to domain models groups into bounded contexts. These classes would model the behaviours the campaign types and not concerned with persistence.

@stevejgordon, 100% agree. I work with bounded context at my current job. Simple logic on the model classes is not necessarily a problem, but I think the ViewModels are WAY too complex, use persistence, etc...
starting with the ViewModels would be a great place to start. Not every mapping between model and ViewModel and vice-versa need the same logic, and I'd argue that some of the logic in the extension methods that do that don't even belong in those extension methods...

@stevejgordon @mgmccarthy Yeup, a bounded approach is a good thing.
Pragmatically speaking, some of what is going on that we don't like (smart model classes, persisty-ish VMs) are artifacts of a) first attempts by early contributors b) symptoms of missing bits from early ASP.NET builds or c) when people were trying to get things in for a demo etc.  So, I'd just like to suggest that even before we make wholesale changes we get our current architecture leveled off. And I do believe we're getting close.
I want to avoid the whole lava-layer architecture bits that tends to happen on projects where the hands on the keyboards change frequently.
We inherited this code base and not all the bits are handsome, but this is our date and the dance is quickly approaching. :dancers:  :laughing:
The move to command/query is a significant jump forward in architecture from the everything-in-one-repository approach, and it may not be perfect...but it is approachable. I can explain command/query in 10 minutes to a junior programmer, walk them through a sample at a code-a-thon and set them loose on the code. It's just a hammer and not always the right tool, but I feel compelled to balance architectural bliss with approach-ability. Does that make sense?
This would be a great conversation over :beers: or other non-alcoholic beverages if that's your tickle.

@MisterJames, CQRS is definitely the way to go here, and like you said, it's not that tough to explain to another programmer. Mediator makes that possible as a framework. Approaching the system this way makes it easier to spot duped code or code in the wrong place b/c we should name commands in terms of the domain. To me, it's a way to clarify what the business meaning is as an operation after how the command/events are named.
One more thing i'd like to add to this is list is to get rid of the entire IAllReadyDataAccess API. CRUD-based API's obfuscate the business intent. The artifacts of this are present in the implementation of these interfaces, where some of these methods are doing a LOT of DB work, checks, rules, update, deletes etc.... does all this need to be in one handler? Should two different areas of the site be sending commands to do the same thing? etc... this ties directly into identification and enforcement of bounded context.
You could argue that the Web API controllers we have are also CRUD-based API's, and as such, COULD be renamed to something clearer (or it they're not being used, done away with altogether) ,but it's not that important in the long run.


Insecure Registry Connection Failed

Problem description
docker pull private registry hostname:imagename results in:
"Error response from daemon: Get registry server hostname:port: dial tcp: lookup registry server hostname on <IP_ADDRESS>:53: no such host "
after an upgrade from <IP_ADDRESS> to <IP_ADDRESS>
Unable to connect to insecure registry on <IP_ADDRESS>, was able to connect previously on <IP_ADDRESS> with same settings.  Registry is set in docker daemon settings.   Providing the server IP directly does not work either.
Project version(s) affected
<IP_ADDRESS>
Suggestions for a fix

Thank you for your feedback. However, it looks like your issue is not about Docker documentation, but that you may be having trouble with Docker itself. Please file a Docker support ticket or check out https://forums.docker.com/ to see if you can find a solution.
If this really is a documentation issue and I have misunderstood,please reopen it with more information.


Update themes.txt

You need to add some images to your repo, see README.

You need to add some images to your repo, see README.

@bep sorry for that, already ben there also change ratio images 🙏 tryng pushing again. before that i also change my repository name


fix: empty div if there's no remaining content

The title is self-explanatory.
I have ambition to contribute to this project more, you have done very good job so far and the plugin is very useful to me. Is it ok if I work on this more? I don't want to interfere with your plans.

@all-contributors please add @chovanecadam for fixing bugs

@chovanecadam Really appreciated! Feel free to contribute. I am happy that this small plugin did help you!
I am building my own site myself and also have 2 ongoing projects. It's possible that I would create more remark plugins to cover my own use case and also share them with the community. :)


Materialize Select is flickering in Firefox

When clicking on a "Materialize Select" in Firefox at least 36 and up, the select is flickering before opened. I can see a scrollbar before it's hidden. Works nicely in Chrome.

Scrollbar is gone, but it still flickers 2-3 times before it opens.

Fixed in 90afeb0070d7270efe72f3edfbca712e7abec156


Merge dask-expr repository into dask

[ ] Closes #xxxx
[ ] Tests added / passed
[ ] Passes pre-commit run --all-files

We will have to use a merge commit here, not squash commit


Distribution Support Request

Name of the Distribution
Slackware
Version of the Distribution
15.0
Package Types

[ ] Deb
[ ] RPM
[X] Tar.gz
[ ] Snap - Stop! Please file your issue in PowerShell-Snap instead.

Processor Architecture
X86-X64
.NET Core Support

[X] The version and architecture of the Distribution is supported by .NET Core.

Distribution Requirements

[X] The version of the Distribution is supported for at least one year.
[ ] The version of the Distribution is not an interim release or equivalent.

Exemption Justification
No response
Progress

[ ] An issue has been filed to create a Docker image in PowerShell-Docker

For PowerShell Team ONLY

[ ] Docker image created
[ ] Docker image published
[ ] Distribution tested
[ ] Update packages.microsoft.com deployment
[ ] Lifecycle updated
[ ] Documentation Updated

Slackware predates all of the shown distributions by many years.  There are many contributors, and I can easily use someone else's packaging tools to create a package for the current release of Powershell for Slackware. But for now building there is not supported. It should be because it is marginally easier.

There is slackbuild (precompiled)

There is slackbuild (precompiled)

That's a build from a compressed tar release of PowerShell. I'm bringing up the problem that someone, myself, possibly can't build from source code because the tools included don't recognize Slackware. At least they did not back in the 14.0 series which was about six years ago. Now I'll go ahead and try again and capture the output and present it here.

Follow up:
I just followed the good steps and it responded:
Get-PowerShell MASTER Installer Version 1.2.0
Installs PowerShell and Optional The Development Environment
Original script is at: https://raw.githubusercontent.com/PowerShell/PowerShell/master/tools\install-powershell.psh
Arguments used:
Operating System Details:
OS: linux
DIST:
DistroBasedOn:
PSUEDONAME:
REV:
KERNEL: <IP_ADDRESS>-microsoft-standard-WSL2
MACH: x86_64
OSSTR:
Sorry, your operating system is based on  and is not supported by PowerShell or this installer at this time.
I'll see if it is possible to invoke the PowerShell script I saw there.

That's a build from a compressed tar release of PowerShell

Yes, I always use it myself (Arch). There is no need to build anything, just unpack, chmod pwsh, symlink and use.

@DoctorWho8 I notice you ticked the box marked "The version and architecture of the Distribution is supported by .NET Core." when Slackware is absent from that list.
I suggest that before there can be a PowerShell for Slackware there needs to be a dotnet. sdk


Fix how git is installed in bundles

What is being addressed
Using apt-get install in bundles should use a cache mount.
How is this addressed

Update all relevant bundles to use a docker cache mount when installing software with apt-get.

/test-extended

/test-extended

/test-extended


Inconsistent info in tooltips

For the same code, I keep getting different stuff in my tooltip hints:
1.

2.

3.

I would prefer for it to always be 3, but most of the time I get the result from image 2. Is there anything I can do to make it so that it is always like image 3?
Here are my Tern user settings:
{
  "tern_argument_hints": true,
  "tern_output_style": "tooltip",
  "tern_argument_completion": true
}

Thanks in advance!

This seems to actually happen if you close the editor. If you create the sum function during the current session and write the comments like so:
sum.js
/**
 * Returns the sum of two numbers
 * @param  {Number} a The first operand
 * @param  {Number} b The second operand
 * @return {Number}   The sum of both operands
 */
function sum (a, b) {
  return a + b
}

module.exports = sum

and then i go into app.js and do the following:
const sum = require('./sum')

sum(

I get the correct tooltip:

However, if I restart the editor and try to type sum( again, I get a different output:

Is there any way around this?

Could you provide me with the relevant part of your math.js file, so that I can see if I can reproduce this?

Ah sorry, it was actually the same thing as sum.js I just named it differently that time.
Steps to reproduce:
1. Create an app.js and sum.js and open them in Sublime
2. Add the following to sum.js:
/**
 * Returns the sum of two operands
 * @param  {Number} a The first operand
 * @param  {Number} b The second operand
 * @return {Number}   The sum of both operands
 */
function sum (a, b) {
  return a + b
}

3. Add the following to sum.js:
const sum = require('./sum')

sum(

At this point, everything works fine:

Now, if you exit out of Sublime and reopen it, and then try doing sum( again, you will get this output instead:

I didn't manage to reproduce this. If I add the module.exports = sum line to the content you describe, I get working argument hints.


Get rid of junk in storage proofs.

Currently block execution storage proofs contain nodes that are not actually necessary to validate the block.
There's a common pattern in the runtime that we use to store intermediate values. Values that need to be passed around between extrinsics. Such values are put in the storage and are removed at the end of block execution with take. Here's one example: https://github.com/paritytech/substrate/blob/df4a58833a650cf37fc97764bf6c9314435e3cb2/frame/system/src/lib.rs#L1361
The problem here is that take attempts to delete the value from the trie, even if it is not there. This attempted deletion query ends up being part of the proof.
A proposed solution would be to use revert rather than take. revert would simply delete the key from the state overlay.
Alternatively we could use static memory instead of storage API for intermediate values. This would also be a significant performance win, although some additional work would be needed to implement transactional semantics with this method.

Putting those values in the memory is a bigger task than it may sound.
The problems is that we generally do not reuse runtime instances. E.g. in the block builder in order to push an extrinsic we create a new runtime instance. The runtime instance is destroyed (semantically  after the call has finished.
 If an extrinsic panics during the inclusion, we should be able to rollback its changes. Rolling back storage overlay changes is easy. However, rolling back changes to the wasm instance itself is not. It would require something like https://github.com/paritytech/substrate/issues/2980. It is achievable, but is very involved.

Putting those values in the memory is a bigger task than it may sound.

Looking at @cheme's suggestion here, as a non-expert in the field, I am actually tempted to say a transient OverlaySetChange with new host functions to manipulate it is a very easy way to solve this.


Add share possibility for bookmark

Add a share button  for the different social platforms (twitter, linkedin)

share for twitter and fb available


iOS landscape mode support

Is there an existing issue for this?

[X] I have searched the existing issues and didn't find mine.

Steps to reproduce
!NOTE! I reopen this issue as far as this is not fixed and changes was rolled back, but a lot of people need this so bad.
iOS testing currently does not support landscape mode.
As a quick test, download the samples:
maestro download-samples
cd ./samples
unzip sample.zip
xcrun simctl install Booted Wikipedia.app

And update the iOS flow to:
appId: org.wikimedia.wikipedia
---
- launchApp
- tapOn: "Search Wikipedia"
- inputText: "tacos"

Launch the simulator and rotate into landscape mode, and then run:
maestro test ios-flow.yaml
The test reports that it passes, but it does not navigate into the search screen to type into the text.
Running maestro studio shows a mismatch where the overlay content appears to be laid out in landscape, but the image of the app's screen is rotated 90 degrees:

Actual results
Test steps marked as passed but nothing is happening on the simulator.
Expected results
Correctly interacting with simulator when actions performed by tests.
About app
Application from samples
About environment
I'm on MacBook Pro M3 running on Sonoma 14.6.1
Logs
No response
Maestro version
1.37.9
How did you install Maestro?
Homebrew
Anything else?
No response

thanks, I remembered about it but forgot to re-open #565. now that it's re-opened, i'll close this as a dupe

@bartekpacia You've reopened #565 but the conversation is locked

Sorry, I forgot to unlock the conversation. Should be good now.
As for iOS landscape support - we plan to re-add it soon.

Sorry, I forgot to unlock the conversation. Should be good now.
As for iOS landscape support - we plan to re-add it soon.

thank you

Sorry, I forgot to unlock the conversation. Should be good now.
As for iOS landscape support - we plan to re-add it soon.

thanks so much, waiting for this coz our app is working in landscape only and we really want to test it using Maestro


Upgrade to support library v26.0.0

The bugfix is available as a beta version: v<IP_ADDRESS>-beta2.
Check out the v26-beta branch for more details.
Once the support library gets the stable version, it will be released to the master branch too.

Support library 26.0.0 is now available. Will be nice if the beta branch can be released

I'll try to upgrade asap but there's a problem with the latest versions of Android Studio 3.0 Canary where it wouldn't find certain constants (like selectable background) so it wouldn't compile.

Nevermind, seems like the new 26.0.0 release version has fixed it. Will release in a few minutes.

Released v<IP_ADDRESS>.


Merge prTesting_201906111034 branch to master branch

Docs Build status updates of commit 0748b1c:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details
Dependent Files

split/updateTestCon.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


prove make_expr_datetime_component and match_datetime_component

[!WARNING]
This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

#2159  👈 (View in Graphite)
#2144 : 2 other dependent PRs (#2104 , #2155 )
#2097 : 1 other dependent PR (#2098 )
#2158
#2096 : 1 other dependent PR (#2156 )
#2157
#2095
#2091 : 1 other dependent PR (#2153 )
#2089
main

This stack of pull requests is managed by Graphite. Learn more about stacking.


Add operation ID header to all outbound requests

What type of PR is this?
(bug)
When testing, we discovered that we are missing the "X-Operation-Id" header from all responses to the new '/clusters' and '/upgrade_policies' endpoints. This means that any non 200 request to consumers of the new OCM Agent API will not be able to track down the source of their bug in the OCM audit logs.
What this PR does / why we need it?
This PR includes "X-Operation-ID" header on all responses from new 'endpoint proxy' routes.
Which Jira/Github issue(s) this PR fixes?
Fixes #
https://issues.redhat.com/browse/OSD-20054
Special notes for your reviewer:
Pre-checks (if applicable):

[ ] Tested latest changes against a cluster
[ ] Ran make generate command locally to validate code changes
[ ] Included documentation changes with PR

Codecov Report

Merging #73 (2ffa478) into master (ba80593) will increase coverage by 0.29%.
The diff coverage is 96.29%.

Additional details and impacted files

@@            Coverage Diff             @@
##           master      #73      +/-   ##
==========================================
+ Coverage   46.23%   46.52%   +0.29%
==========================================
  Files          20       20
  Lines        1261     1268       +7
==========================================
+ Hits          583      590       +7
  Misses        638      638
  Partials       40       40

Files
Coverage Δ

pkg/handlers/cluster.go
81.48% <100.00%> (+0.71%)
:arrow_up:

pkg/handlers/handlers_common.go
100.00% <ø> (ø)

pkg/handlers/upgrade_policies.go
79.43% <95.45%> (+1.22%)
:arrow_up:

/lgtm

Build and run using this PR, tested against a staging cluster yury-addon-test, can get the "X-Operation-ID" header on all responses from new 'endpoint proxy' routes.
curl http://localhost:8081 -vvv | grep -i X-Operation-ID
* Rebuilt URL to: http://localhost:8081/
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying <IP_ADDRESS>...
* TCP_NODELAY set
* Connected to localhost (<IP_ADDRESS>) port 8081 (#0)
> GET / HTTP/1.1
> Host: localhost:8081
> User-Agent: curl/7.61.1
> Accept: */*
>
< HTTP/1.1 200 OK
< Content-Type: application/json

< X-Operation-Id: 46ed8eb6-31ff-4a2f-80ac-b7333e73dba4  <<<<<<

< Date: Mon, 18 Dec 2023 02:22:02 GMT

Also successfully displaying operation ID from OCM, too:
ts=2023-12-18T05:19:34.509128136Z level=info msg="Using OCM as the upgrade config provider"
ts=2023-12-18T05:19:34.51132507Z level=info logger=ocm-config-getter msg="Commencing sync with OCM Spec provider"
ts=2023-12-18T05:19:34.596395638Z level=info logger=ocm-client msg="request to 'http://ocm-agent.openshift-ocm-agent-operator.svc.cluster.local:8081' received response code 200, operation id: '4a0c727c-94c9-4f63-9cf6-4b80580449f2'"
ts=2023-12-18T05:19:34.658742417Z level=info logger=ocm-client msg="request to 'http://ocm-agent.openshift-ocm-agent-operator.svc.cluster.local:8081/upgrade_policies' received response code '200' from OCM upgrade policy service, operation id: 'd7d4674f-2d08-484e-bb5d-01dc55acb030'"
ts=2023-12-18T05:19:34.658864736Z level=info logger=ocm-config-getter msg="No upgrade policies available"
ts=2023-12-18T05:19:34.658895196Z level=info logger=upgrade-config-manager msg="no provider specs found and no UpgradeConfig on cluster, nothing to do"
ts=2023-12-18T05:20:35.202789231Z level=info msg="Using OCM as the upgrade config provider"
ts=2023-12-18T05:20:35.205217415Z level=info logger=ocm-config-getter msg="Commencing sync with OCM Spec provider"
ts=2023-12-18T05:20:35.278538071Z level=info logger=ocm-client msg="request to 'http://ocm-agent.openshift-ocm-agent-operator.svc.cluster.local:8081' received response code 200, operation id: '695f79cb-ee9e-40a0-a04f-a8a4bbd2fd4c'"
ts=2023-12-18T05:20:35.357886592Z level=info logger=ocm-client msg="request to 'http://ocm-agent.openshift-ocm-agent-operator.svc.cluster.local:8081/upgrade_policies' received response code '200' from OCM upgrade policy service, operation id: '2714be1e-e3e8-4d95-ad4f-33ba55e98625'"
ts=2023-12-18T05:20:35.357904718Z level=info logger=ocm-config-getter msg="No upgrade policies available"
ts=2023-12-18T05:20:35.357908262Z level=info logger=upgrade-config-manager msg="no provider specs found and no UpgradeConfig on cluster, nothing to do"

/approve
Thanks a lot Hector and Shawn !!


Add new point to existing polygon

Existing polygon should be deleted and redrawed to make corrections. For this, the new ponit should be added to existing polygon

This is currently available through the Better Polygon plugin, which will probably replace the default polygon tool in the future.


Revert "network: Detect early that NotificationOutSubstream was closed by the remote"

Reverts paritytech/substrate#13396
After the PR the reported peer count dropped to ~0 peers, and sync doesn't really work (best block reported stays the same). I was not able to find a quick fix for this, so let's revert the PR for now.

I wonder why present tests didn't detect it though :thinking:

I don't think this change really broke syncing though. The problem is that the peer count is just reported incorrectly to be 40 because the local node previously didn't know the substream was closed BUT because block request handler in either end is not informed of connected/disconnected sync peers and keeps on serving block requests as long as any connection is open, syncing works. So in other words the syncing works because there's a bug in the block request handler.

I think the same thing. See also my comments here: https://github.com/paritytech/substrate/pull/12434
One node with this bug fixed will indeed have trouble syncing, just like smoldot has trouble syncing today. But if every node on the network had this bug fixed, then all nodes would (normally) sync fine.