Problem with gpu memory allocation

Click to expand!
Issue Type
Others
Source
source
Tensorflow Version
2.8.0
Custom Code
Yes
OS Platform and Distribution
Windows 10
Mobile device
No response
Python version
3.9.8
Bazel version
No response
GCC/Compiler version
No response
CUDA/cuDNN version
11.2
GPU model and memory
notebook version of nvidia gtx 1060 (6GB)
Current Behaviour?
I use tensorflow for image classification (20 classes) with convolutions. My dataset contains about 20000 train images and 5000 test images. Images (RGB) have 200x256 pixels. When I run script to train model using cpu everything seems fine. However when I try run script using gpu, after loading my training and test data I get error on model_fit function.

Num GPUs Available:  1
2022-05-04 17:58:58.482057: I tensorflow/core/platform/cpu_feature_guard.cc:151] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-05-04 17:59:03.655618: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1525] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 4634 MB memory:  -> device: 0, name: NVIDIA GeForce GTX 1060, pci bus id: xxxx:xx:xx.x, compute capability: 6.1
1 Physical GPUs, 1 Logical GPUs
Path: D:/Dataset/seg_train
Loading seg_train
Path: D:/Dataset/seg_test
Loading seg_test
2022-05-04 18:02:48.971100: W tensorflow/core/common_runtime/bfc_allocator.cc:462] Allocator (GPU_0_bfc) ran out of memory trying to allocate 10.44GiB (rounded to<PHONE_NUMBER>0)requested by op _EagerConst
If the cause is memory fragmentation maybe the environment variable 'TF_GPU_ALLOCATOR=cuda_malloc_async' will improve the situation.
Current allocation summary follows.
Current allocation summary follows.
2022-05-04 18:02:48.996013: I tensorflow/core/common_runtime/bfc_allocator.cc:1010] BFCAllocator dump for GPU_0_bfc
2022-05-04 18:02:48.996173: I tensorflow/core/common_runtime/bfc_allocator.cc:1017] Bin (256):  Total Chunks: 16, Chunks in use: 16. 4.0KiB allocated for chunks. 4.0KiB in use in bin. 392B client-requested in use in bin.
2022-05-04 18:02:48.996308: I tensorflow/core/common_runtime/bfc_allocator.cc:1017] Bin (512):  Total Chunks: 1, Chunks in use: 1. 512B allocated for chunks. 512B in use in bin. 512B client-requested in use in bin.
2022-05-04 18:02:48.996473: I tensorflow/core/common_runtime/bfc_allocator.cc:1017] Bin (1024):         Total Chunks: 1, Chunks in use: 1. 1.2KiB allocated for chunks. 1.2KiB in use in bin. 1.0KiB client-requested in use in bin.
2022-05-04 18:02:48.996629: I tensorflow/core/common_runtime/bfc_allocator.cc:1017] Bin (2048):         Total Chunks: 2, Chunks in use: 1. 7.0KiB allocated for chunks. 3.5KiB in use in bin. 3.4KiB client-requested in use in bin.
2022-05-04 18:02:48.996889: I tensorflow/core/common_runtime/bfc_allocator.cc:1017] Bin (4096):         Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2022-05-04 18:02:48.997493: I tensorflow/core/common_runtime/bfc_allocator.cc:1017] Bin (8192):         Total Chunks: 1, Chunks in use: 1. 9.5KiB allocated for chunks. 9.5KiB in use in bin. 9.5KiB client-requested in use in bin.
2022-05-04 18:02:48.997960: I tensorflow/core/common_runtime/bfc_allocator.cc:1017] Bin (16384):        Total Chunks: 1, Chunks in use: 0. 19.0KiB allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2022-05-04 18:02:48.998482: I tensorflow/core/common_runtime/bfc_allocator.cc:1017] Bin (32768):        Total Chunks: 2, Chunks in use: 1. 79.5KiB allocated for chunks. 36.0KiB in use in bin. 36.0KiB client-requested in use in bin.
2022-05-04 18:02:48.999113: I tensorflow/core/common_runtime/bfc_allocator.cc:1017] Bin (65536):        Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2022-05-04 18:02:48.999710: I tensorflow/core/common_runtime/bfc_allocator.cc:1017] Bin (131072):       Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2022-05-04 18:02:49.000273: I tensorflow/core/common_runtime/bfc_allocator.cc:1017] Bin (262144):       Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2022-05-04 18:02:49.000742: I tensorflow/core/common_runtime/bfc_allocator.cc:1017] Bin (524288):       Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2022-05-04 18:02:49.001208: I tensorflow/core/common_runtime/bfc_allocator.cc:1017] Bin (1048576):      Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2022-05-04 18:02:49.001671: I tensorflow/core/common_runtime/bfc_allocator.cc:1017] Bin (2097152):      Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2022-05-04 18:02:49.002131: I tensorflow/core/common_runtime/bfc_allocator.cc:1017] Bin (4194304):      Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2022-05-04 18:02:49.002700: I tensorflow/core/common_runtime/bfc_allocator.cc:1017] Bin (8388608):      Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2022-05-04 18:02:49.004034: I tensorflow/core/common_runtime/bfc_allocator.cc:1017] Bin (16777216):     Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2022-05-04 18:02:49.004682: I tensorflow/core/common_runtime/bfc_allocator.cc:1017] Bin (33554432):     Total Chunks: 1, Chunks in use: 1. 44.56MiB allocated for chunks. 44.56MiB in use in bin. 44.56MiB client-requested in use in bin.
2022-05-04 18:02:49.005383: I tensorflow/core/common_runtime/bfc_allocator.cc:1017] Bin (67108864):     Total Chunks: 1, Chunks in use: 0. 89.12MiB allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2022-05-04 18:02:49.007520: I tensorflow/core/common_runtime/bfc_allocator.cc:1017] Bin (134217728):    Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2022-05-04 18:02:49.008016: I tensorflow/core/common_runtime/bfc_allocator.cc:1017] Bin (268435456):    Total Chunks: 1, Chunks in use: 0. 4.39GiB allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2022-05-04 18:02:49.008477: I tensorflow/core/common_runtime/bfc_allocator.cc:1033] Bin for 10.44GiB was 256.00MiB, Chunk State:
2022-05-04 18:02:49.008888: I tensorflow/core/common_runtime/bfc_allocator.cc:1039]   Size: 4.39GiB | Requested Size: 0B | in_use: 0 | bin_num: 20, prev:   Size: 44.56MiB | Requested Size: 44.56MiB | in_use: 1 | bin_num: -1
2022-05-04 18:02:49.009335: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] Next region of size<PHONE_NUMBER>
2022-05-04 18:02:49.025604: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a00000 of size 256 next 1
2022-05-04 18:02:49.025772: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a00100 of size 1280 next 2
2022-05-04 18:02:49.026373: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a00600 of size 256 next 3
2022-05-04 18:02:49.026991: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a00700 of size 256 next 4
2022-05-04 18:02:49.028407: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a00800 of size 256 next 5
2022-05-04 18:02:49.028560: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a00900 of size 256 next 6
2022-05-04 18:02:49.029196: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a00a00 of size 256 next 9
2022-05-04 18:02:49.029937: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a00b00 of size 256 next 10
2022-05-04 18:02:49.030556: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a00c00 of size 256 next 11
2022-05-04 18:02:49.031054: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a00d00 of size 256 next 14
2022-05-04 18:02:49.031553: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a00e00 of size 256 next 15
2022-05-04 18:02:49.031906: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a00f00 of size 512 next 16
2022-05-04 18:02:49.032334: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a01100 of size 256 next 19
2022-05-04 18:02:49.032719: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a01200 of size 256 next 20
2022-05-04 18:02:49.033158: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a01300 of size 256 next 21
2022-05-04 18:02:49.033523: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a01400 of size 256 next 24
2022-05-04 18:02:49.033892: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a01500 of size 256 next 25
2022-05-04 18:02:49.034323: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a01600 of size 256 next 26
2022-05-04 18:02:49.034824: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] Free  at b03a01700 of size 3584 next 7
2022-05-04 18:02:49.035472: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a02500 of size 3584 next 8
2022-05-04 18:02:49.035923: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] Free  at b03a03300 of size 19456 next 23
2022-05-04 18:02:49.036957: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a07f00 of size 9728 next 22
2022-05-04 18:02:49.039251: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] Free  at b03a0a500 of size 44544 next 13
2022-05-04 18:02:49.039789: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a15300 of size 36864 next 12
2022-05-04 18:02:49.040234: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] Free  at b03a1e300 of size 93454336 next 18
2022-05-04 18:02:49.040779: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b0933e300 of size 46727168 next 17
2022-05-04 18:02:49.041233: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] Free  at b0bfce300 of size<PHONE_NUMBER> next<PHONE_NUMBER>3709551615
2022-05-04 18:02:49.041719: I tensorflow/core/common_runtime/bfc_allocator.cc:1071]      Summary of in-use Chunks by size:
2022-05-04 18:02:49.042440: I tensorflow/core/common_runtime/bfc_allocator.cc:1074] 16 Chunks of size 256 totalling 4.0KiB
2022-05-04 18:02:49.042831: I tensorflow/core/common_runtime/bfc_allocator.cc:1074] 1 Chunks of size 512 totalling 512B
2022-05-04 18:02:49.043889: I tensorflow/core/common_runtime/bfc_allocator.cc:1074] 1 Chunks of size 1280 totalling 1.2KiB
2022-05-04 18:02:49.044474: I tensorflow/core/common_runtime/bfc_allocator.cc:1074] 1 Chunks of size 3584 totalling 3.5KiB
2022-05-04 18:02:49.044901: I tensorflow/core/common_runtime/bfc_allocator.cc:1074] 1 Chunks of size 9728 totalling 9.5KiB
2022-05-04 18:02:49.045330: I tensorflow/core/common_runtime/bfc_allocator.cc:1074] 1 Chunks of size 36864 totalling 36.0KiB
2022-05-04 18:02:49.045784: I tensorflow/core/common_runtime/bfc_allocator.cc:1074] 1 Chunks of size 46727168 totalling 44.56MiB
2022-05-04 18:02:49.046196: I tensorflow/core/common_runtime/bfc_allocator.cc:1078] Sum Total of in-use chunks: 44.62MiB
2022-05-04 18:02:49.046552: I tensorflow/core/common_runtime/bfc_allocator.cc:1080] total_region_allocated_bytes_:<PHONE_NUMBER> memory_limit_:<PHONE_NUMBER> available bytes: 0 curr_region_allocation_bytes_:<PHONE_NUMBER>
2022-05-04 18:02:49.046902: I tensorflow/core/common_runtime/bfc_allocator.cc:1086] Stats:
Limit:<PHONE_NUMBER>
InUse:                        46783232
MaxInUse:                    140225792
NumAllocs:                          34
MaxAllocSize:                 46727168
Reserved:                            0
PeakReserved:                        0
LargestFreeBlock:                    0

2022-05-04 18:02:49.047317: W tensorflow/core/common_runtime/bfc_allocator.cc:474] ***_________________________________________________________________________________________________
Traceback (most recent call last):
  File "D:\DatasetProcessing\ImageClassification.py", line 394, in <module>
    main()
  File "D:\DatasetProcessing\ImageClassification.py", line 387, in main
    first_model()
  File "D:\DatasetProcessing\ImageClassification.py", line 162, in first_model
    history = model.fit(train_images, train_labels, batch_size=2, epochs=4)
  File "D:\WinPython\WPy64-3980\python-3.9.8.amd64\lib\site-packages\keras\utils\traceback_utils.py", line 67, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "D:\WinPython\WPy64-3980\python-3.9.8.amd64\lib\site-packages\tensorflow\python\framework\constant_op.py", line 102, in convert_to_eager_tensor
    return ops.EagerTensor(value, ctx.device_name, dtype)
tensorflow.python.framework.errors_impl.InternalError: Failed copying input tensor from /job:localhost/replica:0/task:0/device:CPU:0 to /job:localhost/replica:0/task:0/device:GPU:0 in order to run _EagerConst: Dst tensor is not initialized.

I have a notebook with nvidia gtx 1060 (6GB) gpu. I've installed the newest driver available for this gpu and cuda version 11.2. I checked gpu values in task manager when script was running but it was 1% up to 5%. It looks like tensorflow is not using gpu at all.

I tried to use:

TF_GPU_ALLOCATOR=cuda_malloc_async

and

memory_limit=4096

and

allow_growth=True

I've also decreased batch_size from 128 to 2. But neither of these options worked.

Standalone code to reproduce the issue
model = tf.keras.Sequential([
            tf.keras.layers.Conv2D(32, (3,3), activation = 'relu', input_shape = (200, 256, 3)),
            tf.keras.layers.MaxPooling2D(2,2),
            tf.keras.layers.Conv2D(32, (3,3), activation = 'relu'),
            tf.keras.layers.MaxPooling2D(2,2),
            tf.keras.layers.Flatten(),
            tf.keras.layers.Dense(128, activation = tf.nn.relu),
            tf.keras.layers.Dense(20, activation = tf.nn.softmax)
        ])
model.compile(optimizer = 'adam', loss = 'sparse_categorical_crossentropy', metrics=['accuracy'])
history = model.fit(train_images, train_labels, batch_size=2, epochs=4)

Relevant log output
No response

@JamesJacquesDiego ,
Usually, this error raises when GPU memory is full.Please clear the memory and test your code.Also please take a look at this issue1 and issue2 with the similar error.Thanks!

Thank you for your suggestions. But I've checked gpu usage in task manager when script was running. And the usage was from 1% to 5%. It looks like tensorflow is not using gpu memory at all.
In the info there was information that tensorflow tried to allocate about 10GB of memory (but I have only 6GB gpu). I've also tried memory_limit=4096 and allow_growth=True.
PS
When I use the same code but with cpu there is no problem.

This looks like a GPU related OOM. Here is a more reproducible example based on what you provided earlier. You could try varying the input size and see if reducing the input size starts the network training.
import tensorflow as tf
import numpy as np
rand = np.random.default_rng()

x = np.random.rand(1000, 200, 256, 3)
y = np.array([rand.integers(20) for i in range(1000)])

model = tf.keras.Sequential([
            tf.keras.layers.Conv2D(32, (3,3), activation = 'relu', input_shape = (200, 256, 3)),
            tf.keras.layers.MaxPooling2D(2,2),
            tf.keras.layers.Conv2D(32, (3,3), activation = 'relu'),
            tf.keras.layers.MaxPooling2D(2,2),
            tf.keras.layers.Flatten(),
            tf.keras.layers.Dense(128, activation = tf.nn.relu),
            tf.keras.layers.Dense(20, activation = tf.nn.softmax)
        ])
model.compile(optimizer = 'adam', loss = 'sparse_categorical_crossentropy', metrics=['accuracy'])
history = model.fit(x, y, batch_size=2, epochs=4)

I've added three lines of code after y variable:
config = tf.compat.v1.ConfigProto()
config.gpu_options.per_process_gpu_memory_fraction=0.5
sess = tf.compat.v1.Session(config=config)
I've run your code and first 5 times it worked, but after that I got error. Here is what i get when it works:
2022-05-11 02:29:31.044577: I tensorflow/core/platform/cpu_feature_guard.cc:151] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-05-11 02:29:31.044577: I tensorflow/core/platform/cpu_feature_guard.cc:151] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-05-11 02:29:37.276618: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1525] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 3071 MB memory:  -> device: 0, name: NVIDIA GeForce GTX 1060, pci bus id: xxxx:xx:xx:x, compute capability: 6.1
2022-05-11 02:29:37.415072: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1525] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 3071 MB memory:  -> device: 0, name: NVIDIA GeForce GTX 1060, pci bus id: xxxx:xx:xx:x, compute capability: 6.1
Epoch 1/4
2022-05-11 02:29:31.044577: I tensorflow/core/platform/cpu_feature_guard.cc:151] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-05-11 02:29:37.276618: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1525] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 3071 MB memory:  -> device: 0, name: NVIDIA GeForce GTX 1060, pci bus id: xxxx:xx:xx:x, compute capability: 6.1
2022-05-11 02:29:37.415072: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1525] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 3071 MB memory:  -> device: 0, name: NVIDIA GeForce GTX 1060, pci bus id: xxxx:xx:xx:x, compute capability: 6.1
2022-05-11 02:29:42.112058: I tensorflow/stream_executor/cuda/cuda_dnn.cc:368] Loaded cuDNN version 8101
500/500 [==============================] - 13s 12ms/step - loss: 3.0854 - accuracy: 0.0520
Epoch 2/4
500/500 [==============================] - 6s 12ms/step - loss: 2.9949 - accuracy: 0.0510
Epoch 3/4
500/500 [==============================] - 6s 12ms/step - loss: 2.9938 - accuracy: 0.0550
Epoch 4/4
500/500 [==============================] - 6s 11ms/step - loss: 2.9932 - accuracy: 0.0580

And here is what I get when it doesn't work:

2022-05-11 02:29:31.044577: I tensorflow/core/platform/cpu_feature_guard.cc:151] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-05-11 02:29:37.276618: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1525] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 3071 MB memory:  -> device: 0, name: NVIDIA GeForce GTX 1060, pci bus id: xxxx:xx:xx.x, compute capability: 6.1
2022-05-11 02:29:37.415072: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1525] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 3071 MB memory:  -> device: 0, name: NVIDIA GeForce GTX 1060, pci bus id: xxxx:xx:xx:x, compute capability: 6.1
2022-05-11 02:29:42.112058: I tensorflow/stream_executor/cuda/cuda_dnn.cc:368] Loaded cuDNN version 8101
2022-05-11 02:31:19.739728: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1525] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 3071 MB memory:  -> device: 0, name: NVIDIA GeForce GTX 1060, pci bus id: xxxx:xx:xx:x, compute capability: 6.1
2022-05-11 02:32:20.290753: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1525] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 3071 MB memory:  -> device: 0, name: NVIDIA GeForce GTX 1060, pci bus id: xxxx:xx:xx:x, compute capability: 6.1
2022-05-11 02:32:50.716189: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1525] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 3071 MB memory:  -> device: 0, name: NVIDIA GeForce GTX 1060, pci bus id: xxxx:xx:xx:x, compute capability: 6.1
2022-05-11 02:33:19.014200: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1525] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 3071 MB memory:  -> device: 0, name: NVIDIA GeForce GTX 1060, pci bus id: xxxx:xx:xx:x, compute capability: 6.1
Traceback (most recent call last):

  File "D:scripts\gpu_test.py", line 33, in <module>
    history = model.fit(x, y, batch_size=2, epochs=4)

  File "D:WinPython\WPy64-3980\python-3.9.8.amd64\lib\site-packages\keras\utils\traceback_utils.py", line 67, in error_handler
    raise e.with_traceback(filtered_tb) from None

  File "D:WinPython\WPy64-3980\python-3.9.8.amd64\lib\site-packages\tensorflow\python\framework\constant_op.py", line 102, in convert_to_eager_tensor
    return ops.EagerTensor(value, ctx.device_name, dtype)

InternalError: Failed copying input tensor from /job:localhost/replica:0/task:0/device:CPU:0 to /job:localhost/replica:0/task:0/device:GPU:0 in order to run _EagerConst: Dst tensor is not initialized.

2022-05-11 02:29:31.044577: I tensorflow/core/platform/cpu_feature_guard.cc:151] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-05-11 02:29:37.276618: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1525] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 3071 MB memory:  -> device: 0, name: NVIDIA GeForce GTX 1060, pci bus id: xxxx:xx:xx:x, compute capability: 6.1
2022-05-11 02:29:37.415072: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1525] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 3071 MB memory:  -> device: 0, name: NVIDIA GeForce GTX 1060, pci bus id: xxxx:xx:xx:x, compute capability: 6.1
2022-05-11 02:29:42.112058: I tensorflow/stream_executor/cuda/cuda_dnn.cc:368] Loaded cuDNN version 8101
2022-05-11 02:31:19.739728: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1525] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 3071 MB memory:  -> device: 0, name: NVIDIA GeForce GTX 1060, pci bus id: xxxx:xx:xx:x, compute capability: 6.1
2022-05-11 02:32:20.290753: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1525] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 3071 MB memory:  -> device: 0, name: NVIDIA GeForce GTX 1060, pci bus id: xxxx:xx:xx:x, compute capability: 6.1
2022-05-11 02:32:50.716189: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1525] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 3071 MB memory:  -> device: 0, name: NVIDIA GeForce GTX 1060, pci bus id: xxxx:xx:xx:x, compute capability: 6.1
2022-05-11 02:33:19.014200: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1525] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 3071 MB memory:  -> device: 0, name: NVIDIA GeForce GTX 1060, pci bus id: xxxx:xx:xx:x, compute capability: 6.1
2022-05-11 02:33:29.241618: W tensorflow/core/common_runtime/bfc_allocator.cc:462] Allocator (GPU_0_bfc) ran out of memory trying to allocate 585.94MiB (rounded to 614400000)requested by op _EagerConst
If the cause is memory fragmentation maybe the environment variable 'TF_GPU_ALLOCATOR=cuda_malloc_async' will improve the situation.
Current allocation summary follows.
Current allocation summary follows.
2022-05-11 02:33:29.241729: I tensorflow/core/common_runtime/bfc_allocator.cc:1010] BFCAllocator dump for GPU_0_bfc
2022-05-11 02:33:29.241792: I tensorflow/core/common_runtime/bfc_allocator.cc:1017] Bin (256): 	Total Chunks: 107, Chunks in use: 107. 26.8KiB allocated for chunks. 26.8KiB in use in bin. 4.6KiB client-requested in use in bin.
2022-05-11 02:33:29.241846: I tensorflow/core/common_runtime/bfc_allocator.cc:1017] Bin (512): 	Total Chunks: 13, Chunks in use: 13. 6.5KiB allocated for chunks. 6.5KiB in use in bin. 6.5KiB client-requested in use in bin.
2022-05-11 02:33:29.241897: I tensorflow/core/common_runtime/bfc_allocator.cc:1017] Bin (1024): 	Total Chunks: 1, Chunks in use: 1. 1.2KiB allocated for chunks. 1.2KiB in use in bin. 1.0KiB client-requested in use in bin.
2022-05-11 02:33:29.241948: I tensorflow/core/common_runtime/bfc_allocator.cc:1017] Bin (2048): 	Total Chunks: 10, Chunks in use: 10. 35.0KiB allocated for chunks. 35.0KiB in use in bin. 33.8KiB client-requested in use in bin.
2022-05-11 02:33:29.241998: I tensorflow/core/common_runtime/bfc_allocator.cc:1017] Bin (4096): 	Total Chunks: 4, Chunks in use: 3. 23.8KiB allocated for chunks. 17.5KiB in use in bin. 10.1KiB client-requested in use in bin.
2022-05-11 02:33:29.242049: I tensorflow/core/common_runtime/bfc_allocator.cc:1017] Bin (8192): 	Total Chunks: 16, Chunks in use: 16. 158.0KiB allocated for chunks. 158.0KiB in use in bin. 153.4KiB client-requested in use in bin.
2022-05-11 02:33:29.242098: I tensorflow/core/common_runtime/bfc_allocator.cc:1017] Bin (16384): 	Total Chunks: 1, Chunks in use: 0. 20.0KiB allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2022-05-11 02:33:29.242343: I tensorflow/core/common_runtime/bfc_allocator.cc:1017] Bin (32768): 	Total Chunks: 14, Chunks in use: 13. 534.0KiB allocated for chunks. 492.0KiB in use in bin. 468.0KiB client-requested in use in bin.
2022-05-11 02:33:29.242411: I tensorflow/core/common_runtime/bfc_allocator.cc:1017] Bin (65536): 	Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2022-05-11 02:33:29.242460: I tensorflow/core/common_runtime/bfc_allocator.cc:1017] Bin (131072): 	Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2022-05-11 02:33:29.242504: I tensorflow/core/common_runtime/bfc_allocator.cc:1017] Bin (262144): 	Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2022-05-11 02:33:29.242548: I tensorflow/core/common_runtime/bfc_allocator.cc:1017] Bin (524288): 	Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2022-05-11 02:33:29.242591: I tensorflow/core/common_runtime/bfc_allocator.cc:1017] Bin (1048576): 	Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2022-05-11 02:33:29.242631: I tensorflow/core/common_runtime/bfc_allocator.cc:1017] Bin (2097152): 	Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2022-05-11 02:33:29.242671: I tensorflow/core/common_runtime/bfc_allocator.cc:1017] Bin (4194304): 	Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2022-05-11 02:33:29.242711: I tensorflow/core/common_runtime/bfc_allocator.cc:1017] Bin (8388608): 	Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2022-05-11 02:33:29.242752: I tensorflow/core/common_runtime/bfc_allocator.cc:1017] Bin (16777216): 	Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2022-05-11 02:33:29.242800: I tensorflow/core/common_runtime/bfc_allocator.cc:1017] Bin (33554432): 	Total Chunks: 14, Chunks in use: 13. 650.33MiB allocated for chunks. 604.50MiB in use in bin. 604.50MiB client-requested in use in bin.
2022-05-11 02:33:29.242845: I tensorflow/core/common_runtime/bfc_allocator.cc:1017] Bin (67108864): 	Total Chunks: 1, Chunks in use: 0. 93.00MiB allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2022-05-11 02:33:29.242890: I tensorflow/core/common_runtime/bfc_allocator.cc:1017] Bin (134217728): 	Total Chunks: 1, Chunks in use: 0. 167.44MiB allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2022-05-11 02:33:29.242937: I tensorflow/core/common_runtime/bfc_allocator.cc:1017] Bin (268435456): 	Total Chunks: 4, Chunks in use: 3. 2.11GiB allocated for chunks. 1.72GiB in use in bin. 1.72GiB client-requested in use in bin.
2022-05-11 02:33:29.242978: I tensorflow/core/common_runtime/bfc_allocator.cc:1033] Bin for 585.94MiB was 256.00MiB, Chunk State:
2022-05-11 02:33:29.243026: I tensorflow/core/common_runtime/bfc_allocator.cc:1039]   Size: 402.54MiB | Requested Size: 2.96MiB | in_use: 0 | bin_num: 20, prev:   Size: 585.94MiB | Requested Size: 585.94MiB | in_use: 1 | bin_num: -1
2022-05-11 02:33:29.243062: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] Next region of size<PHONE_NUMBER>
2022-05-11 02:33:29.243140: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a00000 of size 256 next 1
2022-05-11 02:33:29.243175: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a00100 of size 1280 next 2
2022-05-11 02:33:29.243205: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a00600 of size 256 next 3
2022-05-11 02:33:29.243232: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a00700 of size 256 next 4
2022-05-11 02:33:29.243267: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a00800 of size 256 next 5
2022-05-11 02:33:29.243389: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a00900 of size 256 next 6
2022-05-11 02:33:29.243430: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a00a00 of size 256 next 9
2022-05-11 02:33:29.243460: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a00b00 of size 256 next 10
2022-05-11 02:33:29.243489: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a00c00 of size 256 next 11
2022-05-11 02:33:29.243517: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a00d00 of size 256 next 14
2022-05-11 02:33:29.243544: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a00e00 of size 256 next 15
2022-05-11 02:33:29.243573: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a00f00 of size 512 next 16
2022-05-11 02:33:29.243601: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a01100 of size 256 next 19
2022-05-11 02:33:29.243629: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a01200 of size 256 next 20
2022-05-11 02:33:29.243656: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a01300 of size 256 next 21
2022-05-11 02:33:29.243684: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a01400 of size 256 next 24
2022-05-11 02:33:29.243712: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a01500 of size 256 next 25
2022-05-11 02:33:29.243740: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a01600 of size 256 next 26
2022-05-11 02:33:29.243767: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a01700 of size 256 next 29
2022-05-11 02:33:29.243795: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a01800 of size 256 next 30
2022-05-11 02:33:29.243823: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a01900 of size 256 next 31
2022-05-11 02:33:29.243850: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a01a00 of size 256 next 32
2022-05-11 02:33:29.243878: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a01b00 of size 256 next 33
2022-05-11 02:33:29.243906: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a01c00 of size 256 next 34
2022-05-11 02:33:29.243933: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a01d00 of size 256 next 35
2022-05-11 02:33:29.243961: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a01e00 of size 256 next 36
2022-05-11 02:33:29.243989: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a01f00 of size 256 next 38
2022-05-11 02:33:29.244017: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a02000 of size 256 next 39
2022-05-11 02:33:29.244045: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a02100 of size 512 next 41
2022-05-11 02:33:29.244072: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a02300 of size 256 next 43
2022-05-11 02:33:29.244100: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a02400 of size 256 next 7
2022-05-11 02:33:29.244128: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a02500 of size 3584 next 8
2022-05-11 02:33:29.244156: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a03300 of size 256 next 90
2022-05-11 02:33:29.244184: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a03400 of size 256 next 91
2022-05-11 02:33:29.244211: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a03500 of size 256 next 92
2022-05-11 02:33:29.244239: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a03600 of size 256 next 93
2022-05-11 02:33:29.244276: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a03700 of size 256 next 94
2022-05-11 02:33:29.244305: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a03800 of size 256 next 95
2022-05-11 02:33:29.244337: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a03900 of size 256 next 106
2022-05-11 02:33:29.244368: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a03a00 of size 256 next 112
2022-05-11 02:33:29.244396: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a03b00 of size 256 next 113
2022-05-11 02:33:29.244424: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a03c00 of size 256 next 104
2022-05-11 02:33:29.244452: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a03d00 of size 256 next 108
2022-05-11 02:33:29.244481: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a03e00 of size 5376 next 28
2022-05-11 02:33:29.244508: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a05300 of size 3584 next 37
2022-05-11 02:33:29.244537: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a06100 of size 3584 next 44
2022-05-11 02:33:29.244564: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a06f00 of size 256 next 46
2022-05-11 02:33:29.244593: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a07000 of size 512 next 48
2022-05-11 02:33:29.244620: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a07200 of size 256 next 50
2022-05-11 02:33:29.244648: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a07300 of size 256 next 51
2022-05-11 02:33:29.244676: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a07400 of size 256 next 52
2022-05-11 02:33:29.244703: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a07500 of size 256 next 53
2022-05-11 02:33:29.244731: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a07600 of size 256 next 54
2022-05-11 02:33:29.244758: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a07700 of size 256 next 55
2022-05-11 02:33:29.244786: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a07800 of size 256 next 56
2022-05-11 02:33:29.244813: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a07900 of size 256 next 57
2022-05-11 02:33:29.244841: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a07a00 of size 256 next 58
2022-05-11 02:33:29.244869: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a07b00 of size 256 next 76
2022-05-11 02:33:29.244896: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a07c00 of size 256 next 73
2022-05-11 02:33:29.244924: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a07d00 of size 512 next 67
2022-05-11 02:33:29.244952: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a07f00 of size 256 next 70
2022-05-11 02:33:29.244980: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a08000 of size 256 next 66
2022-05-11 02:33:29.245008: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a08100 of size 256 next 115
2022-05-11 02:33:29.245035: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a08200 of size 256 next 23
2022-05-11 02:33:29.245063: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a08300 of size 10240 next 22
2022-05-11 02:33:29.245092: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a0ab00 of size 43008 next 13
2022-05-11 02:33:29.245121: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a15300 of size 36864 next 12
2022-05-11 02:33:29.245149: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b03a1e300 of size 48758784 next 40
2022-05-11 02:33:29.245176: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b0689e300 of size 10240 next 42
2022-05-11 02:33:29.245204: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b068a0b00 of size 36864 next 45
2022-05-11 02:33:29.245231: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b068a9b00 of size 10240 next 49
2022-05-11 02:33:29.245363: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b068ac300 of size 256 next 64
2022-05-11 02:33:29.245404: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b068ac400 of size 6912 next 60
2022-05-11 02:33:29.245433: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b068adf00 of size 3584 next 68
2022-05-11 02:33:29.245462: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b068aed00 of size 8192 next 74
2022-05-11 02:33:29.245490: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b068b0d00 of size 256 next 72
2022-05-11 02:33:29.245517: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b068b0e00 of size 256 next 77
2022-05-11 02:33:29.245545: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b068b0f00 of size 512 next 79
2022-05-11 02:33:29.245573: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b068b1100 of size 11264 next 75
2022-05-11 02:33:29.245600: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b068b3d00 of size 10240 next 69
2022-05-11 02:33:29.245628: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b068b6500 of size 43008 next 65
2022-05-11 02:33:29.245655: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b068c0d00 of size 36864 next 63
2022-05-11 02:33:29.245683: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b068c9d00 of size 256 next 80
2022-05-11 02:33:29.245710: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b068c9e00 of size 3584 next 81
2022-05-11 02:33:29.245738: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b068cac00 of size 256 next 82
2022-05-11 02:33:29.245766: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b068cad00 of size 36864 next 83
2022-05-11 02:33:29.245794: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b068d3d00 of size 256 next 84
2022-05-11 02:33:29.245822: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b068d3e00 of size 512 next 85
2022-05-11 02:33:29.245850: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b068d4000 of size 10240 next 86
2022-05-11 02:33:29.245877: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b068d6800 of size 256 next 87
2022-05-11 02:33:29.245905: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b068d6900 of size 512 next 59
2022-05-11 02:33:29.245933: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b068d6b00 of size 256 next 88
2022-05-11 02:33:29.245961: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b068d6c00 of size 256 next 111
2022-05-11 02:33:29.245989: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b068d6d00 of size 256 next 105
2022-05-11 02:33:29.246017: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b068d6e00 of size 256 next 97
2022-05-11 02:33:29.246044: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b068d6f00 of size 256 next 100
2022-05-11 02:33:29.246072: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b068d7000 of size 256 next 102
2022-05-11 02:33:29.246100: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b068d7100 of size 512 next 117
2022-05-11 02:33:29.246127: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b068d7300 of size 256 next 118
2022-05-11 02:33:29.246155: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b068d7400 of size 256 next 120
2022-05-11 02:33:29.246183: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b068d7500 of size 256 next 122
2022-05-11 02:33:29.246210: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b068d7600 of size 256 next 103
2022-05-11 02:33:29.246238: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b068d7700 of size 3584 next 109
2022-05-11 02:33:29.246276: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b068d8500 of size 8192 next 99
2022-05-11 02:33:29.246414: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b068da500 of size 12288 next 98
2022-05-11 02:33:29.246455: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b068dd500 of size 10240 next 89
2022-05-11 02:33:29.246484: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b068dfd00 of size 43008 next 101
2022-05-11 02:33:29.246512: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b068ea500 of size 36864 next 96
2022-05-11 02:33:29.246540: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b068f3500 of size 3584 next 119
2022-05-11 02:33:29.246568: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b068f4300 of size 36864 next 121
2022-05-11 02:33:29.246597: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b068fd300 of size 512 next 123
2022-05-11 02:33:29.246624: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b068fd500 of size 10240 next 124
2022-05-11 02:33:29.246652: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b068ffd00 of size 256 next 125
2022-05-11 02:33:29.246680: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b068ffe00 of size 256 next 126
2022-05-11 02:33:29.246708: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b068fff00 of size 256 next 127
2022-05-11 02:33:29.246735: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b06900000 of size 256 next 128
2022-05-11 02:33:29.246763: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b06900100 of size 256 next 129
2022-05-11 02:33:29.246791: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b06900200 of size 256 next 130
2022-05-11 02:33:29.246819: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b06900300 of size 256 next 131
2022-05-11 02:33:29.246846: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b06900400 of size 256 next 132
2022-05-11 02:33:29.246874: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b06900500 of size 256 next 144
2022-05-11 02:33:29.246902: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b06900600 of size 256 next 134
2022-05-11 02:33:29.246930: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b06900700 of size 256 next 141
2022-05-11 02:33:29.246958: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b06900800 of size 512 next 149
2022-05-11 02:33:29.246986: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b06900a00 of size 256 next 139
2022-05-11 02:33:29.247014: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b06900b00 of size 256 next 151
2022-05-11 02:33:29.247042: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b06900c00 of size 256 next 145
2022-05-11 02:33:29.247070: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b06900d00 of size 256 next 138
2022-05-11 02:33:29.247098: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b06900e00 of size 5632 next 148
2022-05-11 02:33:29.247126: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b06902400 of size 3584 next 153
2022-05-11 02:33:29.247154: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b06903200 of size 8192 next 152
2022-05-11 02:33:29.247182: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b06905200 of size 256 next 135
2022-05-11 02:33:29.247209: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b06905300 of size 256 next 142
2022-05-11 02:33:29.247237: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b06905400 of size 512 next 154
2022-05-11 02:33:29.247410: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b06905600 of size 11264 next 146
2022-05-11 02:33:29.247503: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b06908200 of size 10240 next 133
2022-05-11 02:33:29.247584: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b0690aa00 of size 43008 next 150
2022-05-11 02:33:29.247687: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b06915200 of size 36864 next 140
2022-05-11 02:33:29.247751: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b0691e200 of size 256 next 155
2022-05-11 02:33:29.247782: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b0691e300 of size 3584 next 156
2022-05-11 02:33:29.247811: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b0691f100 of size 256 next 157
2022-05-11 02:33:29.247840: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b0691f200 of size 36864 next 158
2022-05-11 02:33:29.247868: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b06928200 of size 256 next 159
2022-05-11 02:33:29.247896: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b06928300 of size 512 next 160
2022-05-11 02:33:29.247924: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b06928500 of size 10240 next 161
2022-05-11 02:33:29.247952: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b0692ad00 of size 256 next 162
2022-05-11 02:33:29.247981: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b0692ae00 of size 256 next 163
2022-05-11 02:33:29.248009: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b0692af00 of size 256 next 164
2022-05-11 02:33:29.248037: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b0692b000 of size 256 next 165
2022-05-11 02:33:29.248065: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b0692b100 of size 256 next 166
2022-05-11 02:33:29.248093: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b0692b200 of size 256 next 167
2022-05-11 02:33:29.248121: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b0692b300 of size 256 next 168
2022-05-11 02:33:29.248149: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b0692b400 of size 256 next 169
2022-05-11 02:33:29.248177: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b0692b500 of size 256 next 170
2022-05-11 02:33:29.248206: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b0692b600 of size 256 next 173
2022-05-11 02:33:29.248233: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b0692b700 of size 256 next 184
2022-05-11 02:33:29.248272: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b0692b800 of size 512 next 179
2022-05-11 02:33:29.248300: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b0692ba00 of size 256 next 174
2022-05-11 02:33:29.248328: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b0692bb00 of size 256 next 171
2022-05-11 02:33:29.248356: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b0692bc00 of size 256 next 188
2022-05-11 02:33:29.248384: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b0692bd00 of size 256 next 182
2022-05-11 02:33:29.248412: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] Free  at b0692be00 of size 6400 next 190
2022-05-11 02:33:29.248440: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b0692d700 of size 3584 next 176
2022-05-11 02:33:29.248468: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] Free  at b0692e500 of size 20480 next 185
2022-05-11 02:33:29.248495: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b06933500 of size 10240 next 181
2022-05-11 02:33:29.248523: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] Free  at b06935d00 of size 43008 next 175
2022-05-11 02:33:29.248551: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b06940500 of size 36864 next 187
2022-05-11 02:33:29.248579: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] Free  at b06949500 of size 48057856 next 18
2022-05-11 02:33:29.248607: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b0971e300 of size 48758784 next 17
2022-05-11 02:33:29.248636: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b0c59e300 of size 48758784 next 116
2022-05-11 02:33:29.248670: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b0f41e300 of size 48758784 next 114
2022-05-11 02:33:29.248701: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b1229e300 of size 48758784 next 107
2022-05-11 02:33:29.248730: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b1511e300 of size 48758784 next 137
2022-05-11 02:33:29.248757: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b17f9e300 of size 48758784 next 143
2022-05-11 02:33:29.248785: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b1ae1e300 of size 48758784 next 147
2022-05-11 02:33:29.248812: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] Free  at b1dc9e300 of size 97517568 next 178
2022-05-11 02:33:29.248840: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b2399e300 of size 48758784 next 186
2022-05-11 02:33:29.248867: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] Free  at b2681e300 of size 175570944 next 27
2022-05-11 02:33:29.248895: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b30f8e300 of size 48758784 next 47
2022-05-11 02:33:29.248923: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b33e0e300 of size 48758784 next 78
2022-05-11 02:33:29.248950: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b36c8e300 of size 48758784 next 61
2022-05-11 02:33:29.248978: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b39b0e300 of size 48758784 next 71
2022-05-11 02:33:29.249006: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b3c98e300 of size 614400000 next 62
2022-05-11 02:33:29.249034: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b6137e300 of size 614400000 next 110
2022-05-11 02:33:29.249061: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] InUse at b85d6e300 of size 614400000 next 136
2022-05-11 02:33:29.249090: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] Free  at baa75e300 of size 422092032 next<PHONE_NUMBER>3709551615
2022-05-11 02:33:29.249119: I tensorflow/core/common_runtime/bfc_allocator.cc:1071]      Summary of in-use Chunks by size:
2022-05-11 02:33:29.249164: I tensorflow/core/common_runtime/bfc_allocator.cc:1074] 107 Chunks of size 256 totalling 26.8KiB
2022-05-11 02:33:29.249202: I tensorflow/core/common_runtime/bfc_allocator.cc:1074] 13 Chunks of size 512 totalling 6.5KiB
2022-05-11 02:33:29.249252: I tensorflow/core/common_runtime/bfc_allocator.cc:1074] 1 Chunks of size 1280 totalling 1.2KiB
2022-05-11 02:33:29.249291: I tensorflow/core/common_runtime/bfc_allocator.cc:1074] 10 Chunks of size 3584 totalling 35.0KiB
2022-05-11 02:33:29.249326: I tensorflow/core/common_runtime/bfc_allocator.cc:1074] 1 Chunks of size 5376 totalling 5.2KiB
2022-05-11 02:33:29.249361: I tensorflow/core/common_runtime/bfc_allocator.cc:1074] 1 Chunks of size 5632 totalling 5.5KiB
2022-05-11 02:33:29.249396: I tensorflow/core/common_runtime/bfc_allocator.cc:1074] 1 Chunks of size 6912 totalling 6.8KiB
2022-05-11 02:33:29.249431: I tensorflow/core/common_runtime/bfc_allocator.cc:1074] 3 Chunks of size 8192 totalling 24.0KiB
2022-05-11 02:33:29.249467: I tensorflow/core/common_runtime/bfc_allocator.cc:1074] 10 Chunks of size 10240 totalling 100.0KiB
2022-05-11 02:33:29.249503: I tensorflow/core/common_runtime/bfc_allocator.cc:1074] 2 Chunks of size 11264 totalling 22.0KiB
2022-05-11 02:33:29.249538: I tensorflow/core/common_runtime/bfc_allocator.cc:1074] 1 Chunks of size 12288 totalling 12.0KiB
2022-05-11 02:33:29.249572: I tensorflow/core/common_runtime/bfc_allocator.cc:1074] 9 Chunks of size 36864 totalling 324.0KiB
2022-05-11 02:33:29.249607: I tensorflow/core/common_runtime/bfc_allocator.cc:1074] 4 Chunks of size 43008 totalling 168.0KiB
![image1-before running script](https://user-images.githubusercontent.com/44622349/167747019-e74d87d2-5612-409c-a945-d3d65631782f.PNG)
![image2-running script](https://user-images.githubusercontent.com/44622349/167747021-1682e710-809e-4e3f-b2d2-dac0e3ab5576.PNG)
![image3-error](https://user-images.githubusercontent.com/44622349/167747022-8cd3dbd1-1541-4b77-9681-a9e12a4d5ab6.PNG)

2022-05-11 02:33:29.249643: I tensorflow/core/common_runtime/bfc_allocator.cc:1074] 13 Chunks of size 48758784 totalling 604.50MiB
2022-05-11 02:33:29.249679: I tensorflow/core/common_runtime/bfc_allocator.cc:1074] 3 Chunks of size 614400000 totalling 1.72GiB
2022-05-11 02:33:29.249716: I tensorflow/core/common_runtime/bfc_allocator.cc:1078] Sum Total of in-use chunks: 2.31GiB
2022-05-11 02:33:29.249757: I tensorflow/core/common_runtime/bfc_allocator.cc:1080] total_region_allocated_bytes_:<PHONE_NUMBER> memory_limit_:<PHONE_NUMBER> available bytes: 0 curr_region_allocation_bytes_:<PHONE_NUMBER>
2022-05-11 02:33:29.249807: I tensorflow/core/common_runtime/bfc_allocator.cc:1086] Stats:
Limit:<PHONE_NUMBER>
InUse:<PHONE_NUMBER>
MaxInUse:<PHONE_NUMBER>
NumAllocs:                      432324
MaxAllocSize:                614400000
Reserved:                            0
PeakReserved:                        0
LargestFreeBlock:                    0

2022-05-11 02:33:29.249874: W tensorflow/core/common_runtime/bfc_allocator.cc:474] **_***********__***____****************************************************************_____________

I used OHM to check gpu memory values. I've added images.

The lines you added have no effect, I assume you read this issue, you need to use the instructions related to TensorFlow 2. However, it seems that you don't have enough memory on the GPU.
To test this assertion you could change the input size( e.g.  x = np.random.rand(1000, 200, 256, 3) to  x = np.random.rand(1000, 100, 100, 3) ) and see if the error persis

So those lines don't work?
config = tf.compat.v1.ConfigProto()
config.gpu_options.per_process_gpu_memory_fraction=0.5
sess = tf.compat.v1.Session(config=config)

The code you've pasted  (with input size you set) works first few times and after 4th or 5th time I get error.
It works after I've change input size to x = np.random.rand(1000, 100, 100, 3) as you advised but reducing resolution of my images in my network is not a solution. What if I will need more images in the future to train my network or if smaller resolution would be bad for classification results?
When I tried to run the script with my network then I get info that tensorflow tries to allocate about 10GB. I have gtx 1060 so I have only 6GB.
@Kubasinska Don't get me wrong. Thanks for your suggestions and help. I am grateful but I'm trying to find the best solution.
I have only about 25000 images in my dataset. Some people train networks with few millions images. I don't know how they do it. Do they divide their data somehow into chunks?

@JamesJacquesDiego ,
In order to expedite the trouble-shooting process, could you please provide the complete code to reproduce the issue reported here.Thanks!

Well, it's almost a complete code. Above I load my train and test images with labels. Below I generate plots to display data. I've decreased resolution of my images and now it works. But it should be some way to load only part of dataset to gpu memory or create additional memory like with ram we can do (virtual ram)?

@JamesJacquesDiego ,
I was facing different error while executing the mentioned code.Please find the gist of it here.

train_images and train_labels are images and classes which i load. Here is used
x = np.random.rand(1000, 200, 256, 3) y = np.array([rand.integers(20) for i in range(1000)])
to simulate those data


Audio Call feature is not working

Issue Type: Bug
I have already tried to uninstall Live Share Audio > close VS code > delete live share audio extension folder >reinstall the audio extension, but still encounter seeing 1 call participant (myself) when I start the 'audio call'. Not sure why I cannot see 2 call participants (a different user connected via visual studio codespaces).
My peer cannot retrieve the audio call I send via VS Code.
Extension version: 1.0.2731
VS Code version: Code 1.48.2 (a0479759d6e9ea56afa657e454193f72aef85bd0, 2020-08-25T10:13:11.295Z)
OS version: Windows_NT x64 10.0.19041

System Info

Item
Value

CPUs
Intel(R) Core(TM) i7-1065G7 CPU @ 1.30GHz (8 x 1498)

GPU Status
2d_canvas: enabledflash_3d: enabledflash_stage3d: enabledflash_stage3d_baseline: enabledgpu_compositing: enabledmultiple_raster_threads: enabled_onoop_rasterization: disabled_offprotected_video_decode: enabledrasterization: enabledskia_renderer: disabled_off_okvideo_decode: enabledviz_display_compositor: enabled_onviz_hit_test_surface_layer: disabled_off_okwebgl: enabledwebgl2: enabled

Load (avg)
undefined

Memory (System)
15.60GB (3.95GB free)

Process Argv

Screen Reader
no

VM
0%

@youkiee Can you try out joining the audio call with yourself and check if you are both participants? Also what version of Live Share Audio do you have installed?

@nandiniYeltiwar Thanks for the comment. When I try joining the audio call myself, I don't see two participants. The version I installed is v.0.1.85

@youkiee We have fixed the issue in the upcoming release. It will be available to you in couple of days.

We released an update to the audio extension that should address this issue, so I’m going to close it as resolved. Please re-open this issue if you’re still running into problems. Thanks!


cms-admin: add helper for creating DocumentInterface root block related functions

This will make updating (once used) less painfull, if functions like replaceDependenciesInOutput are added. This one was not the last we need...
todo

[ ] think about name of this helper
[x] changelog

think about name of this helper

We could name it createDocumentSkeleton, which would be in line with createBlockSkeleton


Unable to get results for different profiles configured

Whenever I try to switch to different profiles to get the required results for those AWS accounts, I always get the results for the "Default" profile only, but not for the switched profile

Hi @justinlazaro-iselect ,
You are using wrong command:
$ aws-azure-login --help
Usage: aws-azure-login [options]

Options:
  -p, --profile <name>             The name of the profile to log in with (or configure)
  -a, --all-profiles               Run for all configured profiles
  -f, --force-refresh              Force a credential refresh, even if they are still valid
  -c, --configure                  Configure the profile
  -m, --mode <mode>                'cli' to hide the login page and perform the login through the CLI (default behavior), 'gui' to perform the login through the Azure GUI (more reliable but only works on GUI operating system), 'debug'
                                   to show the login page but perform the login through the CLI (useful to debug issues with the CLI login)
  --no-sandbox                     Disable the Puppeteer sandbox (usually necessary on Linux)
  --no-prompt                      Do not prompt for input and accept the default choice
  --enable-chrome-network-service  Enable Chromium's Network Service (needed when login provider redirects with 3XX)
  --no-verify-ssl                  Disable SSL Peer Verification for connections to AWS (no effect if behind proxy)
  --enable-chrome-seamless-sso     Enable Chromium's pass-through authentication with Azure Active Directory Seamless Single Sign-On
  --no-disable-extensions          Tell Puppeteer not to pass the --disable-extensions flag to Chromium
  -h, --help                       display help for command

Please either use "-p" or "--profile".
e.g.
aws-azure-login --configure --profile devops


black-formater-plugin only working once

I am using Thonny 4.02 portable on Windows 10.
After installing Black-formater-plugin, for the first time it works.
Using Black a second time it says "Package not found".
Restarting Thonny I can use Black again one more time,
next time it again says "package not found"

Tested now on an other pc (windows 10) with thonny4.02 installed  as standard app. It  shows the same problem as the portable thonny

Thank you for answering to my post!
Kurt

I found a solution: Goto Tools > Manage Packages and install  the package black (which was already installed by the thonny-black-formater-plugin) once more

For me (Thonny 4.1.4), this behavior remains.

thonny-black-formatter version 0.0.4
black  version 24.4.2

(re)installing black, then restarting Thonny, will work exactly once.
Restart Thonny again, and it works the first time, but not thereafter.
I'll add more as (when/if) I poke deeper into what might be happening.


fix implicit convert at src/dlangui/core/parser.d

When I tried building example1, DMD showed these errors.
Erros:
src/dlangui/core/parser.d(162): Error: cannot implicitly convert expression (this._lineText.length) of type ulong to int
src/dlangui/core/parser.d(196): Error: cannot implicitly convert expression (this._lineText.length) of type ulong to int
I fixed these bugs as follows.
Original Code(both of these lines):
_len = _lineText.length;

Fixed Core(both of these lines):
_len = cast(int)_lineText.length;

My envrionments:
OS X Yosemite(10.10.2)
dmd v2.066

Thank you!


update example for tuning

Add examples for tuning using SFT TRainer

@Ssukriti yes let's make it a subfolder.
basically - we have this fine-tuning section in the main READE me with a short high-level/key-content section, then have a "for more details, xxx" linkage to docs/finetuning.md

just to update, while trying the script i realized pad token is needed as padding free does not seem to be supported, hence I have to update the snippet with below code. Still testing as I ran out of memory, but once script works, will move it to subfolder
`special_tokens_dict = {}
if tokenizer.pad_token is None:
print("PAD token set to default, missing in tokenizer")
special_tokens_dict["pad_token"] = ""
def tokenizer_and_embedding_resize(
special_tokens_dict: dict,
tokenizer: PreTrainedTokenizer,
model: PreTrainedModel,
) -> dict:
"""Resize tokenizer and embedding.
Args:
special_tokens_dict: Dict containing special tokens to be added.
tokenizer: transformers.PreTrainedTokenizer.
model: transformers.PreTrainedModel
Return:
dict: Metadata on number of added tokens
"""
num_new_tokens = tokenizer.add_special_tokens(special_tokens_dict)
embedding_size = int(math.ceil(len(tokenizer)))
num_new_tokens = num_new_tokens + embedding_size - len(tokenizer)
model.resize_token_embeddings(embedding_size)
if num_new_tokens > 0:
input_embeddings = model.get_input_embeddings().weight.data
output_embeddings = model.get_output_embeddings().weight.data
    input_embeddings_avg = input_embeddings[:-num_new_tokens].mean(
        dim=0, keepdim=True
    )
    output_embeddings_avg = output_embeddings[:-num_new_tokens].mean(
        dim=0, keepdim=True
    )

    input_embeddings[-num_new_tokens:] = input_embeddings_avg
    output_embeddings[-num_new_tokens:] = output_embeddings_avg`

@Ssukriti @lchu-ibm yes padding free is not supported in the transformers PR, but actually we do support it in the VLLM PR. Its just that we didnt want to complicated the initial PR. We may put it in in a follow up. cc: @ani300
Also do note that the llama3 tokenizer does not have a pad_token, so that has to be set

I have updated the code and tested it with mamba model

@lchu-ibm the PR is ready for review

@lchu-ibm addressed review comments, thank you! Also added a small section on multi-gpu tuning


Adding associations to models and normalizing db

resolves #68

My only note is that we don't need the phrase primary_key: 'id'. Otherwise :shipit:


Enumeration handling

Hi @RehanSaeed! We're porting Schema.NET to PHP and Ruby, and were wondering if you could help us out with a question we had on enumeration handling...
https://schema.org/PaymentCard is both an https://schema.org/Enumeration and a https://schema.org/Service, and in Schema.NET it is treated only as an enum without a corresponding class.
We're really interested in the logic you're applying here and the rationale for it?

That's cool.
Interesting, I was not aware of that. I wonder how many other types are both enumerations and classes. That sounds like a bug. In this case, we should add a suffix to the enumeration or class, and then generate both enum and class types.
We use two JSON-LD files downloaded from schema.org to drive the creation of all the types. Given time, I'd look in there to see how PaymentCard was defined.

Interesting! Maybe a bug... We should raise it with the schema.org folks over at that repo if so? Couldn’t find anything about it in their issues, but you might find something I missed?
Suffix would break implementations like ours that rely on the predictability of names to generate subclasses automatically - perhaps we could use .NET namespaces to keep enums separate and allow them to be predictably referenced? The IDIE would then help users with this too?

The link to the two JSON-LD files can be found in the schema generating Tool project. I would check in there first. It may be a bug with schema.org or it could be our code not handling it.
Namespaces is another way to go. Not sure what I'd choose yet. I'd want to know how often this happens before I decided.

Oh I see, it's definitely coming from schema.org (PaymentMethod subclasses Enumeration, FinancialProduct subclasses Service):
{
  "@id": "http://schema.org/PaymentCard",
  "@type": "rdfs:Class",
  "http://purl.org/dc/terms/source": {
    "@id": "http://www.w3.org/wiki/WebSchemas/SchemaDotOrgSources#FIBO"
  },
  "rdfs:comment": "A payment method using a credit, debit, store or other card to associate the payment with an account.",
  "rdfs:label": "PaymentCard",
  "rdfs:subClassOf": [
    {
      "@id": "http://schema.org/PaymentMethod"
    },
    {
      "@id": "http://schema.org/FinancialProduct"
    }
  ]
},

Looks like this "Enumeration is also a Class" issues applies to:

All the subclasses of https://schema.org/MedicalEnumeration
https://schema.org/PaymentCard
https://schema.org/QualitativeValue and its subclasses (though the https://schema.org/QualitativeValue page seems to have a bug)

So there are quite a few in there... if this isn't something you're aware of perhaps we raise it with the schema.org mailing list / GitHub?

You could ask them to make sure but I'm assuming having an enum and class with the same name is allowed by schema.org.
When you say there are quite a few, how many?

23 that seem to have properties according to the GUI - however: https://schema.org/DrugClass shows that even a straightforward “Enumeration” can be used as a class, without subclassing anything except Enumeration... so not sure whether it would be correct to assume that this applies to all Enumerations?
Perhaps this is where we need guidance?

It might be worth asking them how we should handle this scenario after explaining what Schema.NET has achieved. I'm not certain what the best resolution is at the moment.

Hey @RehanSaeed just to let you know I've reached out to the schema.org mailing list on this question.
For now we're going to implement the following (which seems to match what Schema.NET already does):

Any rdfs:Class that subclasses https://schema.org/Enumeration (either directly or indirectly) should be strictly an enum, and all other inheritance for that rdfs:Class should be ignored

If they come back with anything on the mailing list (specifically around https://schema.org/PaymentCard) we can pick it up then and perhaps discuss how to approach across both projects?

I'd raise an issue on their GitHub too. They do have a lot of open issues and I imagine that they are having a hard time keeping up but I've always got a response from the maintainers.

Please feel free to open a bug once you have some answers.


Copy number reference for CNVkit

Hello. I'm trying to test your germline_wgs.cwl pipeline with germline_wgs.yaml as values file.
I have several problems:

Looks like this line is wrong - cnvkit_vcf_export.cwl doesn't have tumor_bam input.
According to these lines: germline_wgs.cwl#L88, single_sample_sv_callers.cwl#L29 and cnvkit_single_sample.cwl#L24 - cnvkit_reference_cnn is required input.
But according to cnvkit_batch.cwl#L67 and tool help - it is optional.
I ask this question because exome_workflow.tgz doesn't contain this file and germline_wgs.yaml doesn't contain this input.

So - is this reference required? And if yes - how should I get it?
I find this method - but I doesn't have these files either.

Not sure whether this should go here, or a separate ticket opened.
I had the same issue, and using cnvkit, pooling all available normal samples to get a reference.cnn file,

placing it in the analysis-workflows/example_data/ directory and adding the line:

cnvkit_reference_cnn:
    class: File
    path: "exome_workflow/reference.cnn"

to your germline_wgs.yaml file in the analysis-workflows/example_data/ will work.
For me though, that just takes me to the next error - there are a number of fields that are required by the germline_wgs.cwl pipeline that are not present in the example germline_wgs.yaml file. cnvkit_reference_cnn was the first, following on from that are:
merge_estimate_sv_distance
merge_max_distance
merge_min_svs
merge_same_type
merge_same_strand
vep_ensembl_assembly
vep_ensembl_species
vep_ensembl_version
The vep ones are easy to fix by adding to the germline_wgs.yaml:
vep_ensembl_assembly: "GRCh38"
vep_ensembl_species: "homo_sapiens"
vep_ensembl_version: "95"

I haven't been able to figure out what tools the merge variables are parameters for, so I've got nothing sensible to put in them.

@apaul7 your PR #885 references this. Does it also resolve this issue?

Yes, the germline wgs pipeline should no longer have the cnvkit pooled reference.cnn input as a requirement. I'll close the issue.


Refactoring round 1

Alex I've stared at the code but can't figure out right now why it will only delete one CKAN item at a time and then gives up.

Also, apologies up front if this wasn't done "the Python way".... feel free to redo anything

This is looking really good!
Curious -- did you ever look for an external library to do what TimeFormatter does? It's a nice function you wrote, but Django tends to include a lot of these helpers; I might vote for us to switch for ease of maintenance if we wind up using it more than the one place it's called right now: https://github.com/frnhr/django_timedeltatemplatefilter
I have two comments about doing things "the Python way," one is very minor and one is less minor. The first is the use of the allcaps LOG, DB, and LOCK variables in the main script; they're totally fine, but the caps aren't very Pythonic and the extreme shortness of the variable names is a little initially confusing.
The less minor one is the way you've done the importing. Using constructions like from harvester.HarvestRepository import HarvestRepository pulls every function from that class into the global namespace, which is generally frowned upon because it can cause function name conflicts, but more importantly, makes it kind of a pain to chase down where a function like repo.update_stale_records() is being called from, since all someone could tell from looking at the main script is that repo is an instanstiation of CKANRepository. It took me a minute to realize the function was actually in HarvestRepository when I was trying to figure out where this came from in the logs:
2016-08-15 14:21:32,591 INFO Looking for stale records to update
2016-08-15 14:21:32,591 INFO Started processing for 100 records
2016-08-15 14:21:32,791 DEBUG Updating record 0000e36c-9131-5353-8438-fd07d1dc948d from repo at http://open.canada.ca/data
2016-08-15 14:21:33,566 ERROR Updating item failed
2016-08-15 14:21:33,567 ERROR Aborting due to errors after 0 items updated in 0 seconds (0.0 items/sec)

Any chance you'd be willing to change that?

To clarify -- these "library" files don't all have to contain one class each; you can just fill one up with helper functions and import it with the filename. You definitely wouldn't want to do that in every case but there are a couple that might actually be simplified that way.

Right, I saw that the files could easily contain more than one class, but I got to the point where I wanted the different repo types in their own files, and it ended up only the formatter and database stuff were together, so I made it all the same.

The top level import of HarvestRepository was not intentional, it was left over from when everything was in one file at the time!

Ah, OK :)

Well I see the problem now.  Fixed in next commit

Maybe still a problem with OAI repos trying to clean up old records from other repos... just a sec

Yeah that will work better

OK, looks good now! there's still a few aspects of this I'm finding a little unintuitive but it's still a huge improvement on having everything in one file and seems to be working great. Merging so I can work on non-OAI metadata. Thanks a ton.


Reverse ping/curl (for endpoints without exposed ports)

⚠ Please verify that this feature request has NOT been suggested before.

[X] I checked and didn't find similar feature request

🏷 Feature Request Type
New Monitor
🔖 Feature description
Missed this, some ISP blocking ICMP ping
Also would be great in many different cases
✔ Solution
Related thread on Reddit: https://www.reddit.com/r/devops/comments/iftexo/reverse_ping_monitoring/
❓ Alternatives
No response
📝 Additional Context
No response

It already implemented and called 'Push'


I created 5 new processor filters manually, because i wanted to compare the output over the runtime (which REF data was loaded/used) etc for different scenarios.
All of my processor filters ran, but I only ended up with one undeleted Stream and Error Stream, all the other streams/errors from the other filters had been deleted(assume superseded)
My expectation was that I would get multiple Event Streams( and errors)

I am wrong, this is expected behaviour


Better state machine example

If the example plugin is meant to provide a minimum working example of the state machine, I think it needs to be pared down some. Let's make it agnostic to the fact that this is even satellite code. Just create a few tasks with overly verbose names and varying frequencies/delays. Here's a suggestion

repeat_every_5_seconds_task
repeat_every_10_seconds_start_later_task
etc (as needed to demonstrate functionality)

Created a draft pull in #10, still need to proof read the README.
I intentionally left some very minimum satellite things in the code in order to demonstrate that in tasks you can still do things like read battery voltage or get uptime.


Failed peerswap from 63 days ago blocking two peers doing any further swapping

Hello everyone,
Problem statement: we have had a failed ps-swapout attempt from my side which failed, and has a state-cancelled on my end. No neither of us can do another swap with each other.
My side has "state": "State_SwapCanceled",
pscli getswap --id f02f50b2317a9a8dfa8a6a8a7ccab7ec25559f2eb3401d10626d4fdc3c1442f5
{
  "swap": {
    "id": "f02f50b2317a9a8dfa8a6a8a7ccab7ec25559f2eb3401d10626d4fdc3c1442f5",
    "created_at": "1705586782",
    "asset": "lbtc",
    "type": "swap-out",
    "role": "sender",
    "state": "State_SwapCanceled",
    "initiator_node_id": "037f66e84e38fc2787d578599dfe1fcb7b71f9de4fb1e453c5ab85c05f5ce8c2e3",
    "peer_node_id": "023e24602891c28a7872ea1ad5c1bb41abe4206ae1599bb981e3278a121e7895d6",
    "amount": "2000000",
    "channel_id": "812324:2395:1",
    "opening_tx_id": "",
    "claim_tx_id": "",
    "cancel_message": "the prepayment probe was unsuccessful: TemporaryChannelFailure(update=(*lnwire.ChannelUpdate)(0xc005d9f600)({\n Signature: [...removed...],\n ShortChannelID: (lnwire.ShortChannelID) 812324:2395:1,\n Timestamp: (uint32)<PHONE_NUMBER>,\n MessageFlags: (lnwire.ChanUpdateMsgFlags) 00000001,\n ChannelFlags: (lnwire.ChanUpdateChanFlags) 00000001,\n TimeLockDelta: (uint16) 142,\n HtlcMinimumMsat: (lnwire.MilliSatoshi) 10000 mSAT,\n BaseFee: (uint32) 1000,\n FeeRate: (uint32) 0,\n HtlcMaximumMsat: (lnwire.MilliSatoshi)<PHONE_NUMBER> mSAT,\n ExtraOpaqueData: (lnwire.ExtraOpaqueData) {\n }\n})\n)",
    "lnd_chan_id": "893159683678470145"
  }
}

On the peer side from @jvxis, peerswapd still awaits my payment
pscli listactiveswaps{
  "swaps": [    {
      "id": "f02f50b2317a9a8dfa8a6a8a7ccab7ec25559f2eb3401d10626d4fdc3c1442f5",      "created_at": "[1705586789](tel:1705586789)",
      "asset": "lbtc",      "type": "swap-out",
      "role": "receiver",      "state": "State_SwapOutReceiver_AwaitFeeInvoicePayment",
      "initiator_node_id": "037f66e84e38fc2787d578599dfe1fcb7b71f9de4fb1e453c5ab85c05f5ce8c2e3",      "peer_node_id": "037f66e84e38fc2787d578599dfe1fcb7b71f9de4fb1e453c5ab85c05f5ce8c2e3",
      "amount": "2000000",      "channel_id": "812324:2395:1",
      "opening_tx_id": "",      "claim_tx_id": "",
      "cancel_message": "",      "lnd_chan_id": "893159683678470145"
    }  ]
}

and hence every single swap-in from their side, or swap-out from my side is blocked:
My side
pscli swapout --sat_amt 3100566 --channel_id<PHONE_NUMBER>78470145 --asset lbtc
2024/03/28 18:48:12 rpc error: code = Unknown desc = already has an active swap on channel 812324:2395:1: f02f50b2317a9a8dfa8a6a8a7ccab7ec25559f2eb3401d10626d4fdc3c1442f5

Any advice or idea how we can resolve the situation, eg canceling the pending payment?

We both have upgraded to the latest version, and the issue remains.

 The canceled swap seems to be a result of a stuck channel which PeerSwap checks for now. But it seems the swap receiver hasn't canceled the swap for some reason.

Just built via the recent commit a few hours ago to doublecheck whether it resolves it
git branch -v
* master fb7d5f7 [skip ci]docs: transaction labels for on-chain transactions (#284)

I believe @jvxis runs the latest as well.
We do have usual LN routing activity running, whilst it's probably good context to know that the peering between our both nodes isn't 100%, since he's in SouthAmerica and my server's based in Germany. So we do have a connection drop every 10-12 hours for a few minutes

Same here: * master fb7d5f7 [skip ci]docs: transaction labels for on-chain transactions (#284)

What LND versions are you running?

What LND versions are you running?

Here is: LND 0.17.4-beta

What LND versions are you running?

admin@debian-nuc:~$ uname -a
Linux debian-nuc 6.1.0-16-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.67-1 (2023-12-12) x86_64 GNU/Linux
admin@debian-nuc:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description:    Debian GNU/Linux 12 (bookworm)
Release:        12
Codename:       bookworm
admin@debian-nuc:~$ lnd --version
lnd version 0.17.3-beta commit=v0.17.3-beta

It appears that the probe payment failed due to TemporaryChannelFailure.
In that case, a cancel message is also sent to the other peer, but it seems to have failed for some reason.
May need to modify peerswap to resolve.
It may be necessary to continue sending messages until the cancel message of the swap peer is successfully sent, or to add a "recover" process.

Both message sending and recovery cannot be manually operated.
I assume that the peerswap codebase will need to be modified.
I think deleting the target swap as a workaround is an option.
The swaps are managed in BoltDB and I believe they exist as swaps in peerswap datadir.
You can use tool such as boltbrowser to delete the target swap, and then you can run the next swap.

@YusukeShimizu It has worked! Thanks. Now I can do peerswaps again.

Excellent. Thank you @YusukeShimizu for helping us resolving it.
Let us know if you think you need anything for the message sending approach at a future point. Will close this as resolved.

It appears that the probe payment failed due to TemporaryChannelFailure.

This was truly a temporary LND channel problem where the channel worked fine later?
Should we perhaps try a few times instead of giving up immediately?

In that case, a cancel message is also sent to the other peer, but it seems to have failed for some reason.

The cancel message has no ACK? Should we add one and periodically repeat until we hear the ACK?

This was truly a temporary LND channel problem where the channel worked fine later?
Should we perhaps try a few times instead of giving up immediately?

TemporaryChannelFailure is considered a temporary problem that is expected to be recovered.
I think there is a way to retry with Exponential Backoff, etc.
I created draft PR : https://github.com/ElementsProject/peerswap/pull/294

The cancel message has no ACK? Should we add one and periodically repeat until we hear the ACK?

With SendCancelAction, the implementation does not guarantee sending the custom message because the state is set to State_SwapCanceled after the process is complete, regardless of whether the message is sent or not.
It is possible to repeat the process until success, but I think that adding a recovery process on the receiving side to guarantee result consistency would be the solution.
I created draft PR : https://github.com/ElementsProject/peerswap/pull/293


Changed URLs

Hi,
Any chance of script update ? It seems Opera stopped using /v4/register_subscriber for authentication.
Thanks

It looks like it is still used. I just had to update the client version header.


Prettier

Thank you for awesome tool.
I want to participate in development of this tool. Can I add Prettier to this project to have code formatting standard across the project? I think it will help in further development of prerender.
Prettier is a code formatting tool: https://prettier.io/

I wouldn't mind something like that but what's the difference between Prettier and a normal linting tool?

Prettier isn't linting tool, it's a formatting tool.
ESLint is a linting tool, they could work well side by side.

Actually, it will be great to add them both

Prettier looks interesting but we don't have very strict code formatting standards and adding Prettier would mainly just be for forcing things like semicolons, tabs, tab width, etc. I'm not too worried about that so I don't think we'll add Prettier yet but if we get lots of pull requests in the future with bad formatting I might revisit Prettier. Thanks though!


[CLEANUP] Refactor work on PanCommandExecutor/KitchenCommandExecutor …

…to promote code reuse
This pull request holds zero changes to the logic. It's a mere refactor work aimed at promoting code reuse.

Promoted util method convert( Map<String, String> map ) to AbstractBaseCommandExecutor, rather than being duplicated in PanCommandExecutor / KitchenCommandExecutor
executeRepositoryBasedCommand( ... ) was doing both the "print repos", "list repos", etc... type of operations, as well as the action of loading a Trans/Job from the Repository

decoupled the 2: now executeRepositoryBasedCommand( ... ) addresses the "print repos", "list repos", etc... while action of loading the Trans/Job from the Repository is now in loadTransFromRepository( ...) /  loadJobFromRepository( ...)

Centralized and promoted method loadRepositoryDirectory( repo, dirName ) to AbstractBaseCommandExecutor, rather than having its logic copy/pasted in PanCommandExecutor / KitchenCommandExecutor

Also

Checkstyle and copyright header year fixes
Fixed a broken test method in Kitchen test, as it was asserting the output against a COULD_NOT_LOAD_JOB status code, when it should be against a SUCCESS one.

To test Pan / Kitchen

mvn clean verify -Dtest=PanTest -DrunITs -Dit.test=PanIT
mvn clean verify -Dtest=KitchenTest -DrunITs -Dit.test=KitchenIT

@pentaho/rogueone please review

Build Completed
:x: This pull request has errors. They will need to be addressed before it can be accepted. See below for more details. Some links are also available below for further assistance in addressing those issues.
Build Commands
mvn -B -f 'pom.xml' -pl 'engine' -P '!assemblies' -Dsurefire.runOrder=alphabetical -Daudit -amd clean install

Cleanup Commands
mvn -B -f 'pom.xml' -pl 'engine' -P '!assemblies' -amd build-helper:remove-project-artifact

Changed files
  engine/src/main/java/org/pentaho/di/base/AbstractBaseCommandExecutor.java
  engine/src/main/java/org/pentaho/di/kitchen/KitchenCommandExecutor.java
  engine/src/main/java/org/pentaho/di/pan/PanCommandExecutor.java
  engine/src/test/java/org/pentaho/di/kitchen/KitchenTest.java

Unit Test Coverage
These statistics help you identify how your changes have affected the coverage of the following files. If a file is not in this list, then its coverage was not affected by your changes.  Found 2018. Was expecting 2019

Build Completed
:fire: This pull request has some issues. It would be preferable to fix them in order for it to be just perfect. See below for more details. Some links are also available below for further assistance in addressing those issues.
Build Commands
mvn -B -f 'pom.xml' -pl 'engine' -P '!assemblies' -Dsurefire.runOrder=alphabetical -Daudit -amd clean install

Cleanup Commands
mvn -B -f 'pom.xml' -pl 'engine' -P '!assemblies' -amd build-helper:remove-project-artifact

Changed files
  engine/src/main/java/org/pentaho/di/base/AbstractBaseCommandExecutor.java
  engine/src/main/java/org/pentaho/di/kitchen/KitchenCommandExecutor.java
  engine/src/main/java/org/pentaho/di/pan/PanCommandExecutor.java
  engine/src/test/java/org/pentaho/di/kitchen/KitchenTest.java

Unit Test Coverage
These statistics help you identify how your changes have affected the coverage of the following files. If a file is not in this list, then its coverage was not affected by your changes. To get some help interpreting these metrics, please refer to Jacoco's documentation.

:warning: Coverage Changes (click to expand)
org.pentaho.di.trans.steps.checksum.CheckSumMeta

Instruction Change: -0.2% :small_red_triangle_down:

org.pentaho.di.base.AbstractBaseCommandExecutor

Branch Change: -1.1% :small_red_triangle_down:
Complexity Change: -4.7% :small_red_triangle_down:
Instruction Change: -0.6% :small_red_triangle_down:
Method Change: -3.2% :small_red_triangle_down:

org.pentaho.di.base.Params

Complexity Change: -3.1% :small_red_triangle_down:
Instruction Change: -1.7% :small_red_triangle_down:
Line Change: -1.5% :small_red_triangle_down:
Method Change: -3.4% :small_red_triangle_down:

org.pentaho.di.job.entries.evalfilesmetrics.JobEntryEvalFilesMetrics

Branch Change: -0.4% :small_red_triangle_down:
Complexity Change: -0.5% :small_red_triangle_down:
Instruction Change: -0.1% :small_red_triangle_down:

org.pentaho.di.job.entries.waitforsql.JobEntryWaitForSQL

Branch Change: -0.8% :small_red_triangle_down:
Complexity Change: -1% :small_red_triangle_down:
Instruction Change: -0.1% :small_red_triangle_down:

org.pentaho.di.kitchen.KitchenCommandExecutor

Branch Change: -4.3% :small_red_triangle_down:
Complexity Change: -0.6% :small_red_triangle_down:
Line Change: -2.8% :small_red_triangle_down:

org.pentaho.di.pan.PanCommandExecutor

Branch Change: -7.4% :small_red_triangle_down:
Complexity Change: -4.2% :small_red_triangle_down:
Instruction Change: -1% :small_red_triangle_down:
Line Change: -4.8% :small_red_triangle_down:
Method Change: -3.2% :small_red_triangle_down:

org.pentaho.di.trans.steps.dbproc.DBProcMeta

Instruction Change: -0.1% :small_red_triangle_down:

org.pentaho.di.trans.steps.dimensionlookup.DimensionLookupMeta

Method Change: -1% :small_red_triangle_down:

org.pentaho.di.trans.steps.memgroupby.MemoryGroupByMeta

Branch Change: -1.1% :small_red_triangle_down:
Complexity Change: -1.3% :small_red_triangle_down:
Instruction Change: -0.1% :small_red_triangle_down:
Line Change: -0.4% :small_red_triangle_down:

org.pentaho.di.trans.steps.sortedmerge.SortedMergeMeta

Instruction Change: -0.2% :small_red_triangle_down:

org.pentaho.di.trans.steps.symmetriccrypto.symmetriccryptotrans.SymmetricCryptoTransMeta

Branch Change: -3.3% :small_red_triangle_down:
Complexity Change: -1.8% :small_red_triangle_down:
Instruction Change: -0.3% :small_red_triangle_down:

org.pentaho.di.trans.steps.tableoutput.TableOutputMeta

Branch Change: -0.9% :small_red_triangle_down:
Complexity Change: -0.8% :small_red_triangle_down:

org.pentaho.di.ui.repository.pur.repositoryexplorer.model.UIEERepositoryUser

Branch Change: -8.3% :small_red_triangle_down:
Complexity Change: -6.7% :small_red_triangle_down:
Instruction Change: -0.9% :small_red_triangle_down:
Line Change: -3.1% :small_red_triangle_down:

org.pentaho.di.ui.repository.pur.repositoryexplorer.model.UISecurityRole

Branch Change: -1.8% :small_red_triangle_down:
Complexity Change: -1.5% :small_red_triangle_down:
Instruction Change: -0.2% :small_red_triangle_down:
Line Change: -0.5% :small_red_triangle_down:

Note: The list of offending coverage changes extends beyond the presented results and was truncated to prevent reaching the comment size limit.


[stable/grafana] Support multiple types of dashboard provisioning

What this PR does / why we need it:
Support both the "standard" method of dashboard provisioning and the "sidecar" method of dashboard provisioning within this Helm chart, at the same time.
The Grafana chart currently supports both methods of automated dashboard provisioning, however both methods are not supported at the same time. This is a limitation of the Helm chart, which has been lifted by this commit.
This PR was created so that I can use the stable/prometheus-operator Helm chart, version 6.0.0 to deploy Grafana with Kubernetes monitoring dashboards. In addition to the dashboards which are part of stable/prometheus-operator, I wish to have additional dashboards from the Official Grafana Dashboard Hub provisioned automatically. This commit lifts the Helm chart's restriction on doing both of these two types of dashboard provisioning at the same time.
Which issue this PR fixes
No issue was created.
Special notes for your reviewer:
The automatic Grafana dashboard provisioning has been broken since stable/grafana version 3.5.6. Please see Issue #15725 for further information. I have tested this change by using stable/grafana 3.7.0 with my proposed fix in https://github.com/helm/charts/issues/15725#issuecomment-513926149 and the stable/prometheus-operator chart version 6.0.0.
I have mentioned all maintainers listed in Chart.yaml, per the instructions in the PR template:
@zanhsieh
@rtluckie
@maorfr
Checklist
[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

[x] DCO signed
[x] Chart Version bumped
[x] Title of the PR starts with chart name (e.g. [stable/chart])

/ok-to-test
/assign @zanhsieh
can you ptal?

@irasnd may you can include your suggestion https://github.com/helm/charts/issues/15725#issuecomment-513926149 inside the same PR ?

@irasnd may you can include your suggestion #15725 (comment) inside the same PR ?

Ok, I have added another commit which adds this fix.

/retest

/re-test

/test all

/lgtm

Ok, I have added another commit which adds this fix.

Thanks !
:-D


Insiders: JS move to new file refactoring import * as refs don't update

Issue Type: Bug
If you select a named export with your cursor and choose "Move to new file" the export is moved and named imports are updated but imports using import * as myExports from 'myModule;' don't get updated. I would expect a new named import to be added with the correct path to the new module and all the references to myExports.export to be changed to the new import.
VS Code version: Code - Insiders 1.24.0-insider (08537497eecd3172390194693d3d7c0ec8f52b70, 2018-05-31T05:10:58.506Z)
OS version: Darwin x64 17.5.0

System Info

Item
Value

CPUs
Intel(R) Core(TM) i7-3720QM CPU @ 2.60GHz (8 x 2600)

GPU Status
2d_canvas: unavailable_softwareflash_3d: enabledflash_stage3d: enabledflash_stage3d_baseline: enabledgpu_compositing: enabledmultiple_raster_threads: enabled_onnative_gpu_memory_buffers: enabledrasterization: unavailable_softwarevideo_decode: enabledvideo_encode: enabledvpx_decode: enabledwebgl: enabledwebgl2: enabled

Load (avg)
2, 3, 3

Memory (System)
16.00GB (0.82GB free)

Process Argv
/Applications/Visual Studio Code - Insiders.app/Contents/MacOS/Electron -psn_0_1122578

Screen Reader
no

VM
0%

Extensions: none

Screen recording of it not updating * imports.
http://recordit.co/l2dzXnHkHH

Opened https://github.com/Microsoft/TypeScript/issues/24583 to track this upstream


Headers and cookies

Is there a possibility of being able to access the connections headers and cookies?
As this would be very useful for cookie authenticated connections.
I myself rely on the cookie system to allow my users to connect via browser or by what I am working on currently a Xamarin Forms app.
Your websocket dll seems to be the most stable over all the others I have seen.
So I hope that I can end up using this PCL as finding PCLs that do websocket client side is dragging my development time quite a bit

Websockets.PCL is a wrapper around SocketRocket and AndroidAsync. Taking a look at those library we could in theory pass a dictionary of headers and cookies to the underlying implementation.
That said, I am not sure when I will have to make the changes (pretty busy at the moment). I will accept any pull requests if you want to go ahead and implement the feature.
Alternatively, you can pass your auth token as a query string or in an initial message.

Thanks for the reply.
Will look into it when I get home
Can you point me to a good starting point as I got a little lost when I looked at the source initially.
Thanks

https://github.com/NVentimiglia/Websockets.PCL/blob/master/Websockets/IWebSocketConnection.cs
Is the interface that all of the implementations implement. Open will need to include your header dictionary.
We pass this to a C# bridge
https://github.com/NVentimiglia/Websockets.PCL/blob/master/Websockets.Droid/WebsocketConnection.cs
Received by a Java Bridge
https://github.com/NVentimiglia/Websockets.PCL/blob/master/Websockets.DroidBridge/src/websockets/DroidBridge/BridgeController.java#L42
Use the Android Async API
https://github.com/NVentimiglia/Websockets.PCL/blob/master/Websockets.DroidBridge/src/websockets/DroidBridge/BridgeController.java#L65
https://github.com/koush/AndroidAsync

I did some more looking into AndroidAsync (I figured that would be the first gotcha). Im not sure now about the header support for websockets. (The API was not obvious) Will raise a question with them.
https://github.com/loopj/android-async-http/issues/1144

Ah ok thanks.
This seems like it would take some time. More than I have to get my app ready. So on server side I'll get it to recieve a json string and add cookies that way for now then I should be able to work on this once the app is out. Will keep you up to date


Create Metasploit 4.5.0-dev.md

Hello @rohang14
Thanks for your contribution.
Let's connect ->
GitHub -> @Suryansh1720001 (Follow)
LinkedIn -> https://www.linkedin.com/in/suryanshprajapati17
Also, Star the repo ⭐


[MOS-016] View entry

As a
user
I want*
to tap on an entry in the overview
so that
I can view the whole entry.
Acceptance Criteria:

[ ] View the content of the entry
[ ] Add the 'Edit' & 'Delete' buttons