unable to train,

I have an Nvidia 4070 Super 12GB GPU and followed all the installation steps correctly, including trying the Easy Pinokio installer. Despite this, I keep encountering the same error at the end. The script is running with the 'venv' environment activated, and both the required dependencies and Nightly PyTorch are installed.
[2024-09-07 00:45:50] [INFO] F:\fluxgym\env\Lib\site-packages\torch\autograd\graph.py:818: UserWarning: cuDNN SDPA backward got grad_output.strides() != output.strides(), attempting to materialize a grad_output with matching strides... (Triggered internally at C:\actions-runner_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cudnn\MHA.cpp:672.)
[2024-09-07 00:45:50] [INFO] return Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass

Same here, on 4090
NVIDIA-SMI 560.35.03              Driver Version: 560.35.03      CUDA Version: 12.6
OS: Ubuntu 22.10
Brand new conda environment with Python 3.11.9
Torch version: 2.5.0.dev20240906+cu121
Seems to have crashed following 150 steps of generation
steps:   6%|▌         | 150/2560 [02:18<37:05,  1.08it/s, avr_loss=nan]Traceback (most recent call last):
[2024-09-07 08:47:52] [INFO] File "/home/me/ai/fluxgym/sd-scripts/flux_train_network.py", line 519, in <module>
[2024-09-07 08:47:52] [INFO] trainer.train(args)
[2024-09-07 08:47:52] [INFO] File "/home/me/ai/fluxgym/sd-scripts/train_network.py", line 1171, in train
[2024-09-07 08:47:52] [INFO] accelerator.backward(loss)
[2024-09-07 08:47:52] [INFO] File "/home/me/anaconda3/envs/fluxgym/lib/python3.11/site-packages/accelerate/accelerator.py", line 2159, in backward
[2024-09-07 08:47:52] [INFO] loss.backward(**kwargs)
[2024-09-07 08:47:52] [INFO] File "/home/me/anaconda3/envs/fluxgym/lib/python3.11/site-packages/torch/_tensor.py", line 581, in backward
[2024-09-07 08:47:52] [INFO] torch.autograd.backward(
[2024-09-07 08:47:52] [INFO] File "/home/me/anaconda3/envs/fluxgym/lib/python3.11/site-packages/torch/autograd/__init__.py", line 347, in backward
[2024-09-07 08:47:52] [INFO] _engine_run_backward(
[2024-09-07 08:47:52] [INFO] File "/home/me/anaconda3/envs/fluxgym/lib/python3.11/site-packages/torch/autograd/graph.py", line 818, in _engine_run_backward
[2024-09-07 08:47:52] [INFO] return Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass
[2024-09-07 08:47:52] [INFO] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2024-09-07 08:47:52] [INFO] RuntimeError: CUDA error: CUBLAS_STATUS_EXECUTION_FAILED when calling `cublasGemmEx( handle, opa, opb, m, n, k, &falpha, a, CUDA_R_16BF, lda, b, CUDA_R_16BF, ldb, &fbeta, c, CUDA_R_16BF, ldc, compute_type, CUBLAS_GEMM_DEFAULT_TENSOR_OP)`
[2024-09-07 08:47:53] [INFO] steps:   6%|▌         | 150/2560 [02:19<37:19,  1.08it/s, avr_loss=nan]
[2024-09-07 08:47:54] [INFO] Traceback (most recent call last):
[2024-09-07 08:47:54] [INFO] File "/home/me/anaconda3/envs/fluxgym/bin/accelerate", line 8, in <module>
[2024-09-07 08:47:54] [INFO] sys.exit(main())
[2024-09-07 08:47:54] [INFO] ^^^^^^
[2024-09-07 08:47:54] [INFO] File "/home/me/anaconda3/envs/fluxgym/lib/python3.11/site-packages/accelerate/commands/accelerate_cli.py", line 48, in main
[2024-09-07 08:47:54] [INFO] args.func(args)
[2024-09-07 08:47:54] [INFO] File "/home/me/anaconda3/envs/fluxgym/lib/python3.11/site-packages/accelerate/commands/launch.py", line 1106, in launch_command
[2024-09-07 08:47:54] [INFO] simple_launcher(args)
[2024-09-07 08:47:54] [INFO] File "/home/me/anaconda3/envs/fluxgym/lib/python3.11/site-packages/accelerate/commands/launch.py", line 704, in simple_launcher
[2024-09-07 08:47:54] [INFO] raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd)
[2024-09-07 08:47:54] [INFO] subprocess.CalledProcessError: Command '['/home/me/anaconda3/envs/fluxgym/bin/python3.11', 'sd-scripts/flux_train_network.py', '--pretrained_model_name_or_path', '/home/me/ai/fluxgym/models/unet/flux1-dev.sft', '--clip_l', '/home/me/ai/fluxgym/models/clip/clip_l.safetensors', '--t5xxl', '/home/me/ai/fluxgym/models/clip/t5xxl_fp16.safetensors', '--ae', '/home/me/ai/fluxgym/models/vae/ae.sft', '--cache_latents_to_disk', '--save_model_as', 'safetensors', '--sdpa', '--persistent_data_loader_workers', '--max_data_loader_n_workers', '2', '--seed', '42', '--gradient_checkpointing', '--mixed_precision', 'bf16', '--save_precision', 'bf16', '--network_module', 'networks.lora_flux', '--network_dim', '4', '--optimizer_type', 'adamw8bit', '--learning_rate', '1e-4', '--cache_text_encoder_outputs', '--cache_text_encoder_outputs_to_disk', '--fp8_base', '--highvram', '--max_train_epochs', '16', '--save_every_n_epochs', '4', '--dataset_config', '/home/me/ai/fluxgym/dataset.toml', '--output_dir', '/home/me/ai/fluxgym/outputs', '--output_name', 'adiv1', '--timestep_sampling', 'shift', '--discrete_flow_shift', '3.1582', '--model_prediction_type', 'raw', '--guidance_scale', '1.0', '--loss_type', 'l2']' returned non-zero exit status 1.
[2024-09-07 08:47:54] [ERROR] Command exited with code 1
[2024-09-07 08:47:54] [INFO] Runner: <LogsViewRunner nb_logs=161 exit_code=1>

I have similar error with Nvidia RTX 4060 TI 16GB VRAM:
error.txt

I followed official sd-scripts README. I reinstalled PyTorch (2.4.0) as following the step from the below instruction.
https://github.com/kohya-ss/sd-scripts/tree/sd3?tab=readme-ov-file#flux1-training-wip

FLUX.1 training (WIP)
This feature is experimental. The options and the training script may change in the future. Please let us know if you have any idea to improve the training.
Please update PyTorch to 2.4.0. We have tested with torch==2.4.0 and torchvision==0.19.0 with CUDA 12.4. We also updated accelerate to 0.33.0 just to be safe. requirements.txt is also updated, so please update the requirements.
The command to install PyTorch is as follows: pip3 install torch==2.4.0 torchvision==0.19.0 --index-url https://download.pytorch.org/whl/cu124

I followed official sd-scripts README. I reinstalled PyTorch (2.4.0) as following the step from the below instruction. I think the error is gone.
https://github.com/kohya-ss/sd-scripts/tree/sd3?tab=readme-ov-file#flux1-training-wip
FLUX.1 training (WIP) This feature is experimental. The options and the training script may change in the future. Please let us know if you have any idea to improve the training.
Please update PyTorch to 2.4.0. We have tested with torch==2.4.0 and torchvision==0.19.0 with CUDA 12.4. We also updated accelerate to 0.33.0 just to be safe. requirements.txt is also updated, so please update the requirements.
The command to install PyTorch is as follows: pip3 install torch==2.4.0 torchvision==0.19.0 --index-url https://download.pytorch.org/whl/cu124

The install steps prompted on the front README says to install the nightly build which is a dev torch 2.5.0 not 2.4.0.
I just downgraded both Torch and Torchvision to 2.4.0 and 0.19.0
Now I'm getting a new error
[2024-09-07 03:20:06] [INFO] F:\fluxgym\env\Lib\site-packages\torch\utils\checkpoint.py:1399: FutureWarning: torch.cpu.amp.autocast(args...) is deprecated. Please use torch.amp.autocast('cpu', args...) instead.
[2024-09-07 03:20:06] [INFO] with device_autocast_ctx, torch.cpu.amp.autocast(**cpu_autocast_kwargs), recompute_context:  # type: ignore[attr-defined]

I followed official sd-scripts README. I reinstalled PyTorch (2.4.0) as following the step from the below instruction. I think the error is gone.
https://github.com/kohya-ss/sd-scripts/tree/sd3?tab=readme-ov-file#flux1-training-wip
FLUX.1 training (WIP) This feature is experimental. The options and the training script may change in the future. Please let us know if you have any idea to improve the training.
Please update PyTorch to 2.4.0. We have tested with torch==2.4.0 and torchvision==0.19.0 with CUDA 12.4. We also updated accelerate to 0.33.0 just to be safe. requirements.txt is also updated, so please update the requirements.
The command to install PyTorch is as follows: pip3 install torch==2.4.0 torchvision==0.19.0 --index-url https://download.pytorch.org/whl/cu124

The install steps prompted on the front README says to install the nightly build which is a dev torch 2.5.0 not 2.4.0.
I just downgraded both Torch and Torchvision to 2.4.0 and 0.19.0
Now I'm getting a new error
[2024-09-07 03:20:06] [INFO] F:\fluxgym\env\Lib\site-packages\torch\utils\checkpoint.py:1399: FutureWarning: torch.cpu.amp.autocast(args...) is deprecated. Please use torch.amp.autocast('cpu', args...) instead. [2024-09-07 03:20:06] [INFO] with device_autocast_ctx, torch.cpu.amp.autocast(**cpu_autocast_kwargs), recompute_context: # type: ignore[attr-defined]

Does that prevent your lora training?

I followed official sd-scripts README. I reinstalled PyTorch (2.4.0) as following the step from the below instruction. I think the error is gone.
https://github.com/kohya-ss/sd-scripts/tree/sd3?tab=readme-ov-file#flux1-training-wip
FLUX.1 training (WIP) This feature is experimental. The options and the training script may change in the future. Please let us know if you have any idea to improve the training.
Please update PyTorch to 2.4.0. We have tested with torch==2.4.0 and torchvision==0.19.0 with CUDA 12.4. We also updated accelerate to 0.33.0 just to be safe. requirements.txt is also updated, so please update the requirements.
The command to install PyTorch is as follows: pip3 install torch==2.4.0 torchvision==0.19.0 --index-url https://download.pytorch.org/whl/cu124

The install steps prompted on the front README says to install the nightly build which is a dev torch 2.5.0 not 2.4.0.
I just downgraded both Torch and Torchvision to 2.4.0 and 0.19.0
Now I'm getting a new error
[2024-09-07 03:20:06] [INFO] F:\fluxgym\env\Lib\site-packages\torch\utils\checkpoint.py:1399: FutureWarning: torch.cpu.amp.autocast(args...) is deprecated. Please use torch.amp.autocast('cpu', args...) instead. [2024-09-07 03:20:06] [INFO] with device_autocast_ctx, torch.cpu.amp.autocast(**cpu_autocast_kwargs), recompute_context: # type: ignore[attr-defined]

Does that prevent your lora training?

Yes, there is no training steps that begin, this is the last prompt and is just stagnant.
[2024-09-07 03:20:06] [INFO] F:\fluxgym\env\Lib\site-packages\torch\utils\checkpoint.py:1399: FutureWarning: torch.cpu.amp.autocast(args...) is deprecated. Please use torch.amp.autocast('cpu', args...) instead. [2024-09-07 03:20:06] [INFO] with device_autocast_ctx, torch.cpu.amp.autocast(**cpu_autocast_kwargs), recompute_context: # type: ignore[attr-defined]

I followed official sd-scripts README. I reinstalled PyTorch (2.4.0) as following the step from the below instruction. I think the error is gone.
https://github.com/kohya-ss/sd-scripts/tree/sd3?tab=readme-ov-file#flux1-training-wip
FLUX.1 training (WIP) This feature is experimental. The options and the training script may change in the future. Please let us know if you have any idea to improve the training.
Please update PyTorch to 2.4.0. We have tested with torch==2.4.0 and torchvision==0.19.0 with CUDA 12.4. We also updated accelerate to 0.33.0 just to be safe. requirements.txt is also updated, so please update the requirements.
The command to install PyTorch is as follows: pip3 install torch==2.4.0 torchvision==0.19.0 --index-url https://download.pytorch.org/whl/cu124

The install steps prompted on the front README says to install the nightly build which is a dev torch 2.5.0 not 2.4.0.
I just downgraded both Torch and Torchvision to 2.4.0 and 0.19.0
Now I'm getting a new error
[2024-09-07 03:20:06] [INFO] F:\fluxgym\env\Lib\site-packages\torch\utils\checkpoint.py:1399: FutureWarning: torch.cpu.amp.autocast(args...) is deprecated. Please use torch.amp.autocast('cpu', args...) instead. [2024-09-07 03:20:06] [INFO] with device_autocast_ctx, torch.cpu.amp.autocast(**cpu_autocast_kwargs), recompute_context: # type: ignore[attr-defined]

Does that prevent your lora training?

Yes, there is no training steps that begin, this is the last prompt and is just stagnant.
[2024-09-07 03:20:06] [INFO] F:\fluxgym\env\Lib\site-packages\torch\utils\checkpoint.py:1399: FutureWarning: torch.cpu.amp.autocast(args...) is deprecated. Please use torch.amp.autocast('cpu', args...) instead. [2024-09-07 03:20:06] [INFO] with device_autocast_ctx, torch.cpu.amp.autocast(**cpu_autocast_kwargs), recompute_context: # type: ignore[attr-defined]

Same error here and it remains stuck I hope @cocktailpeanut will fix this soon once it works I will be using it a lot.

I have similar error with Nvidia RTX 4060 TI 16GB VRAM:
K:\Users\CRCODE22\pinokio\api\fluxgym.git\env\lib\site-packages\torch\autograd\graph.py:818: UserWarning: cuDNN SDPA backward got grad_output.strides() != output.strides(), attempting to materialize a grad_output with matching strides... (Triggered internally at C:\actions-runner_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cudnn\MHA.cpp:672.)
[2024-09-07 11:23:05] [INFO] return Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass

I think this is a warning and not an error. Unless the process crashes it is still going.
Even I see this warning message but the training works fine. The reason it is not updating quickly is because the script seems to only print all the updates after each epoch (instead of each step).
This means it takes quite some time before the first progress is printed on the screen. So unless the program completely crashes and you don't see any VRAM usage on your task manager, try to keep it running and see if it updates.

Same here, on 4090 NVIDIA-SMI 560.35.03 Driver Version: 560.35.03 CUDA Version: 12.6 OS: Ubuntu 22.10 Brand new conda environment with Python 3.11.9 Torch version: 2.5.0.dev20240906+cu121
Seems to have crashed following 150 steps of generation
steps:   6%|▌         | 150/2560 [02:18<37:05,  1.08it/s, avr_loss=nan]Traceback (most recent call last):
[2024-09-07 08:47:52] [INFO] File "/home/me/ai/fluxgym/sd-scripts/flux_train_network.py", line 519, in <module>
[2024-09-07 08:47:52] [INFO] trainer.train(args)
[2024-09-07 08:47:52] [INFO] File "/home/me/ai/fluxgym/sd-scripts/train_network.py", line 1171, in train
[2024-09-07 08:47:52] [INFO] accelerator.backward(loss)
[2024-09-07 08:47:52] [INFO] File "/home/me/anaconda3/envs/fluxgym/lib/python3.11/site-packages/accelerate/accelerator.py", line 2159, in backward
[2024-09-07 08:47:52] [INFO] loss.backward(**kwargs)
[2024-09-07 08:47:52] [INFO] File "/home/me/anaconda3/envs/fluxgym/lib/python3.11/site-packages/torch/_tensor.py", line 581, in backward
[2024-09-07 08:47:52] [INFO] torch.autograd.backward(
[2024-09-07 08:47:52] [INFO] File "/home/me/anaconda3/envs/fluxgym/lib/python3.11/site-packages/torch/autograd/__init__.py", line 347, in backward
[2024-09-07 08:47:52] [INFO] _engine_run_backward(
[2024-09-07 08:47:52] [INFO] File "/home/me/anaconda3/envs/fluxgym/lib/python3.11/site-packages/torch/autograd/graph.py", line 818, in _engine_run_backward
[2024-09-07 08:47:52] [INFO] return Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass
[2024-09-07 08:47:52] [INFO] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2024-09-07 08:47:52] [INFO] RuntimeError: CUDA error: CUBLAS_STATUS_EXECUTION_FAILED when calling `cublasGemmEx( handle, opa, opb, m, n, k, &falpha, a, CUDA_R_16BF, lda, b, CUDA_R_16BF, ldb, &fbeta, c, CUDA_R_16BF, ldc, compute_type, CUBLAS_GEMM_DEFAULT_TENSOR_OP)`
[2024-09-07 08:47:53] [INFO] steps:   6%|▌         | 150/2560 [02:19<37:19,  1.08it/s, avr_loss=nan]
[2024-09-07 08:47:54] [INFO] Traceback (most recent call last):
[2024-09-07 08:47:54] [INFO] File "/home/me/anaconda3/envs/fluxgym/bin/accelerate", line 8, in <module>
[2024-09-07 08:47:54] [INFO] sys.exit(main())
[2024-09-07 08:47:54] [INFO] ^^^^^^
[2024-09-07 08:47:54] [INFO] File "/home/me/anaconda3/envs/fluxgym/lib/python3.11/site-packages/accelerate/commands/accelerate_cli.py", line 48, in main
[2024-09-07 08:47:54] [INFO] args.func(args)
[2024-09-07 08:47:54] [INFO] File "/home/me/anaconda3/envs/fluxgym/lib/python3.11/site-packages/accelerate/commands/launch.py", line 1106, in launch_command
[2024-09-07 08:47:54] [INFO] simple_launcher(args)
[2024-09-07 08:47:54] [INFO] File "/home/me/anaconda3/envs/fluxgym/lib/python3.11/site-packages/accelerate/commands/launch.py", line 704, in simple_launcher
[2024-09-07 08:47:54] [INFO] raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd)
[2024-09-07 08:47:54] [INFO] subprocess.CalledProcessError: Command '['/home/me/anaconda3/envs/fluxgym/bin/python3.11', 'sd-scripts/flux_train_network.py', '--pretrained_model_name_or_path', '/home/me/ai/fluxgym/models/unet/flux1-dev.sft', '--clip_l', '/home/me/ai/fluxgym/models/clip/clip_l.safetensors', '--t5xxl', '/home/me/ai/fluxgym/models/clip/t5xxl_fp16.safetensors', '--ae', '/home/me/ai/fluxgym/models/vae/ae.sft', '--cache_latents_to_disk', '--save_model_as', 'safetensors', '--sdpa', '--persistent_data_loader_workers', '--max_data_loader_n_workers', '2', '--seed', '42', '--gradient_checkpointing', '--mixed_precision', 'bf16', '--save_precision', 'bf16', '--network_module', 'networks.lora_flux', '--network_dim', '4', '--optimizer_type', 'adamw8bit', '--learning_rate', '1e-4', '--cache_text_encoder_outputs', '--cache_text_encoder_outputs_to_disk', '--fp8_base', '--highvram', '--max_train_epochs', '16', '--save_every_n_epochs', '4', '--dataset_config', '/home/me/ai/fluxgym/dataset.toml', '--output_dir', '/home/me/ai/fluxgym/outputs', '--output_name', 'adiv1', '--timestep_sampling', 'shift', '--discrete_flow_shift', '3.1582', '--model_prediction_type', 'raw', '--guidance_scale', '1.0', '--loss_type', 'l2']' returned non-zero exit status 1.
[2024-09-07 08:47:54] [ERROR] Command exited with code 1
[2024-09-07 08:47:54] [INFO] Runner: <LogsViewRunner nb_logs=161 exit_code=1>

This is a different problem than the others because this one actually fails and crashes.
I have experienced this a few times too, and don't know when it exactly happens. But when I run the exact same thing one more time it works. So I've been assuming it's some edge case that has to do with the script.
The good news is it at least was training for a bit before it crashed, which is way better than it just failing to begin with.
So I recommend just trying one more time and see if it works this time. Let me know how it goes

Oh also, by the way everyone on this thread, I pushed a fix a few hours ago for customized training https://github.com/cocktailpeanut/fluxgym/commit/a118913e3b18c91d143ec334662cca3ccd1859a4
Basically if you were trying anything other than the default config it was not actually picking up those settings, so for example if you tried the 12G or 16G vram option, it may have been just trying to use the default 20G vram option.
So just to make sure, try pulling in the changes and retry.

Oh also, by the way everyone on this thread, I pushed a fix a few hours ago for customized training a118913
Basically if you were trying anything other than the default config it was not actually picking up those settings, so for example if you tried the 12G or 16G vram option, it may have been just trying to use the default 20G vram option.
So just to make sure, try pulling in the changes and retry.

Thank you I am doing a clean install and will report if it works.

Oh also, by the way everyone on this thread, I pushed a fix a few hours ago for customized training a118913
Basically if you were trying anything other than the default config it was not actually picking up those settings, so for example if you tried the 12G or 16G vram option, it may have been just trying to use the default 20G vram option.
So just to make sure, try pulling in the changes and retry.

@cocktailpeanut that has not solved the problem "[2024-09-07 14:39:09] [INFO] K:\Users\CRCODE22\pinokio\api\fluxgym.git\env\lib\site-packages\torch\autograd\graph.py:818: UserWarning: cuDNN SDPA backward got grad_output.strides() != output.strides(), attempting to materialize a grad_output with matching strides... (Triggered internally at C:\actions-runner_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cudnn\MHA.cpp:672.)
[2024-09-07 14:39:09] [INFO] return Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass"
error.txt

Also it is not an out of memory problem as it does not hit the limit it will just remain stuck on that error and continue using GPU for hours without the training progressing and without new logs.

I have similar error with Nvidia RTX 4060 TI 16GB VRAM:
K:\Users\CRCODE22\pinokio\api\fluxgym.git\env\lib\site-packages\torch\autograd\graph.py:818: UserWarning: cuDNN SDPA backward got grad_output.strides() != output.strides(), attempting to materialize a grad_output with matching strides... (Triggered internally at C:\actions-runner_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cudnn\MHA.cpp:672.)
[2024-09-07 11:23:05] [INFO] return Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass

I think this is a warning and not an error. Unless the process crashes it is still going.
Even I see this warning message but the training works fine. The reason it is not updating quickly is because the script seems to only print all the updates after each epoch (instead of each step).
This means it takes quite some time before the first progress is printed on the screen. So unless the program completely crashes and you don't see any VRAM usage on your task manager, try to keep it running and see if it updates.

I had it running for hours and it will not increase epoch or provide any updates.

@cocktailpeanut it works now thank you.
working.txt
| 256/4096 [19:46<4:56:36,  4.63s/it, avr_loss=0.344] [2024-09-07 14:58:26] [INFO] epoch 2/16

I have similar error with Nvidia RTX 4060 TI 16GB VRAM:
K:\Users\CRCODE22\pinokio\api\fluxgym.git\env\lib\site-packages\torch\autograd\graph.py:818: UserWarning: cuDNN SDPA backward got grad_output.strides() != output.strides(), attempting to materialize a grad_output with matching strides... (Triggered internally at C:\actions-runner_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cudnn\MHA.cpp:672.)
[2024-09-07 11:23:05] [INFO] return Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass

I think this is a warning and not an error. Unless the process crashes it is still going.
Even I see this warning message but the training works fine. The reason it is not updating quickly is because the script seems to only print all the updates after each epoch (instead of each step).
This means it takes quite some time before the first progress is printed on the screen. So unless the program completely crashes and you don't see any VRAM usage on your task manager, try to keep it running and see if it updates.

Hey, thank you for the prompt response and trying to help troubleshoot this issue. I followed your instructions. It has been over 30 minutes with the training running, it doesn't seem to resolve, no GPU usage in task manager from the Command Prompt. I'm unsure how to go about this, as I have mentioned, I reinstalled this multiple times, step by step and still have this code. The actual command prompt only last shows the Florence 2 captioning, I don't know if training data should appear in there as well? I did try both PowerShell and CMD.
[2024-09-07 17:30:25] [INFO] steps:   0%|          | 0/3200 [00:00<?, ?it/s]2024-09-07 17:30:25 INFO     unet dtype:                  train_network.py:1046
[2024-09-07 17:30:25] [INFO] torch.float8_e4m3fn, device:
[2024-09-07 17:30:25] [INFO] cuda:0
[2024-09-07 17:30:25] [INFO] INFO     text_encoder [0] dtype:      train_network.py:1052
[2024-09-07 17:30:25] [INFO] torch.float8_e4m3fn, device:
[2024-09-07 17:30:25] [INFO] cuda:0
[2024-09-07 17:30:25] [INFO] INFO     text_encoder [1] dtype:      train_network.py:1052
[2024-09-07 17:30:25] [INFO] torch.bfloat16, device: cpu
[2024-09-07 17:30:26] [INFO]
[2024-09-07 17:30:26] [INFO] epoch 1/16
[2024-09-07 17:30:34] [INFO] 2024-09-07 17:30:34 INFO     epoch is incremented.            train_util.py:668
[2024-09-07 17:30:34] [INFO] current_epoch: 0, epoch: 1
[2024-09-07 17:30:34] [INFO] 2024-09-07 17:30:34 INFO     epoch is incremented.            train_util.py:668
[2024-09-07 17:30:34] [INFO] current_epoch: 0, epoch: 1
[2024-09-07 17:30:43] [INFO] F:\fluxgym\env\Lib\site-packages\torch\utils\checkpoint.py:1399: FutureWarning: torch.cpu.amp.autocast(args...) is deprecated. Please use torch.amp.autocast('cpu', args...) instead.
[2024-09-07 17:30:43] [INFO] with device_autocast_ctx, torch.cpu.amp.autocast(**cpu_autocast_kwargs), recompute_context:  # type: ignore[attr-defined]

After a "git pull" on the latest update, a few more commands lines have added to this error, hopefully this helps.
[2024-09-07 18:12:47] [INFO] steps:   0%|          | 0/3200 [00:00<?, ?it/s]2024-09-07 18:12:47 INFO     unet dtype:                  train_network.py:1046
[2024-09-07 18:12:47] [INFO] torch.float8_e4m3fn, device:
[2024-09-07 18:12:47] [INFO] cuda:0
[2024-09-07 18:12:47] [INFO] INFO     text_encoder [0] dtype:      train_network.py:1052
[2024-09-07 18:12:47] [INFO] torch.float8_e4m3fn, device:
[2024-09-07 18:12:47] [INFO] cuda:0
[2024-09-07 18:12:47] [INFO] INFO     text_encoder [1] dtype:      train_network.py:1052
[2024-09-07 18:12:47] [INFO] torch.bfloat16, device: cpu
[2024-09-07 18:12:47] [INFO]
[2024-09-07 18:12:47] [INFO] epoch 1/16
[2024-09-07 18:12:50] [INFO] F:\fluxgym\env\Lib\site-packages\diffusers\utils\outputs.py:63: FutureWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
[2024-09-07 18:12:50] [INFO] torch.utils._pytree._register_pytree_node(
[2024-09-07 18:12:50] [INFO] F:\fluxgym\env\Lib\site-packages\diffusers\utils\outputs.py:63: FutureWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
[2024-09-07 18:12:50] [INFO] torch.utils._pytree._register_pytree_node(
[2024-09-07 18:12:54] [INFO] F:\fluxgym\env\Lib\site-packages\diffusers\utils\outputs.py:63: FutureWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
[2024-09-07 18:12:54] [INFO] torch.utils._pytree._register_pytree_node(
[2024-09-07 18:12:54] [INFO] F:\fluxgym\env\Lib\site-packages\diffusers\utils\outputs.py:63: FutureWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
[2024-09-07 18:12:54] [INFO] torch.utils._pytree._register_pytree_node(
[2024-09-07 18:12:54] [INFO] 2024-09-07 18:12:54 INFO     epoch is incremented.            train_util.py:668
[2024-09-07 18:12:54] [INFO] current_epoch: 0, epoch: 1
[2024-09-07 18:12:54] [INFO] 2024-09-07 18:12:54 INFO     epoch is incremented.            train_util.py:668
[2024-09-07 18:12:54] [INFO] current_epoch: 0, epoch: 1
[2024-09-07 18:13:03] [INFO] F:\fluxgym\env\Lib\site-packages\torch\utils\checkpoint.py:1399: FutureWarning: torch.cpu.amp.autocast(args...) is deprecated. Please use torch.amp.autocast('cpu', args...) instead.
[2024-09-07 18:13:03] [INFO] with device_autocast_ctx, torch.cpu.amp.autocast(**cpu_autocast_kwargs), recompute_context:  # type: ignore[attr-defined]

yeah so i see why people think that warning is a crash...becuase there is nothing to show you that training is still going until it decides to make a sample image. everyone is so use to seeing the training steps incrementing and loss rates...


Fisher & Paykel RF605QDAA (Quad Door) Fridge

The fridge is not displaying properly. This is what I see in the GE App:

This is what I see in HA:

The top is double door fridge, bottom left is freezer and bottom right can be changed between:  Freeze,  Soft Freeze, Chill and Fridge
If you tell me what you need and how to get it, I will provide more information...

I have the same fridge & the same information (identical) being reported.
I am also happy to provide logs to help with the development of this integration.
It would also be great if there was current temperatures of the compartments.
My freezer has stopped working and is about +10 Deg C despite the -18 Deg C set point. I have put my Netatmo monitors in the compartments to  monitor the temperatures. The Fisher Paykel tech is booked in for the 10-October.

I think that the integration can provide only what's in the native App. Tbh, I don't think it provides current temperatures...

I expect that might be the case. However with my Daikin air conditioners, the app does not have the compressor speed, but the Home Assistant integration does have compressor speed, which I have found useful. I expect the current temperatures would not been in the SmartHQ app as customers might be concerned with the temperature not being exactly on the set point, however as an engineer, I understand that the temperature will fluctuate even under great control.

I expect that might be the case. However with my Daikin air conditioners, the app does not have the compressor speed, but the Home Assistant integration does have compressor speed, which I have found useful. I expect the current temperatures would not been in the SmartHQ app as customers might be concerned with the temperature not being exactly on the set point, however as an engineer, I understand that the temperature will fluctuate even under great control.

Yes, one would reasonably expect that here must be current temperature reading somewhere in the fridge. After all, how else would it maintain the set temperature? But if it is exposed to API/Cloud or whatever SmartHQ is using is so far a mystery.
If you find something, please let me know. I too had the freezer fail and knowing that earlier could safe some food and time with servicing.

Hi guys, Fpa ref products do not expose compartment temperatures to the app as we don't want people getting upset when their target temp != compartment temp.  You'd be surprised how many people get upset when their product temp doesn't exactly match the target temp at all times.
Internally the product knows the compartment temperatures.

Hi guys, Fpa ref products do not expose compartment temperatures to the app as we don't want people getting upset when their target temp != compartment temp. You'd be surprised how many people get upset when their product temp doesn't exactly match the target temp at all times.
Internally the product knows the compartment temperatures.

Exactly what I was expecting.  If I was at F&P, I would be developing SMART functionality in a way that technicians could possible remotely troubleshoot. So, there must be a backdoor somewhere, or a way to read these... Perhaps they even go to the cloud and are just not exposed to the App... Time to set up a sniffer and capture all the traffic... Nice hack challenge... 🤣😂😁

Hi guys, Fpa ref products do not expose compartment temperatures to the app as we don't want people getting upset when their target temp != compartment temp. You'd be surprised how many people get upset when their product temp doesn't exactly match the target temp at all times.
Internally the product knows the compartment temperatures.

Exactly what I was expecting. If I was at F&P, I would be developing SMART functionality in a way that technicians could possibly troubleshoot remotely. So, there must be a backdoor somewhere, or a way to read these... Perhaps they even go to the cloud and are just not exposed to the App... Time to set up a sniffer and capture all the traffic... Nice hack challenge... 🤣😂😁

There isn't a service mode that exposes these values to wifi, yet.  There will be in the future though ;)

Per my original message, I have had a problem with my Quad door Fisher & Paykel fridge recently.
In order to see what the temperatures are doing in the fridge, I have had to put my Netamo sensors in each compartment.
The fridge is doing as expected with a set point of 3 Deg C and the actual varying between 3 & 4 Deg C. The variable temperature compartment with a set point of -18 Deg C does vary a lot. It does get down to about -15 Deg C but does get close to zero. The big problem is the freezer with the set point being -18 Deg C but the current temperature being about +11 Deg C (yes positive). Has been like this for nearly 2 weeks. The Fisher Paykel technician should be here Thursday to look at it. I was hoping to have some actual temperatures in the app or Home Assistant, but I do understand customers would be concerned about normal fridge/freezer temperature behaviour.
I plan to have the sensors back in the compartments for 24hrs before the Technician is due to arrive.
I also had the problem that the freezer was well and truly defrosted before I knew there was a problem. An alarm on the fridge or app would have been helpful. I have had lots to do with alarming in my work. I understand that not getting an alarm right can cause a lot more problems. My thoughts on an alarm would be something like the compartment temperature has not been within 4 degrees C of set point for 6 hours. So for my freezer, with a set point of -18 Deg C, after it was warmer than -14 Deg C for 6 hours would have alarmed. Even if the notification was not active on the fridge or in the app, if it was available in Home Assistant would have helped.
The app has lot of potential. I do understand that more information for those who do not understand could be very problematic.
I look forward to see how this develops to help service technicians and end users.

That's exactly what happened to my fridge on 8 March (that one was not having the smart module in). I was lucky that technician came next day and quickly determined that the problem was in sealed internal part and the only possible fix was to replace the fridge under warranty. That was quick... But then it took 52 days of waiting and over 7 hours on the phone with various parts of F&P, shipping company and local delivery/service person to have the replacement in my home...
So good luck to you...
Disclaimer: I have had various F&P products since 1992 and this was the first one that had a problem, ever. I hope the experience above was one off - just unlucky combination of bad luck (apparently the replacement fridge was shipped to wrong address initially), Easter, Anzac Day and me being in Tasmania.


Docker AWS Provisioning Issue - Cannot Create Swarm Master AWS instance with External Key/Value Store - Needed for Overlay Networking

I am working through the Overlay portion of this example:  https://docs.docker.com/engine/userguide/networking/get-started-overlay/ and I am getting an error (see below).
I am using Docker for Mac, my Mac environment:
{"Client":{"Version":"1.12.2","ApiVersion":"1.24","GitCommit":"bb80604","GoVersion":"go1.7.1","Os":"darwin","Arch":"amd64","BuildTime":"Sun Oct 16 07:57:31 UTC 2016"},"Server":null}
NOTE:  I had a colleague try this same build on their AWS and they ran it with Docker installed on an Ubuntu instance they had running on a VM, they received the same error.
Terminal Output with Error:
docker-machine create \

-d amazonec2
--swarm --swarm-master
--swarm-discovery="consul://$(docker-machine ip my-keystore):8500"
--engine-opt="cluster-store=consul://$(docker-machine ip my-keystore):8500"
--engine-opt="cluster-advertise=eth1:2376"
--amazonec2-region us-east-1
--amazonec2-instance-type c3.2xlarge
--amazonec2-ami ami-2d39803a
--amazonec2-access-key **************** --amazonec2-secret-key ***************
swarm-master
Running pre-create checks...
Creating machine...
(swarm-master) Launching instance...
Waiting for machine to be running, this may take a few minutes...
Detecting operating system of created instance...
Waiting for SSH to be available...
Detecting the provisioner...
Provisioning with ubuntu(upstart)...
Installing Docker...
Copying certs to the local machine directory...
Copying certs to the remote machine...
Setting Docker configuration on the remote daemon...
Configuring swarm...
Error creating machine: Error running provisioning: Unable to pull image: Post https://52.201.x.x:2376/v1.15/images/create?fromImage=swarm%3Alatest: read tcp 192.168.x.x:59968->52.201.x.x:2376: read: connection reset by peer

Has anyone else run into this?  The pull request always fails and I am not sure where to go from here.  Any help is appreciated at this point.
Thanks,
Monica

This specific issue was caused by the naming of the ethernet interface on the AWS instance.
Changing from: eth1 to ens3 solved the problem (see modified command below).
docker-machine create
-d amazonec2
--swarm --swarm-master
--swarm-discovery="consul://$(docker-machine ip my-keystore):8500"
--engine-opt="cluster-store=consul://$(docker-machine ip my-keystore):8500"
--engine-opt="cluster-advertise=ens3:2376"
--amazonec2-region us-east-1
--amazonec2-instance-type c3.2xlarge
--amazonec2-ami ami-2d39803a
--amazonec2-access-key **************** --amazonec2-secret-key ***************
swarm-master

$ docker-machine create \

-d virtualbox
--swarm --swarm-master
--engine-env NO_PROXY=localhost,.us.oracle.com,.oraclecorp.com,.oracle.com
--swarm-discovery="consul://$(docker-machine ip mh-keystore):8500"
--engine-opt="cluster-store=consul://$(docker-machine ip mh-keystore):8500"
--engine-opt="cluster-advertise=eth1:2376"
mgr
Running pre-create checks...
(mgr) Unable to get the latest Boot2Docker ISO release version:  Get https://api.github.com/repos/boot2docker/boot2docker/releases/latest: dial tcp <IP_ADDRESS>:44
3: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because conne
cted host has failed to respond.
Creating machine...
(mgr) Unable to get the latest Boot2Docker ISO release version:  Get https://api.github.com/repos/boot2docker/boot2docker/releases/latest: dial tcp <IP_ADDRESS>:44
3: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because conne
cted host has failed to respond.
(mgr) Copying C:\Users\mrmahesh.docker\machine\cache\boot2docker.iso to C:\Users\mrmahesh.docker\machine\machines\mgr\boot2docker.iso...
(mgr) Creating VirtualBox VM...
(mgr) Creating SSH key...
(mgr) Starting the VM...
(mgr) Check network to re-create if needed...
(mgr) Waiting for an IP...
Waiting for machine to be running, this may take a few minutes...
Detecting operating system of created instance...
Waiting for SSH to be available...
Detecting the provisioner...
Provisioning with boot2docker...
Copying certs to the local machine directory...
Copying certs to the remote machine...
Setting Docker configuration on the remote daemon...
Configuring swarm...
Error creating machine: Error running provisioning: Unable to pull image: Get https://registry-1.docker.io/v2/: net/http: request canceled while waiting for connecti
on (Client.Timeout exceeded while awaiting headers)


Java 17

According to reliable sources, it may require in the call to Java
--add-opens java.base/sun.net.www.protocol.jar=ALL-UNNAMED
See https://docs.oracle.com/en/java/javase/16/migrate/migrating-jdk-8-later-jdk-releases.html
This command may help diagnose problems related to missing classes:
mvn dependency:build-classpath

To add new Scala code but still work in Maven, one might need things like this:
              <plugin>
                <groupId>net.alchim31.maven</groupId>
                <artifactId>scala-maven-plugin</artifactId>
                <version>4.6.1</version>
                <executions>
                    <execution>
                        <id>scala-compile-first</id>
                        <phase>process-resources</phase>
                        <goals>
                            <goal>add-source</goal>
                            <goal>compile</goal>
                        </goals>
                    </execution>
                    <execution>
                        <id>scala-test-compile</id>
                        <phase>process-test-resources</phase>
                        <goals>
                            <goal>testCompile</goal>
                        </goals>
                    </execution>
                </executions>
                <configuration>
                    <scalaVersion>2.12.15</scalaVersion>
                </configuration>
            </plugin>

        <dependency>
            <groupId>org.scala-lang</groupId>
            <artifactId>scala-library</artifactId>
            <version>2.12.15</version>
        </dependency>
        <dependency>
            <groupId>org.clulab</groupId>
            <artifactId>processors-main_2.12</artifactId>
            <version>8.4.8</version>
        </dependency>
    ...
    <repositories>
        <repository>
            <id>artifactory.cs.arizona</id>
            <url>http://artifactory.cs.arizona.edu:8081/artifactory/sbt-release</url>
        </repository>
    </repositories>

Nice @kwalcock !
Should we add a sample pom.xml file somewhere in the documentation?

This issue is here mostly to preserve Becky's findings after Slack forgets it and until we advance to more recent Java versions.  If I understand it correctly, the XML above is mostly helpful for people wanting to add a few Scala source files to their otherwise Java project built with Maven.  It isn't directly related to processors.  We usually work the other way around with a Scala project built with sbt to which a few Java files are added.  I don't think we need to document for others how this is done.  We don't document gradle, either, for example.

Fair enough.

We now have SSL at https://artifactory.clulab.org.


TFLite GPU Delegate will block the thread who is calling interpreter.run()

System information

OS Platform and Distribution: Linux Ubuntu 16.04
Mobile device: OnePlus 3, One Plus 5 and Pixel 2 XL
TensorFlow Lite version on Android: 0.0.0-gpu-experimental
Have I written custom code: a GitHub repo contains the codes to reproduce the issue.
https://github.com/dailystudio/ml/tree/master/deeplab
DeepLab v3 TFLite model: DeepLab segmentation (257x257)

Describe the current behavior
Using the following code snippet to create an Interpreter with GPU delegate
        Interpreter.Options options = new Interpreter.Options();

        GpuDelegate delegate = new GpuDelegate();
        options.addDelegate(delegate);

        Interpreter interpreter = new Interpreter(mModelBuffer, options);

Calling the run() of the Interpreter with following lines of codes:
        interpreter.run(mImageData, mOutputs);

If these two code snippets are called in two different threads, the thread which calls interpreter.run() will be blocked. interpreter.run() will never return.
If these two code snippets are called in the same thread, interpreter.run() will be executed properly and output correct results.
Describe the expected behavior
Developers needn't care about which threads are used for calling these APIs. Even these APIs are called in different threads, interpreter.run() should return correctly with blocking issue.
Code to reproduce the issue
The full code can be found here:
https://github.com/dailystudio/ml/blob/master/deeplab/app/src/main/java/com/dailystudio/deeplab/ml/DeepLabLite.java
Currently, the code in repository works fine because the new Interpreter() and interpreter.run() are called in the same thread.
The DeepLabLite class has two important functions: initialize() and segment(). In intialize(), we read TFLite model from asset/ directory into a MappedByteBuffer:

    @Override
    public boolean initialize(Context context) {
        if (context == null) {
            return false;
        }

        mModelBuffer = loadModelFile(context, MODEL_PATH);
        if (mModelBuffer == null) {
            return false;
        }

        ...
    }

In segment(), we use that MappedByteBuffer to create an Interpreter and call run() for inference:
        ...
        Interpreter.Options options = new Interpreter.Options();

        if (USE_GPU) {
            GpuDelegate delegate = new GpuDelegate();
            options.addDelegate(delegate);
        }

        Interpreter interpreter = new Interpreter(mModelBuffer, options);
        ...
        final long start = System.currentTimeMillis();
        interpreter.run(mImageData, mOutputs);
        final long end = System.currentTimeMillis();
        ...

The DeepLabLite.initialize() is called in an AsyncTask after application is launched, while the DeepLabLite.segment() is called a Loader after users pick an image for segmentation. These codes will be no problem.
But if we keep the codes of calling these two methods unchanged and move the following line from segment() to initialize():
        Interpreter interpreter = new Interpreter(mModelBuffer, options);

P.S.: Of course, we need to declare a class member to hold this Interpreter for future using in segment().

Then the calling of interpreter.run() will be blocked forever.
Other information
With my tests, I suspect this problem is independent of devices. It would happen on all Android devices. It should be related to GpuDelegate. If you do not call options.addDelegate() to add a GpuDelegate, the interpreter.run() will also run well.

@dailystudio

Developers needn't care about which threads are used for calling these APIs.

Unfortunately, that is not the case when using OpenGL.  OpenGL is a state machine, and a proper GL context needs to be kept around.  This GL context is bound to the thread that it was created on.
https://www.khronos.org/opengl/wiki/OpenGL_and_multithreading
There are mechanisms to transfer GL contexts or use parent / children GL contexts for multithreaded architectures, but at the current level of developer preview, the exposed APIs probably don't give you enough control to do this.  Open sourcing GPU is just around the corner, at which point you will have more fine-grained control of the GPU processing with respect to your multithreaded programming model.

@dailystudio

Developers needn't care about which threads are used for calling these APIs.

Unfortunately, that is not the case when using OpenGL. OpenGL is a state machine, and a proper GL context needs to be kept around. This GL context is bound to the thread that it was created on.
https://www.khronos.org/opengl/wiki/OpenGL_and_multithreading
There are mechanisms to transfer GL contexts or use parent / children GL contexts for multithreaded architectures, but at the current level of developer preview, the exposed APIs probably don't give you enough control to do this. Open sourcing GPU is just around the corner, at which point you will have more fine-grained control of the GPU processing with respect to your multithreaded programming model.

Hmm..., but I am not using OpenGL in my demo application. I understand your points, but from my point of view, as a developer who is using TFLite, I only care about how to use your APIs to achieve my own goals. In my codes, there are no OpenGL related codes. So, let me learn and understand the concept of OpenGL contexts or multi-threads architectures seems like an imposition.
I think my case is quite typical. First, you load a model from assets and create an Interpreter for using in the future. Then when you really need it, you call run() to inference the results. Calling these two steps in the main thread is absolutely not acceptable, especially in a real product. That means in most cases, these two steps will be called in different threads and probably be called in two different threads You couldn't suppose every developer who uses TFLite will have acknowledgment about everything. If I am using OpenGL to write the application, yes, I may be aware that there would be some multi-thread issues. But I am just a developer who is writing a standard application which is using TF to segment image. To be honest, I had used an entire afternoon to find out the root cause of this issue. Just because I am quite interested and have enthusiastic in Tensorflow.
I am not challenging the work what you have already done. I just want it to be better and can be accepted by more people. My suggestions are:

You can handle the OpenGL context issues in the implementation of TFLite libraries. Of course, I am not an expert in this direction and that may be perfect but impossible. ;)
You can throw a runtime exception to warn the developer that they are using the API incorrectly and they should keep creation and inference in the same thread.
But no matter which solution you finally decide to use, just simply adding some tips on the related section of the Tensorflow official website to tell the developers about this.

Thanks for your patient to pay attention to my issue and I hope my advice could help the TFLite become better in the future.

@dailystudio
That's actually pretty good feedback and one of the reason's why we put out a developer "preview" to gather feedback like yours.  We really appreciate it.

You can handle the OpenGL context issues in the implementation of TFLite libraries.

The fact that you have to be mindful of the GL context is inevitable, especially when you work in multithreaded settings.  We actually tried our best to hide that away from the users (and that's why you don't see the GL context in the API), but maybe hiding that was a bad thing.  If the API requires you to provide the GL context (or maybe the thread that owns the GL context), maybe that might have been better.

You can throw a runtime exception to warn the developer that they are using the API incorrectly and they should keep creation and inference in the same thread.

That's a great idea.  We'll see how we can add that check without losing performance.  Doing a GL context check before every runInference is probably not the right way to go ;)

But no matter which solution you finally decide to use, just simply adding some tips on the related section of the Tensorflow official website to tell the developers about this.

Will do.
For now, I guess the easiest trick you can employ (if you want to go down the path of multithreading) is to have a dedicated thread that does initialization and inference all there, and you send a signal to the thread to run the inference.

@impdji
Great! I am looking forward to these updates. ; )

+1 to adding some sort of warning or runtime exception that @dailystudio already mentioned. I ran into this today so luckily there was already an issue open for it :) When I hit the issue while trying out gpu delegate, my app just froze and after debugging, saw that it was on tflite.run.

+1 as well, took me quite a while to find out why .run is not returning. anyway guys, keep up the good job, it is really appreciated and it is really cool to watch how TF is evolving. 2 yrs ago something ondevice GPU support was hard to imagine for me.

One note to this that might or might NOT relate to the single thread issue.
On most older devices (however with Open GL 3.2 capable), when inference is run on GPU, preview frame rate tend to drop after few seconds. So even though it runs inference faster than on CPU, it probably blocks Texture View somehow. Is there some general recommendation on which GPUs is makes sense to use GPUDelegate?

@bazinac
We have seen frame rate dropping when the device overheats.  Otherwise, we have not experienced performance degradations from other factors.  I mostly work on C++ layer, so I don't know whether Java can cause any issues, but given that the MobileNet demo app just runs fine in Java, I'm wondering whether it's device overheating.
re: recommendation.  The ideal use case is the following:

You get the camera input in the form of a surface texture.
Create an OpenGL shader storage buffer object (SSBO).
Use GPUDelegate.bindGlBufferToTensor() to associate that SSBO with the input tensor.
Write a small shader program to dump surface texture of [1] into that SSBO of [2] efficiently.
Run inference.

There is also similar optimization you can do for output.
Once project is fully open sourced, you should even have access to the command buffer queue, and can directly render the output of the network, if your network's output is something that can be directly rendered on screen.  We didn't expose that through API, because it would be too complicated without showing the code what's going on.

Thanks for prompt answer. However I am not refering to situation, that could be caused by overheating. Even when running here provided demo on some older devices (like Samsung Galaxy J5 2017, Galaxy Tab S2), frame rate drops right after few seconds (like 3-5) when you switch to GPU. When using CPU, this does not happen.
Also thanks for recommendation for some input feeding optimalization, will try.

@bazinac
If it's not an overheat issue, we have seen slowdowns sometimes:

On some Huawei devices, GPU frequency drops to half after boot up, comes back after a while, and repeats this fluctuation.
On some devices, we had a driver bug that would slowly leak memory, and eventually blows up after several hundreds of thousand calls.

However, none of these is really applicable to your situation =/
Is your phone's OpenGL driver up to date?

Anyone who has subscribed to this:
I have just submitted the change with checking whether init's EGLContext is the same as invoke's EGLContext.  The phrase "submitted" applies to the internal code for now.  I'm not sure when this will go live to the public; we are trying to decide whether we should do another dev preview release, or whether we should just go with open source, as we're pretty close ;)

Not officially announced yet, but FYI: GPU code is now visible at:
https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/delegates/gpu
if you need the code for better insight what is happening.

Now that the code of checking gl context is live, I'm gonna close this issue.  Please reopen if things don't work as expected.

@impjdi Could you provide some example on how to work with SSBO in TFLite classification android app??

@SanthoshRajendiran
https://github.com/tensorflow/tensorflow/issues/26297
has some shader code and its invocation around it.  The shader code there is mapping GlTexture to SSBO.

But it only works sometimes... :/

@jsolves
From past reports, we know that it hangs when you don't have the right OpenGL context than when it was initialized.  Make sure that your interpreter initialization & interpreter invoke (well, run in Java) is happening on the same thread.  We had a way of throwing an exception, but that collided with something else, so that we had to revert that change :(

Yes, I know. I was refering to #26297.


Fixed parser

This PR fixes the parser issue. It allows us now to import the parser plugging to use on other projects by adding a $log and a return statement. I also removed the ts configuration from the project, maybe we can add it on a new feature as soon we stabilize our plugin.

😍

😍

@Fausto95 Have you tested it?

@Fausto95 Have you tested it?

Yes


Hide group action checkboxes and selectbox if group action is not allowed on any row

With this fix is just ->

What if someone has perfect styles for particular datagrid table and suddenly a table header and one column disappears?
Arrgh, i like it, but it could mean broken datagrid table appearance. I will leave it here for a bit longer, until i decide which approach is better.

Can you please resolve the conflicts? I will merge it then.

@dakorpar Thanks


Bug in process_datetime_data() converting datetime to int

Hi @avsolatorio,
I hope you are well.
The fix #72 is correct and allows to use latest pandas package. However, I am still blocked because of the line:
https://github.com/worldbank/REaLTabFormer/blob/main/src/realtabformer/data_utils.py#L265
There are cases where that could fail, e.g. I have tested in a Windows conda env and failed because bare int was translated to int32. Don't ask me why! My last conclusion was that it is related to Windows implementation of things as I have tested the same code and data and it succeeded in Google Colab and in an Ubuntu Linux container on the same Windows host (64bit machine) using WSL.
I think we can be more explicit and use int64 as datetimes are actually 64bit values, this will be in consistency with the following as well:
https://github.com/worldbank/REaLTabFormer/blob/main/src/realtabformer/data_utils.py#L271
So, I suggest to change it from
series = (series.astype(int) / 1e9)

to:
series = (series.astype('int64') / 1e9)

Can you help me on this? I will need a new PyPI version also (1.0.7).
Thank you!

Hello @efstathios-chatzikyriakidis , thanks for letting me know about the root cause likely being because of windows env. The patch is already published.
I highly recommend that you create a PR if you find some of these changes in the future! 😀

Hi @avsolatorio,
Yes, in the future in case I'll find some bug and it is easy to suggest a solution like this one, I will provide a PR.
Thank you so much!


Verify python 3.7 still works and add it to github actions CI

It has been a while since I tested Python 3.7, so it is possible/likely something could be broken.
We should:

Verify it still works, and fix any issues in tests/torchbench
Add a github actions workflow similar to the python 3.8 one to prevent it from getting broken in the future.

I set up Python 3.7 env, run all unit tests and found the following 11 failed:
==================================================================================================== short test summary info ====================================================================================================
FAILED tests/test_dynamic_shapes.py::DynamicShapesMiscTests::test_no_grad_dynamic_shapes - torchdynamo.exc.InternalTorchDynamoError
FAILED tests/test_dynamic_shapes.py::DynamicShapesReproTests::test_get_parameter_dtype_dynamic_shapes - torchdynamo.exc.Unsupported: missing: SETUP_EXCEPT
FAILED tests/test_dynamic_shapes.py::DynamicShapesReproTests::test_maml_dynamic_shapes - AssertionError: 6 != 5
FAILED tests/test_dynamic_shapes.py::DynamicShapesReproTests::test_reformer_sorting_dynamic_shapes - torchdynamo.exc.InternalTorchDynamoError
FAILED tests/test_misc.py::MiscTests::test_no_grad - torchdynamo.exc.InternalTorchDynamoError
FAILED tests/test_repros.py::ReproTests::test_get_parameter_dtype - torchdynamo.exc.Unsupported: missing: SETUP_EXCEPT
FAILED tests/test_repros.py::ReproTests::test_maml - AssertionError: 5 != 4
FAILED tests/test_repros.py::ReproTests::test_reformer_sorting - torchdynamo.exc.InternalTorchDynamoError
FAILED tests/test_subgraphs.py::SubGraphTests::test_resume_with_no_grad1 - AssertionError: 1 != 2
FAILED tests/test_subgraphs.py::SubGraphTests::test_resume_with_no_grad2 - AssertionError: 1 != 3
FAILED tests/test_subgraphs.py::SubGraphTests::test_resume_with_no_grad3 - AssertionError: 1 != 2
========================================================================================== 11 failed, 429 passed, 9 skipped in 15.56s ===========================================================================================

These failures are caused by this CPython bytecode change between 3.7 & 3.8. We have to:

Replace BEGIN_FINALLY with any Python 3.7 instruction.
As WITH_CLEANUP_START and END_FINALLY changed behavior between 3.7 & 3.8, we need to extend the implementation to support both version.

Thanks for looking into that @yanboliang.
The BEGIN_FINALLY/WITH_CLEANUP_START/END_FINALLY stuff is just trying to support:
with torch.no_grad():
   ...

Feel free to reach out to me if you get stuck.

Fixed by #143


Mis-Match Results

Hello Brother,
Thanks for the amazing work.
I have done inference of ISTD test data set images with the the checkpoint provided by you. I am unable to get the good results.
The model is performing extremally well when there is a "grey" color image at its input.

But when there is a combination of colors i.e. even 2 colors in an image, there is hardly any difference between the input image and the output shadow image produced. Few of the results are attached.

yeah,there is a problem haven't been solved, I think the ISTD dataset has a large variety of scenes, and there are not so many scene samples. I suggest that you can train on ISTD+ or improve the number of samples through data enhancement


Layer selector vizjson 3 migration

This closes #10551

@javitonino CR this, please.
cc @nobuti

@javitonino next CR, please.

@javitonino next CR, please. reload failed to me during last tests because it's not a sequel method, but I didn't noticed it before. I didn't noticed because I tried migrating only the layer selector, and it doesn't pop up an error message, so request fails silently (but the migration worked). I saw the error when I added unsupported overlays.
I think that it's not actually needed. In addition, I've added error traces to that controller for errors to be properly logged, so I'll ping @CartoDB/builder-backend for everybody to be aware about likely new error traces (not new errors).

@javitonino last? CR, please

@javitonino CR, please U_U. I haven't changed anything related to key/symbols, because I think that it works. You can test it in ded06, let me know if you manage to break it, please. cc @nobuti

Thanks!


basenc implementation

The --z85 encoding is one I had not heard of before. Fixes several issues with base64 interop implementations according to the RFC.
The base16, base32hex, binary (lsbf & msbf) and z85 implementations are not present in the V's standard library.  Should they? The ones in this program are straightforward and could be added. I'm willing to do the work.
Talk it over with the team if you're interested and let me know.

Couldn't hurt to open a PR to add them to stdlib.  They can be discussed there.
Personally, I've never needed any of them, but someone does or they wouldn't exist.  :-)


Add Kooperative Gesamtschule Leeste

Main homepage is https://www.kgs-leeste.de/
The school has a department for computer science.
Users got email in this scheme<EMAIL_ADDRESS>

@SimonIT Pull request merged. Thank you!


Kuun-Lan's engines flashing when firing

As title, I've found that the engines of the Kuun-Lan's starts to flash when the ship it's firing with her point defense turrets. I don't know if this is a bug present in the vanilla game, but I report it anyway.

Pretty sure this happens in the vanilla game, yes. It also occurs with most capital ships in the game, if I recall correctly.

Yeah, that happens in HW1 Classic as well iirc. As much as I'd like this to be fixed in CET for Cataclysm, its inconsequential and has no bearing on anything, other than being funky I guess.

I wonder if it is the same issue that I fixed in the HW1 engine. The mothership engine kept flickering in the intro sequence. Turns out there is a check to see if velocity == 0 to turn off the engine effect and as soon as velocity != 0 the effect is started again. Due to physics engine floating point precision the velocity is switching back and forth between very small values and 0. Introducing a small epsilon helped fix that.
https://gitlab.com/gardens-of-kadesh/gardens-of-kadesh/-/commit/b0a4daa19368afc0e128e4aab57f07f69d8b0c90
I just noticed my fix is only for NIS playback - but I assume there is similar code somewhere else.
A quick search through the code later: there is a macro for setting an object to moving which then causes the engine effect to be drawn:
SET_MOVING_LINEARLY(obj->posinfo.isMoving);
It is used in multiple places in Physics.c and the test for if something is moving looks like this:
if (Real32ToUdword(obj->posinfo.velocity.x) | Real32ToUdword(obj->posinfo.velocity.y) | Real32ToUdword(obj->posinfo.velocity.z))
So as above velocity needs to be perfectly 0 for it to turn off the engine effect.


[Question] Threading ?

I'm noticing an increasing "lag" when a new state is dispatched, because I have a more and more complex state implementing Equatable and using skipRepeat. Is there a way to make my state comparison on a background queue? Would making a custom Subscription a good place?

Okay silly me. I can just subscribe in a custom queue and my subscription will be called in that queue. Perfect!

This is an interesting bottleneck nevertheless. skipRepeats wouldn't work well out of the box for larger collections. That's why I compare in my subscriber using not == but a "similar enough" operator instead whenever applicable, kind of like a shallow comparison. But you'll have to write that for every type that can have large objects.
For another part of the same app, I added versioning (an incrementing number) to user-initiated requests to discard outdated responses. I wonder if versioning would be a useful for ReSwift states in general through an extension maybe.

I'll add some more do the discussion after my research:
So about Equatable, I implement it everywhere it make sense, and for collections I have some very specific implementation case by case, sometimes it make sense to do a deep comparaison on every objects, sometime I need it to be fast, so just comparing ids/string is ok. The downside is that my states and objects are very bloated and it's hard to see what equatable are really doing. So I use the == of Equatable because it's convenient, but it's some case it's more about being similar enough.
I also have custom operator for merging operations in my reducer, but that's another story.
About threading, I'm doing everything on a serial high priority background queue now, in short, it looks like that:
let FLUX_QUEUE = DispatchQueue(label: "xxx.appName.xxx.store.queue")

extension DispatchQueue {
    public class func flux() -> DispatchQueue {
        return FLUX_QUEUE
    }
}

and
extension Store {

    open func dispatchQueue(_ action: Action) {
        DispatchQueue.flux().async {
            self.dispatchFunction(action)
        }
    }
}

So I use dispatchQueue whenever I would have called a store.dispatch.
The only downside is that I don't see how to shortcut StoreSubscriber to be called on the main thread, I guess I have so shortcut subscriber first and register my custom class etc... So for now I need to remember to use the main queue in all my new state function call.
Do you think of a better way of doing that @DivineDominion ?

@DivineDominion about versionning, you have exactly the same problem as me for my live search!
I store a state that is define by a [queryString: [results]], so whenever you would go back to a query that is already in the state, the newState would never be called with the skipRepeat implementation. I ended up storing a timestamp and a lastSearch in my searchState which is updated every time a search is fired, so I'm sure the subscriber is called so I can refresh my UI.
I could manually pull my results from my store from my viewController but it kinda breaks my workflow.

@Dimillian since people type fast in my app and results are supposed to provide instant filtering of in-memory results, I use an incrementing integer instead of a timestamp just for good measure, but yeah, this is like server--client architecture in a single app. In other words, cumbersome :)
Have a look at ReSwift.Subscription. Maybe it makes more sense to provide an extension first where you have a custom transformation (like select or skipRepeats) that wraps passing the result back in your DispatchQueue. If that works, and if writing this transformation everywhere gets on your nerves, you could pull the combination into a subscribeOnXYZQueue method.
This sounds much like RxSwift subscribeOn, by the way. And I guess this could be a useful extension to the core library. At least it can make sense to discuss this some more!

@DivineDominion Well I debounce it, so it's not triggered at every keystroke; but still fairly often. Server query -> reduce results ||&& local search -> dispatch state change -> etc...  I get pretty good performance and a nice flow now.
I may get something wrong, but subscribing on a specific queue and dispatching action on a specific queue is something different right? I can subscribe on any queue/main thread, but as long as I trigger my dispatch actions in a specific queue, the subscriber will be called on that queue right and not on the subscriber one, right?

@Dimillian  Oh you're right, ReSwift doesn't schedule anything special. It looks like the subscriber will be called from the same queue where the action is dispatched. Doing this from anywhere is not encouraged, so I assume you dispatch from the main queue.
Let's hope I get the rest of this right :)
Now you want to process the equality check (Subscription transformation) on a different queue. That would be possible if you wrote a transformation like the built-in select:
public func select<Substate>(
    _ selector: @escaping (State) -> Substate
    ) -> Subscription<Substate>
{
    return Subscription<Substate> { sink in
        self.observe { oldState, newState in
            sink(oldState.map(selector) ?? nil, newState.map(selector) ?? nil)
        }
    }
}

Sadly, writing transformations like this is not possible from the client app's side at the moment. I'll try to incorporate this in #263 but for now you'd have to write this in a ReSwift fork:
// Untested
extension Subscription {
    public func onQueue(_ queue: DispatchQueue) -> Subscription<State> {
        return Subscription<Substate> { sink in
            self.observe { oldState, newState in
                queue.async {
                    sink(oldState.map(selector) ?? nil, newState.map(selector) ?? nil)
                }
            }
        }
    }
}

Then call it like this:
store.subscribe(foo) { $0.onQueue(.flux()).skipRepeats(==) }

Yeah it might be better when I'll be able to too that hehe. But as of now, I'm doing everything on my flux queue, and get back to the main thread on the new state callback. It works, just a bit more code :)

I tried with dispatching actions on my custom serial queue, but there are race conditions when I dispatch 2 actions at almost the same time, both are trying to modifying the same state value. So I guess I am supposed to dispatch actions only from the main thread?

@MichaelGuoXY All actions must be dispatched in a serial manner. If you are experiencing a race condition then the actions are not being dispatched serially. Perhaps your queue is setup incorrectly. If you believe it to be setup correctly, please create a separate github issue including a small sample project for us to replicate the issue with.

@MichaelGuoXY All actions must be dispatched in a serial manner. If you are experiencing a race condition then the actions are not being dispatched serially. Perhaps your queue is setup incorrectly. If you believe it to be setup correctly, please create a separate github issue including a small sample project for us to replicate the issue with.

I've created a new issue here https://github.com/ReSwift/ReSwift/issues/443 please take a look, thanks


FactoryBot/IdSequence error occurred while cop was inspecting file

Expected behavior
FactoryBot/IdSequence does not raise an error while inspecting spec/factories/sequences.rb
Actual behavior
Output for bundle exec rubocop spec/factories/sequences.rb --debug
For /private/var/www/my_project: configuration from /private/var/www/my_project/.rubocop.yml
configuration from<EMAIL_ADDRESS>configuration from<EMAIL_ADDRESS>Default configuration from<EMAIL_ADDRESS>configuration from<EMAIL_ADDRESS>configuration from<EMAIL_ADDRESS>configuration from<EMAIL_ADDRESS>configuration from<EMAIL_ADDRESS>configuration from<EMAIL_ADDRESS>configuration from<EMAIL_ADDRESS>configuration from<EMAIL_ADDRESS>configuration from<EMAIL_ADDRESS>Inheriting configuration from /private/var/www/my_project/.rubocop_todo.yml
Use parallel by default.
Skipping parallel inspection: only a single file needs inspection
Inspecting 1 file
Scanning /private/var/www/my_project/spec/factories/sequences.rb
An error occurred while FactoryBot/IdSequence cop was inspecting /private/var/www/my_project/spec/factories/sequences.rb:28:4.
undefined method `value' for s(:lvar, :name):RuboCop::AST::Node
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-factory_bot-2.24.0/lib/rubocop/cop/factory_bot/id_sequence.rb:27:in `on_send'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-1.56.3/lib/rubocop/cop/commissioner.rb:143:in `public_send'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-1.56.3/lib/rubocop/cop/commissioner.rb:143:in `block (2 levels) in trigger_restricted_cops'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-1.56.3/lib/rubocop/cop/commissioner.rb:171:in `with_cop_error_handling'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-1.56.3/lib/rubocop/cop/commissioner.rb:142:in `block in trigger_restricted_cops'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-1.56.3/lib/rubocop/cop/commissioner.rb:141:in `each'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-1.56.3/lib/rubocop/cop/commissioner.rb:141:in `trigger_restricted_cops'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-1.56.3/lib/rubocop/cop/commissioner.rb:70:in `on_send'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-ast-1.29.0/lib/rubocop/ast/traversal.rb:154:in `on_block'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-1.56.3/lib/rubocop/cop/commissioner.rb:71:in `on_block'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-ast-1.29.0/lib/rubocop/ast/traversal.rb:161:in `on_defs'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-1.56.3/lib/rubocop/cop/commissioner.rb:71:in `on_defs'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-ast-1.29.0/lib/rubocop/ast/traversal.rb:137:in `block in on_dstr'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-ast-1.29.0/lib/rubocop/ast/traversal.rb:137:in `each'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-ast-1.29.0/lib/rubocop/ast/traversal.rb:137:in `on_dstr'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-1.56.3/lib/rubocop/cop/commissioner.rb:71:in `on_begin'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-ast-1.29.0/lib/rubocop/ast/traversal.rb:158:in `on_block'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-1.56.3/lib/rubocop/cop/commissioner.rb:71:in `on_block'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-ast-1.29.0/lib/rubocop/ast/traversal.rb:20:in `walk'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-1.56.3/lib/rubocop/cop/commissioner.rb:87:in `investigate'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-1.56.3/lib/rubocop/cop/team.rb:156:in `investigate_partial'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-1.56.3/lib/rubocop/cop/team.rb:98:in `investigate'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-1.56.3/lib/rubocop/runner.rb:345:in `block in inspect_file'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-1.56.3/lib/rubocop/runner.rb:344:in `each'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-1.56.3/lib/rubocop/runner.rb:344:in `flat_map'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-1.56.3/lib/rubocop/runner.rb:344:in `inspect_file'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-1.56.3/lib/rubocop/runner.rb:287:in `block in do_inspection_loop'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-1.56.3/lib/rubocop/runner.rb:321:in `block in iterate_until_no_changes'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-1.56.3/lib/rubocop/runner.rb:314:in `loop'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-1.56.3/lib/rubocop/runner.rb:314:in `iterate_until_no_changes'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-1.56.3/lib/rubocop/runner.rb:283:in `do_inspection_loop'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-1.56.3/lib/rubocop/runner.rb:164:in `block in file_offenses'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-1.56.3/lib/rubocop/runner.rb:189:in `file_offense_cache'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-1.56.3/lib/rubocop/runner.rb:163:in `file_offenses'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-1.56.3/lib/rubocop/runner.rb:154:in `process_file'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-1.56.3/lib/rubocop/runner.rb:135:in `block in each_inspected_file'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-1.56.3/lib/rubocop/runner.rb:134:in `each'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-1.56.3/lib/rubocop/runner.rb:134:in `reduce'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-1.56.3/lib/rubocop/runner.rb:134:in `each_inspected_file'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-1.56.3/lib/rubocop/runner.rb:120:in `inspect_files'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-1.56.3/lib/rubocop/runner.rb:73:in `run'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-1.56.3/lib/rubocop/cli/command/execute_runner.rb:26:in `block in execute_runner'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-1.56.3/lib/rubocop/cli/command/execute_runner.rb:52:in `with_redirect'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-1.56.3/lib/rubocop/cli/command/execute_runner.rb:25:in `execute_runner'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-1.56.3/lib/rubocop/cli/command/execute_runner.rb:17:in `run'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-1.56.3/lib/rubocop/cli/command.rb:11:in `run'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-1.56.3/lib/rubocop/cli/environment.rb:18:in `run'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-1.56.3/lib/rubocop/cli.rb:118:in `run_command'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-1.56.3/lib/rubocop/cli.rb:125:in `execute_runners'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-1.56.3/lib/rubocop/cli.rb:51:in `block in run'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-1.56.3/lib/rubocop/cli.rb:77:in `profile_if_needed'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-1.56.3/lib/rubocop/cli.rb:43:in `run'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-1.56.3/exe/rubocop:19:in `block in <top (required)>'
/Users/taylorrf/.rvm/rubies/ruby-3.0.5/lib/ruby/3.0.0/benchmark.rb:308:in `realtime'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/gems/rubocop-1.56.3/exe/rubocop:19:in `<top (required)>'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/bin/rubocop:25:in `load'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/bin/rubocop:25:in `<main>'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/bin/ruby_executable_hooks:22:in `eval'
/Users/taylorrf/.rvm/gems/ruby-3.0.5@my_project/bin/ruby_executable_hooks:22:in `<main>'

Steps to reproduce the problem
bundle exec rubocop spec/factories/sequences.rb
1 error occurred:
An error occurred while FactoryBot/IdSequence cop was inspecting /home/circleci/my_project/spec/factories/sequences.rb:28:4.
Errors are usually caused by RuboCop bugs.

Mention the following information in the issue report:
1.56.3 (using Parser <IP_ADDRESS>, rubocop-ast 1.29.0, running on ruby 3.0.5) [x86_64-linux]

RuboCop version
$ [bundle exec] rubocop -V
1.56.3 (using Parser <IP_ADDRESS>, rubocop-ast 1.29.0, running on ruby 3.0.5) [x86_64-darwin22]
  - rubocop-performance 1.19.0
  - rubocop-rails 2.21.2
  - rubocop-rspec 2.24.1

Can you please share the code surrounding spec/factories/sequences.rb:28?

@pirj Sure, here's the relevant parts of the code surrounding spec/factories/sequences.rb:28:
27:  def self.formatted_sequence(name, digits: 6, separator: '-')
28:    sequence(name) { |n|
29:      formatted_identifier = _format_n(n, digits: digits, separator: separator)
30:      yield formatted_identifier
31:    }
32:  end
33:
34:   def self._format_n(n, digits: 6, separator: '-')
36:    format_string = "%0#{digits}d" # see Kernel#sprintf
37:    (format_string % n) \
38:      .reverse.scan(/\d{3}/).join(separator).reverse
39:  end

It’s a matter of adding a check here that node.first_argument.sym_type? and adding a couple of tests.
I’m uncertain how widespread dstr/str type arguments are and if it’s worth supporting that.
Will you be up to submit a PR, @taylorrf ?

Just to add, I am also experiencing this issue in our organization's project. The lines that crashes RuboCop (twice in fact) look like this:
model.collection.select do |li|
  (li.sequence == 0 || li.sequence.blank?) && (li.retry_number == 0 || li.retry_number.blank?)
end

This is written in a trait block. Curiously, there is no call to sequence in the factory for the li's type.
This occurs when running bundle exec rubocop --auto-gen-config from the terminal.
Here is the tail of the output from that command:
2 errors occurred:
An error occurred while FactoryBot/IdSequence cop was inspecting /.../spec/factories/[the_factory_file].rb:99:11.
An error occurred while FactoryBot/IdSequence cop was inspecting /.../spec/factories/[the_factory_file]:99:31.
Errors are usually caused by RuboCop bugs.
Please, report your problems to RuboCop's issue tracker.
https://github.com/rubocop/rubocop/issues

Mention the following information in the issue report:
1.58.0 (using Parser <IP_ADDRESS>, rubocop-ast 1.30.0, running on ruby 3.0.6) [arm64-darwin22]

Also, here's the version information:
% bundle exec rubocop -V
1.58.0 (using Parser <IP_ADDRESS>, rubocop-ast 1.30.0, running on ruby 3.0.6) [arm64-darwin22]
  - rubocop-performance 1.19.1
  - rubocop-rails 2.22.2
  - rubocop-rake 0.6.0
  - rubocop-rspec 2.25.0

For the initial case it would be sufficient to add a check that the first_argument.sym_type? here https://github.com/rubocop/rubocop-factory_bot/blob/5a92242927b9d90386ccb97cc1f9888e2aa30f9d/lib/rubocop/cop/factory_bot/id_sequence.rb#L27
For the second - that there’s no explicit receiver.
It makes sense to add a node pattern matcher, as it would incorporate both of those checks.
Would anyone be interested in sending a PR provided any necessary support?


Create sfdx file namespace commands

Is your feature request related to a problem? Please describe.
Nope. I just thought that since we have sfdx data we could also have sfdx file to manipulate any type of file storage salesforce has, such as contentDocuments, attachments and libraries
What are you trying to do
Upload, download, delete files to/from Salesforce
Describe the solution you'd like
There could exist a new namespace command called file/doc for uploading/downloading/deleting all types of files Salesforce has, like contentDocument and attachments
sfdx file:upload|download|delete
This command must work with streams if the API supports it... we don't want to load an entire file in memory just to start uploading it...
This command should work with multiple files
Describe alternatives you've considered
Use 3rd party solutions or create my own integration
Additional context
N/A

https://github.com/forcedotcom/cli/issues/1993

Thanks @AllanOricil, it's in our backlog 👍


Support meta plugins

Some plugins (e.g. csharp-plugin, php-plugin) are meta plugin with dependencies to maven submodules.
Hence, there is no http://repo1.maven.apache.org/maven2/org/codehaus/sonar-plugins/php/parent/2.4/parent-2.4.jar - only a parent pom.
It would be nice if puppet-sonarqube would be able to install such meta plugin.
Thanks!

Hi,
You can install such plugins by overriding the groupid parameter.
Here's how to install the latest version of the PHP plugin for example:
  sonarqube::plugin { 'sonar-php-plugin' :
    groupid => 'org.codehaus.sonar-plugins.php',
    version => '2.4.1',
  }

Regards,


Fix to #430 in dev branch broke matrixproto.scale()

Hey there, just a quick note that updates recently pushed to the dev branch to fix issue #430  (specifically line 3784 here: https://github.com/adobe-webplatform/Snap.svg/commit/b14d35dd346d0d398e4513a5534001506ef0aa9a) have broken the matrixproto.scale function for us. I don't understand what the fix accomplishes, or where the variable sy is supposed to be defined now, but I had to roll back our version to fix our app.
Sorry I don't have time to research exactly why this broke snap.svg for us, but I wanted to mention it in case others are noticing this problem and searching for a fix.

Fixed


Add info on expected host etc. to X509_STORE_CTX_print_verify_cb()

This is a spin-off of #11470 due to a review comment by @mattcaswell: https://github.com/openssl/openssl/pull/11470#discussion_r416677920.
Strengthen X509_STORE_CTX_print_verify_cb() diagnostics regarding expected host etc.
Add X509_VERIFY_PARAM_get0_host(), X509_VERIFY_PARAM_get0_email(),
and X509_VERIFY_PARAM_get1_ip_asc() to support this,
as well as internal helper functions ipv4_to_asc() and ipv6_to_asc(),
which are used also for simplifying other IP address output functions.

[x] documentation is added or updated

I've rebased this PR again but one Travis run keeps failing this way:
No output has been received in the last 10m0s, this potentially indicates a stalled build or something wrong with the build itself

which is certainly unrelated.

This pull request is ready to merge

merged - thanks!

@DDvO   Had  the cmp && crmp feature   done  in master openssl ？


duplicate information

Hi,
I installed MethylDackel successfully and runned it without error.
However, I did't found any duplicate information in the output *.log file.
I wanted to ask a help for obtaining the duplicate information, like the number of duplicate reads.

Maybe you should ask the author of MethylDackel first. In the meantime, I will also test MethylDackel to find if it can output duplicate information. Out of curiosity, why the duplicate information is important?

Hi,
I installed MethylDackel successfully and runned it without error.
However, I did't found any duplicate information in the output *.log file.
I wanted to ask a help for obtaining the duplicate information, like the number of duplicate reads.

Maybe you should ask the author of MethylDackel first. In the meantime, I will also test MethylDackel to find if it can output duplicate information.
Out of curiosity, why the duplicate information is important?

Hi,
I installed MethylDackel successfully and runned it without error.
However, I did't found any duplicate information in the output *.log file.
I wanted to ask a help for obtaining the duplicate information, like the number of duplicate reads.

Maybe you could use samtools to obtain the duplicate information?

Considering PCR in construction of genomic DNA library, it was important to remove duplicated mappings to reduce PCR bias. I tried to use rm_dup and markdup of samtools, and the removal effect is not very good. By comparison，Picard was better for removing thoes duplicated reads.

Considering PCR in construction of genomic DNA library, it was important to remove duplicated mappings to reduce PCR bias. I tried to use rm_dup and markdup of samtools, and the removal effect is not very good. By comparison，Picard was better for removing thoes duplicated reads.

Thansks for your help.
MethylDackel is able to remove the duplicate information. It uses the pileup of samtools to extract methylation metrics. It just does not output the statistical information of duplicated reads.
Is this statistical information important for downstream analysis？If it important, I will output the duplicated information in my methylation information extraction algorithm.

Thank you for your reply.
For assessing quality and depth of sequencing data, this statistical information was important. And I suggested to output the duplicated information.


Fixes #1351 - State-dependent method bodies part 2

Context
Completes remaining tasks for #1351.

Different modifiers now generates warning 82 (added associated manual files)
Top-level state machines now enabled
Fixed bug with extra space (space is now trimmed in modifiers)

Tests performed

Tested more complicated state nesting
Tested multiple top-level state machines
Tested the new warning generation

@pwang347 Are you able to look into the conflicting file UmpleParserTest.java that was also, I think modified recently by another PR. You simply need to merge your changes into what is current. I am not sure why it can't happen automatically.

Great job!


More utilities

Others

[x] Is the given input an alphabet?
[x] Is the given input numeric or not?

Lists

[x] Convert numeric list to list of strings.
[x] Convert a list of strings to a numeric list.
[x] Convert nested to a single list.
[x] Convert a single list to a nested list.
[x] Join N lists
[x] Replace elements in a list

[x] At index
[x] At values

[x] Is one list contained within the other?
[x] Sort a list and get its indices back

[ ] Fix issue with sorted in get_object_vount


Create Moderno, Ensino, Educação, Cultura

Institution/School Name
Moderno, Ensino, Educação, Cultura
Instiution/School Website
https://www.google.com/url?sa=t&source=web&rct=j&url=https://portalmoderno.com.br/new/&ved=2ahUKEwib8o7NnsP9AhUwqpUCHcw1D0UQFnoECA0QAQ&usg=AOvVaw1_5-85wmqN7qvc013tk26u
Email Users
Please place an x between the square brackets if yes

[ x ] Students
[ ] Academic/Teaching Staff
[ ] Other/Support Staff
[ ] Alumni

Education Levels
Please place an x between the square brackets if yes

[ ] Post-graduate research
[ ] Graduate School
[ ] College (University) or Undergraduate School or equivalent
[ ] Community College or equivalent
[ ] High School or equivalent
[ x ] Elementary School or equivalent

@Notiongirl I am sorry, this domain seems to represent an elementary school, not a specific university or a college. Please apply for free educational license with an official document which proves that you are a student. You can do it using "OFFICIAL DOCUMENT" tab at https://www.jetbrains.com/shop/eform/students.
I will review this request later. Reviewing elementary schools takes much more time, because we generally do not recommend using professional software development tools in elementary schools. It is often not necessary for students, at least under 13 of age.
Thank you for your understanding!

@Notiongirl Please provide us with a proof, which shows that the school recognizes the domain you are submitting as an official email domain for the students. https://portalmoderno.com.br website does not mention the domain positivosolucoes.com.br.
You can provide us with any of the following:

A copy of instruction (a hard-copy or electronic one) given to you by the school that tells you to use an email address in this email domain
An URL of a page at the official school website that says that students use this email domain
A reference to a recognized journal where an article was published by someone from the school, with explicit affiliation to the school, and e-mail address from the email domain
A WHOIS data that link the email domain to the school somehow
A copy of official letter from the school representative saying that the school provides e-mail addresses in this domain to the students
Anything else that can prove this fact in similar way, i.e. can be verified independently.

@Notiongirl You might miss my previous comment. Please react.

https://positivoon.com.br
Em qua., 15 de mar. de 2023 12:48 PM, Philip Torchinsky <
@.***> escreveu:

@Notiongirl https://github.com/Notiongirl You might miss my previous
comment. Please react.
—
Reply to this email directly, view it on GitHub
https://github.com/JetBrains/swot/pull/16809#issuecomment-1470286483,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/A5G6J2MD4654P434UBXE2BTW4HQC7ANCNFSM6AAAAAAVQLC5HQ
.
You are receiving this because you were mentioned.Message ID:
@.***>

@Notiongirl Regrettably, the information you provided about the school is insufficient. The link you shared in your last comment directs me to a login page without any clear indication of what the website is about.
Without more reliable information about your school, I regret to inform you that I am unable to fulfill your request.
Please apply for free educational license with an official document which proves that you are a student. You can do it using "OFFICIAL DOCUMENT" tab at https://www.jetbrains.com/shop/eform/students.
I have to close the request without merging.
Thank you for your understanding!


Fix the pytest version to be less than 8.0.0

What does this PR do?
Fixes the pytest version.
We're getting errors such as this one:
https://github.com/huggingface/accelerate/actions/runs/7958684877/job/21725397566?pr=2450

Not sure whom to ping, as Zach is off this week, maybe @ydshieh with reference to https://github.com/huggingface/transformers/pull/28758.

In the linked PR, the version was pinned to <8.0.0, not sure why it's done partly including and partly excluding 8.0.0. I can change this to <=8.0.0 if we're fine with that.

@BenjaminBossan I don't think it really matters, let's leave it like is

In the https://github.com/huggingface/transformers/pull/28758, the version was pinned to <8.0.0, not sure why it's done partly including and partly excluding 8.0.0

In transformers, they had other issues with 8.0.0 for some tests. I will merge this since some checks are failing because without this specific PR.


Release Google.Cloud.Dialogflow.Cx.V3 version 2.11.0

Changes in this release:
New features

Added generative AI, safety/security and speech endpointing settings (commit f62319c)

Created release for Google.Cloud.Dialogflow.Cx.V3-2.11.0


Search Bug

Issue Type: Bug
On Mac,using cmd + F trigger search， cursor don't focus on search input, while previous version do.
VS Code version: Code 1.40.0 (86405ea23e3937316009fc27c9361deee66ffbf5, 2019-11-06T17:09:34.601Z)
OS version: Darwin x64 18.6.0

Any news regarding this bug? Its very annoying :-(

Could you please provide more information, like screenshots, gif to help trouble?

Could you please provide more information, like screenshots, gif to help trouble?

Same bug here, not breaking but very annoying, anything we can do for fix it till next update?

@JsanchezSeyte confirmed. This actually solved my issue too.

When I reset Find With Selection it defaulted to CMD+E which made it worse for me.

@rebornix any updates on this?

https://github.com/microsoft/vscode/issues/44093#issuecomment-546157573
..................................................................

@rebornix @xiaoshude open pull request pending: https://github.com/microsoft/vscode/pull/95314

@yordis are you still seeing this issue in current Insiders? I can't reproduce this issue, sorry.

@rebornix problem fixed in Insiders version.
Should I close #95314?

Thanks for closing it :)


Optimize logo image

Quality same but file smaller.  The app I'm using sometimes does mess up though just going to put that out there.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.1 out of 2 committers have signed the CLA.:white_check_mark: RDIL:x: ImgBotAppYou have signed the CLA already but the status is still pending?  I made that mistake a while ago

@kerbybit smh, he signed it, but the bot is the one that actually made the commit so the bot has to sign it

It’s an application I can’t sign into it

Hey uh @FalseHonesty is there a chance you could take a look at my 14 PRs:
https://github.com/HyperiumClient/Hyperium/pulls


🐛 Searching "getUserMedia" breaks site

When searching for getUserMedia (https://caniuse.com/?search=getUserMedia) the site is broken.
I can reproduce this issue in all browsers
data-manager.ts:178 Uncaught (in promise) TypeError: Cannot read property 'title' of null
    at data-manager.ts:178
    at Array.map (<anonymous>)
    at data-manager.ts:177
    at data-manager.ts:408
    at Array.forEach (<anonymous>)
    at N (data-manager.ts:402)
    at data-manager.ts:450

Thanks for reporting, should now be fixed!


New verticalOffset prop

When used with NavigatorIOS it should be possible to push the section list down a bit. This is just a first idea, maybe I can find a better solution for this?

This should already be possible using the sectionListStyle prop.


Extend builder with tupled map and filter

[x] Tests added for any new code
[x] Ran make fmt-fix (or had formatting run automatically on all files edited)
[ ] Documentation added for any new functionality
[ ] Entries added to ./unreleased/ for any new functionality

closes  #2470
As discussed, only map and filter are added, but not except and funDef

Codecov Report

Merging #2504 (f09e5b3) into main (d527474) will decrease coverage by 0.01%.
The diff coverage is 100.00%.

:exclamation: Current head f09e5b3 differs from pull request most recent head 3e5e4a2. Consider uploading reports for the commit 3e5e4a2 to get more accurate results

@@            Coverage Diff             @@
##             main    #2504      +/-   ##
==========================================
- Coverage   78.43%   78.42%   -0.01%
==========================================
  Files         441      441
  Lines       15467    15469       +2
  Branches     2445     2439       -6
==========================================
+ Hits        12131    12132       +1
- Misses       3336     3337       +1

Impacted Files
Coverage Δ

...palache/tla/typecomp/unsafe/UnsafeSetBuilder.scala
98.07% <100.00%> (+0.07%)
:arrow_up:

... and 4 files with indirect coverage changes
:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


Circular double linked list implementation

Circular double  linked list implementation

this program help to use double circular linked lists


New API proposal-Verified Caller

This API allows accurate operator-certified "Verified Caller" information, including SMS and video clips, to be displayed on the phone screen of the called user before they answer the call. This API enables enterprise customers to vividly present their multimedia business cards and reduces the incidence of malicious calls.
It is applicable to customers in industries such as government, finance, express delivery, healthcare, education, etc., who have outbound call requirements.
For instance, when a hospital arranges follow-ups and communication between customer service and rehabilitation patients, patients might doubt the hospital's identity and typically refuse the calls. By utilizing this API, the hospital's identity can be displayed, thereby improving connection rates and significantly enhancing customer service satisfaction.

Supporting material available here

Hello @DanXu-ChinaTelecom, @Jason-ZTE
As for 16-05 TSC Minutes the API is approved and I need you to provide the initial maintainers and codeowners from your side for the upcoiming Verified Caller GitHub Repository

thx @TEF-RicardoSerr ，we proposed initial codeowners： DanXu-ChinaTelecom and Jason-ZTE @Jason-ZTE ，
and  initial maintainers as follows：

Org
Name
GitHub Username

China Telecom
Dan Xu
DanXu-ChinaTelecom

China Telecom
Xianyang Zhang
zxyxwz

ZTE
Tu Jiashun
Jason-ZTE

China Unicom
Yang Fan
chinaunicomyangfan

China Mobile
Hanbai Wang
HanbaiWang

HuaWei
Shuting Qing
ShutingQing

Whale Cloud Technology Co.,Ltd
Yun Wang
WangYunyunA

Subproject have been created here. Closing this issue