WIP: Add vulkan backend

issue: https://github.com/leejet/stable-diffusion.cpp/issues/256
Looks like theyre doing some changes to vulkan shader generation in ggml repo, and its currently broken. Will keep and eye on it and update the pr accordingly.

Related issue: https://github.com/ggerganov/llama.cpp/issues/5356
(im new to this, so I might have made some mistakes. I would be grateful for any guidance or feedback)

Hey, nice to see someone working on this. I'd like to get this to work. There's probably some ops that need to be supported by Vulkan upstream, right? I can help with that.

@0cc4m Thanks for offering help.
Currently the hpp file generated by ggml_vk_generate_shaders.py does not have types like mul_mat_vec_id_q3_k_f32_len, div_f32_len etc
Also some types were renamed eg: dequant_q5_k_len is imported in ggml/src/ggml-vulkan.cpp but header file has dequant_q5_K_len
Im assuming these issues will be solved by your work in llama.cpp? please correct me if Im wrong

@0cc4m Thanks for offering help.
Currently the hpp file generated by ggml_vk_generate_shaders.py does not have types like mul_mat_vec_id_q3_k_f32_len, div_f32_len etc
Also some types were renamed eg: dequant_q5_k_len is imported in ggml/src/ggml-vulkan.cpp but header file has dequant_q5_K_len
Im assuming these issues will be solved by your work in llama.cpp? please correct me if Im wrong
Also let me know if I can help with anything

It is working in Llama.cpp. I'll take a look at the status in ggml, maybe that needs an update.

I manually wired up Vulkan and compiled SD.cpp with the latest ggml modified with llama.cpp's modifications to Vulkan. It runs and loads a model, but their Vulkan shaders do not implement CONCAT and it fails.
./sd -m ~/Desktop/Misc/stable_diffusion/a1111/models/Stable-diffusion/ponyDiffusionV6XL_v6.safetensors --prompt "score_9, score_8_up, score_7_up, score_6_up, score_5_up, rainbow dash" -W 1024 -H 1024 -v
Option:
    n_threads:         8
    mode:              txt2img
    model_path:        /home/david/Desktop/Misc/stable_diffusion/a1111/models/Stable-diffusion/ponyDiffusionV6XL_v6.safetensors
    wtype:             unspecified
    vae_path:
    taesd_path:
    esrgan_path:
    controlnet_path:
    embeddings_path:
    stacked_id_embeddings_path:
    input_id_images_path:
    style ratio:       20.00
    normzalize input image :  false
    output_path:       output.png
    init_img:
    control_image:
    clip on cpu:       false
    controlnet cpu:    false
    vae decoder on cpu:false
    strength(control): 0.90
    prompt:            score_9, score_8_up, score_7_up, score_6_up, score_5_up, rainbow dash
    negative_prompt:
    min_cfg:           1.00
    cfg_scale:         7.00
    clip_skip:         -1
    width:             1024
    height:            1024
    sample_method:     euler_a
    schedule:          default
    sample_steps:      20
    strength(img2img): 0.75
    rng:               cuda
    seed:              42
    batch_count:       1
    vae_tiling:        false
    upscale_repeats:   1
System Info:
    BLAS = 1
    SSE3 = 1
    AVX = 1
    AVX2 = 1
    AVX512 = 1
    AVX512_VBMI = 1
    AVX512_VNNI = 1
    FMA = 1
    NEON = 0
    ARM_FMA = 0
    F16C = 1
    FP16_VA = 0
    WASM_SIMD = 0
    VSX = 0
[DEBUG] stable-diffusion.cpp:158  - Using Vulkan backend
ggml_vulkan: Found 1 Vulkan devices:
Vulkan0: NVIDIA RTX A4000 Laptop GPU (NVIDIA) | uma: 0 | fp16: 1 | warp size: 32
[INFO ] stable-diffusion.cpp:178  - loading model from '/home/david/Desktop/Misc/stable_diffusion/a1111/models/Stable-diffusion/ponyDiffusionV6XL_v6.safetensors'
[INFO ] model.cpp:737  - load /home/david/Desktop/Misc/stable_diffusion/a1111/models/Stable-diffusion/ponyDiffusionV6XL_v6.safetensors using safetensors format
[DEBUG] model.cpp:803  - init from '/home/david/Desktop/Misc/stable_diffusion/a1111/models/Stable-diffusion/ponyDiffusionV6XL_v6.safetensors'
[INFO ] stable-diffusion.cpp:201  - Stable Diffusion XL
[INFO ] stable-diffusion.cpp:207  - Stable Diffusion weight type: f16
[DEBUG] stable-diffusion.cpp:208  - ggml tensor size = 400 bytes
[WARN ] stable-diffusion.cpp:213  - !!!It looks like you are using SDXL model. If you find that the generated images are completely black, try specifying SDXL VAE FP16 Fix with the --vae parameter. You can find it here: https://huggingface.co/madebyollin/sdxl-vae-fp16-fix/blob/main/sdxl_vae.safetensors
[DEBUG] ggml_extend.hpp:884  - clip params backend buffer size =  1564.36 MB(VRAM) (713 tensors)
[DEBUG] ggml_extend.hpp:884  - unet params backend buffer size =  4900.07 MB(VRAM) (1680 tensors)
[DEBUG] ggml_extend.hpp:884  - vae params backend buffer size =  94.47 MB(VRAM) (140 tensors)
[DEBUG] stable-diffusion.cpp:309  - loading vocab
[DEBUG] clip.hpp:164  - vocab size: 49408
[DEBUG] clip.hpp:175  -  trigger word img already in vocab
[DEBUG] stable-diffusion.cpp:329  - loading weights
[DEBUG] model.cpp:1380 - loading tensors from /home/david/Desktop/Misc/stable_diffusion/a1111/models/Stable-diffusion/ponyDiffusionV6XL_v6.safetensors
[INFO ] stable-diffusion.cpp:413  - total params memory size = 6558.89MB (VRAM 6558.89MB, RAM 0.00MB): clip 1564.36MB(VRAM), unet 4900.07MB(VRAM), vae 94.47MB(VRAM), controlnet 0.00MB(VRAM), pmid 0.00MB(VRAM)
[INFO ] stable-diffusion.cpp:432  - loading model from '/home/david/Desktop/Misc/stable_diffusion/a1111/models/Stable-diffusion/ponyDiffusionV6XL_v6.safetensors' completed, taking 4.34s
[INFO ] stable-diffusion.cpp:449  - running in eps-prediction mode
[DEBUG] stable-diffusion.cpp:482  - finished loaded file
[DEBUG] stable-diffusion.cpp:1452 - txt2img 1024x1024
[DEBUG] stable-diffusion.cpp:1207 - prompt after extract and remove lora: "score_9, score_8_up, score_7_up, score_6_up, score_5_up, rainbow dash"
[INFO ] stable-diffusion.cpp:565  - Attempting to apply 0 LoRAs
[INFO ] stable-diffusion.cpp:1212 - apply_loras completed, taking 0.00s
[DEBUG] clip.hpp:1312 - parse 'score_9, score_8_up, score_7_up, score_6_up, score_5_up, rainbow dash' to [['score_9, score_8_up, score_7_up, score_6_up, score_5_up, rainbow dash', 1], ]
[DEBUG] clip.hpp:1152 - token length: 77
[DEBUG] ggml_extend.hpp:838  - clip compute buffer size: 2.56 MB(VRAM)
ggml_vulkan: Error: Missing op: CONCAT
GGML_ASSERT: /home/david/Desktop/Dev/ggml/stable-diffusion.cpp/ggml/src/ggml-vulkan.cpp:5533: false
Aborted (core dumped)

After adding CONCAT to the relevant place (probably not the solution for that?), it makes it a little further but still fails here:
ggml_backend_vk_graph_compute: error: op not supported  (view) (UNARY)
GGML_ASSERT: /home/david/Desktop/Dev/ggml/stable-diffusion.cpp/ggml/src/ggml-vulkan.cpp:6227: ok

At this point it's beyond my knowledge/skill.

@Cloudwalk9 Thank you for trying it, I can add the missing ops. Can you upload your progress to a branch that I can access?

@0cc4m Done, but it's pretty crude. I updated the submodule to point to my fork of ggml with the imported Vulkan stuff, also had to fix some headers. https://github.com/Cloudwalk9/stable-diffusion.cpp

@0cc4m Done, but it's pretty crude. I updated the submodule to point to my fork of ggml with the imported Vulkan stuff, also had to fix some headers. https://github.com/Cloudwalk9/stable-diffusion.cpp

I got the following error with this fork
[INFO ] stable-diffusion.cpp:432  - loading model from 'v2.gguf' completed, taking 0.89s
ggml_vulkan: Error: Missing op: IM2COL
GGML_ASSERT: /home/theo/Downloads/stable-diffusion.cpp/ggml/src/ggml-vulkan.cpp:5533: false
Aborted (core dumped)

@0cc4m They synced the newer Vulkan shader code (split into individual files) from llama.cpp to upstream ggml, so you could probably target ggml directly.

@0cc4m They just synced the newer Vulkan shader code (split into individual files) from llama.cpp to upstream ggml, so you could probably target ggml directly, instead of my forked submodule.

Yeah, my WIP branch is here: https://github.com/0cc4m/ggml/tree/vulkan-stable-diffusion-ops
I implemented all the ops, but there's still some bug that makes the image not adhere to the prompt. I'll investigate that later.

@0cc4m They just synced the newer Vulkan shader code (split into individual files) from llama.cpp to upstream ggml, so you could probably target ggml directly, instead of my forked submodule.

Yeah, my WIP branch is here: https://github.com/0cc4m/ggml/tree/vulkan-stable-diffusion-ops
I implemented all the ops, but there's still some bug that makes the image not adhere to the prompt. I'll investigate that later.

Great work, thank you!
Some ops appear to still be missing when I try to use TAESD:
ggml_vulkan: Error: Missing op: ADD for f16 and f32 to f16
D:\a\stable-diffusion.cpp\stable-diffusion.cpp\ggml\src\ggml-vulkan.cpp:4149: fatal error

We're finally about to see Stable Diffusion where the only major dependency is your graphics driver...

@SkutteOleg Thank you, those should be easy to add. I fixed the first bug that caused issues, but I ran into another matmul bug that I have to find in the shader code. I hope I can find it soon.

LORA and TAESD should work now. I also fixed the matmul bug. It's generating images correctly in my tests, but not that fast yet.

LORA and TAESD should work now. I also fixed the matmul bug. It's generating images correctly in my tests, but not that fast yet.

It is amazing, actually. It's 2.5 times faster than CUDA12 on my end 😲 (perhaps due to lower memory usage, but i'm not sure)

On which hardware?

On which hardware?

NVIDIA GeForce GTX 1660 SUPER

It's 2.5 times faster than CUDA12 on my end 😲 (perhaps due to lower memory usage, but i'm not sure)

I've had the time to some further testing. I was comparing the speed to a previous build of sd.cpp. It turns out CUDA12 image generation speed also got faster after ggml update. Even still, Vulkan is 20% faster.
I was wrong about memory, however. It appears that Vulkan uses more memory as I can no longer fit both llama.cpp and stable-diffusion.cpp on the GPU at the same time.

LORA and TAESD should work now. I also fixed the matmul bug. It's generating images correctly in my tests, but not that fast yet.

Excellent work, for me works fine, tested with intel ARC a580

UPD: I was testing at 512x512 before. When trying 1024x1024 Vulkan is indeed 15% slower for me. Also, at 1024x1024 it produces broken outputs on my hardware.

This is a problem with a very large buffer that sd.cpp requests for VAE decoding (?). I cannot fix that on the Vulkan side, but I am throwing an exception now so that it crashes instead of just generating garbage output. Maybe @leejet can think of a solution? Vulkan has a restriction on how large VRAM buffers can be (usually 4GB), and 1024x1024 VAE decoding requests a buffer larger than that.

there should be VAE-tiling available, or fallback to cpu (not exposed as a cli option afaik).

This is a problem with a very large buffer that sd.cpp requests for VAE decoding (?). I cannot fix that on the Vulkan side, but I am throwing an exception now so that it crashes instead of just generating garbage output. Maybe @leejet can think of a solution? Vulkan has a restriction on how large VRAM buffers can be (usually 4GB), and 1024x1024 VAE decoding requests a buffer larger than that.

Shouldn't VAE tiling help with that? This occurs for me even with VAE tiling enabled.
Also, I'm curious, what is the reason for images looking slightly different on different backends?

CUDA12:

Vulkan:

Excellent work, well done.  Pictures are generated at 384x384 on my Intel i5-1035G1.

Tried the vulkan repo from Skuttle,
vulkan sdcpp -> 2.12 it/s
cuda sdcpp -> 3.95 it/s
comfyui -> 1.27 it/s
Nvidia gtx 1650 ti mobile
Fedora 40
nearly identical images, though why are some patches different b/w cuda and vulkan?

This is a problem with a very large buffer that sd.cpp requests for VAE decoding (?). I cannot fix that on the Vulkan side, but I am throwing an exception now so that it crashes instead of just generating garbage output. Maybe @leejet can think of a solution? Vulkan has a restriction on how large VRAM buffers can be (usually 4GB), and 1024x1024 VAE decoding requests a buffer larger than that.

Shouldn't VAE tiling help with that? This occurs for me even with VAE tiling enabled.

It should, and it does in my tests. I can generate 1024x1024 images with SDXL by using --vae-tiling or --vae-on-cpu.

why are some patches different b/w cuda and vulkan?

There are slight differences in how the CUDA and Vulkan backends calculate, for example the CUDA backend uses tensor cores for matrix multiplication, while the Vulkan backend (on Nvidia GPUs) uses the regular CUDA cores. That can change the results slightly. There might also be some minor differences in other operations that contribute to that, too.

I tried the img2img mode but it immediately raises an error
ggml_vulkan: Error: Missing op: PAD

I tried the img2img mode but it immediately raises an error ggml_vulkan: Error: Missing op: PAD

Thank you for reporting that, I forgot to check img2img. It should work now.

When trying to load the easynegative embedding I get this missing vulkan operator:
ggml_vulkan: Error: Missing op: CONCAT for f16 and f16 to f16

When trying to load any embedding I get this missing vulkan operator:
ggml_vulkan: Error: Missing op: CONCAT for f16 and f16 to f16

I can implement that, but it's odd considering that f16 CONCAT is not even implemented for CPU or CUDA. Do embeddings work with those?

At least with cpu it doesn't work, I'm unable to test cuda

At least with cpu it doesn't work, I'm unable to test cuda

You can try to build with this branch: https://github.com/0cc4m/ggml/tree/vulkan-stable-diffusion-ops-concat-f16

I've cherry-picked the changes from your branch and the embeddings now work fine, I'm not sure why on cpu the program crashed.

1.5 it/s on an SD 1.5 model at 512x512 Euler A 20 steps. The performance drops quadratically with resolution. I get 7 it/s on ComfyUI with same settings.
BUT, CUDA backend here is 3 it/s on the same settings and performance also tanks with resolution in the same manner. RTX A4000 Mobile, which is about equivalent to a desktop 3060 Ti.
There's still a LOT of room for optimization.
Also I found a bug I intend to submit upstream. I encountered one of my model weights interpreted as "F64"(?!) that works in standard web UIs but not ggml/sd.cpp, regardless of --type setting.

@0cc4m , sorry to bother you again but I've found another missing op, this time related to upscaling:
upscaler.cpp:49   - upscaling from (512 x 512) to (2048 x 2048)
Errors: ggml_vulkan: Found 1 Vulkan devices:
Vulkan0: AMD Radeon Graphics (RADV RENOIR) (radv) | uma: 1 | fp16: 1 | warp size: 64
ggml_vulkan: Error: Missing op: LEAKY_RELU

sorry to bother you again but I've found another missing op, this time related to upscaling:

No problem, I added LEAKY_RELU. Please try it.

Upscaling works fine now, however I've also just found out that quantized models don't seem to work on my pc:
Vulkan0: AMD Radeon Graphics (RADV RENOIR) (radv) | uma: 1 | fp16: 1 | warp size: 64
/home/daniandtheweb/Applications/stable-diffusion/test/stable-diffusion.cpp/ggml/src/ggml-vulkan.cpp:4145: GGML_ASSERT(op == GGML_OP_GET_ROWS || (!ggml_is_quantized(src0->type) && (src1 == nullptr || !ggml_is_quantized(src1->type)))) failed
ptrace: Operation not permitted.
No stack.
The program is not being run.

No problem, I added LEAKY_RELU. Please try it.

Upscaling works fine now, however I've also just found out that quantized models don't seem to work on my pc:
Vulkan0: AMD Radeon Graphics (RADV RENOIR) (radv) | uma: 1 | fp16: 1 | warp size: 64
/home/daniandtheweb/Applications/stable-diffusion/test/stable-diffusion.cpp/ggml/src/ggml-vulkan.cpp:4145: GGML_ASSERT(op == GGML_OP_GET_ROWS || (!ggml_is_quantized(src0->type) && (src1 == nullptr || !ggml_is_quantized(src1->type)))) failed
ptrace: Operation not permitted.
No stack.
The program is not being run.

The quantized models don't work only when an embedding or a lora are used, without any of them the program runs just fine.

Yeah, I think embedding and lora only working on non-quantized models is expected. Does that work on CPU/CUDA?

On cpu lora + quantized model works (I can't test embeddings as for some reason I can't get them to work at all on cpu)

https://github.com/ggerganov/ggml/pull/904
the required ops where merged upstream

i got a error when using a fp8 model
ggml/src/ggml-vulkan.cpp:2111: GGML_ASSERT(idx < vk_instance.device_indices.size()) failed

That's a generic error that means the vulkan device you requested doesn't exist. If you didn't request one manually (with the environment variable GGML_VK_VISIBLE_DEVICES) then it means you don't have a Vulkan device. Missing driver maybe? Try running vulkaninfo --summary to check what devices are available.

Attempting to compile this, getting quite a lot of errors all of a similar type:
/tmp/guix-build-stable-diffusion-cpp-vulkan-15553084e5d1a7d6abd3aac176bee95b0247c379.drv-0/source/ggml/src/ggml-vulkan.cpp:1108:109: error: ‘matmul_f32_f16_aligned_len’ was not declared in this scope; did you mean ‘matmul_f32_aligned_len’?
 1108 |         ggml_vk_create_pipeline(ctx, ctx->device->pipeline_matmul_f32_f16->a_l, "matmul_f32_f16_aligned_l", matmul_f32_f16_aligned_len, matmul_f32_f16_aligned_data, "main", 3, sizeof(vk_mat_mat_push_constants), l_wg_denoms, warptile_l, l_align);
      |                                                                                ^~~~~~~~~~~~~~~~~~~~~~~~~~
      |                                                                                matmul_f32_aligned_len
/tmp/guix-build-stable-diffusion-cpp-vulkan-15553084e5d1a7d6abd3aac176bee95b0247c379.drv-0/source/ggml/src/ggml-vulkan.cpp:1108:137: error: ‘matmul_f32_f16_aligned_data’ was not declared in this scope; did you mean ‘matmul_f32_aligned_data’?
 1108 |         ggml_vk_create_pipeline(ctx, ctx->device->pipeline_matmul_f32_f16->a_l, "matmul_f32_f16_aligned_l", matmul_f32_f16_aligned_len, matmul_f32_f16_aligned_data, "main", 3, sizeof(vk_mat_mat_push_constants), l_wg_denoms, warptile_l, l_align);
      |                                                                                ^~~~~~~~~~~~~~~~~~~~~~~~~~~
      |                                                                                matmul_f32_aligned_data
/tmp/guix-build-stable-diffusion-cpp-vulkan-15553084e5d1a7d6abd3aac176bee95b0247c379.drv-0/source/ggml/src/ggml-vulkan.cpp:1196:97: error: ‘matmul_id_f32_f32_len’ was not declared in this scope; did you mean ‘matmul_f32_fp32_len’?
 1196 |         ggml_vk_create_pipeline(ctx, ctx->device->pipeline_matmul_id_f32->l, "matmul_id_f32_l", matmul_id_f32_f32_len, matmul_id_f32_f32_data, "main", 4, sizeof(vk_mat_mat_id_push_constants), l_wg_denoms, warptile_l, 1);
      |                                                                                ^~~~~~~~~~~~~~~~~~~~~
      |                                                                                matmul_f32_fp32_len
/tmp/guix-build-stable-diffusion-cpp-vulkan-15553084e5d1a7d6abd3aac176bee95b0247c379.drv-0/source/ggml/src/ggml-vulkan.cpp:1196:120: error: ‘matmul_id_f32_f32_data’ was not declared in this scope; did you mean ‘matmul_f32_fp32_data’?
 1196 |         ggml_vk_create_pipeline(ctx, ctx->device->pipeline_matmul_id_f32->l, "matmul_id_f32_l", matmul_id_f32_f32_len, matmul_id_f32_f32_data, "main", 4, sizeof(vk_mat_mat_id_push_constants), l_wg_denoms, warptile_l, 1);
      |                                                                                ^~~~~~~~~~~~~~~~~~~~~~
      |                                                                                matmul_f32_fp32_data

I did run the python script to generate shader headers, so those are in ggml's src/ directory, as expected. Perhaps an issue with the script not generating the right shaders for my device? I'm using an Nvidia GTX 1050 Ti with the proprietary drivers (I hope to try this with Nouveau using NVK at a later date as well).
Full build log using Guix
Any help to get this working would be appreciated.

@msglm That looks like the ggml-version you're using is outdated. You have to use an up-to-date one that contains the Vulkan operators I've implemented to allow running stable diffusion. One part of that is that the Python script to generate shaders has been deleted and replaced with a C++ program that is automatically called by Make/CMake as part of the build process.

@0cc4m Seems that your suggestion didn't fix the core of the issue, since it's still not able to properly generate (seemingly, for similar reasons). The problem seemingly lies in the actual generation of the shaders.
The failed build log in full:
guix-build-vanilla.log
The version of GLSLC I have on my machine is 2<IP_ADDRESS><IP_ADDRESS>0.0 (Target: SPIR-V 1.0) (since that's the tool doing the generating and heavy lifting). I decided to change FILE* spv = fopen(path.c_str(), "rb"); to FILE* spv = fopen(path.c_str(), "wb+"); in vulkan-shaders-gen.cpp to see if I could force the files to be generated if they don't exist, but The result was the following warnings:
/tmp/guix-build-stable-diffusion-cpp-vulkan-15553084e5d1a7d6abd3aac176bee95b0247c379.drv-0/build/ggml/src/ggml-vulkan-shaders.hpp:660:41: warning: ISO C++ forbids zero-size array ‘get_rows_q4_1_data’ [-Wpedantic]
  660 | extern unsigned char get_rows_q4_1_data[0];
      |                                         ^
/tmp/guix-build-stable-diffusion-cpp-vulkan-15553084e5d1a7d6abd3aac176bee95b0247c379.drv-0/build/ggml/src/ggml-vulkan-shaders.hpp:663:36: warning: ISO C++ forbids zero-size array ‘gelu_f32_data’ [-Wpedantic]
  663 | extern unsigned char gelu_f32_data[0];
      |                                    ^
/tmp/guix-build-stable-diffusion-cpp-vulkan-15553084e5d1a7d6abd3aac176bee95b0247c379.drv-0/build/ggml/src/ggml-vulkan-shaders.hpp:666:41: warning: ISO C++ forbids zero-size array ‘get_rows_q5_0_data’ [-Wpedantic]
  666 | extern unsigned char get_rows_q5_0_data[0];
      |                                         ^
/tmp/guix-build-stable-diffusion-cpp-vulkan-15553084e5d1a7d6abd3aac176bee95b0247c379.drv-0/build/ggml/src/ggml-vulkan-shaders.hpp:669:41: warning: ISO C++ forbids zero-size array ‘get_rows_q5_1_data’ [-Wpedantic]
  669 | extern unsigned char get_rows_q5_1_data[0];
      |                                         ^
/tmp/guix-build-stable-diffusion-cpp-vulkan-15553084e5d1a7d6abd3aac176bee95b0247c379.drv-0/build/ggml/src/ggml-vulkan-shaders.hpp:672:39: warning: ISO C++ forbids zero-size array ‘cpy_f16_f16_data’ [-Wpedantic]
  672 | extern unsigned char cpy_f16_f16_data[0];
      |                                       ^
/tmp/guix-build-stable-diffusion-cpp-vulkan-15553084e5d1a7d6abd3aac176bee95b0247c379.drv-0/build/ggml/src/ggml-vulkan-shaders.hpp:675:54: warning: ISO C++ forbids zero-size array ‘matmul_id_q4_0_f32_aligned_data’ [-Wpedantic]
  675 | extern unsigned char matmul_id_q4_0_f32_aligned_data[0];
      |                                                      ^
/tmp/guix-build-stable-diffusion-cpp-vulkan-15553084e5d1a7d6abd3aac176bee95b0247c379.drv-0/build/ggml/src/ggml-vulkan-shaders.hpp:678:50: warning: ISO C++ forbids zero-size array ‘mul_mat_vec_nc_f16_f32_data’ [-Wpedantic]
  678 | extern unsigned char mul_mat_vec_nc_f16_f32_data[0];
      |                                                  ^
/tmp/guix-build-stable-diffusion-cpp-vulkan-15553084e5d1a7d6abd3aac176bee95b0247c379.drv-0/build/ggml/src/ggml-vulkan-shaders.hpp:681:51: warning: ISO C++ forbids zero-size array ‘matmul_id_q4_1_f32_fp32_data’ [-Wpedantic]
  681 | extern unsigned char matmul_id_q4_1_f32_fp32_data[0];
      |                                                   ^
/tmp/guix-build-stable-diffusion-cpp-vulkan-15553084e5d1a7d6abd3aac176bee95b0247c379.drv-0/build/ggml/src/ggml-vulkan-shaders.hpp:684:46: warning: ISO C++ forbids zero-size array ‘matmul_id_q5_0_f32_data’ [-Wpedantic]
  684 | extern unsigned char matmul_id_q5_0_f32_data[0];

These warnings make me believe that vulkan-shaders-gen.cpp is just improperly generating the shaders on my machine (as they're zero-sized arrays).
Full build log for the attempt with removed error-checking can be found here:
guix-build-modded.log

These are problems arising from the package manager you are using, which I am unfamiliar with. You'd have to look into why vulkan-shaders-gen fails to compile the SPIR-V files correctly (issue with glslc?) or in the correct location on your system:
[ 15%] Built target vulkan-shaders-gen
make  -f ggml/src/CMakeFiles/ggml.dir/build.make ggml/src/CMakeFiles/ggml.dir/depend
make[2]: Entering directory '/tmp/guix-build-stable-diffusion-cpp-vulkan-15553084e5d1a7d6abd3aac176bee95b0247c379.drv-0/build'
[ 21%] Generate vulkan shaders
cd /tmp/guix-build-stable-diffusion-cpp-vulkan-15553084e5d1a7d6abd3aac176bee95b0247c379.drv-0/build/ggml/src && ../../bin/vulkan-shaders-gen --glslc /gnu/store/hkkwk7dhcsfy9r43zqi82yi3blhdb4zf-shaderc-2024.0/bin/glslc --input-dir /tmp/guix-build-stable-diffusion-cpp-vulkan-15553084e5d1a7d6abd3aac176bee95b0247c379.drv-0/source/ggml/src/vulkan-shaders --output-dir /tmp/guix-build-stable-diffusion-cpp-vulkan-15553084e5d1a7d6abd3aac176bee95b0247c379.drv-0/build/ggml/src/vulkan-shaders.spv --target-hpp /tmp/guix-build-stable-diffusion-cpp-vulkan-15553084e5d1a7d6abd3aac176bee95b0247c379.drv-0/build/ggml/src/ggml-vulkan-shaders.hpp --target-cpp /tmp/guix-build-stable-diffusion-cpp-vulkan-15553084e5d1a7d6abd3aac176bee95b0247c379.drv-0/build/ggml/src/ggml-vulkan-shaders.cpp --no-clean
ggml_vulkan: Generating and compiling shaders to SPIR-V
Error opening SPIR-V file: /tmp/guix-build-stable-diffusion-cpp-vulkan-15553084e5d1a7d6abd3aac176bee95b0247c379.drv-0/build/ggml/src/vulkan-shaders.spv/matmul_f32_f16_fp32.spv (No such file or directory)

I see also that this branch is out of date. @sohzm Do you want to update this (@SkutteOleg has opened a PR in your fork)? Otherwise this should be closed and a new PR should be opened (maybe by @SkutteOleg )

Thanks @0cc4m for pointing out the pr, I had missed it.
Ive merged it and also added changes to make it work with the recent ggml commit https://github.com/ggerganov/ggml/commit/fc31d407910e0ebeee5f1c0f65cada2ad211d1da in this commit: https://github.com/leejet/stable-diffusion.cpp/pull/291/commits/41ca4d509d005213b6be95ebd46f6b9bee90263b
Im fixing the merge conflicts to get this pr ready but can you confirm if the changes related to the epsilon as a parameter for group_norm are correct as I dont know what its about.

@leejet could you please review and merge this PR? If there's anything you'd like me to change or improve, please let me know. Thanks!

Seconding the problem here: https://github.com/leejet/stable-diffusion.cpp/pull/291#issuecomment-2267560268
Quant with LORA does not play nice with the vulkan backend. Works on CPU though.

Anyone tested whether Vulkan will noticeably improve performance on AMD APU's?

Anyone tested whether Vulkan will noticeably improve performance on AMD APU's?

I just did a quick test: This DOES appear to speed up performance significantly on an AMD APU (tested with a 5600G).
Using SD 1.5, it cut my inference time > 50% (~4 secs per iteration compared to ~9 secs per iteration for a 512x512).

Good news:
This implementation works pretty well (to an extent), and the generation speed is much, much faster than CPU or than the directML implementation of Stable Diffusion that Windows+AMD users have to deal with most of the time.
Bad news:
The memory management is behaving strangely, which is aggresively limiting the resolution I can generate at.

Hidden for compactness
Problem 1: the vae
Images up to 512x576 resolution can generate just fine with fp16 sd1.5 models. Reported vram usage stays around 3GB for the whole time (including vae decoding).
But anything bigger (512x640 or bigger) and the vae suddenly runs out of memory (quantization doesn't really help):
.\build\bin\Release\sd.exe -m ..\ComfyUI\models\checkpoints\dreamshaper_8.safetensors --cfg-scale 5 --steps 4 --sampling-method euler  -H 512 -W 640 --seed 42 -p "fantasy medieval village world inside a glass sphere , high detail, fantasy, realistic, light effect, hyper detail, volumetric lighting, cinematic, macro, depth of field, blur, red light and clouds from the back, highly detailed epic cinematic concept art cg render made in maya, blender and photoshop, octane render, excellent composition, dynamic dramatic cinematic lighting, aesthetic, very inspirational, world inside a glass sphere by james gurney by artgerm with james jean, joe fenton and tristan eaton by ross tran, fine details, 4k resolution"
ggml_vulkan: Found 1 Vulkan devices:
Vulkan0: AMD Radeon RX 5700 XT (AMD proprietary driver) | uma: 0 | fp16: 1 | warp size: 64
[INFO ] stable-diffusion.cpp:180  - loading model from '..\ComfyUI\models\checkpoints\dreamshaper_8.safetensors'
[INFO ] model.cpp:741  - load ..\ComfyUI\models\checkpoints\dreamshaper_8.safetensors using safetensors format
[INFO ] stable-diffusion.cpp:203  - Stable Diffusion 1.x
[INFO ] stable-diffusion.cpp:209  - Stable Diffusion weight type: f16
[INFO ] stable-diffusion.cpp:431  - total params memory size = 1969.78MB (VRAM 1969.78MB, RAM 0.00MB): clip 235.06MB(VRAM), unet 1640.25MB(VRAM), vae 94.47MB(VRAM), controlnet 0.00MB(VRAM), pmid 0.00MB(VRAM)
[INFO ] stable-diffusion.cpp:435  - loading model from '..\ComfyUI\models\checkpoints\dreamshaper_8.safetensors' completed, taking 1.70s
[INFO ] stable-diffusion.cpp:455  - running in eps-prediction mode
[INFO ] stable-diffusion.cpp:573  - Attempting to apply 0 LoRAs
[INFO ] stable-diffusion.cpp:1032 - apply_loras completed, taking 0.00s
[INFO ] stable-diffusion.cpp:1156 - get_learned_condition completed, taking 72 ms
[INFO ] stable-diffusion.cpp:1177 - sampling using Euler method
[INFO ] stable-diffusion.cpp:1181 - generating image: 1/1 - seed 42
  |==================================================| 4/4 - 1.12s/it
[INFO ] stable-diffusion.cpp:1212 - sampling completed, taking 4.56s
[INFO ] stable-diffusion.cpp:1220 - generating 1 latent images completed, taking 4.57s
[INFO ] stable-diffusion.cpp:1223 - decoding 1 latents
ggml_vulkan: Device memory allocation of size<PHONE_NUMBER> failed.
ggml_vulkan: Requested buffer size exceeds device memory allocation limit: ErrorOutOfDeviceMemory
ggml_gallocr_reserve_n: failed to allocate AMD Radeon RX 5700 XT buffer of size<PHONE_NUMBER>
[ERROR] ggml_extend.hpp:935  - vae: failed to allocate the compute buffer

Problem 2: ??
With --vae-on-cpu or --vae-tiling, I can generate images up to 768x640 resolution just fine, the reported vram usage peaks at 4.3 GB out of the 8GB I have available.
.\build\bin\Release\sd.exe -m ..\ComfyUI\models\checkpoints\dreamshaper_8.safetensors --cfg-scale 5 --steps 30 --sampling-method euler  -H 768 -W 640 --seed 42 -p "fantasy medieval village world inside a glass sphere , high detail, fantasy, realistic, light effect, hyper detail, volumetric lighting, cinematic, macro, depth of field, blur, red light and clouds from the back, highly detailed epic cinematic concept art cg render made in maya, blender and photoshop, octane render, excellent composition, dynamic dramatic cinematic lighting, aesthetic, very inspirational, world inside a glass sphere by james gurney by artgerm with james jean, joe fenton and tristan eaton by ross tran, fine details, 4k resolution"--vae-on-cpu
ggml_vulkan: Found 1 Vulkan devices:
Vulkan0: AMD Radeon RX 5700 XT (AMD proprietary driver) | uma: 0 | fp16: 1 | warp size: 64
[INFO ] stable-diffusion.cpp:180  - loading model from '..\ComfyUI\models\checkpoints\dreamshaper_8.safetensors'
[INFO ] model.cpp:741  - load ..\ComfyUI\models\checkpoints\dreamshaper_8.safetensors using safetensors format
[INFO ] stable-diffusion.cpp:203  - Stable Diffusion 1.x
[INFO ] stable-diffusion.cpp:209  - Stable Diffusion weight type: f16
[INFO ] stable-diffusion.cpp:268  - VAE Autoencoder: Using CPU backend
[INFO ] stable-diffusion.cpp:431  - total params memory size = 1969.78MB (VRAM 1875.32MB, RAM 94.47MB): clip 235.06MB(VRAM), unet 1640.25MB(VRAM), vae 94.47MB(RAM), controlnet 0.00MB(VRAM), pmid 0.00MB(VRAM)
[INFO ] stable-diffusion.cpp:435  - loading model from '..\ComfyUI\models\checkpoints\dreamshaper_8.safetensors' completed, taking 1.67s
[INFO ] stable-diffusion.cpp:455  - running in eps-prediction mode
[INFO ] stable-diffusion.cpp:573  - Attempting to apply 0 LoRAs
[INFO ] stable-diffusion.cpp:1032 - apply_loras completed, taking 0.00s
[INFO ] stable-diffusion.cpp:1156 - get_learned_condition completed, taking 75 ms
[INFO ] stable-diffusion.cpp:1177 - sampling using Euler method
[INFO ] stable-diffusion.cpp:1181 - generating image: 1/1 - seed 42
  |==================================================| 30/30 - 1.79s/it
[INFO ] stable-diffusion.cpp:1212 - sampling completed, taking 53.60s
[INFO ] stable-diffusion.cpp:1220 - generating 1 latent images completed, taking 53.62s
[INFO ] stable-diffusion.cpp:1223 - decoding 1 latents
[INFO ] stable-diffusion.cpp:1233 - latent 1 decoded, taking 16.50s
[INFO ] stable-diffusion.cpp:1237 - decode_first_stage completed, taking 16.50s
[INFO ] stable-diffusion.cpp:1337 - txt2img completed in 70.20s
save result image to 'output.png'

But if i try to get slightly past this, it runs out of memory again:
.\build\bin\Release\sd.exe -m ..\ComfyUI\models\checkpoints\dreamshaper_8.q4_0.gguf --cfg-scale 5 --steps 30 --sampling-method euler  -H 768 -W 704 --seed 42 -p "fantasy medieval village world inside a glass sphere , high detail, fantasy, realistic, light effect, hyper detail, volumetric lighting, cinematic, macro, depth of field, blur, red light and clouds from the back, highly detailed epic cinematic concept art cg render made in maya, blender and photoshop, octane render, excellent composition, dynamic dramatic cinematic lighting, aesthetic, very inspirational, world inside a glass sphere by james gurney by artgerm with james jean, joe fenton and tristan eaton by ross tran, fine details, 4k resolution"--vae-on-cpu
ggml_vulkan: Found 1 Vulkan devices:
Vulkan0: AMD Radeon RX 5700 XT (AMD proprietary driver) | uma: 0 | fp16: 1 | warp size: 64
[INFO ] stable-diffusion.cpp:180  - loading model from '..\ComfyUI\models\checkpoints\dreamshaper_8.q4_0.gguf'
[INFO ] model.cpp:738  - load ..\ComfyUI\models\checkpoints\dreamshaper_8.q4_0.gguf using gguf format
WARNING: Behavior may be unexpected when allocating 0 bytes for ggml_calloc!
[INFO ] stable-diffusion.cpp:203  - Stable Diffusion 1.x
[INFO ] stable-diffusion.cpp:209  - Stable Diffusion weight type: q4_0
[INFO ] stable-diffusion.cpp:268  - VAE Autoencoder: Using CPU backend
[INFO ] stable-diffusion.cpp:431  - total params memory size = 1431.11MB (VRAM 1336.65MB, RAM 94.47MB): clip 66.61MB(VRAM), unet 1270.04MB(VRAM), vae 94.47MB(RAM), controlnet 0.00MB(VRAM), pmid 0.00MB(VRAM)
[INFO ] stable-diffusion.cpp:435  - loading model from '..\ComfyUI\models\checkpoints\dreamshaper_8.q4_0.gguf' completed, taking 1.22s
[INFO ] stable-diffusion.cpp:455  - running in eps-prediction mode
[INFO ] stable-diffusion.cpp:573  - Attempting to apply 0 LoRAs
[INFO ] stable-diffusion.cpp:1032 - apply_loras completed, taking 0.00s
[INFO ] stable-diffusion.cpp:1156 - get_learned_condition completed, taking 70 ms
[INFO ] stable-diffusion.cpp:1177 - sampling using Euler method
[INFO ] stable-diffusion.cpp:1181 - generating image: 1/1 - seed 42
ggml_vulkan: Device memory allocation of size<PHONE_NUMBER> failed.
ggml_vulkan: Requested buffer size exceeds device memory allocation limit: ErrorOutOfDeviceMemory
ggml_gallocr_reserve_n: failed to allocate AMD Radeon RX 5700 XT buffer of size<PHONE_NUMBER>
[ERROR] ggml_extend.hpp:935  - unet: failed to allocate the compute buffer

Vulkan devices info
vulkaninfo --summary
WARNING: [Loader Message] Code 0 : Layer VK_LAYER_RTSS uses API version 1.1 which is older than the application specified API version of 1.3. May cause issues.
==========
VULKANINFO
==========

Vulkan Instance Version: 1.3.261

Instance Extensions: count = 13
-------------------------------
VK_EXT_debug_report                    : extension revision 10
VK_EXT_debug_utils                     : extension revision 2
VK_EXT_swapchain_colorspace            : extension revision 4
VK_KHR_device_group_creation           : extension revision 1
VK_KHR_external_fence_capabilities     : extension revision 1
VK_KHR_external_memory_capabilities    : extension revision 1
VK_KHR_external_semaphore_capabilities : extension revision 1
VK_KHR_get_physical_device_properties2 : extension revision 2
VK_KHR_get_surface_capabilities2       : extension revision 1
VK_KHR_portability_enumeration         : extension revision 1
VK_KHR_surface                         : extension revision 25
VK_KHR_win32_surface                   : extension revision 6
VK_LUNARG_direct_driver_loading        : extension revision 1

Instance Layers: count = 17
---------------------------
VK_LAYER_AMD_switchable_graphics    AMD switchable graphics layer                 1.3.280  version 1
VK_LAYER_EOS_Overlay                Vulkan overlay layer for Epic Online Services 1.2.136  version 1
VK_LAYER_EOS_Overlay                Vulkan overlay layer for Epic Online Services 1.2.136  version 1
VK_LAYER_KHRONOS_profiles           Khronos Profiles layer                        1.3.283  version 1
VK_LAYER_KHRONOS_shader_object      Khronos Shader object layer                   1.3.283  version 1
VK_LAYER_KHRONOS_synchronization2   Khronos Synchronization2 layer                1.3.283  version 1
VK_LAYER_KHRONOS_validation         Khronos Validation Layer                      1.3.283  version 1
VK_LAYER_LUNARG_api_dump            LunarG API dump layer                         1.3.283  version 2
VK_LAYER_LUNARG_gfxreconstruct      GFXReconstruct Capture Layer Version 1.0.4    1.3.283  version 4194308
VK_LAYER_LUNARG_monitor             Execution Monitoring Layer                    1.3.283  version 1
VK_LAYER_LUNARG_screenshot          LunarG image capture layer                    1.3.283  version 1
VK_LAYER_OBS_HOOK                   Open Broadcaster Software hook                1.3.216  version 1
VK_LAYER_RENDERDOC_Capture          Debugging capture layer for RenderDoc         1.2.131  version 17
VK_LAYER_ROCKSTAR_GAMES_social_club Rockstar Games Social Club Layer              1.0.70   version 1
VK_LAYER_RTSS                       RTSS overlay hook bootstrap                   1.1.73   version 1
VK_LAYER_VALVE_steam_fossilize      Steam Pipeline Caching Layer                  1.3.207  version 1
VK_LAYER_VALVE_steam_overlay        Steam Overlay Layer                           1.3.207  version 1

Devices:
========
GPU0:
      apiVersion         = 1.3.280
      driverVersion      = 2.0.302
      vendorID           = 0x1002
      deviceID           = 0x731f
      deviceType         = PHYSICAL_DEVICE_TYPE_DISCRETE_GPU
      deviceName         = AMD Radeon RX 5700 XT
      driverID           = DRIVER_ID_AMD_PROPRIETARY
      driverName         = AMD proprietary driver
      driverInfo         = 24.6.1 (AMD proprietary shader compiler)
      conformanceVersion = <IP_ADDRESS>
      deviceUUID         = 00000000-2800-0000-0000-000000000000
      driverUUID         = 414d442d-5749-4e2d-4452-560000000000

The memory management is behaving strangely, which is aggresively limiting the resolution I can generate at.

I had this problem also (512x512 was fine, but 1024x1024 failed on SDXL IIRC), but assumed it was because I was an APU (with a unified-memory-like architecture). This may mean that my drivers (or bios?) might constrain me to a max GPU memory allocation size.
I think GGML (on Llama.cpp at least) will actually show a warning for this:
https://github.com/ggerganov/llama.cpp/blob/master/ggml/src/ggml-vulkan.cpp#L989
    if (size > device->max_memory_allocation_size) {
        throw vk::OutOfDeviceMemoryError("Requested buffer size exceeds device memory allocation limit");
    }

No idea if it's possibly to actually change this limit. Your issue might be different as it looks like you have dedicated GPU? Maybe it is actually just that you're out of GPU mem?

Your issue might be different as it looks like you have dedicated GPU? Maybe it is actually just that you're out of GPU mem?

I do have a dedicated GPU with 8GB worth of dedicated memory. According to windows task manager, less than half the vram is used (for 768x640px images). I can also generate 1024x1536px images with the directML implementation, so I don't believe I'm actually running out of memory.

so I don't believe I'm actually running out of memory.

Yeah, I looked at your logs above again and noticed that it's the same error I hit: Requested buffer size exceeds device memory allocation limit.
Doing some research quickly, it seems it might be possible to allow exceeding the Max Allocation Limit on some devices with this extension. This should probably be done in the GGML implementation though.
https://registry.khronos.org/vulkan/specs/1.3-extensions/man/html/VkMemoryOverallocationBehaviorAMD.html
Anyone tried working with the above?

https://github.com/leejet/stable-diffusion.cpp/pull/291#issuecomment-2259896691

Vulkan has a restriction on how large VRAM buffers can be (usually 4GB), and 1024x1024 VAE decoding requests a buffer larger than that.

@stduhpf I had a quick play around with that extension, but I couldn't get proper results on my machine (it all ran fine, but parts of the resulting image were just blank).
If you want to try on your end, I've linked a patch file you can apply to ggml/src/ggml-vulkan.cpp which should allow you to go above the memory allocation limit (whether it'll actually work for you is another question).
ggml-vulkan.patch
If anyone else wants to take a gander and see if I missed something, please do.

@jimtendo it works, thank you.
I also get the blank image at some point.

@jimtendo it works, thank you!
But I do also get the blank images at higher resolutions. (>960x768px)

Good news:
This implementation works pretty well (to an extent), and the generation speed is much, much faster than CPU or than the directML implementation of Stable Diffusion that Windows+AMD users have to deal with most of the time.
Bad news:
The memory management is behaving strangely, which is aggresively limiting the resolution I can generate at.
Hidden for compactness
Vulkan devices info

That's a quirk with the AMD Windows driver and with amvlk (and probably the proprietary one too) on Linux. They report 2GB max allocation size, but I think up to 4GB is fine (which is also what is reported and working on the same hardware if you use the community RADV driver).
We might have to add a workaround for those drivers to up the limit to 4GB, but I really don't know why they report the lower value.

I know FLUX support isn't merged yet, just throwing this out there. I tried to apply Vulkan support on top of #356, but it just gradually eats all of my RAM and VRAM without doing anything.
Logs:
sd.exe --diffusion-model  flux1-schnell.q2_k.gguf --vae ae.q8_0.gguf --clip_l clip_l.q8_0.gguf --t5xxl t5xxl.q4_0.gguf -p "a lovely cat holding a sign that says 'flux.cpp'" --cfg-scale 1.0 --sampling-method euler -v --steps 4
Option:
    n_threads:         3
    mode:              txt2img
    model_path:
    wtype:             unspecified
    clip_l_path:       clip_l.q8_0.gguf
    t5xxl_path:        t5xxl.q4_0.gguf
    diffusion_model_path:   flux1-schnell.q2_k.gguf
    vae_path:          ae.q8_0.gguf
    taesd_path:
    esrgan_path:
    controlnet_path:
    embeddings_path:
    stacked_id_embeddings_path:
    input_id_images_path:
    style ratio:       20.00
    normzalize input image :  false
    output_path:       output.png
    init_img:
    control_image:
    clip on cpu:       false
    controlnet cpu:    false
    vae decoder on cpu:false
    strength(control): 0.90
    prompt:            a lovely cat holding a sign that says 'flux.cpp'
    negative_prompt:
    min_cfg:           1.00
    cfg_scale:         1.00
    guidance:          3.50
    clip_skip:         -1
    width:             512
    height:            512
    sample_method:     euler
    schedule:          default
    sample_steps:      4
    strength(img2img): 0.75
    rng:               cuda
    seed:              42
    batch_count:       1
    vae_tiling:        false
    upscale_repeats:   1
System Info:
    BLAS = 1
    SSE3 = 1
    AVX = 1
    AVX2 = 1
    AVX512 = 0
    AVX512_VBMI = 0
    AVX512_VNNI = 0
    FMA = 1
    NEON = 0
    ARM_FMA = 0
    F16C = 1
    FP16_VA = 0
    WASM_SIMD = 0
    VSX = 0
[DEBUG] stable-diffusion.cpp:166  - Using Vulkan backend
ggml_vulkan: Found 1 Vulkan devices:
Vulkan0: NVIDIA GeForce GTX 1660 SUPER (NVIDIA) | uma: 0 | fp16: 1 | warp size: 32
[INFO ] stable-diffusion.cpp:203  - loading clip_l from 'clip_l.q8_0.gguf'
[INFO ] model.cpp:738  - load clip_l.q8_0.gguf using gguf format
[DEBUG] model.cpp:755  - init from 'clip_l.q8_0.gguf'
WARNING: Behavior may be unexpected when allocating 0 bytes for ggml_calloc!
[INFO ] stable-diffusion.cpp:210  - loading t5xxl from 't5xxl.q4_0.gguf'
[INFO ] model.cpp:738  - load t5xxl.q4_0.gguf using gguf format
[DEBUG] model.cpp:755  - init from 't5xxl.q4_0.gguf'
WARNING: Behavior may be unexpected when allocating 0 bytes for ggml_calloc!
[INFO ] stable-diffusion.cpp:217  - loading diffusion model from 'flux1-schnell.q2_k.gguf'
[INFO ] model.cpp:738  - load flux1-schnell.q2_k.gguf using gguf format
[DEBUG] model.cpp:755  - init from 'flux1-schnell.q2_k.gguf'
WARNING: Behavior may be unexpected when allocating 0 bytes for ggml_calloc!
[INFO ] stable-diffusion.cpp:224  - loading vae from 'ae.q8_0.gguf'
[INFO ] model.cpp:738  - load ae.q8_0.gguf using gguf format
[DEBUG] model.cpp:755  - init from 'ae.q8_0.gguf'
WARNING: Behavior may be unexpected when allocating 0 bytes for ggml_calloc!
[INFO ] stable-diffusion.cpp:236  - Version: Flux Schnell
[INFO ] stable-diffusion.cpp:267  - Weight type:                 q2_K
[INFO ] stable-diffusion.cpp:268  - Conditioner weight type:     q8_0
[INFO ] stable-diffusion.cpp:269  - Diffsuion model weight type: q2_K
[INFO ] stable-diffusion.cpp:270  - VAE weight type:             q8_0
[DEBUG] stable-diffusion.cpp:272  - ggml tensor size = 400 bytes
[INFO ] stable-diffusion.cpp:311  - set clip_on_cpu to true
[INFO ] stable-diffusion.cpp:314  - CLIP: Using CPU backend
[DEBUG] clip.hpp:171  - vocab size: 49408
[DEBUG] clip.hpp:182  -  trigger word img already in vocab
[DEBUG] ggml_extend.hpp:1033 - clip params backend buffer size =  125.20 MB(RAM) (196 tensors)
[DEBUG] ggml_extend.hpp:1033 - t5 params backend buffer size =  4826.11 MB(RAM) (219 tensors)
[DEBUG] ggml_extend.hpp:1033 - flux params backend buffer size =  3732.51 MB(VRAM) (776 tensors)
[DEBUG] ggml_extend.hpp:1033 - vae params backend buffer size =  94.57 MB(VRAM) (138 tensors)
[DEBUG] stable-diffusion.cpp:399  - loading weights
[DEBUG] model.cpp:1463 - loading tensors from clip_l.q8_0.gguf
[DEBUG] model.cpp:1463 - loading tensors from t5xxl.q4_0.gguf
[INFO ] model.cpp:1609 - unknown tensor 'text_encoders.t5xxl.encoder.embed_tokens.weight | q4_0 | 2 [4096, 32128, 1, 1, 1]' in model file
[DEBUG] model.cpp:1463 - loading tensors from flux1-schnell.q2_k.gguf
[DEBUG] model.cpp:1463 - loading tensors from ae.q8_0.gguf
[INFO ] stable-diffusion.cpp:498  - total params memory size = 8778.40MB (VRAM 3827.08MB, RAM 4951.31MB): clip 4951.31MB(RAM), unet 3732.51MB(VRAM), vae 94.57MB(VRAM), controlnet 0.00MB(VRAM), pmid 0.00MB(RAM)
[INFO ] stable-diffusion.cpp:502  - loading model from '' completed, taking 115.93s
[INFO ] stable-diffusion.cpp:519  - running in Flux FLOW mode
[DEBUG] stable-diffusion.cpp:564  - finished loaded file
[DEBUG] stable-diffusion.cpp:1370 - txt2img 512x512
[DEBUG] stable-diffusion.cpp:1119 - prompt after extract and remove lora: "a lovely cat holding a sign that says 'flux.cpp'"
[INFO ] stable-diffusion.cpp:647  - Attempting to apply 0 LoRAs
[INFO ] stable-diffusion.cpp:1124 - apply_loras completed, taking 0.00s
[DEBUG] conditioner.hpp:1037 - parse 'a lovely cat holding a sign that says 'flux.cpp'' to [['a lovely cat holding a sign that says 'flux.cpp'', 1], ]
[DEBUG] clip.hpp:311  - token length: 77
[DEBUG] t5.hpp:397  - token length: 256
[DEBUG] ggml_extend.hpp:984  - t5 compute buffer size: 68.25 MB(RAM)
[DEBUG] conditioner.hpp:1156 - computing condition graph completed, taking 17989 ms
[INFO ] stable-diffusion.cpp:1248 - get_learned_condition completed, taking 17996 ms
[INFO ] stable-diffusion.cpp:1271 - sampling using Euler method
[INFO ] stable-diffusion.cpp:1275 - generating image: 1/1 - seed 42
[DEBUG] ggml_extend.hpp:984  - flux compute buffer size: 398.50 MB(VRAM)

vk::Device::allocateDescriptorSets: ErrorOutOfDeviceMemory

Fork with FLUX + Vulkan: https://github.com/SkutteOleg/stable-diffusion.cpp/tree/master
Same parameters work well with CUDA12 and CPU builds.

@SkutteOleg I get a similar issue (CPU works fine). Adding --type q4_0 removed the OutOfDeviceMemory error for me, but the symptom I get now is that my GPU seems to die/reset.
[DEBUG] ggml_extend.hpp:981  - flux compute buffer size: 103.57 MB(VRAM)
terminate called after throwing an instance of 'vk::DeviceLostError'
  what():  vk::Device::waitForFences: ErrorDeviceLost
Aborted (core dumped)

I'm on Linux and if I run dmesg I can see the following:
[  309.811487] amdgpu 0000:29:00.0: amdgpu: ring gfx uses VM inv eng 0 on hub 0
[  309.811489] amdgpu 0000:29:00.0: amdgpu: ring gfx_low uses VM inv eng 1 on hub 0
[  309.811491] amdgpu 0000:29:00.0: amdgpu: ring gfx_high uses VM inv eng 4 on hub 0
[  309.811493] amdgpu 0000:29:00.0: amdgpu: ring comp_1.0.0 uses VM inv eng 5 on hub 0
[  309.811495] amdgpu 0000:29:00.0: amdgpu: ring comp_1.1.0 uses VM inv eng 6 on hub 0
[  309.811496] amdgpu 0000:29:00.0: amdgpu: ring comp_1.2.0 uses VM inv eng 7 on hub 0
[  309.811498] amdgpu 0000:29:00.0: amdgpu: ring comp_1.3.0 uses VM inv eng 8 on hub 0
[  309.811500] amdgpu 0000:29:00.0: amdgpu: ring comp_1.0.1 uses VM inv eng 9 on hub 0
[  309.811501] amdgpu 0000:29:00.0: amdgpu: ring comp_1.1.1 uses VM inv eng 10 on hub 0
[  309.811503] amdgpu 0000:29:00.0: amdgpu: ring comp_1.2.1 uses VM inv eng 11 on hub 0
[  309.811505] amdgpu 0000:29:00.0: amdgpu: ring comp_1.3.1 uses VM inv eng 12 on hub 0
[  309.811507] amdgpu 0000:29:00.0: amdgpu: ring kiq_<IP_ADDRESS> uses VM inv eng 13 on hub 0
[  309.811509] amdgpu 0000:29:00.0: amdgpu: ring sdma0 uses VM inv eng 0 on hub 8
[  309.811510] amdgpu 0000:29:00.0: amdgpu: ring vcn_dec uses VM inv eng 1 on hub 8
[  309.811512] amdgpu 0000:29:00.0: amdgpu: ring vcn_enc0 uses VM inv eng 4 on hub 8
[  309.811514] amdgpu 0000:29:00.0: amdgpu: ring vcn_enc1 uses VM inv eng 5 on hub 8
[  309.811515] amdgpu 0000:29:00.0: amdgpu: ring jpeg_dec uses VM inv eng 6 on hub 8
[  309.813283] amdgpu 0000:29:00.0: amdgpu: recover vram bo from shadow start
[  309.813285] amdgpu 0000:29:00.0: amdgpu: recover vram bo from shadow done
[  309.813296] amdgpu 0000:29:00.0: amdgpu: GPU reset(1) succeeded!
[  373.907561] [drm:amdgpu_job_timedout [amdgpu]] *ERROR* ring comp_1.1.0 timeout, signaled seq=2, emitted seq=2
[  373.907873] [drm:amdgpu_job_timedout [amdgpu]] *ERROR* Process information: process sd pid 2622 thread sd pid 2622
[  373.908163] amdgpu 0000:29:00.0: amdgpu: GPU reset begin!
[  373.948119] amdgpu 0000:29:00.0: amdgpu: MODE2 reset
[  373.948180] amdgpu 0000:29:00.0: amdgpu: GPU reset succeeded, trying to resume

... so it looks like it's crashing and restarting my GPU.
I'm not sure what I did differently, but I also managed to get this output from dmesg. In this case, Flux didn't crash but just remained frozen (looks like it kept resetting the GPU):
[  615.579270] INFO: task kworker/u65:8:3109 blocked for more than 122 seconds.
[  615.579272]       Not tainted 6.8.0-41-generic #41-Ubuntu
[  615.579273] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  615.579275] task:kworker/u65:8   state:D stack:0     pid:3109  tgid:3109  ppid:2      flags:0x00004000
[  615.579278] Workqueue: ttm ttm_bo_delayed_delete [ttm]
[  615.579284] Call Trace:
[  615.579286]  <TASK>
[  615.579288]  __schedule+0x27c/0x6b0
[  615.579293]  schedule+0x33/0x110
[  615.579295]  schedule_timeout+0x157/0x170
[  615.579299]  dma_fence_default_wait+0x1e1/0x220
[  615.579303]  ? __pfx_dma_fence_default_wait_cb+0x10/0x10
[  615.579306]  dma_fence_wait_timeout+0x116/0x140
[  615.579310]  dma_resv_wait_timeout+0x7f/0xf0
[  615.579314]  ttm_bo_delayed_delete+0x2a/0xc0 [ttm]
[  615.579320]  process_one_work+0x16f/0x350
[  615.579324]  worker_thread+0x306/0x440
[  615.579328]  ? __pfx_worker_thread+0x10/0x10
[  615.579331]  kthread+0xf2/0x120
[  615.579334]  ? __pfx_kthread+0x10/0x10
[  615.579337]  ret_from_fork+0x47/0x70
[  615.579340]  ? __pfx_kthread+0x10/0x10
[  615.579343]  ret_from_fork_asm+0x1b/0x30
[  615.579348]  </TASK>

@SkutteOleg Same here. Even when using q2_k  quants (3.8GB) to make a tiny 64x64 image, it tries to allocate over 8GB of vram, and crashes. The cpu backend doesn't even need 3.9GB during diffusion with these settings.

This looks like a memory leak. Vram consumption starts shooting up as soon as the GPU starts working, after everything is loaded.

https://github.com/leejet/stable-diffusion.cpp/pull/356#issuecomment-2305946307

The Flux Vulkan issue is the result of an inefficient GGML_OP_REPEAT implementation in Vulkan that I already fixed in llama.cpp https://github.com/ggerganov/llama.cpp/pull/8959/commits/0645ed5c97d0dc7c77c038fa966316ebc6c4bd65 . Once the Vulkan changes are synced back to ggml it'll work.

The Flux Vulkan issue is the result of an inefficient GGML_OP_REPEAT implementation in Vulkan that I already fixed in llama.cpp ggerganov/llama.cpp@0645ed5 . Once the Vulkan changes are synced back to ggml it'll work.

That's great to hear. Thanks for the update and all your contributions. Do you think the same thing happens in the cuda backend? When using resolutions above 512x512, the vram requirements get a stale increase, even just for a q4_0 quantization that would otherwise work smoothly at high resolutions in ComfyUI and FORGE on 8GB vram.

The Flux Vulkan issue is the result of an inefficient GGML_OP_REPEAT implementation in Vulkan that I already fixed in llama.cpp ggerganov/llama.cpp@0645ed5 . Once the Vulkan changes are synced back to ggml it'll work.

Thanks @0cc4m , I was able to patch ggml based on the commit you linked, and It works! I get a bit under 6 s/it on my rx 5700xt, which isn't lightning fast, but still much faster than on CPU. Memory usage seems perfectly fine.

The Flux Vulkan issue is the result of an inefficient GGML_OP_REPEAT implementation in Vulkan that I already fixed in llama.cpp ggerganov/llama.cpp@0645ed5 . Once the Vulkan changes are synced back to ggml it'll work.

Thanks @0cc4m , I was able to patch ggml based on the commit you linked, and It works! I get a bit under 6 s/it on my rx 5700xt, which isn't lightning fast, but still much faster than on CPU. Memory usage seems perfectly fine.

Hit there, that's good to know. May i ask what your system specs are and on which quants you've tested?

The Flux Vulkan issue is the result of an inefficient GGML_OP_REPEAT implementation in Vulkan that I already fixed in llama.cpp ggerganov/llama.cpp@0645ed5 . Once the Vulkan changes are synced back to ggml it'll work.

Thanks @0cc4m , I was able to patch ggml based on the commit you linked, and It works! I get a bit under 6 s/it on my rx 5700xt, which isn't lightning fast, but still much faster than on CPU. Memory usage seems perfectly fine.

Hit there, that's good to know. May i ask what your system specs are and on which quants you've tested?

For testing, I used .\build\bin\Release\sd.exe --diffusion-model  ..\ComfyUI\models\unet\flux1-schnell-Q2_k.gguf --vae ..\ComfyUI\models\vae\ae.q2_k.gguf --clip_l ..\ComfyUI\models\clip\clip_l.q8_0.gguf --t5xxl ..\ComfyUI\models\clip\t5xxl_q4_k.gguf  -p "a lovely cat holding a sign says 'flux.cpp'" --cfg-scale 1.0 --sampling-method euler -v --steps 4 -t 24 --clip-on-cpu --vae-on-cpu. So that a 512x512 image. I think I could get away with bigger quants for the diffusion model, since q2 is a bit degraded.
System specs: Ryzen9 3900x + 32GB DDR4 + RX 5700 XT (8GB VRAM)

The Flux Vulkan issue is the result of an inefficient GGML_OP_REPEAT implementation in Vulkan that I already fixed in llama.cpp ggerganov/llama.cpp@0645ed5 . Once the Vulkan changes are synced back to ggml it'll work.

Thanks @0cc4m , I was able to patch ggml based on the commit you linked, and It works! I get a bit under 6 s/it on my rx 5700xt, which isn't lightning fast, but still much faster than on CPU. Memory usage seems perfectly fine.

Hit there, that's good to know. May i ask what your system specs are and on which quants you've tested?

For testing, I used .\build\bin\Release\sd.exe --diffusion-model ..\ComfyUI\models\unet\flux1-schnell-Q2_k.gguf --vae ..\ComfyUI\models\vae\ae.q2_k.gguf --clip_l ..\ComfyUI\models\clip\clip_l.q8_0.gguf --t5xxl ..\ComfyUI\models\clip\t5xxl_q4_k.gguf -p "a lovely cat holding a sign says 'flux.cpp'" --cfg-scale 1.0 --sampling-method euler -v --steps 4 -t 24 --clip-on-cpu --vae-on-cpu. So that a 512x512 image. I think I could get away with bigger quants for the diffusion model, since q2 is a bit degraded.
System specs: Ryzen9 3900x + 32GB DDR4 + RX 5700 XT (8GB VRAM)

Thanks for answering and yes, i think you should test q4_0 or q4_k as that would still fully fit 8GB vram. And could you also test resolutions like 1024x1024? It was mostly when using higher resolution that the vram requirements shut o[ drastically for me and swap to shared vram starts. But then stable-diffusion.cpp just hangs at the sampling stage and does nothing. It also looks like the clip model and text encoder get unloaded from ram when this happens.

@stduhpf any  chance you might be able to push a fork up of your working code? Tried to cherry-pick from llama.cpp, but my git-fu isn't up to scratch.
Pretty keen to see if this increases performance on AMD APUs.

Thanks for answering and yes, i think you should test q4_0 or q4_k as that would still fully fit 8GB vram. And could you also test resolutions like 1024x1024? It was mostly when using higher resolution that the vram requirements shut o[ drastically for me and swap to shared vram starts. But then stable-diffusion.cpp just hangs at the sampling stage and does nothing. It also looks like the clip model and text encoder get unloaded from ram when this happens.

I was running out of memory when using q3 at 1024x1024, but it works with q2 (34 s/it)

I can't seem to run a q4_k quantization at any resolution, Somehow it's allocating 6.389 GB of vram to load the model, despite the file size being only 4.2 GB. This discrepency doesn't happen with q2_k or q3_k.

@stduhpf any chance you might be able to push a fork up of your working code? Tried to cherry-pick from llama.cpp, but my git-fu isn't up to scratch.
Pretty keen to see if this increases performance on AMD APUs.

I've updated my forks with Vulkan REPEAT fix:
https://github.com/SkutteOleg/ggml/tree/master
https://github.com/SkutteOleg/stable-diffusion.cpp/tree/master

@stduhpf any chance you might be able to push a fork up of your working code? Tried to cherry-pick from llama.cpp, but my git-fu isn't up to scratch.
Pretty keen to see if this increases performance on AMD APUs.

I've updated my forks with Vulkan REPEAT fix: https://github.com/SkutteOleg/ggml/tree/master https://github.com/SkutteOleg/stable-diffusion.cpp/tree/master

Ah, thanks. I was just about to make a fork of ggml with the fix myself.

Thanks for answering and yes, i think you should test q4_0 or q4_k as that would still fully fit 8GB vram. And could you also test resolutions like 1024x1024? It was mostly when using higher resolution that the vram requirements shut o[ drastically for me and swap to shared vram starts. But then stable-diffusion.cpp just hangs at the sampling stage and does nothing. It also looks like the clip model and text encoder get unloaded from ram when this happens.

I was running out of memory when using q3 at 1024x1024, but it works with q2 (34 s/it)
I can't seem to run a q4_k quantization at any resolution, Somehow it's allocating 6.389 GB of vram to load the model, despite the file size being only 4.2 GB. This discrepency doesn't happen with q2_k or q3_k.

I see, thanks for confirming, so that's still a problem. It also seems happens on cuda for me.

Using @SkutteOleg 's fork above, Flux now works with Vulkan on my AMD 5600G.
Performance for the Unet is around 2.5x better (~30s vs ~75s per iteration) - and seemingly uses far less power (fan doesn't go wild like it does on CPU).
I can't get LoRa's to load, but I think that's a problem with the LoRa loader itself (fails on CPU & GPU). It seems to be a mismatch between the tensor names in Flux models and what the SD.cpp implementation looks for. That said, it might just be the particular GGUF quant I'm using. If not, am going to see if I can patch that up later.

I can't get LoRa's to load

https://github.com/leejet/stable-diffusion.cpp/pull/356/commits/46eeff55f55a849fd804883652a1d6f36dd8d716

https://github.com/leejet/stable-diffusion.cpp/commit/46eeff55f55a849fd804883652a1d6f36dd8d716 try with this

I've done a bit of playing around with it. This does fix the mappings, but I get another error (I was getting this before too, but had assumed that it was due to the failed LoRa mappings).
[WARN ] lora.hpp:164  - PATH: IS F32 OR F16
[DEBUG] lora.hpp:190  - (152 / 152) LoRA tensors applied successfully
[WARN ] lora.hpp:194  - Made it here.
/home/jimtendo/Projects/stable-diffusion.cpp2/ggml/src/ggml-backend.c:224: GGML_ASSERT(buf != NULL && "tensor buffer not set") failed

Strangely, I get the same issue on CPU - unless I explicitly specify with --type something that is not FP16 or FP32.
But, if I do that on Vulkan via something like the --type q4_0, I then get the following (which probably makes sense?):
Missing CPY op for types: f32 q4_0
/home/jimtendo/Projects/stable-diffusion.cpp2/ggml/src/ggml-vulkan.cpp:2967: fatal error

I've been trying to trace where this occurs (without --type specified) exactly. I suspect it's in the call to GGMLRunner::compute in lora.hpp but am still stepping through.

https://github.com/leejet/stable-diffusion.cpp/commit/8847114abfd900898e78d0257f5f9086f2473601 this should fix it

I'm getting OutOfPoolMemoryError with vulkan and an 8gb 6600 xt card - the model is stable_diffusion-ema-pruned-v2-1_768.q5_0.gguf.

Thank you for your contribution!

Hey @leejet, this PR requires https://github.com/ggerganov/llama.cpp/commit/0645ed5c97d0dc7c77c038fa966316ebc6c4bd65 to be synced into ggml. Without it, FLUX doesn't work, and performance is worse in general.

@0cc4m I've noticed that performing quantization directly while using txt2img on loras causes the following issue:
Missing CPY op for types: f32 q8_0

The error comes out for any quantizations I try to convert to (8_0, 4_0, 5_0...) when I apply a lora called more_details. Since it's a missing op type of error maybe it could be implemented in ggml to fix this?

Since it's a missing op type of error maybe it could be implemented in ggml to fix this?

Yes, that is possible, basically you have to implement a GPU kernel that does quantization. One for each quant you want to support. Not a priority for me at this time (might be annoying to implement), but if someone gives it a shot I'm happy to assist and review.

Well I'm late to the party as I only found out now that sd.cpp has Vulkan support :smile:. I use a Firepro W8100 on Linux with the Mesa/RADV driver. Like what @stduhpf saw I'm getting a memory allocation error when generating images past a certain size. For instance here's what happens when I run with a Q8_0 SDXL Lightning model generating 1024x1024 images.
ggml_vulkan: Device memory allocation of size<PHONE_NUMBER> failed.
ggml_vulkan: Requested buffer size exceeds device memory allocation limit: ErrorOutOfDeviceMemory
ggml_gallocr_reserve_n: failed to allocate AMD Radeon FirePro W8100 (RADV HAWAII) buffer of size<PHONE_NUMBER>
[ERROR] ggml_extend.hpp:991  - vae: failed to allocate the compute buffer

If I use vae tiling it takes longer but actually works! Here are some numbers with and without tiling on a 768x768 generation, which is small enough to work both with and without tiling.
With tiling
[INFO ] stable-diffusion.cpp:1283 - generating image: 1/1 - seed<PHONE_NUMBER>
  |==================================================| 4/4 - 3.66s/it
[INFO ] stable-diffusion.cpp:1315 - sampling completed, taking 14.81s
[INFO ] stable-diffusion.cpp:1323 - generating 1 latent images completed, taking 14.82s
[INFO ] stable-diffusion.cpp:1326 - decoding 1 latents
[INFO ] ggml_extend.hpp:531  - processing 36 tiles
  |==================================================| 36/36 - 1.05it/s
[INFO ] stable-diffusion.cpp:1336 - latent 1 decoded, taking 24.87s
[INFO ] stable-diffusion.cpp:1340 - decode_first_stage completed, taking 24.87s
[INFO ] stable-diffusion.cpp:1449 - txt2img completed in 40.25s

Without tiling
[INFO ] stable-diffusion.cpp:1283 - generating image: 1/1 - seed<PHONE_NUMBER>
  |==================================================| 4/4 - 3.71s/it
[INFO ] stable-diffusion.cpp:1315 - sampling completed, taking 14.88s
[INFO ] stable-diffusion.cpp:1323 - generating 1 latent images completed, taking 14.88s
[INFO ] stable-diffusion.cpp:1326 - decoding 1 latents
[INFO ] stable-diffusion.cpp:1336 - latent 1 decoded, taking 14.35s
[INFO ] stable-diffusion.cpp:1340 - decode_first_stage completed, taking 14.35s
[INFO ] stable-diffusion.cpp:1449 - txt2img completed in 29.79s

I haven't dug into the code yet but it looks like each tile is 128 by 128 pixels regardless of the resolution I generate at. Would using a larger tile size speed things up in this case, considering how the GPU should have enough memory to handle 512x512 tiles when generating a 1024x1024 image?


Add docs with image for Eyedropper.

For WindowsCommunityToolkit PR(windows-toolkit/WindowsCommunityToolkit#2843).

Docs Build status updates of commit 0336a90:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/controls/Eyedropper.md
:white_check_mark:Succeeded
View

docs/resources/images/Controls/Eyedropper.png
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Docs Build status updates of commit 6ace290:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/controls/Eyedropper.md
:white_check_mark:Succeeded
View

docs/resources/images/Controls/Eyedropper.png
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Docs Build status updates of commit d9e6eba:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/controls/Eyedropper.md
:white_check_mark:Succeeded
View

docs/resources/images/Controls/Eyedropper.png
:white_check_mark:Succeeded
View

docs/toc.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Add links to component guide

What

make first 'how it looks' heading link to the default example in the component guide for that component
add 'preview' link for the first example into the same heading, before 'preview all'
add 'preview' link to the 'how it looks' heading on the default example page

Why
I've found the lack of this navigation occasionally frustrating for quite a while now. It's not a huge problem, and I know the URLs to type, but I think this makes navigating the component guide fractionally easier.
Visual Changes

Before
After

Could you update the part of the link that says "preview" to "preview default"? I like the idea but don't think it's very clear what you're getting with the current implementation.

Could you update the part of the link that says "preview" to "preview default"? I like the idea but don't think it's very clear what you're getting with the current implementation.

I think we should follow the convention of the other examples, which also have 'preview' links e.g.

It's possible that the 'preview all' link isn't now in the best place following this change. Not sure where would be better to put it though.


why there is UIkit?

Same question, is the UIKit for design only or does it also provides a VOIP server from CometChat


Privacy Error on PirateWeather Website

Describe the issue
Trying to visit https://pirateweather.net/en/latest/ gives me the browser privacy error message. Message is as follows:

Did the URL for the API documentation change or did the certificate that was used for the site expire? It was working fine when I looked a few days ago so it's probably just an expired certificate.
Acknowledgements

[X] I have searched this repository and Home Assistant Repository to see if the issue has already been reported.
[X] I have read through the API documentation before opening this issue.
[X] I have written an informative title.

@alexander0042 Any update here? Am still seeing this when trying to visit the documentaion.

@alexander0042 Any updates here? Am still seeing this issue this morning. I think search engines take people to the http version of the page so at least most people aren't likely to see the error.

Can confirm this is fixed now. Is there no way in cloudfront to force SSL? It's not a big deal if it isn't possible though.


Pull in upstream updates from cisagov/skeleton-ansible-role

🗣 Description
In this pull request I pull in the upstream updates from cisagov/skeleton-ansible-role.
💭 Motivation and Context
I'm going to be making changes to this repository, so I may as well make sure it is up to date.
🧪 Testing
All pre-commit hooks and GitHub Actions pass.
🚥 Types of Changes

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (causes existing functionality to change)

✅ Checklist

[x] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[x] I have read the CONTRIBUTING document.
[ ] I have added tests to cover my changes.
[x] All new and existing tests passed.

All the checks pass now, except for the BS JS LGTM check that insists on launching for no reason.  @dav3r and @mcdonnnj, you can review now.

Looks good. I'm confident that when the tests pass this will still be a good PR.

I find your surfeit of faith disturbing, @felddy!


Release 2024.15

This is last release candidate in the style of git flow.
This will NOT be released as it has been done before, it's just for testing purposes until new workflow is finished. @hyrsky is working on that.
NOTE that this still points to MAIN to see changes.
DO. NOT. MERGE. THIS

Closed in favor of https://github.com/City-of-Helsinki/hel-fi-drupal-grants/pull/1528


fix a typo in README

found a small typo in the README

That's hacktoberfest hitting us?

Please avoid useless PRs, thanks


Injection crash/Error pop-up

Describe the bug
A clear and concise description of what the bug is.
upon injecting, csgo crashes and a small window with the error message pops up:
"Failed to initialize interfaces"
To Reproduce
Steps to reproduce the behavior.
just inject (error happened on 2020-06-09)
Expected behavior
A clear and concise description of what you expected to happen.
the cheat to work, open menu etc.
Screenshots
If applicable, add screenshots to help explain your problem.
https://imgur.com/xinnLnw
Additional context
Add any other context about the problem here.
no

Update client_panorama.dll to client.dll

@atheleon This does fix the popup issue indeed, any direction in updating structs?

	static auto SimulatePlayerSimulatedEntities = pattern::find< void(__thiscall*)(c_base_entity*) >(g_csgo.m_client_dll, "56 8B F1 57 8B BE ?? ?? ?? ?? 83 EF 01");```

	static auto PostThinkVPhysics = pattern::find< bool(__thiscall*)(c_base_entity*) >(g_csgo.m_client_dll, "55 8B EC 83 E4 F8 81 EC ?? ?? ?? ?? 53 8B D9");
	static auto SimulatePlayerSimulatedEntities = pattern::find< void(__thiscall*)(c_base_entity*) >(g_csgo.m_client_dll, "56 8B F1 57 8B BE ?? ?? ?? ?? 83 EF 01");```

This works well, and doesn't make the game crash anymore, however the aimbot doesn't seem to shoot at all, can this be due to outdated offsets?

This works well, and doesn't make the game crash anymore, however the aimbot doesn't seem to shoot at all, can this be due to outdated offsets?

Possibly offsets, check hardcoded offsets in player class, and update vtable indexes.

Fixed.


Improve child process spawning

- Summary
If the path to binary is not quoted on Windows and contains a space, it will get the path wrong and fail. This change makes sure that the path to app binary is quoted.
Fixes: https://github.com/netlify/cli/issues/817

- Test plan

Have your app binary in path with spaces
Run netlify dev

- Description for the changelog
Quote app binary path before spawning it.

- A picture of a cute animal (not mandatory but encouraged)
🐬

Note: an alternative would be not to use shell: true. This means users won't be able to use shell features like $variable expansion or globbing, but it also means no escaping issues. It also is safer. See more information.

Thanks for the tip @ehmicky. I attempted that yesterday but could make the addons tests work on Windows. Found out the problem today, we were trying to invoke a JS file without shell, which isn't supported. Updated now. Please take another look!


Please tag v1.x.x

Could you please release and tag a semver v1.x.x?

Major version zero (0.y.z) is for initial development. Anything may change at any time. The public API should not be considered stable.
Version 1.0.0 defines the public API. The way in which the version number is incremented after this release is dependent on this public API and how it changes.
— http://semver.org/

No meaningful relationship between v0.x.x releases can be inferred. There's no guarantee that v0.7.4 will be in any way compatible with v0.7.3.
This means library authors need to lock their dependencies to the patch-version, making interoperability painful. For example landrush.gemspec locks to rubydns 0.7.0 while vagrant-dns.gemspec locks to rubydns 0.6.x, so they can't both be loaded at once.
A v1.0.0 release doesn't need to be finished or perfect, it just defines the backwards-compatibility semantics moving forwards, and should be released pretty much as soon as the code is being used in the wild.
Thanks!

Nice, thanks again for your efforts @ioquatix.


typo?

In docs. https://docs.taxilang.org/language-reference/querying-with-taxiql/
| `<`    | Less than                |
| `<`    | Less than or equal to    |

Is that right? both are the same? should the second one not be:
| `<=`    | Less than or equal to    |

Great catch - thanks! Yes, that's a typo. I'll fix shortly.


Disable AMD switchable graphics device to avoid enumerating error

The problem is described in this link: vkEnumeratePhysicalDevices error.
Tested on the following three platforms:

only AMD graphics card
only NVIDIA graphics card
AMD CPU with graphics card and NVIDIA graphics card

Should we add "setenv("DISABLE_LAYER_AMD_SWITCHABLE_GRAPHICS_1", "1", 1)"?

I think this is fine here, because the specific graphics card selection is ordered in code based on priority and does not depend on external software settings. If there is a better solution to the problem you can post it here

Should we add "setenv("DISABLE_LAYER_AMD_SWITCHABLE_GRAPHICS_1", "1", 1)"?

I think this is fine here, because the specific graphics card selection is ordered in code based on priority and does not depend on external software settings. If there is a better solution to the problem you can post it here


Follow comments coding conventions

Replaced ':' at the end of comments with '.' to follow conventions mentioned  here

End comment text with a period.

@rpetrusha Can you look at this PR? The change does put this in compliance with our stated guidelines. The original seems better to me for these examples. Should the guidelines change?

I think that the guidelines are fine, @BillWagner. It's assumed that a comment applies to the following line or lines of code, so using a colon rather than a period doesn't seem to me to make any real difference. The ambiguity is really about how many lines of code a comment applies to, and I don't think the difference between a period-ending comment and a colon-ending comment is going to add much clarification.

I think the change looks good. Changing the convention would be a lot of work since most of our samples use periods to end comments.


Compiler error with swift 4.2

I installed the data cache via pod (Installing DataCache 1.4). It shows that it downloads 1.4 i.e latest version. I am trying to compiler it but it gives error in datacache.swift file

Please update your project to Swift version 4.2, or use previous version of DataCache


Daisy: Remove disk.IsWindows

We added disk.IsWindows in #935 to assist feature development in the CLI tool. The isWindows param is no longer required since we handle it manually now:
https://github.com/GoogleCloudPlatform/compute-image-tools/blob/7964ff4bbecbb5c6e253be250d8083821f9a194a/cli_tools/common/image/importer/daisy_inflater.go#L121

I didn't notice this " addFeatureToDisk(wf, "WINDOWS", inflationDiskIndex) " - is there anythig corresponding fro apiInflator?

@dntczdx , the API inflater has the following:
After the disk is inflated, there's a step that conditionally adds UEFI_COMPATIBLE and/or WINDOWS, depending on the results of inspection, and the guest_os_features that are currently on the disk:
https://github.com/GoogleCloudPlatform/compute-image-tools/blob/1bd9907d6bf176cfb789898ff001e3e149a5ab74/cli_tools/common/image/importer/process_planner.go#L107
https://github.com/GoogleCloudPlatform/compute-image-tools/blob/f8c570e578c0359d667da21475b4d9ae6ab300ba/cli_tools/common/image/importer/processor.go#L63


MetricCollection just recomputes the first metric after first .compute call if compute_groups is enabled

🐛 Bug
I think the MetricCollection was designed to be used as follows:
metrics = MetricCollection([...])
for _ in range(...):
    metrics.update(...)
metrics.compute()
metrics.reset()

But I used it the following way:
metrics = MetricCollection([...])
for _ in range(...):
    metrics.update(...)
    metrics.compute()
metrics.reset()

This does not work. After the first compute call only the first metric in the metric collection will return a new value. This is because after calling .compute of all metrics in the metric collection the respective ._computed property is set to a value. If the metrics are in the same compute_group then the next .update call of the metric collection will invoke the .update method only for the first metric. However it is exactly this method that sets ._computed to None again. For the other metrics it is still not None and thus will return the cached value of the first computation the whole time.
To Reproduce
from torch import Tensor
import numpy as np
from torchmetrics import MetricCollection
from torchmetrics.classification import BinaryPrecision, BinaryAccuracy, BinaryRecall, BinarySpecificity
# Generate random predictions and labels for a binary classification problem
np.random.seed(42)  # For reproducibility
n_samples = 100  # Number of samples

# Metrics calculation
metrics = MetricCollection([
    BinaryRecall(),
    BinaryAccuracy(),
    BinaryPrecision(),
    BinarySpecificity()
])

n_epochs = 10

for _ in range(n_epochs):
    # Generate random predictions (0 or 1)
    predictions = Tensor(np.random.randint(2, size=n_samples))

    # Generate random true labels (0 or 1)
    true_labels = Tensor(np.random.randint(2, size=n_samples))

    metrics.update(predictions, true_labels)
    print(metrics.compute())

Expected behavior
The compute method should return the new calculations for all metrics.
Environment

TorchMetrics 1.2.0, installed with pip
Python 3.11
PyTorch 2.1.0
MacOs

Additional context

MetricCollection just recomputes the first metric after first .compute call if compute_groups is enabled #2206


Bug#277 fix one unit test

Thank you, this has been merged into master.

Thank you, this has been merged into master.


Verify bootstrap success and instance health

Instances sometimes fail to bootstrap:

Start hooks can fail to download [1]
SSH public keys can fail to download

Instances sometimes become unhealthy in ways that aren't measured by our health checks:

Certain Docker commands hang forever [2]
Abusive jobs can hog CPU (to be addressed in https://github.com/travis-ci/worker/pull/366)

We need a way to ensure instance health at bootstrap and on an ongoing basis. I'd like to use this issue as a place to brainstorm on design. (If a similar issue already exists somewhere, please point me to it!)
I think if I needed such a check on a bunch of my own servers, I'd use an approach like the following:

Create a /tmp/health directory
make the cloud init script write results to this directory, e.g. /tmp/health/cloud-init.ok if everything completed successfully, /tmp/health/cloud-init.nok if any errors were encountered
Use a cron job to occasionally check the status of required services (docker, travis-worker) and take appropriate action (e.g. restarting Docker, imploding the instance)

One problem: The only way I know to confirm that docker isn't working as expected is to try a command, e.g. docker ps, and observe that it just hangs forever. I'm not sure how to check this in a script without making the script hang forever, too. Maybe we could:

run docker ps&, wait a few seconds, then check if a process with that PID is still running?
check the modification date on docker log file?

Thoughts?

I would love to add another layer of health monitoring.  Based on what you're writing above, I believe this would be specific to the instances brought up in autoscaling groups on EC2, and that we would add such a layer of health monitoring at the terraform-config level.
With regard to the specific case of docker ps not coming back, I am in favor of treating a timeout as a failure condition and imploding the host.  I think it's great if we can do this with bash, but I'm also happy to use a different programming language that's already present on the system such as python.

I think that some of the other health checks mentioned could be integrated into the worker "prestart hook" script, too, so that workers that are unhealthy never come into service: https://github.com/travis-infrastructure/terraform-config/blob/master/modules/aws_asg/prestart-hook.bash

As a temporary workaround to locate dead workers, here's a script I'm using:
#!/bin/bash
# Usage:
# get instance ips: $HOME/git/travis/bin/private-ec2-ips.sh > ips.txt
# Then, from a bastion:
# parallel-scp -h ips.txt check-health.sh /tmp/check-health.sh
# parallel-ssh -x '-tt' -O RequestTTY=force -h ips.txt -o outdir -e errdir bash -c /tmp/check-health.sh
# grep NOK outdir/*

# In some cases, the services can be successfully restarted via SSH to the instance:
# service travis-worker restart
# sudo restart docker

my_ip="$(curl -s http://<IP_ADDRESS>/latest/meta-data/local-ipv4)"

sudo docker ps  > /dev/null 2>&1 &
sleep 3
jobs=$(jobs -l)

docker_ps_pid=$(echo "${jobs}" | grep -v Done | grep "sudo docker ps" | awk '{printf $2}')

if [ ! -z "${docker_ps_pid}" ]; then
    echo "[NOK] $my_ip 'docker ps' is stalled; 'docker ps' PID is ${docker_ps_pid}"
    exit 1
else
    echo " [OK] 'docker ps'"
fi

services="
    travis-worker
    docker
"

for service in $services; do
    type -a systemctl > /dev/null 2>&1
    if [ $? -eq 0 ]; then
        status_cmd="systemctl show $service --property=SubState --value"
        expected_result="running"
    else
        status_cmd="status $service"
        expected_result="running"
    fi

    service_status="$(eval $status_cmd)"
    service_ok=$(echo "${service_status}" | grep "$expected_result")

    if [ -z "${service_ok}" ]; then
        echo "[NOK] $my_ip $service. status is: ${service_status}"
        exit 1
    else
        echo " [OK] $service"
    fi
done


NavigationDuplicated on navigating to the same page with different params

Hi.
I have a page with '/user/2' URL. I want to change to a different user from the page with 'this.$router.push'
I tried
this.$router.push({ path: '/user/3' })
and
this.$router.push({ path: 'user', params: { userId: 3 } })
but both of them throw NavigationDuplicated error event though the params are defrrent

Hi.
I have a page with '/user/2' URL. I want to change to a different user from the page with 'this.$router.push'
I tried
this.$router.push({ path: '/user/3' })
and
this.$router.push({ path: 'user', params: { userId: 3 } })
but both of them throw NavigationDuplicated error event though the params are defrrent

Same issue . any update?

https://github.com/vuejs/vue-router/issues/2872#issuecomment-542555350


Allow not using either HOST_AUTH_METHOD or PASSWORD for initialisation

Hello everyone,
I suggest adding a option that would allow skipping of authentication method init to the image for people that extend based on this image. I have to currently disable docker_verify_minimum_env in docker-entrypoint.sh to add my own init scripts and generate my own hba file without anything else. It would be great if this could be something like POSTGRES_AUTH_INIT=disable which would skip these steps.

Yep, the entrypoint is designed to be source-able and thus composable, so skipping behavior it has that you don't want is going to be something we suggest writing your own entrypoint for (sourcing ours and adjusting the invoked functions, as seen in #1150).


BasicTextField: The cursor jumps to the previous line when typing at the beginning of a empty line

compose.version=1.1.0-alpha04

The cursor jumps to the previous line when typing at the beginning of a blank line. See demo:
https://user-images.githubusercontent.com/5214214/153717861-4f121afd-1f3e-42e2-9ba7-70aee803262d.mp4
Steps to reproduce:

Input the text into the BasicTextField:

Line 1
Line 3

Put the cursor after "Line 1" and press enter to get a new empty line
Switch input method to Chinese input mode
Enter some Chinese content
Delete all Chinese content so that the current line is empty
At this point, keep the Chinese input mode, and enter new Chinese, when the keyboard key is pressed, the cursor will jump to the previous line, and our empty line is automatically deleted.

Seems to be the same issue as #672.

TODO: recheck this bug after https://github.com/JetBrains/androidx/pull/193 is merged.

Can be reproduced even easier:
Input English test, switch to Chinese, press enter, start typing. Cursor will jump back.

still happen with compose.version=1.1.0-alpha05

@VeselovAlex It looks like this bug has been fixed on v1.1.0-rc01. So I am closing this issue...

Good job! Thank you so much @VeselovAlex


[publisher] API Product search bar

When API product view is selected, search box  placeholder in the API product appears same as search APIs.

Placeholder of the API should be changed as Search Products or Search API products.

We can't search API Products when routing the API Product tab. But we have to make proper placeholders here. you can use  'type:APIPRODUCT' property search to search API products at here


AMBARI-24113. Actions for components adding are absent at the Druid s…

…ervice page
What changes were proposed in this pull request?
Actions for components adding are absent at the Druid service page
How was this patch tested?
Manually
Please review Ambari Contributing Guide before opening a pull request.

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/Ambari-Github-PullRequest-Builder/2727/
Test FAILed.
Test FAILured.


bump dependencies

This is creating version conflicts in pscid because purescript-arrays: "~1.0.0" doesn't resolve to "1.1.0".

Could you release this as a patch version? Trying to keep pscid's dependencies green. Thanks!

I'm on my phone atm. I believe you have commit access, so you are more than welcome to make a release if you feel comfortable doing so.

Seems to have worked :) Thanks! :+1:


Prevents SQL Error when either subtract_stock field is NULL

This might only affect me?
Upgrading a development env kept failing due to $menuItem->subtract_stock and $optionValue->subtract_stock both being nullable, while is_tracked is not, throwing an error
SQLSTATE[23000]: Integrity constraint violation: 1048 Column 'is_tracked' cannot be null (SQL: insert into `ti_stocks` (`location_id`, `stockable_id`, `stockable_type`, `quantity`, `is_tracked`, `created_at`, `updated_at`) values (2, 4, menu_option_values, 10, ?, 2022-03-08 14:07:37, 2022-03-08 14:07:37))

though it's possible that in the regular course of operation, these fields won't ever actually be null.

Thanks for the PR, can you create this on the 3.x branch instead?

Yep, recreated at https://github.com/tastyigniter/TastyIgniter/pull/952


[5.4] Fix bad method name in FilesystemServiceProvider

This PR fixes a bad method name (typo) in FilesystemServiceProvider class.

This is not a typo - it's the name of the underlying PHP library that is used:
https://flysystem.thephpleague.com/

Oh, my bad. Sorry :)


⚠ Fortnite has degraded performance

In 6f2018d, Fortnite (https://fortnite.com) experienced degraded performance:

HTTP code: 200
Response time: 1621 ms

Resolved: Fortnite performance has improved in 33c8248.


a bit of numba to be faster in numpy

a bit of numba to be faster in numpy

This change is

Weird because when using codecov function are now considered as not covered


There was a problem in sending this request. Please try again 403.

Hi, it may be my fault but every time the server starts after a while I get a notification back from the script I included in the title. Is this a common error or is it my fault? Any advice on how to fix this?
Greetings

Hi there,
Thank you for reaching out to us regarding this issue. Unfortunately, this issue happened some time ago, and it's not feasible for us to investigate it now, as we don't have enough information to replicate the issue.
If you encounter a similar issue again, please feel free to create a new GitHub ticket and provide us with as much detail as possible so we can investigate it.
Thank you for your understanding, and we apologise for any inconvenience this issue may have caused.
Best regards,
The Integrations Team.


add large cursor support, posix shm

Jay - 32-bit CI is fixed (#2674).

Apologies - I've just merged a conflicting change to os_calls.c


Derive Clone for ObjectStore builders and Make URL Parsing Stricter (#3419)

Which issue does this PR close?

Closes #3419
Rationale for this change

What changes are included in this PR?

Are there any user-facing changes?
No, this does not making any breaking changes to released APIs

FYI @roeap perhaps you would be able to review this :pray:

Many thanks! I have been looking into the verde support. The structs re-exported from http::headers do not work out of the box but I am close, I think. Will submit a PR after this one, unless @tustvold already started on it.


Add missing engine f/w for CAR.RAV4H_TSS2

Togo#7636  https://discord.com/channels/469524606043160576/524327905937850394/782969064989392896
[also fixed some alpha-sorting]

68658f44486a2b0b|2020-08-06--06-47-49 has the fw versions we need

DongleID/route 68658f44486a2b0b|2020-08-06--06-47-49