Loading in Float32 vs Float16 has very different speed

John6666
Thu, 20 Feb 2025 16:47:34 +0000

This is very strange. If you are using a GeForce from the 20x0 generation, it can be easily explained (it does not support bfloat16. If it is 1xx0, it does not support float16), but you are using an A100.

Perhaps your version of the CUDA Toolkit or PyTorch is very old?

Read full topic