llama-cpp-turboquant/ggml/src
Salvatore Mesoraca cb00020504
vulkan : mul_mat: fix UB with small warps (ggml/952)
When the device's warp size is less than 16,
it is possible for loadstride_a (mul_mm.comp:114)
and loadstride_b (mul_mm.comp:115) to be set to 0.
Because they are calculated as: the workgroup size,
multiplied by LOAD_VEC_* (which can be 1) and divided by 16.
And the workgroup size is set to be the same as the
warp/subgroup size.

The loadstride_* variables are used as increments in the
loops that populate the buffers used for the multiplication.

When they are 0 they cause an infinite loop.
But infinite loops without side-effects are UB and the
values of loadstride_* are known at compile time.
So, the compiler quietly optimizes all the loops away.
As a consequence, the buffers are not populated and
the multiplication result is just a matrix with all elements
set to 0.

We prevent the UB by making sure that the workgroup size
will never be less than 16, even if our device has a
smaller warp size (e.g. 8).

Signed-off-by: Salvatore Mesoraca <s.mesoraca16@gmail.com>
2024-10-01 16:07:39 +03:00
..
ggml-cann cann: fix crash when llama-bench is running on multiple cann devices (#9627) 2024-09-25 11:30:38 +08:00
ggml-cuda CUDA: remove bad assert (ggml/972) 2024-09-29 21:15:37 +03:00
ggml-sycl Revert "[SYCL] fallback mmvq (#9088)" (#9579) 2024-09-23 11:28:06 +08:00
kompute@4565194ed7 llama : reorganize source code + improve CMake (#8006) 2024-06-26 18:33:02 +03:00
kompute-shaders ggml : move rope type enum to ggml.h (#8949) 2024-08-13 21:13:15 +02:00
llamafile ggml : move common CPU backend impl to new header (#9509) 2024-09-16 16:22:07 +02:00
vulkan-shaders vulkan : argsort barriers must be under uniform control flow (ggml/951) 2024-09-29 21:15:37 +03:00
CMakeLists.txt ggml : add AVX512DQ requirement for AVX512 builds (#9622) 2024-09-24 11:03:21 +03:00
ggml-aarch64.c ggml : add run-time detection of neon, i8mm and sve (#9331) 2024-09-28 15:06:16 +03:00
ggml-aarch64.h ggml : minor naming changes (#8433) 2024-07-12 10:46:02 +03:00
ggml-alloc.c ggml-alloc : fix list of allocated tensors with GGML_ALLOCATOR_DEBUG (#9573) 2024-09-21 14:24:23 +02:00
ggml-backend-impl.h ggml/examples: add backend support for numerical optimization (ggml/949) 2024-09-20 21:15:05 +03:00
ggml-backend.c ggml : fix trailing whitespace (#0) 2024-09-20 21:15:05 +03:00
ggml-blas.cpp ggml : hide ggml_object, ggml_cgraph, ggml_hash_set (#9408) 2024-09-12 14:23:49 +03:00
ggml-cann.cpp ggml/examples: add backend support for numerical optimization (ggml/949) 2024-09-20 21:15:05 +03:00
ggml-common.h ggml-quants : ternary packing for TriLMs and BitNet b1.58 (#8151) 2024-09-05 21:48:47 -04:00
ggml-cpu-impl.h ggml : move common CPU backend impl to new header (#9509) 2024-09-16 16:22:07 +02:00
ggml-cuda.cu mtgpu: enable VMM (#9597) 2024-09-26 03:27:40 +02:00
ggml-impl.h ggml : move common CPU backend impl to new header (#9509) 2024-09-16 16:22:07 +02:00
ggml-kompute.cpp ggml/examples: add backend support for numerical optimization (ggml/949) 2024-09-20 21:15:05 +03:00
ggml-metal.m metal : reduce command encoding overhead (#9698) 2024-10-01 16:00:25 +03:00
ggml-metal.metal metal : use F32 prec for K*Q in vec FA (#9595) 2024-09-23 11:27:47 +03:00
ggml-quants.c ggml : add run-time detection of neon, i8mm and sve (#9331) 2024-09-28 15:06:16 +03:00
ggml-quants.h ggml : add run-time detection of neon, i8mm and sve (#9331) 2024-09-28 15:06:16 +03:00
ggml-rpc.cpp ggml/examples: add backend support for numerical optimization (ggml/949) 2024-09-20 21:15:05 +03:00
ggml-sycl.cpp Revert "[SYCL] fallback mmvq (#9088)" (#9579) 2024-09-23 11:28:06 +08:00
ggml-vulkan.cpp vulkan : mul_mat: fix UB with small warps (ggml/952) 2024-10-01 16:07:39 +03:00
ggml.c ggml : fix ggml_cast (ggml/973) 2024-10-01 16:07:39 +03:00