thek0tyara/llama-cpp-turboquant
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 4

Better tiling implementation, fixed tiling asserts

Browse source
This commit is contained in:
Corentin 2021-06-24 19:36:08 +09:00
parent 425380f1a1
commit 6f04eb9db2
2 changed files with 142 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

2
examples/python_naive_matmul/2_tiled_matmul.py
View file

@ -13,7 +13,7 @@ class MatMulOp:
max_workgroup_size = props['max_work_group_size']
if tile_size < 0:
tile_size = 1
while (2 * tile_size * tile_size <= max_workgroup_invocation
while (4 * tile_size * tile_size <= max_workgroup_invocation
and 2 * tile_size <= max_workgroup_size[0]
and 2 * tile_size <= max_workgroup_size[1]):
tile_size *= 2