thek0tyara/llama-cpp-turboquant
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 4

Merge pull request #256 from lopuhin/fix-python-examples

Browse source 
Fix first two python examples in the docs
This commit is contained in:
Alejandro Saucedo 2021-10-31 14:37:29 +00:00 • committed by GitHub
commit a73b33c0ef
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
1 changed files with 10 additions and 11 deletions
Download patch file Download diff file Expand all files Collapse all files

21
docs/overview/python-examples.rst
View file

@ -26,7 +26,7 @@ Then you can interact with it from your interpreter. Below is the same sample as
sq = mgr.sequence() sq = mgr.sequence()
sq.eval(OpTensorSyncLocal([tensor_in_a, tensor_in_b, tensor_out])) sq.eval(OpTensorSyncDevice([tensor_in_a, tensor_in_b, tensor_out]))
# Define the function via PyShader or directly as glsl string or spirv bytes # Define the function via PyShader or directly as glsl string or spirv bytes
@python2shader @python2shader
@ -37,11 +37,11 @@ Then you can interact with it from your interpreter. Below is the same sample as
i = index.x i = index.x
data3[i] = data1[i] * data2[i] data3[i] = data1[i] * data2[i]
algo = mgr.algorithm([tensor_in_a, tensor_in_b, tensor_out], compute_shader_multiply.to_spirv()) algo = mgr.algorithm([tensor_in_a, tensor_in_b, tensor_out], compute_shader_multiply.to_spirv())
# Run shader operation synchronously # Run shader operation synchronously
sq.eval(OpAlgoDispatch(algo)) sq.eval(OpAlgoDispatch(algo))
sq.eval(OpAlgoSyncLocal([tensor_out])) sq.eval(OpTensorSyncLocal([tensor_out]))
assert tensor_out.data().tolist() == [2.0, 4.0, 6.0] assert tensor_out.data().tolist() == [2.0, 4.0, 6.0]
@ -61,13 +61,12 @@ Similarly you can find the same extended example as above:
mgr = Manager(0, [2]) mgr = Manager(0, [2])
# Can be initialized with List[] or np.Array # Can be initialized with List[] or np.Array
tensor_in_a = Tensor([2, 2, 2]) tensor_in_a = mgr.tensor([2, 2, 2])
tensor_in_b = Tensor([1, 2, 3]) tensor_in_b = mgr.tensor([1, 2, 3])
tensor_out = Tensor([0, 0, 0]) tensor_out = mgr.tensor([0, 0, 0])
mgr.eval_tensor_create_def([tensor_in_a, tensor_in_b, tensor_out]) seq = mgr.sequence()
seq.eval(kp.OpTensorSyncDevice([tensor_in_a, tensor_in_b, tensor_out]))
seq = mgr.create_sequence("op")
# Define the function via PyShader or directly as glsl string or spirv bytes # Define the function via PyShader or directly as glsl string or spirv bytes
@python2shader @python2shader
@ -81,8 +80,8 @@ Similarly you can find the same extended example as above:
algo = mgr.algorithm([tensor_in_a, tensor_in_b, tensor_out], compute_shader_multiply.to_spirv()) algo = mgr.algorithm([tensor_in_a, tensor_in_b, tensor_out], compute_shader_multiply.to_spirv())
# Run shader operation asynchronously and then await # Run shader operation asynchronously and then await
mgr.eval_async(kp.OpAlgoDispatch(algo))) seq.eval_async(kp.OpAlgoDispatch(algo))
mgr.eval_await() seq.eval_await()
seq.record(kp.OpTensorSyncLocal([tensor_in_a])) seq.record(kp.OpTensorSyncLocal([tensor_in_a]))
seq.record(kp.OpTensorSyncLocal([tensor_in_b])) seq.record(kp.OpTensorSyncLocal([tensor_in_b]))