thek0tyara/llama-cpp-turboquant
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 4

Merge pull request #154 from EthicalML/glslang_implementation

Browse source 
Glslang implementation for online shader compilation
This commit is contained in:
Alejandro Saucedo 2021-02-21 10:14:50 +00:00 • committed by GitHub
commit cb79948bb5
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
53 changed files with 3722 additions and 3462 deletions
Download patch file Download diff file Expand all files Collapse all files

10
docs/index.rst
View file

@ -15,12 +15,12 @@ Documentation Index (as per sidebar)
:caption: C++ Documentation:
C++ Examples <overview/advanced-examples>
C++ Memory Management Principles <overview/memory-management>
C++ Build System Deep Dive <overview/build-system>
C++ Converting GLSL/HLSL Shaders to Cpp Headers <overview/shaders-to-headers>
C++ Extending Kompute with Custom Operations <overview/custom-operations>
Memory Management Principles <overview/memory-management>
Build System Deep Dive <overview/build-system>
Processing Shaders (Online & Offline) <overview/shaders-to-headers>
Extending Kompute with Custom Operations <overview/custom-operations>
C++ Class Documentation & Reference <overview/reference>
C++ Code Coverage <https://kompute.cc/codecov/>
Test Code Coverage <https://kompute.cc/codecov/>
.. toctree::
:titlesonly:

2
docs/overview/async-parallel.rst
View file

@ -27,7 +27,7 @@ This basically provides further granularity on Vulkan Fences, which is its means
It is important that submitting tasks asynchronously, does not mean that these will be executed in parallel. Parallel execution of operations will be covered in the following section.
Asynchronous operation submission can be achieved through the kp::Manager, or directly through the kp::Sequence. Below is an example using the Kompute manager.
Asynchronous operation submission can be achieved through the :class:`kp::Manager`, or directly through the :class:`kp::Sequence`. Below is an example using the Kompute manager.
Conceptual Overview
^^^^^^^^^^^^^^^^^^^^^

2
docs/overview/build-system.rst
View file

@ -65,6 +65,8 @@ Compile Flags
- Enable debug build including debug flags (enabled by cmake debug build)
* - -DKOMPUTE_DISABLE_VK_DEBUG_LAYERS
- Disable the debug Vulkan layers, mainly used for android builds
* - -DKOMPUTE_DISABLE_SHADER_UTILS
- Disable the shader utils and skip adding glslang as dependency
Dependencies

2
docs/overview/custom-operations.rst
View file

@ -11,7 +11,7 @@ These nuances are important for more advanced users of Kompute, as this will pro
Flow of Function Calls
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The top level operation which all operations inherit from is the `kp::OpBase` class. Some of the "Core Native Operations" like `kp::OpTensorCopy`, `kp::OpTensorCreate`, etc all inherit from the base operation class.
The top level operation which all operations inherit from is the :class:`kp::OpBase` class. Some of the "Core Native Operations" like :class:`kp::OpTensorCopy`, :class:`kp::OpTensorCreate`, etc all inherit from the base operation class.
The `kp::OpAlgoBase` is another base operation that is specifically built to enable users to create their own operations that contain custom shader logic (i.e. requiring Vulkan Compute Pipelines, DescriptorSets, etc). The next section contains an example which shows how to extend the OpAlgoBase class.

2
docs/overview/memory-management.rst
View file

@ -21,7 +21,7 @@ The memory ownership is hierarchically outlined in the component architecture -
Optional Memory Management
-------------
As outlined above, resource memory is only managed by Kompute if the resources are created by Kompute. Each of the Kompute components can also be initialised with externally managed resources. The kp::Manager for example can be initialized with an external Vulkan Device. The first principle ensures that all memory ownership is explicitly defined when managing and creating Kompute resources.
As outlined above, resource memory is only managed by Kompute if the resources are created by Kompute. Each of the Kompute components can also be initialised with externally managed resources. The :class:`kp::Manager` for example can be initialized with an external Vulkan Device. The first principle ensures that all memory ownership is explicitly defined when managing and creating Kompute resources.

26
docs/overview/reference.rst
View file

@ -12,7 +12,7 @@ Below is a diagram that provides insights on the relationship between Vulkan Kom
Manager
-------
The Kompute Manager provides a high level interface to simplify interaction with underlying kp::Sequences of kp::Operations.
The Kompute Manager provides a high level interface to simplify interaction with underlying :class:`kp::Sequences` of :class:`kp::Operations`.
.. image:: ../images/kompute-vulkan-architecture-manager.jpg
:width: 100%
@ -23,7 +23,7 @@ The Kompute Manager provides a high level interface to simplify interaction with
Sequence
-------
The Kompute Sequence consists of batches of kp::Operations, which are executed on a respective GPU queue. The execution of sequences can be synchronous or asynchronous, and it can be coordinated through its respective vk::Fence.
The Kompute Sequence consists of batches of :class:`kp::Operations`, which are executed on a respective GPU queue. The execution of sequences can be synchronous or asynchronous, and it can be coordinated through its respective vk::Fence.
.. image:: ../images/kompute-vulkan-architecture-sequence.jpg
:width: 100%
@ -34,7 +34,7 @@ The Kompute Sequence consists of batches of kp::Operations, which are executed o
Tensor
-------
The kp::Tensor is the atomic unit in Kompute, and it is used primarily for handling Host and GPU Device data.
The :class:`kp::Tensor` is the atomic unit in Kompute, and it is used primarily for handling Host and GPU Device data.
.. image:: ../images/kompute-vulkan-architecture-tensor.jpg
:width: 100%
@ -45,7 +45,7 @@ The kp::Tensor is the atomic unit in Kompute, and it is used primarily for handl
Algorithm
-------
The kp::Algorithm consists primarily of the components required for shader code execution, including the relevant vk::DescriptorSet relatedresources as well as vk::Pipeline and all the relevant Vulkan resources as outlined in the architectural diagram.
The :class:`kp::Algorithm` consists primarily of the components required for shader code execution, including the relevant vk::DescriptorSet relatedresources as well as vk::Pipeline and all the relevant Vulkan resources as outlined in the architectural diagram.
.. image:: ../images/kompute-vulkan-architecture-algorithm.jpg
:width: 100%
@ -56,7 +56,7 @@ The kp::Algorithm consists primarily of the components required for shader code
OpBase
-------
The kp::OpBase provides a top level class for an operation in Kompute, which is the step that is executed on a GPU submission. The Kompute operations can consist of one or more kp::Tensor.
The :class:`kp::OpBase` provides a top level class for an operation in Kompute, which is the step that is executed on a GPU submission. The Kompute operations can consist of one or more :class:`kp::Tensor`.
.. image:: ../images/kompute-vulkan-architecture-operations.jpg
:width: 100%
@ -78,7 +78,7 @@ The vk::OpAlgoBase extends the vk::OpBase class, and provides the base for shade
OpMult
-------
The kp::OpMult operation is a sample implementation of the kp::OpAlgoBase class. This class shows how it is possible to create a custom vk::OpAlgoBase that can compile as part of the binary. The kp::OpMult operation uses the shader-to-cpp-header-file script to convert the script into cpp header files.
The :class:`kp::OpMult` operation is a sample implementation of the :class:`kp::OpAlgoBase` class. This class shows how it is possible to create a custom vk::OpAlgoBase that can compile as part of the binary. The :class:`kp::OpMult` operation uses the shader-to-cpp-header-file script to convert the script into cpp header files.
.. image:: ../images/kompute-vulkan-architecture-opmult.jpg
:width: 100%
@ -90,7 +90,7 @@ The kp::OpMult operation is a sample implementation of the kp::OpAlgoBase class.
OpTensorCopy
-------
The kp::OpTensorCopy is a tensor only operation that copies the GPU memory buffer data from one kp::Tensor to one or more subsequent tensors.
The :class:`kp::OpTensorCopy` is a tensor only operation that copies the GPU memory buffer data from one :class:`kp::Tensor` to one or more subsequent tensors.
.. doxygenclass:: kp::OpTensorCopy
:members:
@ -98,7 +98,7 @@ The kp::OpTensorCopy is a tensor only operation that copies the GPU memory buffe
OpTensorSyncLocal
-------
The kp::OpTensorSyncLocal is a tensor only operation that maps the data from the GPU device memory into the local host vector.
The :class:`kp::OpTensorSyncLocal` is a tensor only operation that maps the data from the GPU device memory into the local host vector.
.. doxygenclass:: kp::OpTensorSyncLocal
:members:
@ -106,11 +106,19 @@ The kp::OpTensorSyncLocal is a tensor only operation that maps the data from the
OpTensorSyncDevice
-------
The kp::OpTensorSyncDevice is a tensor only operation that maps the data from the local host vector into the GPU device memory.
The :class:`kp::OpTensorSyncDevice` is a tensor only operation that maps the data from the local host vector into the GPU device memory.
.. doxygenclass:: kp::OpTensorSyncDevice
:members:
Shader
--------
The :class:`kp::Shader` class contains a set of utilities to compile and process shaders.
.. doxygenclass:: kp::Shader
:members:

23
docs/overview/shaders-to-headers.rst
View file

@ -1,9 +1,28 @@
Converting Shaders to C++ Headers
Processing Shaders with Kompute
=====================
Kompute allows for shaders to be loaded directly through the kp::OpAlgoBase as either raw strings (through shaderc) or compiled SPIRV bytes. For this latter, the traditional method of including the SPIRV bytes is by loading the SPIRV file directly and passing the contents.
Kompute allows for two main ways of interacting with shaders - namely:
* Integration with [glslang](https://github.com/KhronosGroup/glslang) for online/runtime shader compilation
* A CLI that coverts shaders into C++ header files
Processing Shaders Online via Kompute Shader Utils
---------------
Kompute provides a set of helper functions that expose the C++ functionality of the glslang Khronos framework to process shader sources online during runtime.
It's worth emphasising that the suggested approach is to process shaders offline, so the section below is suggested to convert shaders to either their respective SPV format, or convert them into C++ sources that would be embedded as part of the resulting binary.
The Shader utility function can be skipped on build time through compiler flags - for more information on this you should read the `build section <build-system.rst>`_.
More details on the shader utils can be found in the :class:`kp::Shader` section of the `C++ reference page <reference.rst>`_.
Converting Shaders into C / C++ Header Files
----------------------------------
Kompute allows for shaders to be loaded directly through the :class:`kp::OpAlgoBase` as either raw strings (through shaderc) or compiled SPIRV bytes. For this latter, the traditional method of including the SPIRV bytes is by loading the SPIRV file directly and passing the contents.
The Kompute codebase has a utility that allows you to convert shader files into C++ header files containing the SPIRV header data. This is useful as it enables developers to compile the SPIRV shaders into the final binary, which avoids the need for multiple files being required.