thek0tyara/llama-cpp-turboquant
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 4

Added optensorSyncDevice and optensorcopy tests

Browse source
This commit is contained in:
Alejandro Saucedo 2020-09-06 12:52:45 +01:00
parent ec89fc6d56
commit 1b4e4b6b18
12 changed files with 398 additions and 116 deletions
Download patch file Download diff file Expand all files Collapse all files

8
src/include/kompute/operations/OpTensorSyncDevice.hpp
View file

@ -35,21 +35,23 @@ class OpTensorSyncDevice : public OpBase
~OpTensorSyncDevice() override;
/**
* Performs basic checks such as ensuring that there is at least one tensor provided, that they are initialized and that they are not of type TensorTpes::eStaging.
* Performs basic checks such as ensuring that there is at least one tensor provided, that they are initialized and that they are not of type TensorTpes::eStaging. For staging tensors in host memory, the map is performed during the init function.
*/
void init() override;
/**
* Records the copy commands from teh first tensor into all the other tensors provided. Also optionally records a barrier.
* For device tensors, it records the copy command to the device tensor from the temporary staging tensor.
*/
void record() override;
/**
* Copies the local vectors for all the tensors to sync the data with the gpu.
* Does not perform any further sync functions. Frees the staging tensors together with their respective memory.
*/
void postSubmit() override;
private:
// Never owned resources
std::vector<std::shared_ptr<Tensor>> mStagingTensors;
};
} // End namespace kp