thek0tyara/llama-cpp-turboquant
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 4

llama : add --no-host to disable host buffers (#16310)

Browse source 
* implement --no-host to disable host buffer

* fix equal_mparams

* move no-host enumeration order together with other model params

---------

Co-authored-by: slaren <slarengh@gmail.com>
This commit is contained in:
Gadflyii 2025-10-06 12:55:53 -05:00 • committed by GitHub
parent c08002a198
commit 3df2244df4
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
6 changed files with 56 additions and 10 deletions
Download patch file Download diff file Expand all files Collapse all files

1
common/common.cpp
View file

@ -1133,6 +1133,7 @@ struct llama_model_params common_model_params_to_llama(common_params & params) {
mparams.use_mlock = params.use_mlock;
mparams.check_tensors = params.check_tensors;
mparams.use_extra_bufts = !params.no_extra_bufts;
mparams.no_host = params.no_host;
if (params.kv_overrides.empty()) {
mparams.kv_overrides = NULL;