thek0tyara/llama-cpp-turboquant
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 4

server : do not default to multiple slots with speculative decoding (#17017)

Browse source 
* server : do not default to multiple slots with speculative decoding

* cont : fix
This commit is contained in:
Georgi Gerganov 2025-11-05 14:32:55 +02:00 • committed by GitHub
parent 2f0c2db43e
commit 13b339bcd9
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
2 changed files with 14 additions and 4 deletions
Download patch file Download diff file Expand all files Collapse all files

4
common/common.h
View file

@ -507,6 +507,10 @@ struct common_params {
// return false from callback to abort model loading or true to continue
llama_progress_callback load_progress_callback = NULL;
void * load_progress_callback_user_data = NULL;
bool has_speculative() const {
return !speculative.model.path.empty() || !speculative.model.hf_repo.empty();
}
};
// call once at the start of a program if it uses libcommon