thek0tyara/llama-cpp-turboquant
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 4

server : pre-calculate EOG logit biases (#14721)

Browse source 
ggml-ci
This commit is contained in:
Georgi Gerganov 2025-07-16 14:04:12 +03:00 • committed by GitHub
parent e4841d24d3
commit 6ffd4e9c44
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
3 changed files with 17 additions and 15 deletions
Download patch file Download diff file Expand all files Collapse all files

3
common/common.h
View file

@ -177,7 +177,8 @@ struct common_params_sampling {
std::vector<common_grammar_trigger> grammar_triggers; // optional triggers (for lazy grammars)
std::set<llama_token> preserved_tokens;
std::vector<llama_logit_bias> logit_bias; // logit biases to apply
std::vector<llama_logit_bias> logit_bias; // logit biases to apply
std::vector<llama_logit_bias> logit_bias_eog; // pre-calculated logit biases for EOG tokens
// print the parameters into a string
std::string print() const;