thek0tyara/llama-cpp-turboquant
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 4

vocab : mark EOT token for Granite models (#16499)

Browse source 
* vocab : mark EOT token for Granite models

* sampling : fallback to EOS when EOT is not found
This commit is contained in:
Georgi Gerganov 2025-10-10 17:17:31 +03:00 • committed by GitHub
parent 68ee98ae18
commit 81086cd6a3
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
2 changed files with 6 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

5
src/llama-sampling.cpp
View file

@ -2541,8 +2541,13 @@ static void llama_sampler_infill_apply(struct llama_sampler * smpl, llama_token_
if (n_non_eog == 0) {
cur_p->size = 1;
cur_p->data[0].id = ctx->vocab->token_eot();
if (cur_p->data[0].id == LLAMA_TOKEN_NULL) {
cur_p->data[0].id = ctx->vocab->token_eos();
}
cur_p->data[0].logit = 1.0f;
GGML_ASSERT(cur_p->data[0].id != LLAMA_TOKEN_NULL);
return;
}