thek0tyara/llama-cpp-turboquant
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 4

common : inhibit lazy grammar sampler while reasoning is active (#20970)

Browse source 
* common : inhibit grammar while reasoning budget is active

* cont : update force_pos in accept

* cont : fix tests

* cont : tweak should apply logic

* cont : return early not using grammar sampler

* Add tests

* cont : prevent backend sampling when reasoning budget enabled

* cont : fix typo

---------

Co-authored-by: Piotr Wilkin <piotr.wilkin@syndatis.com>
This commit is contained in:
Aldehir Rojas 2026-03-27 12:30:40 -05:00 • committed by GitHub
parent ff934e29bc
commit 59d840209a
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
8 changed files with 295 additions and 106 deletions
Download patch file Download diff file Expand all files Collapse all files

2
tools/server/server-common.cpp
View file

@ -1110,7 +1110,7 @@ json oaicompat_chat_params_parse(
reasoning_budget = json_value(body, "thinking_budget_tokens", -1);
}
if (reasoning_budget >= 0 && !chat_params.thinking_end_tag.empty()) {
if (!chat_params.thinking_end_tag.empty()) {
llama_params["reasoning_budget_tokens"] = reasoning_budget;
llama_params["reasoning_budget_start_tag"] = chat_params.thinking_start_tag;
llama_params["reasoning_budget_end_tag"] = chat_params.thinking_end_tag;