thek0tyara/llama-cpp-turboquant
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 4

server: return_progress to also report 0% processing state (#18305)

Browse source
This commit is contained in:
Xuan-Son Nguyen 2025-12-23 21:49:05 +01:00 • committed by GitHub
parent 5b6c9bc0f3
commit 5ee4e43f26
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
2 changed files with 17 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

6
tools/server/server-context.cpp
View file

@ -2313,6 +2313,12 @@ private:
slot.n_prompt_tokens_processed = 0;
slot.prompt.tokens.keep_first(n_past);
// send initial 0% progress update if needed
// this is to signal the client that the request has started processing
if (slot.task->params.stream && slot.task->params.return_progress) {
send_partial_response(slot, {}, true);
}
}
if (!slot.can_split()) {