thek0tyara/llama-cpp-turboquant
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 4

server : Cmdline arg -to changes http read timeout from current 600sec default (#18279)

Browse source 
* Prevent crash if TTFT >300sec, boosted to 90 days

* server : allow configurable HTTP timeouts for child models

* server : pass needed timeouts from params only

---------

Co-authored-by: Greg Slocum <fromgit@wbtek.slocum.net>
This commit is contained in:
wbtek 2025-12-30 01:12:48 +09:00 • committed by GitHub
parent 3595ae5963
commit 5b1248c9af
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
2 changed files with 14 additions and 3 deletions
Download patch file Download diff file Expand all files Collapse all files

5
tools/server/server-models.h
View file

@ -183,7 +183,10 @@ public:
const std::string & path,
const std::map<std::string, std::string> & headers,
const std::string & body,
const std::function<bool()> should_stop);
const std::function<bool()> should_stop,
int32_t timeout_read,
int32_t timeout_write
);
~server_http_proxy() {
if (cleanup) {
cleanup();