thek0tyara/llama-cpp-turboquant
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 4

server: (router) add stop-timeout option (#18350)

Browse source 
* server: (router) add stop-timeout option

* also allow stop while loading

* add docs

* unload_lru: also wait for unload to complete
This commit is contained in:
Xuan-Son Nguyen 2025-12-24 23:47:49 +01:00 • committed by GitHub
parent 4cbafad4f0
commit f5acfb2ffa
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
5 changed files with 133 additions and 64 deletions
Download patch file Download diff file Expand all files Collapse all files

1
common/arg.h
View file

@ -10,6 +10,7 @@
// pseudo-env variable to identify preset-only arguments
#define COMMON_ARG_PRESET_LOAD_ON_STARTUP "__PRESET_LOAD_ON_STARTUP"
#define COMMON_ARG_PRESET_STOP_TIMEOUT "__PRESET_STOP_TIMEOUT"
//
// CLI argument parsing