thek0tyara/llama-cpp-turboquant
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 4

scripts: benchmark for HTTP server throughput (#14668)

Browse source 
* scripts: benchmark for HTTP server throughput

* fix server connection reset
This commit is contained in:
Johannes Gäßler 2025-07-14 13:14:30 +02:00 • committed by GitHub
parent 0f4c6ec0f1
commit 494c5899cb
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
4 changed files with 218 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

2
tools/server/utils.hpp
View file

@ -11,6 +11,8 @@
// increase max payload length to allow use of larger context size
#define CPPHTTPLIB_FORM_URL_ENCODED_PAYLOAD_MAX_LENGTH 1048576
// increase backlog size to avoid connection resets for >> 1 slots
#define CPPHTTPLIB_LISTEN_BACKLOG 512
// disable Nagle's algorithm
#define CPPHTTPLIB_TCP_NODELAY true
#include <cpp-httplib/httplib.h>