runtests: add options to set minimum number of tests, use them

To detect mistakes made in the runtests framework that reduce
the number of test runs. Before this patch it could go undetected with
a green CI.

The minimum thresholds will need light maintenance going forward (either
bumping them periodically, or adjust if some may fell below minimums for
justified reasons). We may also make minimums tighter or looser, or more
job-specific.

Latest number of test runs for each job can be seen at Test Clutch:
https://testclutch.curl.se/static/reports/feature-matrix.html

Also:
- GHA: set minimums.

Assisted-by: Dan Fandrich

Follow-up to f2a75a14dd
Follow-up to bb1391f943 #19510

Closes #19942
This commit is contained in:
Viktor Szakats 2025-12-12 01:55:30 +01:00
parent eb39fee40b
commit 3f1cd809ee
No known key found for this signature in database
GPG key ID: B5ABD165E2AEF201
9 changed files with 66 additions and 22 deletions

View file

@ -52,6 +52,7 @@ permissions: {}
env:
MAKEFLAGS: -j 5
CURL_CI: github
CURL_TEST_MIN: 1500
DEBIAN_FRONTEND: noninteractive
jobs: