The HTTP/3 tests did send 20 transfers against nghttpx with a backend
that failed the uploads with a 400 and an incomplete response body. This
causes stream resets.
Apache keeps the connection open, but newer nghttpx closes the front
connection after "too many" reset. When that bites, it depends on the
number of transfers ongoing how the test case fails. This led to flaky
outcomes.
Reduce the transfers to just a single one and check the result of
that one. Parallelism is not important here.
refs #19489Closes#19530
Sync outliers with the rest of the code.
Also:
- return error in some failed init cases, instead of `CURLE_OK`:
1908, 1910, 1913, 2082, 3010
- lib1541: delete unused struct member.
Closes#19515
Rename `Curl_timeleft()` to `Curl_timeleft_ms()` to make the units in
the returned `timediff_t` clear. (We used to always have ms there, but
with QUIC started to sometimes calc ns as well).
Rename some assigned vars without `_ms` suffix for clarity as well.
Closes#19486
Overhaul of the internal cookie APIs and an attempt to better return
errors for OOM and similar critical problems, separate from ordinary and
benign parsing problems.
Closes#19493
Some curl command-lines are long, often repetitive, and difficult
to read or write:
Before this patch (1 test == 1 line):
- <=78 characters: 1099 tests
- 79-132 characters: 500 tests
- 133+ characters: 217 tests: patch meant to help with some of these.
After this patch:
- <=78 characters: 1288 lines
- 79-132 characters: 526 lines
- 133+ characters: 190 lines
After this patch it's possible to fold long lines into multiple ones.
Folding can reduce greppability, thus this is primarily useful for cases
when the options are repetitive, e.g. a list of form options, headers,
mail parameters and the like.
Closes#19500
To untangle the different curl/server requirements of these tests.
Also to make this test run for non-H2 builds. Searching the cause of
the flakiness documented in #19481.
Also:
- fix the callback function prototype. Detected by ASAN with this patch,
though the issue was pre-existing.
```
lib/cw-out.c:211:14: runtime error: call to function emptyWriteFunc
through pointer to incorrect function type
'unsigned long (*)(char *, unsigned long, unsigned long, void *)'
tests/libtest/lib2405.c:72: note: emptyWriteFunc defined here
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior lib/cw-out.c:211:14
```
Ref: https://github.com/curl/curl/actions/runs/19296686908/job/55180334364?pr=19487#step:44:3768
Follow-up to 96a5ce5a82#19481Closes#19487
To better reject junk and detect overflows. There were already
additional precautions and protections in place, but this is cleaner.
Extended the 1614 unit tests with some more bad syntax cases.
Closes#19475
Also:
- replace `manpage` with `man page`, add to `badwords.txt`.
- badwords.pl: import `-w` feature from curl-www, syncing the two
scripts fully.
- badwords.txt: import missing items from curl-www, syncing the two
files fully.
- pyspelling.words: drop `cURL` allowed word.
Closes#19468
When asking for the last N bytes of a file, and that size was larger
than the file size, it would miss the first byte due to a logic error.
The fixed range parser is now made a common function in the file now
renamed to vssh.c (from curl_path.c) - used by both backends.
Unit test 2605 verifies the parser.
Reported-by: Stanislav Fort (Aisle Research)
Closes#19460
- allow 02_28 to run in HTTP/1.1 without H2 support again.
Follow-up to 3752de465d#19412
- fix to skip 02_28 for all protocols for curl without compression
support (either zlib, brotli or ztsd).
Closes#19458
Data files no longer depend on mixed newline styles. Before this
patch the harness still assumed data files to use LF newlines,
ensured by `.gitattribute` and distributing sources with LF newlines.
To allow using platform native newlines (CRLF on Windows typically),
update the test harness to support data files with any newline style
on disk. And delete `.gitattributes`.
Fix to:
- load original data files (from test/data) so that their newline-style
doesn't matter on the checked out source repo, meaning it works
when its CRLF on Windows, just like any other file.
(if a BOM slips in, it's caught by `spacecheck.pl` as binary content.)
- do the same in `util.py` used by `smbserver.py` (for test 1451).
- also fix `util.py` to use us-ascii encoding for data files, replacing utf-8.
Also:
- runtests: rework the stray CR checker to allow full CRLF data files,
and keep warning for mixed newlines.
Follow-up to 904e7ecb66#19347Closes#19398
To debug the Windows CI fails further. Acting on the suspicions that
`taskkill` may sometimes be applied to the wrong process.
- log task info, and task child info before calling `taskkill` on a PID.
(on native Windows.)
One of the calls needs PowerShell.
- add env `CURL_TEST_NO_TASKKILL` to disable using `taskkill`.
- add env `CURL_TEST_NO_TASKKILL_TREE` to use `taskkill` without
`-t`, meaning to kill the process, but not child processes.
- GHA/windows: disable `taskkill` calls, to see what happens.
I'll revert or tweak this in a future commit depending on results.
Ref: https://github.com/curl/curl/discussions/14854#discussioncomment-13062859
Ref: https://github.com/curl/curl/discussions/14854#discussioncomment-14913014Closes#19421
Before this patch servers were loading the original data source file
(from `tests/data/test*`) if they failed to open the preprocessed data
file.
It was causing issues in many (most?) tests, because original data files
are not preprocessed, thus may be incomplete and/or come with wrong
newline characters. It's also causing difficult to diagnose issues when
a test accidentally references another test's data, which by chance
makes the test pass initially, until either that or the executed test
data gets an update, and breaking it, as seen in #19329.
Historically, the fallback existed first, then the preprocessed copy.
The fallback is no longer used by tests (except by stray accidents).
Fix it by dropping the fallback logic and relying on the preprocessed
data file saved there by the runtests framework.
Also fix two remaining test data cross-references:
- test1565: reference own server input data instead of test1's.
- test3014: reference own server input data instead of test1439's.
Ref: #19398
Follow-up to aaf9522a2c#19329Closes#19429
To allow running pytests on more curl configurations.
Also delete a redundant H3 feature check from test_17_14_expired_cert.
Cherry-picked from #19407Closes#19412
A regression in curl 8.17.0 led to a customer CAPATH set by the
application (or the curl command) to be ignored unless licurl was built
with a default CAPATH.
Add test cases using `--capath` on the custom pytest CA, generated with
the help of the openssl command when available.
Fixes#19401
Reported-by: Brad King
Closes#19308
curl is for transfers so disabling all protocols has to be a mistake.
Previously it would allow this to get set (even if curl_easy_setopt()
returns an error for it) and then let libcurl return error instead.
Updated 1474 accordingly.
Closes#19388
There is no more mixed-newline file in the repository after this patch.
Except for`.bat` and `.sln` files (4 in total), all files use LF
newlines.
Also:
- `spacecheck.pl`: drop mixed-EOL exception for test data.
- runtests: add option `-w` to check if test data has stray CR bytes in
them.
- build: enable the option above in test targets, except the CI-specific
one where `spacecheck.pl` does this job already.
- tested OK (with expected failures) in CI with stray CRs added.
- cmake: enable option `-a` for the `tests` target. To continue testing
after a failed test.
Follow-up to 63e9721b63#19313
Follow-up to 6cf3d7b1b1#19318
Follow-up to 4d2a05d3fe#19284Closes#19347
Use the %VERSION instead. The user-agent stripping was introduced at the
time before we had %VERSION (introduced in e6b21d4). The tests would
then remove the user-agent header to make them possible to be compared
in a version independent way.
Fixes#19355
Reported-by: Stefan Eissing
Closes#19356
- `reply/data*`, `verify/stdout`, `verify/stderr`, `verify/file*`,
`verify/proxy`:
- make `crlf="yes"` force CRLF to all lines, instead of just applying
to HTTP protocol headers.
- add support for `crlf="headers"` that only converts HTTP protocol
header lines to CRLF. (previously done via `crlf="yes"`.)
- use `crlf="headers"` where possible.
- `reply/connect*`:
- add support for `crlf="yes"` and `crlf="headers"`.
- use them where possible.
- `client/file*`, `client/stdin`:
- add support for `crlf="yes"`.
- use it where possible.
- `reply/data*`, `verify/protocol`:
- replace existing uses of `crlf="yes"` with `crlf="headers`" where it
does not change the result.
Reducing the number of `tests/data/test*`:
- CRLF newlines from 10295 to 1985. (119985 lines total)
- files with mixed newlines from 656 to 113. (1890 files total)
After this patch there remain 141 sections with mixed newlines, where
the mixing is not split between headers/non-headers. There is no obvious
pattern here. Some of the CRLF uses might be accidental, or
non-significant. They will be tackled in a future patch.
Follow-up to 6cf3d7b1b1#19318
Follow-up to 4d2a05d3fe#19284Closes#19313
The test FTP server returned LF newlines for 227/229 replies, instead of
the CRLF used for the rest.
Test data added later were explicitly made to expect an LF in these
response lines.
After this patch the FTP server returns CRLF newlines, allowing
to delete this special case in test data.
Follow-up to 3bfff57e1f
Follow-up to a7937ed49cCloses#19330
Test 696 and 556 share the same libtest code. Make sure to issue
the `GET` request to the correct runtime test number instead of using
the hard-wired "556".
It makes the `sws` test server read the response string from `test696`
`<data>` section, instead of reading it from `test556`. To avoid this
hidden interaction between test data.
AFAICS there is no other similar hard-coded string in reused libtests.
Ref: https://github.com/curl/curl/pull/19313#issuecomment-3477448933
Follow-up to be82a3605a#16003Closes#19329
- fix regex to not catch CR (from CRLF), in `PORT`, `EPRT`
commands, allowing to use `crlf="yes"` more.
- add `crlf="headers"` mode for `protocol` sections.
To call `subnewlines()` without its force option.
This is the mode used in `data` sections when `crlf="yes"`.
(This confusion may be subject to a future commit.)
- subnewlines: apply CRLF to `HEAD` and `CONNECT` HTTP requests.
- subnewlines: apply CRLF to RTSP requests.
- delete remaining empty `protocol` sections.
Reducing the number of `tests/data/test*`:
- CRLF newlines from 11325 to 10295. (119984 lines total)
- files with mixed newlines from 707 to 656. (1890 files total)
Follow-up to 4d2a05d3fe#19284Closes#19318
