Since a16485a42e, the test servers build with a different set of
options than the tool/lib - for example a different CURLDEBUG. To make
buildinfo better reflect the curl build, move it to src/ and build it
here using the local CURLDEBUG. Renamed to curlinfo to not get confused
with buildinfo.txt
I chose src/ and not lib/ because the file also uses tool-specific headers.
Assisted-by: Viktor Szakats
Closes#17187
This tool now contains ON/OFF information about features in the build.
This way, runtests gets both positive and negative feature presence with
this. Allows for more flexibility and avoids having to duplicate the
names.
Closes#17180
Add `--retry=<num>` option to tell runtests to retry the first `<num>`
tests that failed. Retries aren't run right away, but added to the end
of the test queue. Once all retry slots are used, test fail as normal.
In CI, typically a single test fails for flakiness, and rarely over 5.
Make the `ci-test` targets default to `--retry=5`.
Closes#17091
- test2088 verifies that mutual tls works
This adds a new certificate to generate which has the clientAuth key
usage enabled, and uses it to connect to a https-mtls server.
Closes#16923
Before this patch, autotools disabled building unit tests for
non-debug-enabled (`DEBUGBUILD`) builds. runtests skipped running this
combination, though they were built in cmake builds. There seems to be
no technical reason anymore for these restrictions. This patch removes
them, allowing to build and run unit tests for non-debug-enabled builds.
To improve unit test build and run coverage.
- autotools: do not disable building unit tests for non-debug-enabled
build. Bringing behavior closer to cmake builds. (There are still
exceptions in autotools, something for another PR)
- runtests: run unit tests for non-debug-enabled builds.
It extends coverage by 50 tests.
- `lib/altsvc.c`: fix to include `CURL_TIME` support in libcurlu, for
unit tests. It fixes test 1654, and syncs `CURL_TIME` behavior with
test 1660 and `lib/hsts.c`.
Ref: 10a7d05be3
Ref: fc8e0dee30#13694
Ref: 99f78cbf6e#16770Closes#16771
To run curl, tests and servers via `wine`:
```shell
export CURL_TEST_EXE_RUNNER=wine
```
runtests prefixes commands with the specified runner. For systems where
this isn't automatic or supported, e.g. macOS.
Closes#16785
In the cases observed throughout the last year, `handle64` run once per
test run, but with no action (match or task kill). It did not help with
flakiness and seems redundant.
runtests launched it (if present) in Cygwin/MSYS jobs too, where it
probably shouldn't have, because we have seen no flakiness there. In CI
the tool was present and launched in MSYS2 jobs, but not in Cygwin.
After this patch the "clearlocks" warning remain in the log. They are
consistently appearing once in every MSVC CI log, early in the tests:
```
test 3207 SKIPPED: curl lacks OpenSSL support
[...START-OF-TESTS...]
test 0003...[HTTP POST with auth and contents but with content-length set to 0]
--pd---e--- OK (3 out of 1596, remaining: 17:50, took 1.423s, duration: 00:02)
test 0007...[HTTP with cookie parser and header recording]
--pd--oe--- OK (7 out of 1596, remaining: 07:51, took 1.485s, duration: 00:02)
test 0006...[HTTP with simple cookie send]
--pd---e--- OK (6 out of 1596, remaining: 09:11, took 1.488s, duration: 00:02)
test 0005...[HTTP over proxy]
--pd---e--- OK (5 out of 1596, remaining: 11:03, took 1.491s, duration: 00:02)
CUSTOMBUILD : error : 169: cleardir(log/8/lock) failed [D:\a\curl\curl\bld\tests\test-ci.vcxproj]
test 0001...[HTTP GET]
--pd---e--- OK (1 out of 1596, remaining: 55:34, took 1.466s, duration: 00:02)
test 0004...[Replaced internal and added custom HTTP headers]
```
Ref: https://github.com/curl/curl/actions/runs/13546192228/job/37858323380?pr=16484#step:14:167
Ref: e53523fef0#14859
Ref: 311c31ec8e#6179
Follow-up to 3a8920e5ed#16600Closes#16484
If libtests, units and servers binaries are all present, auto-enable
bundle mode.
Drop manual runtests option.
Note: Make sure to "make clean" before changing the test bundle build
setting.
Also fix to append executable extension to all libtest and unit test
executables when launching them. This should make it a tiny bit faster
on Windows.
Follow-up to f4f25505df#15000
Follow-up to 71cf0d1fca#14772Closes#16750
This commit introduces support for features in the test selection
process by adding them to the keywords list with the `feat:` prefix. It
allows users to specify features to run only tests with them, or exclude
tests using `!feat:<feature>`, similar to how keywords are handled.
Fixes#16533Closes#16619
Signed-off-by: Aquila Macedo <aquilamacedo@riseup.net>
To make it apparent which CI jobs are missing this tool, so we can
install it to improve the runtests log.
Correction to the followed-up commit: `diff` is not installed via the
`gcc` package but via `automake`. Meaning it needs be installed manually
for MSYS cmake jobs.
Follow-up to e6c78e18da#16571Closes#16578
Support multi-target cmake builds via `CURL_DIRSUFFIX` env. For example:
`export CURL_DIRSUFFIX=Debug/`.
Multi-target generators place their output to `src/<subdir>/`,
`lib/<subdir>/`, `tests/server/<subdir>`, `tests/libtest/<subdir>` and
`tests/unit/<subdir>/` by default. Before this patch, `runtests.pl`
couldn't run on such builds because it expected the binaries under the
their `<subdir>`-less directories. This patch allows to set such subdir
and make `runtests.pl` find the binaries. In CI we use multi-target
builds with tests for MSVC. It also helps Xcode-generator builds, though
in CI we don't have such job running tests.
There may be better solutions to configure this, but passing a custom
value to `runtests.pl` including its subprocesses is somewhat tricky.
The reason the configuration value expects the slash at the end is
because MSYS is automagically expanding the env to a (wrong) absolute
path if the slash is in the front.
Also:
- drop the `-DCMAKE_RUNTIME_OUTPUT_DIRECTORY_*=` workaround from CI.
- replace `resolve` references in tests with a new `%RESOLVE` variable.
It didn't use a filename extension before. After this patch it uses
`exe_ext('TOOL')`. I'm not sure if this is the correct choice vs.
`exe_ext('SRV')`.
- fix `-c` option format in manual.
- fix some whitespace.
Note, in CI we still tweak `CMAKE_RUNTIME_OUTPUT_DIRECTORY_*` in jobs
which share steps between `./configure` and cmake. It's easier that way.
Ref: #15000
Cherry-picked from #16394Closes#16452
Reapply an accidental, and unrelated, change committed in
3c128966ed which unwantedly reverted
the preceding commit fcd5c2baff.
Caused by doing a local rebase when `git am` did not apply cleanly,
and most likely using a local base behind master.
Extend output filter to pick up backslashes. This makes them pass in CI
when run in the vcpkg MSVC job, for example.
Also:
- cmake: allow TrackMemory, aka `ENABLE_CURLDEBUG` again. Drop FIXME.
- cmake: drop stale TODO.
- runtests: include the word 'TrackMemory' in the message disabling it.
Follow-up to 9f23c8f201#14541
Follow-up to 94c596bbc5#16283Closes#16289
It should be done if "threaded-resolver" is enabled, but the detection
was slightly broken for builds that add c-ares for the asyn-rr feature.
Closes#16226
The condition `!$cmdtype eq "perl"` (introduced in a4765b0551) is always
false. It checks whether a logical negation (giving true/false) is equal
to the string `"perl"`. This is impossible, so the logging never worked.
The intent was probably to negate the result of the string
comparison:`!($cmdtype eq "perl")` or simply `$cmdtype ne "perl"`.
Fixes#16128
Reported-by: Igor Todorovski
Closes#16129
While this is useful on CI to highlight issues in the scripts, it's a
recipe for pain when enabled on users' builds.
Suggested-by: Leon Timmermans
Ref: #16128
Works better if we later introduce another way to do the asynch RR
resolves (together with the threaded resolver) that does not use c-ares.
Closes#16090
Allow building with c-ares and yet use threaded resolver for the main
host A/AAAA resolving:
`--with-ares` provides the c-ares install path and defaults to use
c-ares for name resolving
`--with-threaded-resolver` still uses c-ares in the build (for HTTPS)
but uses the threaded resolver for "normal" resolves.
It works similarly for cmake: ENABLE_ARES enables ares, and if
ENABLE_THREADED_RESOLVER also is set, c-ares is used for HTTPS RR and
the threaded resolver for "normal" resolves.
HTTPSRR and c-ares-rr are new features return by curl_version_info() and
thus shown by curl -V.
The c-ares-rr feature bit is there to make it possible to distinguish
between builds using c-ares for all name resolves and builds that use
the threaded resolves for the regular name resolves and c-ares for
HTTPSRR only. "c-ares-rr" means it does not use c-ares for "plain" name
resolves.
HTTPSRR support is EXPERIMENTAL only.
Closes#16054
When enabled in the build.
Update test2100: verify with HTTPS RR included
Adjust runtests and server/disabled.c to include "HTTPSRR" as a feature
in the test suite.
Also, decode the ALPN list in HTTPS records straight into IDs. There's
no point in storing everything in string format. Skip ALPNs we do not
support.
Closes#16007
lib : remove all hyper code
configure: stop detecting hyper
docs: no more mention of hyper
tests: mo more special-handling of hyper builds
CI: no jobs using hyper
Closes#15120
Designed to aid converting off from sscanf parsers. sscanf is hard to
use right, easy to mess up and often makes for sloppy error checking.
The new parsers allow more exact and pedandic parsing.
This new set of functions should be possible to use (and extend) and
switch over other libcurl parser code to use going forward.
Adapts the following to use the new functions:
- altsvc.c
- hsts.c
- http_aws_sigv4.c
Bonus: fewer memory copies, fewer stack buffers.
Test: Unit test1664
Docs: docs/internals/STRPARSE.md
Closes#15692
Using this option (only available in debug builds) makes curl always
call curl_easy_duphandle() on the handle before using it.
To help us catch curl_easy_duphandle() mistakes better.
Add a CI job using this.
Bonus: the previous runtests option -e is now also supported as
--test-event
Closes#15504
Sort TESTINFO lines by description within the number of skipped test.
It makes the list of skipped test groups easier to diff/compare between
jobs and runs.
Closes#15374
- add variable for the null device filename and use that in Perl code.
- initialize this variable with `NUL` on native Windows.
- add `%DEV_NULL` variable and use it in tests.
Fixes `The system cannot find the path specified.` messages seen when
running `runtests.pl` with native Windows Perl.
Also adjust code to not break mcedit syntax highlighting.
Cherry-picked from #14949Closes#15098
Also introduce 'notexists' for verifying that directory entries do not
exist after a test. Now an explicit supported feature instead of using
"funny" perl in postcheck.
Closes#15046
Collect all ready runners from select() and process in a loop. This
assures fairness in processing among all runners.
Formerly, only the first ready runner in the list of all was processed,
leading to later runners being delayed in processing and reporting
overly long test durations.
Also, reduce the backend idle timeout for the h2/h3 test servers so that
process shutdowns take less time.
Closes#14967
- cmake: drop `configure.os`.
This also includes OS version, but thus far it's not important enough
to include it.
- autotools: drop redundant, autotools-only `{target|host}.vendor`.
(it's part of the triplet in `{target|host}`.)
- swap order to `*.cpu` -> `*.os` to match triplet-order.
- cmake: drop redundant `target`.
It's manually filled and only in a (so far) few CI jobs. Let's revisit
when this becomes useful.
- move `buildinfo.txt` to build root.
- dist: add `buildinfo.txt` to `DISTCLEANFILES`.
- autotools: detect human readable compiler version.
- autotools: replace `XXYY` `compiler.version` with "X.Y"-style.
(also to match cmake.)
- autotools: use distinct `compiler_id` for Apple clang: `APPLECLANG`.
To match cmake and also because the the "X.Y"-style version number
is the Apple version, while `XXYY` was a value roughly translated to
mainline llvm/clang version.
- show buildinfo at the end of the configure stage, when run in CI, or
when `CURL_BUILDINFO` or `CURL_CI` env is set.
Follow-up to 1fdea16846#14802
Assisted-by: Dan Fandrich
Ref: https://github.com/curl/curl/pull/14802#issuecomment-2334942991Closes#14822
- perform torture tests with '-j2' for shorter runtime
- when waiting on test results overly long, log the tests
waited for and eventually log the test log directories
for easier analysis what is wrong in CI jobs.
- sockfilt.c: treat the windows errno 109 (ERROR_BROKEN_PIPE)
as a socket closed by the client and do not exit.
- when verifying https server, do not in addition check
the http server behind it also
- when tearing down the stunnel of a non-responsive https
server, tear down the http server with it
Closes#14960
CI:
- GHA/windows: enable OpenSSH server, SysInternals `handle`, `impacket`.
Skip `impacket` on MSYS2 due to install failure.
Skip OpenSSH server for old/standalone mingw-w64 (building curl
without SSH there.)
- GHA/windows: make test tool installs a separate step.
- GHA/cygwin: enable OpenSSH server.
Skip `impacket`: it's compiling for 7 minutes then breaks.
Skip `stunnel` due to sluggish test run performance.
(This update is unrelated to Azure jobs.)
- GHA/linux: migrate Linux jobs from Azure CI.
- GHA/linux: migrate scanbuild job from Azure CI.
- GHA/linux: enable libssh2 in a job. Also enable valgrind.
- CI/windows: enable SSPI in two jobs.
- CI/windows: disable zlib in one more job.
- CI/windows: improve `if` condition checking GnuTLS.
- CI/windows: ignore SFTP/SCP tests as necessary.
- universally ignore SCP tests, they fail everywhere.
- ignore test 612.
- ignore test 613 616 618 with MSYS2 mingw-w64.
- ignore test 614 with libssh.
- ignore all SFTP with MSYS2 native.
- ignore all SFTP with vcpkg with `libssh2[core,zlib]`.
- ignore a couple of SFTP tests with MSYS2 mingw-w64.
(This matches settings on Azure CI.)
- GHA/windows: ignore failing 1451 'Basic SMB request' test for
old mingw-w64 7.3.0 (but not for 9.5.0!):
```
2024-09-11 21:45:59,738 ERROR smbComNegotiate: b'NT LM 0.12\x00' is not in list
[...]
curl: (7) Could not connect to server
[...]
FAIL 1451: 'Basic SMB request' SMB
```
Ref: https://github.com/curl/curl/actions/runs/10816280746/job/30007130770#step:13:3546
- CI/Azure: delete, now moved to GHA.
sshserver:
- sshserver: fix permissions for SSH host key on Windows, allowing sshd
to launch.
- sshserver: fix initializing config paths with Cygwin/MSYS2 sshd.
- sshserver: fix initializing config paths in parallel builds.
- sshserver: delete redundant `DenyUsers`. This also opens the way
to allow multiple usernames.
- sshserver: fix `AllowUsers` for GHA/windows, by allowing the
domainless username again.
Follow-up to 3ee7c676ec#5721
- sshserver: fix `Deprecated option` warnings in `sshd_config`:
```
D:/a/curl/curl/bld/tests/log/2/server/curl_sshd_config line 7: Deprecated option AuthorizedKeysFile2
D:/a/curl/curl/bld/tests/log/2/server/curl_sshd_config line 25: Deprecated option KeyRegenerationInterval
D:/a/curl/curl/bld/tests/log/2/server/curl_sshd_config line 35: Deprecated option RhostsRSAAuthentication
D:/a/curl/curl/bld/tests/log/2/server/curl_sshd_config line 36: Deprecated option RSAAuthentication
D:/a/curl/curl/bld/tests/log/2/server/curl_sshd_config line 37: Deprecated option ServerKeyBits
D:/a/curl/curl/bld/tests/log/2/server/curl_sshd_config line 41: Deprecated option UseLogin
```
- sshserver: fix `Deprecated option` warnings in `ssh_config` with
Cygwin/MSYS2.
- sshserver: fix dumping config files due to the filenames missing their
full paths.
- sshserver: add workaround to make `logmsg` messages visible.
Before this patch they only went to a file and never shown.
runtests:
- runtests: log details when these Windows commands are called:
`handle`, `taskkill`, `tasklist`.
- runtests: add documentation links to Windows tools:
`handle`, `taskkill`, `tasklist`, `icacls`.
- runtests: add `-t` (kill whole tree) option to `taskkill` in
`servers.pm`, syncing it with the other `taskkill` call.
Follow-up to bc72a78a11#14488
- runtests: show warning if Sysinternals `handle` tool is missing.
- runtests: drop Windows XP Home compatibility `tskill` call.
The call was made on all Windows versions. It's possibly overkill to
do this, because XP Home is probably rarely used for running curl
tests these days. In case it's needed, it'd be better to put it under
an explicit option.
- runtests: show Perl version and path.
Fix/silence fallouts:
- unit2603: fix building with disabled HTTP support.
- unit2604: silence `-Woverlength-strings` warnings in C89 mode.
- test437, test1614: fix to pass with no-IPv6 builds.
Closes#14859
Since ASCII transfers on FTP means sending CRLF line endings, we should
still keep converting them to LF-only on platforms where text files
typically do not use CRLF.
This also DOES NOT convert existing CRLF line endings on ASCII uploads
but only does stand-alone LF => CRLF.
Regression from eeb7c12807 shipped in 8.10.0
Reported-by: finkjsc on github
Fixes#14873Closes#14875
Also:
- read `buildinfo.txt` from `runtests.pl` and dump it to the log.
- cmake: show `CROSS` target flag for cross-builds.
- cmake: add logic to detect arguments passed via the command-line.
It is meant to help filling out missing datapoints in the testclutch
matrix.
Closes#14802
All sections defined with the mode="text" attribute now get line endings
normalized so that comparisons become line ending agnostic. Removes the
previous problem of figuring out how exactly different Windows
environments should be treated in this regard.
Closes#14717
