After restricting OpenSSH-Windows to a single job, and bumping it to
the pre-release version, that job started hanging then timing out with
reasonable consistency.
Since we saw similar hangs before with OpenSSH-Windows stable, in all
jobs, drop OpenSSH-Windows from CI, and replace it with MSYS openssh.
After this patch, all Windows jobs use MSYS2 or Cygwin openssh.
Follow-up to 0ec72c1ef8#16672Closes#16704
MSVC:
- switch jobs to standard openssh server. Reduce exceptions.
- make the SCP/SFTP ignore list more specific and comment with details.
- keep using OpenSSH-Windows for the OpenSSL job, and bump to the
prerelease version.
- disable `ENABLE_DEBUG` for BoringSSL to have such build tested. (This
is the first Windows non-ENABLE_DEBUG build with test runs.)
Takeaways:
- test 612 broken on Windows.
- test 613 broken on Windows with the standard openssh server.
- test 614 broken with libssh and OpenSSH-Windows.
- test 3022 broken with libssh2 and OpenSSH-Windows.
- tests broken with OpenSSH-Windows:
601 603 617 619 621 641 665 2004.
- vcpkg `libssh2[core,zlib]` broken due to:
curl: (67) Authentication failure
MSVC prep steps:
- install base msys2 package to simplify configuration, align with other
jobs and allow to use msys2 packages for tests.
- add support for msys2 openssh server. Keep OpenSSH-Windows as per-job
option. Add support for OpenSSH prerelease versions.
Prerelease does not make a difference in test results, but, stable was
last updated in 2019 (v8.0.0.1) and it seems better to use maintained
release track, with its latest from April 2024 (v9.5.0).
https://community.chocolatey.org/packages/openssh/8.0.0.1https://community.chocolatey.org/packages/opensshhttps://github.com/PowerShell/Win32-OpenSSHhttps://github.com/PowerShell/openssh-portable
- add 'libssh' to its job name.
- make `ENABLE_DEBUG` a per-job option.
msys/mingw:
- install `openssh` later and only when necessary.
- downgrade msys2 runtime later. (to follow other jobs)
- disable `CheckSpace` earlier. Also to untie it from the runtime
downgrade step, which we would hopefully drop.
Closes#16672
Replace `Cwd::abs_path()` with `File::Spec->rel2abs()`. The former
requires the file to exist, but in some cases, it's missing.
Seen in MSVC vcpkg jobs using Chocolatey OpenSSH v8.0.0.1 ending up with
`$path=/d/a/curl/curl/bld/tests/log/3/server/ssh_server.pid`, which does
not exist while converting to an absolute path (the path is already
absolute, but the conversion is done unconditionally):
```
Use of uninitialized value in subroutine entry at D:/a/curl/curl/tests/pathhelp.pm line 128.
can't convert empty path at D:/a/curl/curl/tests/pathhelp.pm line 128.
```
Ref: https://github.com/curl/curl/actions/runs/13747741797/job/38444844173#step:14:1233 (master)
Ref: https://github.com/curl/curl/actions/runs/13751862952/job/38453816737#step:14:3185 (trace)
Also ignore 3 new libssh2 jobs failing due to memleak.
Partial revert of 1bd5ac998b#16570Closes#16636
It fixes test 2302, 2303, 2307 with MSVC and clang on Windows.
GCC Windows builds were not affected.
Failure was caused by stack overflow due to a 1MB+ sized test struct on
stack. Replace it with dynamic allocation.
Also unignore affected tests in GHA/windows.
As seen under WINE with llvm-mingw:
```
$ wine64 libtests.exe lib2302 ws://127.0.0.1:59964/2302 > stdout2302 2> stderr2302
Test: lib2302
URL: ws://127.0.0.1:59964/2302
wine: Unhandled stack overflow at address 000000014007486A (thread 0024), starting debugger...
Unhandled exception: stack overflow in 64-bit code (0x000000014007486a).
```
Ref: #16629 (discovery)
Ref: 1bd5ac998b#16570Closes#16630
The GnuTLS MSVC/vcpkg build doesn't actually work on Windows. Let's
restore the build itself, to keep it fit for more testing. With disabled
tests (and examples) to keep it fast and not add to flakiness.
Also:
- enable GnuTLS in the MultiSSL job.
- limit building examples to one normal and one UWP job. It saves
6 x 1-1.5 minutes. Coverage remains the same, because example builds
only depend on the toolchain / target, not on the actual features
(except IPv6, but that's enabled for all.)
Closes#16623
- enable zstd in Cygwin and MSYS jobs.
- dl-mingw: use Ninja in the 9.5.0 (winlibs-mingw) job.
The download package is shipping with it. Saves 15s build time.
Keep testing GNU Makefiles with the two mingw-builds jobs.
- dl-mingw: split `env` prop to `env` and `ver` to aid integrating with
MSYS2.
- dl-mingw: install MSYS2 with options to make it quick (<20s).
It allows to use MSYS2 dependency packages with the downloaded
toolchains. It also makes configuration cleaner. Install libpsl.
- dl-mingw: enable mbedTLS in the 7.3.0 job.
(OpenSSL took a long time to install, wolfSSL misses features.)
Assisted-by: Jeremy Drake
Closes#16429
Add support for running pytest in GHA/macos jobs.
Experimental, with caveats:
- slow.
- `httpd` often fails to start.
- 10-15 tests (depending on C compiler) fail consistently:
02_20, 02_33, 02_34, 03_01, 03_03, 05_04, 07_42.
- Homebrew build of vsftpd misses TLS support.
- `nghttpx` temporarily disabled for pytest.
You can test pytest by adding `install_steps: pytest` to a job.
Closes#16518
Before this patch `--http2` did not work in gcc builds with Secure
Transport, because ALPN relied on a compiler supporting the
`HAVE_BUILTIN_AVAILABLE` aka `__builtin_available()` feature. This
is clang-specific and missing from gcc (as of gcc v14).
Add support for ALPN and HTTP/2 when this compiler feature is missing.
Also drop test exceptions from GHA/macos in CI.
Follow-up to 092f6815c8
Ref: c349bd668c#14097 (issue 15.)
Ref: #4314Closes#16581
Re-enable running tests 19, 504, 704, 705, 1233 in CI MSYS jobs.
We carried over these exceptions from AppVeyor CI, where they have been
present for a long time. Cygwin jobs do not need these exceptions and
Cygwin and MSYS are similar envs. Time to re-evaluate if skipping them
is still necessary on MSYS.
Closes#16592
It runs fine now. Tested in all gcc-12 jobs after temporarly enabling
debug in them all (test 2100 requires debug-enabled).
Ref: c349bd668c#14097 (issue 15.)
Closes#16582
It fixes tests 1539, and 2402, 2404 (for non-Secure Transport), on macOS
with the gcc compiler.
Also unignore these tests in GHA/macos for non-secure transport.
Ref: c349bd668c#14097 (issue 15.)
Ref: 7b0240c077#16539
Ref: 2ec00372a1#16482Closes#16580
Bump msys2/setup-msys2 from 2.26.0 to 2.27.0. It brings the perf
regression experienced earlier with GfW and the pre-installed
MSYS2 on the GHA runner. Apply the runtime downgrade trick as
a workaround.
Fixes#16440Fixes#16547Closes#16574
These tests seem to be running no less stable now than others.
Stop ignoring their results to catch real issues.
These are consistently failing and remain on the ignore list:
in MSVC / vcpkg jobs:
```
FAIL-IGNORED 2302: 'WebSockets via callback (frame mode) + curl_ws_send()' WebSockets
FAIL-IGNORED 2303: 'WebSockets but gets a 200 back' WebSockets
FAIL-IGNORED 2307: 'WebSockets, overlong PING payload' WebSockets
```
https://github.com/curl/curl/actions/runs/13674664461/job/38233949942?pr=16570#step:14:4089
- Likely curl issues either in tests, server, or in WebSockets support.
in tests running under MSYS, affecting native mingw Windows builds only:
```
FAIL-IGNORED 612: 'SFTP post-quote remove file' SFTP, post-quote
[...]
curl: (21) rm command failed: Operation failed
```
https://github.com/curl/curl/actions/runs/13674664461/job/38233952699?pr=16570#step:14:1378
in tests running under MSYS, affecting both MSYS and native mingw Windows builds:
```diff
FAIL-IGNORED 613: 'SFTP directory retrieval' SFTP, directory
[...]
--- log/7/check-expected 2025-03-05 11:19:54.119658000 +0000
+++ log/7/check-generated 2025-03-05 11:19:54.119658000 +0000
@@ -1,3 +1,3 @@
d????????? N U U N ??? N NN:NN asubdir[LF]
--rw?rw?rw? 1 U U 37 Jan 1 2000 plainfile.txt[LF]
+-rw?r-?r-? 1 U U 37 Jan 1 2000 plainfile.txt[LF]
-r-?r-?r-? 1 U U 47 Dec 31 2000 rofile.txt[LF]
```
https://github.com/curl/curl/actions/runs/13674664461/job/38233950866?pr=16570#step:14:1316
- Possibly a curl test portabibility, Perl or MSYS issue.
in Cygwin tests:
```
FAIL-IGNORED 615: 'SFTP put remote failure' SFTP, SFTP put, FAILURE
```
https://github.com/curl/curl/actions/runs/13674664461/job/38233949428?pr=16570#step:12:3817
Follow-up to adcfd4fb3e#16553
Ref: #14854Closes#16570
To include the expected/generated diffs in the error log.
Also make it explicit for pure MSYS, though it was installed by `gcc`
before this patch.
Closes#16571
This job was never stable. Bumping to -j8 (from -j4) possibly made it
flakier: 032447e624#16271
Keep this job for build tests and drop running tests to improve the CI
experience and save CI time.
It's also a simple build with no dependencies. CI continues to build
a similar job with 9.5.0, which is more stable.
It remains a puzzle why builds with this toolchain (7.3.0 win32 threads
mingw-builds) is flakier and requires more test exceptions than the
indentical build with a slightly different build/version of
the toolchain (9.5.0 posix threads winlibs_mingw).
Ref: #14854Closes#16564
- enable quictls in autotools and cmake jobs. autotools requires
a workaround due to wrong libpath in the quictls pkg-config.
nghttp3 is offered by Homebrew, but not ngtcp2, to enable H3.
- install `libnghttp2` rather than `nghttp2`.
`libnghttp2` is preinstalled and smaller. It also avoids detecting
`nghttpx`, which confuses `pytest`.
- limit `brew unlink openssl` to libressl/quictls jobs.
Closes#16517
This now makes sure to trim off exact matches for curl symbols and long
curl commanad line options instead of using pattern matching as before.
This should catch typoed names (that still follow the pattern) better.
The cleanspell.pl script is no longer used. cleancmd.pl is used for all
markdown files.
Closes#16504
- replace `--parallel <n>` and `-j<n>` for individual commands with
`MAKEFLAGS`, for jobs not yet doing it.
This enables parallel builds in distcheck / maketgz-and-verify-in-tree,
where `-j` option was missing.
- add `--parallel` for iOS Xcode job for improved performance.
- drop redundant `-j5` for Android jobs.
- drop stray `cmake --config` options from single-target jobs (cygwin,
msys/mingw, dl-mingw, non-native). Drop redundant
`CMAKE_RUNTIME_OUTPUT_DIRECTORY_*` settings too.
- GHA/windows: add timeout for package install steps where missing.
- GHA/non-native: specify target type explicitly for iOS cmake jobs.
Xcode default was already Debug, single-target default was generic,
now it's Release, with unity batch to keep it fast.
`MAKEFLAGS` is necessary for autotools jobs and CMake jobs using
the default (GNU Make) generator. It's ignored by Ninja and other tools.
`cmake --parallel` is still necessary for jobs with Visual Studio or
Xcode generators. Parallelism is 5 for GHA Linux and Windows runners,
4 for macOS, 3 for VMs, 2 for AppVeyor.
Closes#16502
- GHA/windows/WinCE:
- set `-O3 -DNDEBUG` C flags manually for the CMake mingw32ce build.
CMake doesn't recognize the platform and fails to add them. To match
autotools (using `-O2`), and hit similar compiler warnings.
- enable parallel builds for cmake.
- tune parallelism for cmake using unity batches.
- tune parallelism for autotools.
Follow-up to 2a292c3984#15975
- tests: fix potentially uninitialized value in `readline()` in
`getpart.c`. Detected by gcc 4.4.0 `-O2` (Windows CE) jobs:
```
tests/server/getpart.c: In function 'getpart':
tests/server/getpart.c:298: error: 'datalen' may be used uninitialized in this function
```
Ref: https://github.com/curl/curl/actions/runs/13522595237/job/37785147505?pr=16476#step:11:25
Follow-up to 592880a3ca
- vtls_scache: rework returning pointer to avoid compiler warning seen
with `-O3` gcc 4.4.0 builds (Windows CE/schannel):
```
lib/vtls/schannel.c: In function 'schannel_connect_step1':
lib/vtls/vtls_scache.c:975: error: dereferencing pointer 'old_cred.4474' does break strict-aliasing rules
lib/vtls/vtls_scache.c:985: error: dereferencing pointer 'old_cred.4474' does break strict-aliasing rules
lib/vtls/schannel.c:959: note: initialized from here
```
Ref: https://github.com/curl/curl/actions/runs/13523868335/job/37789610845#step:9:25
Follow-up to fa0ccd9f1f#15774Closes#16476
`./configure` mingw32ce builds enable C99 mode automatically, that
triggers compiler warnings in gcc 4.4.0. We initially worked it around
in CI by suppressing the detection of C99 with `ac_cv_prog_cc_c99=no`.
Replace it with automatically silencing the bogus warnings in C99 mode,
for all build systems:
```
lib/ftp.c: In function 'Curl_GetFTPResponse':
lib/ftp.c:726: error: format '%zd' expects type 'signed size_t', but argument 4 has type 'ssize_t'
lib/ws.c: In function 'ws_dec_pass_payload':
lib/ws.c:304: error: format '%zd' expects type 'signed size_t', but argument 3 has type 'ssize_t'
lib/ws.c: In function 'ws_enc_write_head':
lib/ws.c:581: error: format '%zd' expects type 'signed size_t', but argument 3 has type 'long int'
lib/vtls/schannel.c: In function 'schannel_connect_step1':
lib/vtls/schannel.c:1122: error: format '%zd' expects type 'signed size_t', but argument 3 has type 'ssize_t'
lib/vtls/schannel.c: In function 'schannel_connect_step2':
lib/vtls/schannel.c:1311: error: format '%zd' expects type 'signed size_t', but argument 3 has type 'ssize_t'
lib/vtls/schannel.c: In function 'schannel_send':
lib/vtls/schannel.c:1793: error: format '%zd' expects type 'signed size_t', but argument 3 has type 'ssize_t'
lib/vtls/schannel.c:1810: error: format '%zd' expects type 'signed size_t', but argument 3 has type 'ssize_t'
lib/vtls/schannel.c: In function 'schannel_shutdown':
lib/vtls/schannel.c:2286: error: format '%zd' expects type 'signed size_t', but argument 4 has type 'ssize_t'
lib/vtls/vtls.c: In function 'ssl_cf_recv':
lib/vtls/vtls.c:1422: error: format '%zd' expects type 'signed size_t', but argument 5 has type 'ssize_t'
```
Ref: https://github.com/curl/curl/actions/runs/13533841306/job/37821720902?pr=16492#step:9:20
Also: simplify Windows CE job configuration in GHA/windows.
Follow-up to 2a292c3984#15975Closes#16492
- appveyor: restore VS2008 job, after fixing its issues.
Enable OpenSSL in it. It takes 1 minute.
Follow-up to 9b0467b169#16453
Follow-up to edfa537100#16456
- appveyor: make a copy of OpenSSL DLLs to have them picked up as an
artifact (disabled by default) to aid local tests.
- appveyor: dump CMake configuration logs on failure.
- appveyor: tidy up job parameter defaults.
- GHA/windows: add pre-fill check option for dl-mingw jobs.
- GHA/windows: fix pre-fill check option for MSYS jobs by installing
`diffutils`.
Follow-up to e7adf3e837#15841
- GHA/windows: de-duplicate to `PATH` commands for Cygwin.
- GHA/windows: drop `$SYSTEMROOT/System32` from `PATH` for Cygwin
configure. It's not needed.
Follow-up to 36fd2dd6ee#13599
- list `.pdb` files in curl version step for MSVC.
Ref: #16439
Cherry-picked from #16394Closes#16458
Support multi-target cmake builds via `CURL_DIRSUFFIX` env. For example:
`export CURL_DIRSUFFIX=Debug/`.
Multi-target generators place their output to `src/<subdir>/`,
`lib/<subdir>/`, `tests/server/<subdir>`, `tests/libtest/<subdir>` and
`tests/unit/<subdir>/` by default. Before this patch, `runtests.pl`
couldn't run on such builds because it expected the binaries under the
their `<subdir>`-less directories. This patch allows to set such subdir
and make `runtests.pl` find the binaries. In CI we use multi-target
builds with tests for MSVC. It also helps Xcode-generator builds, though
in CI we don't have such job running tests.
There may be better solutions to configure this, but passing a custom
value to `runtests.pl` including its subprocesses is somewhat tricky.
The reason the configuration value expects the slash at the end is
because MSYS is automagically expanding the env to a (wrong) absolute
path if the slash is in the front.
Also:
- drop the `-DCMAKE_RUNTIME_OUTPUT_DIRECTORY_*=` workaround from CI.
- replace `resolve` references in tests with a new `%RESOLVE` variable.
It didn't use a filename extension before. After this patch it uses
`exe_ext('TOOL')`. I'm not sure if this is the correct choice vs.
`exe_ext('SRV')`.
- fix `-c` option format in manual.
- fix some whitespace.
Note, in CI we still tweak `CMAKE_RUNTIME_OUTPUT_DIRECTORY_*` in jobs
which share steps between `./configure` and cmake. It's easier that way.
Ref: #15000
Cherry-picked from #16394Closes#16452
