This makes `runtests.pl` run the final executables directly.
Before this patch it called the autotools/libtool wrapper tool, which
then called the final executables.
This solution was already used for `curl.exe`.
Applies to tests run in the `mingw, AM x86_64 c-ares U` job, which still
shows unexplained flakiness.
Also makes tests finish 45 seconds faster.
Ref: #14854
Follow-up to 1a2d38c47c#15437Closes#15662
The MSVC UWP job in CI did not actually enable UWP. Fix this and
the fallouts discovered after enabling it.
- GHA/windows: make sure to enable UWP in MSVC vcpkg UWP job.
Use the CMake options and C flags already used for mingw-w64, but use
`WINAPI_FAMILY_PC_APP` instead of the deprecated `WINAPI_FAMILY_APP`.
(The former is not supported by mingw-w64, so leave it there as-is.)
Follow-up to cb22cfca69#14077
- GHA/windows: by default the MSVC UWP job became 2x-3x slower than
others after actually enabling UWP. Most of it is caused by
CMake/MSBuild automatically building full APPX containers for each
`.exe` target. This includes 21 CMake feature detections. Each
detection app is built into a 15MB APPX project, with code signing,
logos, etc. Example:
https://github.com/curl/curl/actions/runs/12056968170/job/33620610958
Disable this overhead for curl build targets via custom
`CMAKE_VS_GLOBALS` options. I've found no way to apply them to feature
detection targets, so those remain slow.
- cmake: automatically enable Unicode for UWP builds. It's required.
Also stop enabling it manually in the existing CI job.
- tests: fix `getpid()` use for Windows UWP:
```
tests\server\util.c(281,21): warning C4013: 'getpid' undefined; assuming extern returning int
```
Ref: https://github.com/curl/curl/actions/runs/12061215311/job/33632904249#step:11:38
- src/tool_doswin: disable `GetLoadedModulePaths()` for UWP.
mingw-w64 UWP was okay with this, but MS SDK headers are not.
This makes `--dump-module-paths` return empty for UWP builds.
```
src\tool_doswin.c(620,3): error C2065: 'MODULEENTRY32': undeclared identifier
src\tool_doswin.c(626,11): warning C4013: 'CreateToolhelp32Snapshot' undefined; assuming extern returning int
src\tool_doswin.c(626,36): error C2065: 'TH32CS_SNAPMODULE': undeclared identifier
src\tool_doswin.c(632,7): warning C4013: 'Module32First' undefined; assuming extern returning int
```
Ref: https://github.com/curl/curl/actions/runs/12055081933/job/33614629930#step:9:35
- examples: fix `websocket.c` to include `winsock2.h` before `windows.h`
to make it build with MSVC UWP:
```
include\curl\curl.h(143,16): error C2061: syntax error: identifier 'curl_socket_t'
include\curl\curl.h(143,16): error C2059: syntax error: ';'
include\curl\curl.h(417,52): error C2146: syntax error: missing ')' before identifier 'curlfd'
include\curl\curl.h(417,38): error C2081: 'curl_socket_t': name in formal parameter list illegal
```
Ref: https://github.com/curl/curl/actions/runs/12055317910/job/33615644427#step:14:126
- GHA/windows: silence linker warning with MSVC UWP builds:
```
LINK : warning LNK4075: ignoring '/INCREMENTAL' due to '/OPT:ICF' specification
```
Ref: https://github.com/curl/curl/actions/runs/12055696808/job/33616629610#step:11:38
- GHA/windows: set `/INCREMENTAL:NO` for all MSVC jobs to improve
performance a little.
- cmake: show `UWP` platform flag.
Ref: #15652Closes#15657
The ECH feature cannot be built without HTTPS RR.
ECH automatically implied HTTPS RR in `./configure` but not in CMake,
winbuild, documentation.
Also update documentation and CI configs.
Follow-up to a362962b72#11922Closes#15648
Merge cmake and autotools build steps for cygwin, msys2 and
cross-linux jobs.
Advantages:
- makes it easier to keep the two build tracks in sync.
- uses the same steps across jobs.
- avoids scrolling through greyed out steps.
- syncs steps with other workflows already merged like this.
- less code.
Also:
- stop ignoring WebSockets tests results for msys2/mingw-w64 cmake jobs,
except for 2301 2302 that were also ignored for autotools. Syncing the
two build methods.
- drop 'cmake' from step names where cmake was the only build tool.
This was redundant as "CM" already indicates it in the job name.
Closes#15643
- GHA/windows: switch mingw-w64 UWP CI job to use UCRT.
`msvcr120_app` was missing `getch()` for example.
Follow-up to f988842d85#15637
This job tests compiling for UWP correctly, but the the resulting
`curl.exe` still doesn't look like a correct UWP app, now exiting
on startup with: `curl: error initializing curl library`.
- tool_getpass: restore `getch()` for UWP builds.
Follow-up to f988842d85#15637
- schannel: silence `-Werror=null-dereference` warning in mingw-w64 UWP:
```
lib/vtls/schannel_verify.c: In function 'Curl_verify_host':
lib/vtls/schannel_verify.c:558:33: error: null pointer dereference [-Werror=null-dereference]
558 | for(i = 0; i < alt_name_info->cAltEntry; ++i) {
| ~~~~~~~~~~~~~^~~~~~~~~~~
lib/vtls/schannel_verify.c:559:50: error: null pointer dereference [-Werror=null-dereference]
559 | PCERT_ALT_NAME_ENTRY entry = &alt_name_info->rgAltEntry[i];
| ~~~~~~~~~~~~~^~~~~~~~~~~~
```
Ref: https://github.com/curl/curl/actions/runs/12022656065/job/33515255397?pr=15638#step:19:27
Follow-up to 9640a8ef6f#15421
- GHA/windows: fix `find` command in MSVC job step.
Follow-up to 5f9411f953#15380
- GHA/windows: drop unnecessary `windowsappcompat` lib from mingw-w64
UWP job. Also drop related MSYS2 package.
- GHA/windows: cmake 3.31.0 still invokes `windres` with wrong options
with mingw-w64 UPW. Update curl version in comment accordingly.
- GHA/windows: tidy up mingw-w64 UWP spec logic, limit it to gcc.
- GHA/windows: update comments on `curl.exe` UWP startup errors.
Closes#15638
Avoid the `curl.exe` wrapper binary created by libtool, and run the real
`curl.exe` directly for tests and version information.
This solution was used in Azure jobs. I missed it when migrating jobs
to GHA.
Applies to tests run in the `mingw, AM x86_64 c-ares U` job, which has
seen unexplained flakiness.
Ref: 354afc891d#6049
Follow-up to e53523fef0#14859Closes#15437
- Create a new macro SCH_DEV() to manage verbose debug messages that are
only useful for debugging Schannel recv decryption.
schannel_recv contains a lot of useful debug messages to help debug the
function, however in practice they are not otherwise useful and showing
them in debug builds adds a lot of noise.
To show these messages curl must now be built with
CURL_SCHANNEL_DEV_DEBUG defined.
Prior to this change many, but not all, extra-verbose messages were
wrapped in DEBUGF() so they were only shown in debug builds.
Ref: https://github.com/curl/curl/issues/14807Closes#14826
Fix the significant perf regression for vcpkg jobs by switching to the
MSYS2 shell environment from Git for Windows. This env is already used
for old-mingw-w64 job that remained unaffected by this issue.
The issue began with the windows-runner update 20241015.1.0. It bumped
Git for Windows from Git 2.46.2.windows.1 to Git 2.47.0.windows.1. GfW
bumped its MSYS2 components, including `msys-2.0.dll`. That's Cygwin
code, which may have contributed to this. Pipes were involved and
`runtests.pl` relies on pipes heavily in parallel mode. (The issue was
not seen with parallel tests disabled, in retrospect.)
This is useful as a permanent solution too. It drop GfW as a dependency
and makes Windows jobs use one less shell/env flavour.
Long term it might help to use native Windows Perl to avoid the MSYS
layer completely, if there is a way to make that work.
Assortment of possibly related links:
https://cygwin.com/pipermail/cygwin/2024-August/256398.htmlf78009cb1c7f3c225325https://github.com/actions/runner-images/issues/10843https://github.com/git-for-windows/git/issues/5199https://github.com/git-for-windows/msys2-runtime/pull/757913a41703555afcb2f31c5f4dcdc5
Follow-up to c33174d42f#15364
Follow-up to 1e0305973c#15356Closes#15380
- Linux: move test and pytest prereqs right before test run.
- returns build phase results faster.
- allows skipping steps for jobs that don't need them.
- makes dependencies more transparent.
- sync prereq install step names.
- use `tests/requirements.txt` more.
Closes#15275
- bump github/codeql-action from 3.26.10 to 3.26.11
- bump vmactions/omnios-vm from 1.0.7 to 1.0.8
- bump actions/cache from 4.0.2 to 4.1.1
Closes#15178Closes#15179Closes#15244
Cygwin jobs took 30 minutes and was the reason for moving them to their
own workflow. After recent changes, they finish in 8 minutes, which is
on par or lower than the longest Windows jobs. Thus, Cygwin can now be
moved back to the Windows workflow without causing extra wait for all
the workflow jobs to complete (and allowing a manual retry for the
remaining flaky jobs).
Notice that Cygwin still uses the cygwin/cygwin-install-action action to
install Cygwin itself. This action uses no caching and the time it takes
fluctuates widely depending on mirror speed, congestion and other
things. If Cygwin tends to hold up jobs again for this or other reasons,
this patch may be reverted.
Follow-up to 80a8e24956#15083
Follow-up to 2d8464c4cb#14366Closes#15086
Move test runs from autotools jobs to cmake ones for Cygwin and MSYS.
This makes producing test results and finishing the workflows faster,
because the cmake build steps are much faster than autotools in these
envs.
Also:
- drop building examples with Cygwin autotools. It takes almost
4 minutes. Keep building them with cmake, taking 20 seconds.
- drop building examples in MSYS autotools jobs that run tests.
Keep building them in autotools jobs without tests. It makes
the longest running job 2 minutes shorter.
After this patch Cygwin job times are on par with or lower than Windows
ones. It means Cygwin doesn't cause extra delay to finish the whole
workflow, allowing to re-merge these jobs into the Windows workflow.
Closes#15083
- http3-linux: add newlines for readability.
- http3-linux: use `make pytest` to run pytest.
- checksrc: use `|` multiline to sync with other uses.
- checksrc: prefer `$()`.
- prefer `>-` over `>` for folded option lists.
- cygwin: drop a `grep` no longer necessary.
Closes#15081
- reduce job timeouts for msys2 and old-mingw-w64 jobs.
- fold long lines and reorder options in MSVC jobs.
- make some options default in MSVC jobs.
Closes#15017
- CI default: raise parallelism to 20
- CI valgind: set parallelism to 6
- CI non-native: adapt parallelism for OS builds
- CI Windows: no longer ignore FTP, TFTP, MQTT and SMTP
- CI Windows: restrict test timeout to 10 minutes
- CI Windows: do not run tests for msh3 build
- tests, various: restrict curl invocation to ipv4 to avoid talking to
any ipv6 test server running in parallel
- tests: requiring http/2 server no longer needs to mention http server
- test 190: use a fixed timeout of 10 seconds instead of %FTPTIME2, as
that value rises under parallel load beyond what the server is waiting
and then produces different results
- test 1540: add debug logging
- testrunner: add verify checks for http/2 and http/3 server
Closes#15040
Add `CURL_CA_SEARCH_SAFE` build-time option to enable CA bundle search
in the `curl` tool directory. The lookup method was already used to find
`.curlrc` and `_curlrc` (on Windows). On Windows it overrides the unsafe
default `SearchPath()` method.
Enable with:
- cmake: `-DCURL_CA_SEARCH_SAFE=ON`
- autotools: `--enable-ca-search-safe`
- raw: `CPPFLAGS=-DCURL_CA_SEARCH_SAFE`
On Windows, before this patch the whole `PATH` was searched for
a CA bundle. `PATH` may contain unwanted or world-writable locations,
including the current directory. Searching them all is convenient to
pick up any CA bundle, but not secure.
The Muldersoft curl distro implements such CA search via a custom
patch for Windows:
cd652d4792/patch/curl_tool_doswin.diff (L50)
MSYS2/mingw-w64 distro has also been rolling a patch solving this:
https://github.com/msys2/MINGW-packages/blob/master/mingw-w64-curl/0001-Make-cURL-relocatable.patchhttps://github.com/msys2/MINGW-packages/blob/master/mingw-w64-curl/pathtools.c
Also add option to fully disable Windows CA search:
- cmake: `-DCURL_DISABLE_CA_SEARCH=ON`
- autotools: `--disable-ca-search`
- raw: `CPPFLAGS=-DCURL_DISABLE_CA_SEARCH`.
Both options are considered EXPERIMENTAL, with possible incompatible
changes or even (partial) removal in the future, depending on feedback.
An alternative, secure option is to embed the CA bundle into the binary.
Safe search can be extended to other platforms if necessary or useful,
by using `_NSGetExecutablePath()` (macOS),
`/proc/self/exe` (Linux/Cygwin), or `argv[0]`.
Closes#14582
- linux: bump up test parallelism for valgrind tests to `-j4`
(from `-j2`). (EXPERIMENTAL)
- linux: drop `apt-get update` for the default architecture on the GHA
native runner. It makes prereq install steps complete faster.
The runner image gets weekly updates, and that should be enough to
guarantee fresh packages in most cases:
https://github.com/actions/runner-images/commits/main/images/ubuntu/Ubuntu2204-Readme.md
- aws-lc: use ninja with cmake.
- aws-lc: build examples with cmake.
- aws-lc: drop `apt update`.
- aws-lc, wolfssl, linux32, http3-linux: move building examples to
the last step.
Follow-up to 45202cbba4#14906
- windows: formatting.
Closes#14992
They are still slow in these jobs/combinations.
- non-native/FreeBSD/arm64 autotools +36s
- non-native/FreeBSD/arm64 cmake +1m
- windows/linux-cross-mingw-w64 autotools +33s
These ones remain:
- linux/aws-lc cmake +6s
- windows/linux-cross-mingw-w64 cmake +12s
Follow-up to 71cf0d1fca#14772
- Move `docs/examples` builds under a separate target.
- Make `BUILD_EXAMPLES` default to `ON`. It means to generate the rules
for `docs/examples` by default, but not build them. To build them,
an explicit `make curl-examples` (or ninja, etc) command is necessary.
This syncs behaviour with autotools, and also how both cmake and
autotools are building tests.
- GHA: update cmake jobs to use the new way of building examples.
- GHA: move examples build step at the end of the job, after building
and running tests. This allows to have build and test run results
faster, and leave the seldom-changing examples build to the end.
Building examples is the slowest build step with no practical way to
make them fast.
- appveyor: enable building examples in two old-MSVC jobs.
- examples: fix examples to build cleanly with old MSVC versions.
- GHA/non-native: move example build log under a GHA foldable section.
- GHA/windows: move building examples into separate step for Linux cross
jobs.
Follow-up to dfdd978f7c#13491Closes#14906
Because it reduces test coverage significantly:
```
TESTINFO: "curl has SSPI support" 109 times (64, 65, 67, 68, 69, 70, 72, 81, 88 and 100 more)
```
Follow-up to e53523fef0#14859Closes#14947
CI:
- GHA/windows: enable OpenSSH server, SysInternals `handle`, `impacket`.
Skip `impacket` on MSYS2 due to install failure.
Skip OpenSSH server for old/standalone mingw-w64 (building curl
without SSH there.)
- GHA/windows: make test tool installs a separate step.
- GHA/cygwin: enable OpenSSH server.
Skip `impacket`: it's compiling for 7 minutes then breaks.
Skip `stunnel` due to sluggish test run performance.
(This update is unrelated to Azure jobs.)
- GHA/linux: migrate Linux jobs from Azure CI.
- GHA/linux: migrate scanbuild job from Azure CI.
- GHA/linux: enable libssh2 in a job. Also enable valgrind.
- CI/windows: enable SSPI in two jobs.
- CI/windows: disable zlib in one more job.
- CI/windows: improve `if` condition checking GnuTLS.
- CI/windows: ignore SFTP/SCP tests as necessary.
- universally ignore SCP tests, they fail everywhere.
- ignore test 612.
- ignore test 613 616 618 with MSYS2 mingw-w64.
- ignore test 614 with libssh.
- ignore all SFTP with MSYS2 native.
- ignore all SFTP with vcpkg with `libssh2[core,zlib]`.
- ignore a couple of SFTP tests with MSYS2 mingw-w64.
(This matches settings on Azure CI.)
- GHA/windows: ignore failing 1451 'Basic SMB request' test for
old mingw-w64 7.3.0 (but not for 9.5.0!):
```
2024-09-11 21:45:59,738 ERROR smbComNegotiate: b'NT LM 0.12\x00' is not in list
[...]
curl: (7) Could not connect to server
[...]
FAIL 1451: 'Basic SMB request' SMB
```
Ref: https://github.com/curl/curl/actions/runs/10816280746/job/30007130770#step:13:3546
- CI/Azure: delete, now moved to GHA.
sshserver:
- sshserver: fix permissions for SSH host key on Windows, allowing sshd
to launch.
- sshserver: fix initializing config paths with Cygwin/MSYS2 sshd.
- sshserver: fix initializing config paths in parallel builds.
- sshserver: delete redundant `DenyUsers`. This also opens the way
to allow multiple usernames.
- sshserver: fix `AllowUsers` for GHA/windows, by allowing the
domainless username again.
Follow-up to 3ee7c676ec#5721
- sshserver: fix `Deprecated option` warnings in `sshd_config`:
```
D:/a/curl/curl/bld/tests/log/2/server/curl_sshd_config line 7: Deprecated option AuthorizedKeysFile2
D:/a/curl/curl/bld/tests/log/2/server/curl_sshd_config line 25: Deprecated option KeyRegenerationInterval
D:/a/curl/curl/bld/tests/log/2/server/curl_sshd_config line 35: Deprecated option RhostsRSAAuthentication
D:/a/curl/curl/bld/tests/log/2/server/curl_sshd_config line 36: Deprecated option RSAAuthentication
D:/a/curl/curl/bld/tests/log/2/server/curl_sshd_config line 37: Deprecated option ServerKeyBits
D:/a/curl/curl/bld/tests/log/2/server/curl_sshd_config line 41: Deprecated option UseLogin
```
- sshserver: fix `Deprecated option` warnings in `ssh_config` with
Cygwin/MSYS2.
- sshserver: fix dumping config files due to the filenames missing their
full paths.
- sshserver: add workaround to make `logmsg` messages visible.
Before this patch they only went to a file and never shown.
runtests:
- runtests: log details when these Windows commands are called:
`handle`, `taskkill`, `tasklist`.
- runtests: add documentation links to Windows tools:
`handle`, `taskkill`, `tasklist`, `icacls`.
- runtests: add `-t` (kill whole tree) option to `taskkill` in
`servers.pm`, syncing it with the other `taskkill` call.
Follow-up to bc72a78a11#14488
- runtests: show warning if Sysinternals `handle` tool is missing.
- runtests: drop Windows XP Home compatibility `tskill` call.
The call was made on all Windows versions. It's possibly overkill to
do this, because XP Home is probably rarely used for running curl
tests these days. In case it's needed, it'd be better to put it under
an explicit option.
- runtests: show Perl version and path.
Fix/silence fallouts:
- unit2603: fix building with disabled HTTP support.
- unit2604: silence `-Woverlength-strings` warnings in C89 mode.
- test437, test1614: fix to pass with no-IPv6 builds.
Closes#14859
