- Done with the new top-complexity script which uses the pmccabe tool.
- Any function scoring over 100 makes the test fail
- The script outputs all functions scoring over 70
- Two >100 functions are whitelisted by name, but they are not allowed
to increase their scores.
Closes#17398
Reduce Curl_ossl_ctx_init() complexity by splitting it up into
sub functions.
While splitting if ECH, add pytest fixed for AWS-LC and enable
it in CI.
Closes#17404
To avoid redundant work in CI and to avoid a single checksrc issue make
all autotools jobs fail. After this patch checksrc issues make fail
the checksrc job, the `dist / verify-out-of-tree-autotools-debug`,
`dist / maketgz-and-verify-in-tree` jobs and the fuzzer job (if run).
Of these, the `dist` jobs replicate local builds, also testing the build
logic.
Also add a script to check the complete local repository, optionally
with the build tree to verify generated C files.
Also:
- automatically run checksrc in subdirectories having a `checksrc`
target. (examples, OS400, tests http/client, unit and tunit)
- tests/libtest: make sure to run `checksrc` on generated `lib1521.c`.
(requires in-tree autotools build.)
- tests: run `checksrc` on targets also for non-`DEBUGBUILD`
builds. It ensures to check `lib1521.c` in CI via job
`dist / maketgz-and-verify-in-tree`.
- src: drop redundant `$(builddir)` in autotools builds.
- scripts: add `checksrc-all.sh` script to check all C sources and
the build directory as an option.
- use the above from CI, also make it verify all generated sources.
- silence `checksrc` issues in generated C sources.
- checksrc: add `-v` option to enable verbose mode.
- checksrc: make verbose mode show checked filename and fix to only
return error on failure.
- make sure that generated C files pass `checksrc`.
Assisted-by: Daniel Stenberg
Closes#17376
They take about 4.5 minutes of CI time in GHA/macos.
Also:
- autotools: improve `caddy`, `vsftpd` detection.
Bringing it closer to cmake.
- autotools: fix `--with-test-caddy=no`, `--with-test-vsftps=no`,
`--with-test-nghttpx=no` options.
- cmake: sync `nghttpx` default with autotools.
- pytest: disable failing mbedTLS tests on macOS CI.
- pytest: disable failing earlydata tests on macOS CI.
- GHA/macos: keep vsftpd pytests disabled due to lengthy run times.
- pytest: fix test_05_04 for LibreSSL. Ref: #17367
Authored-by: Stefan Eissing
Remaining issues:
- some unidentified tests taking a long time with mbedTLS:
`================= 462 passed, 278 skipped in 347.93s (0:05:47) =================`
Ref: https://github.com/curl/curl/actions/runs/15073354301/job/42374999041#step:17:1536
Workaround: not enabling pytest for mbedTLS jobs
- 17 FTP tests taking a long time (affecting all TLS backends):
without vsftpd:
`====================== 496 passed, 244 skipped in 56.15s =======================`
Ref: https://github.com/curl/curl/actions/runs/15073354301/job/42374998230#step:17:1536
with vsftpd:
`================= 513 passed, 227 skipped in 409.37s (0:06:49) =================`
Ref: https://github.com/curl/curl/actions/runs/15073678568/job/42376039672?pr=17362#step:17:1537
Workaround: force-disable vsftpd.
- 100 tests failing with SecureTransport. Let's ignore that due to imminent deprecation.
Ref: https://github.com/curl/curl/actions/runs/15055652333/job/42320873732#step:17:15362
Follow-up to 30ef79ed93#17295
Follow-up to 9147903366#16518Closes#17362
Require now pytest-xdist from tests/http/requirements.txt and
run pytest in 'auto' parallel mode (counts cpu cores).
For CI runs, set the worker count to 4, overriding the
core count of 2 exposed in the images.
- use Filelock to generate allocated ports at start for all
workers and have subsequent workers just read the file and
take the ports for their slot
- make httpd config clearing a function fixture so every test
starts with a clean httpd config
- have fixture `configures_httpd` as parameter of test cases
that configure httpd anyway, saving one reload
- add pytest-xdist and filelock to required pyhton modules
- add installs to ruff CI
- give live checks waiting for a server to start up longer time
- add fixtures to tests that rely on a server
- do not stop servers unnecessarily. failures may not start them
properly again, leading to unexpected fails in whatever follows
- add a https: port to httpd that is *not* back by QUIC to allow
failover tests without stopping the QUIC server
Closes#17295
This makes it easier for users to run it with the correct options
locally. Also, update the ruff, pytype and cmakelint versions to the
latest current ones.
Closes#17343
Tests with old cmake are slow. (no Ninja, no unity, and running slower
than recent versions.)
It also revealed that 3.7.2 2017-01-13 is too old to consume curl via
`find_package()` due to:
```
CMake Error at bld-curl/_pkg/lib/cmake/CURL/CURLConfig.cmake:69 (add_library):
add_library cannot create ALIAS target "CURL::libcurl" because target
"CURL::libcurl_shared" is IMPORTED.
Call Stack (most recent call first):
CMakeLists.txt:48 (find_package)
CMake Error at bld-curl/_pkg/lib/cmake/CURL/CURLConfig.cmake:69 (add_library):
add_library cannot create ALIAS target "CURL::libcurl" because target
"CURL::libcurl_shared" is IMPORTED.
Call Stack (most recent call first):
CMakeLists.txt:49 (find_package)
```
The mitigation for this issue requires 3.11.
Also:
- rename a few existing envs to use the `TEST_` prefix.
- make the `find_package` test provider stage verbose.
- fix issue when consuming with cmake 3.7.2 (all platforms):
```
CMake Error at /home/runner/cmake-3.7.2-Linux-x86_64/share/cmake-3.7/Modules/CMakeFindDependencyMacro.cmake:25 (message):
Invalid arguments to find_dependency. VERSION is empty
Call Stack (most recent call first):
bld-curl/_pkg/lib/cmake/CURL/CURLConfig.cmake:52 (find_dependency)
CMakeLists.txt:48 (find_package)
```
Ref: https://github.com/curl/curl/actions/runs/14906066962/job/41868621979?pr=17293#step:9:1199Closes#17293
The pedantic level is experimental. If it causes issues, we may just
disable it alongside the ignore comments.
Also:
- silence error:
```
INFO audit: zizmor: completed label.yml
error[dangerous-triggers]: use of fundamentally insecure workflow trigger
--> label.yml:13:1
|
13 | 'on': [pull_request_target]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^ pull_request_target is almost always used insecurely
|
= note: audit confidence -> Medium
```
- fix pedantic warning:
```
INFO audit: zizmor: completed label.yml
warning[excessive-permissions]: overly broad permissions
--> label.yml:1:1
... |
24 | | with:
25 | | repo-token: '${{ secrets.GITHUB_TOKEN }}'
| |____________________________________________________- default permissions used due to no permissions: block
|
= note: audit confidence -> Medium
```
- silence `template-injection` false positives like:
```
- note: ${{ matrix.build.torture && 'test-torture' || 'test-ci' }} may expand into attacker-controllable code
- note: ${{ contains(matrix.build.install_steps, 'pytest') && 'caddy httpd vsftpd' || '' }} may expand into attacker-controllable code
```
It doesn't seem like these could be controlled by an attacker.
Let me know if I'm missing something.
Closes#17278
The unplanned dropping of the granular vcpkg binary cache indeed fell
into the cracks between Microsoft's various departments. The old method
is now official dropped, without replacement either on the vcpkg side or
the GitHub cache provider side.
Without a granular cache, vcpkg is impractical for builds larger than
a small dependency tree in CI, for performance reasons.
A granular cache is critical for CI use. Building dependencies is not
a goal of this CI, so a more desirable option would be pre-built binary
downloads. This would also allow keeping job timeouts low, which is
important for quick iteration in GHA when a flaky job requiring a manual
retry needs all other jobs to finish first. (GHA often disregards
step timeouts, which is another contributing factor here.)
Windows remains tested extensively with MSYS2, curl-for-win, and via
AppVeyor CI with MSVC + OpenSSL, and also in GHA via scaled back vcpkg
jobs that perform well without caching. What's lost is the recently
added Android OpenSSL build tests.
We may consider building/cachine important dependencies manually as in
GHA/linux-http3, and/or try integrating MSVC jobs with MSYS2 UCRT DLLs.
Ref: https://github.com/microsoft/vcpkg-tool/pull/1662
Ref: https://github.com/microsoft/vcpkg/issues/45073
Follow-up to cd0ec4784c#17089
Follow-up to e3912f0f9f#17086
Follow-up to 15fb1dc7f8#17069Closes#17200
- GHA: add cmake integration tests for Windows.
- make them run faster with prefill, unity, Ninja, omitting curl tool.
- also test static libcurl.
- add old-cmake support with auto-detection.
- auto-detect Ninja.
- run consumer test apps to see if they work.
- add support for Windows.
- make it more verbose.
- re-add `ExternalProject` cmake consumer test. It's broken.
- tidy up terminology.
Cherry-picked from #16973Closes#17203
This step could take from 5 seconds to 5 minutes, sometimes making it
run out of its time slot. It affected 60 CI jobs.
Saving an estimated minimum of 5 minutes per CI run.
Also fixing:
```
Fri, 25 Apr 2025 06:19:14 GMT
Processing triggers for man-db (2.12.0-4build2) ...
Fri, 25 Apr 2025 06:23:40 GMT
Running kernel seems to be up-to-date.
[...]
Error: The action 'install packages' has timed out after 5 minutes.
```
Ref: https://github.com/curl/curl/actions/runs/14658212268/job/41136971525?pr=17180#step:2:169Closes#17181
They typically finish (well) within 10 minutes.
A notable exception was vcpkg jobs when a rebuild was triggered.
With caching lost and reducing them to short builds, this is not
an issue at the moment.
The advantage of shorter timeouts is hung/crashed jobs giving back
control earlier for a manual retry.
Closes#17173
Build/install the man page. Install the script.
This an "import" rather than just adding the script into the release tarball
with maketgz to make sure the tarball build is completely reproducible.
wcurl development, issues etc is still to be done at
https://github.com/curl/wcurl
Assisted-by: Viktor Szakats
Assisted-by: Samuel Henrique
Closes#17035
- fix filter expression for cache step.
It did not cause an issue because `wolfssh` is always paired with
a `wolfssl*`.
- build `wolfssh` against `wolfssl-opensslextra` (was: `wolfssl-all`).
It makes `wolfssh` builds `OPENSSL_COEXIST`-compatible, and clarifies
its use in the clang-tidy job. The earlier mixup didn't cause issues
because the clang-tidy job is compile-only (using their headers only.)
Useful side-effect is making the wolfssh build valgrind-tested.
Reported-by: bo0tzz on github
Ref: https://github.com/curl/curl/discussions/16970#discussioncomment-12752019Closes#17109
To keep testing these with MSVC:
- UWP !ssl
- arm64 build on the `windows-11-arm` runner
- examples
- OpenSSH-Windows (fix install on `windows-11-arm`)
- `windows-2025` runner
- cmake pre-fill checker
Surprise: UWP doesn't support SSPI, which is required by curl's Schannel
backend. Thus, no TLS support for this UWP build. It also suggests
the Schannel UWP mingw-w64 binaries may be broken and just a happy build
accident thanks to mingw-w64 headers being inaccurate.
Building zlib + libssh2 might actually take up to 2+ minutes with vcpkg,
instead of the previously estimated <1.5 minutes.
Follow-up to e3912f0f9f#17086
Follow-up to 15fb1dc7f8#17069Closes#17089
With the new addition of OpenSSL QUIC API support and the support in
ngtcp2 main branch, make the necessary adjustments in curl to support
this combination.
- add support in configure.ac to detect the feature OPENSSL_QUIC_API2 in
openssl
- initialise ngtcp2 properly in this combination
- add a Curl_vquic_init() for global initialisation that ngtcp2 likes
for performance reasons
- add documentation on how to build in docs/HTTP3.md
- add CI testing in http3-linux.yml
Assisted-by: Viktor Szakats
Closes#17027
In the last 1-2 days, curl's vcpkg binary cache entries stored by GHA
apparently disappeared, and each vcpkg job is now rebuilding everything
on every run. This takes up to 40 minutes per job. New cache entries
fail on upload with messages like:
```
Completed submission of nghttp2:arm64-android@1.65.0 to 0 binary cache(s) in 172 ms
```
(note the zero.)
This coincided with my adding support for windows-11-arm and touching
some jobs affected. Yet, unrelated. It also affects Android jobs. The root
cause is a runner image update bumping the vcpkg tool.
The latest vcpkg tool also has this issue.
As a temporary workaround, stop using vcpkg in CI.
Last good run: https://github.com/curl/curl/actions/runs/14473789417/job/40594304523
First bad run: https://github.com/curl/curl/actions/runs/14474616982/job/40597095025
Assisted-by: Tal Regev
Ref: #17070Closes#17069
- bump an MSYS2/mingw job to windows-2025 runner.
(MSVC is possible, but vcpkg needs to build for windows-2025, and
can't share these with windows-2022 builds, so not optimal for
a single canary job.)
- skip installing OpenSSH-Windows-builtin on windows-2025.
It's preinstalled:
```
ssh client found /c/Windows/System32/OpenSSH/ssh.exe is OpenSSH-Windows 9.5.0
ssh server found /c/Windows/System32/OpenSSH/sshd.exe is OpenSSH-Windows 9.5.0
```
Still older than the manual preview install (9.8.1), so keep using that.
Closes#17066
To allow configuring paths styles for SCP and SFTP servers separately.
- make `scp://` URLs use `%SCP_PWD` (was: `%SSH_PWD`).
- make `%SCP_PWD` equal to `%POSIX_PWD`.
To fix test 3022 with OpenSSH-Windows 9.8.0 server.
The fix works on a local machine. Remains broken in CI.
Before this patch, it was equal to `%FILE_PWD` when using
OpenSSH-Windows, otherwise it was `%POSIX_PWD`.
Notice that no matter what path-style we pass, test 3022
was and still is broken with earlier OpenSSH-Windows versions.
(as tested with 9.5.0, 9.5.0-beta20240403, 8.0.0.1)
- rename rest of `%SSH_PWD` uses to `%SFTP_PWD`.
- drop unused `%POSIX_PWD`.
- GHA/windows: test with OpenSSH-Windows server again.
In the LibreSSL MSVC job. This job is short enough to fit the slow
install of the built-in OpenSSH-Windows tools, if needed.
Follow-up to 1abb087a9c#5298
Ref: #16803Closes#17041
On the windows-2022 runner it installs these client/server versions:
```
ssh client found /c/Windows/System32/OpenSSH/ssh.exe is OpenSSH-Windows 9.5.0
ssh server found /c/Windows/System32/OpenSSH/sshd.exe is OpenSSH-Windows 8.1.0
```
Not currently enabled. Slight downside (when enabled) that Windows needs
over 1 minute to install these two tiny programs.
Closes#17046
Found by improving verify-examples.pl:
- Operate directly on markdown files to remove the need to render nroff files
first.
- Add -Wall as a compiler option to find more issues
Closes#17028
