- URLs specified outside of the markdown []() are now extracted and
checked
- also check TODO, FAQ and KNOWN_BUGS
- more aggressive avoiding to check github.com/curl/curl, all uses of
example domains and some more established URLs on the curl.se site
- list all errors in the end to make them easier to spot in CI logs
Closes#19848
To reduce to amount of Debian packages to install, which hopefully
removes some flakiness due to sometimes very slow Azure package
distro servers. Possible also making these jobs finish 20s faster.
Windows from Debian | llvm | gcc
:------------------ | :----------------: | :----------------:
build time | 2m41s -> 2m20s | 3m19s -> 2m57s
installed packages | 288 -> 142 | 247 -> 99
downloads | 403 MB -> 240 MB | 297 MB -> 134 MB
disk space | 2132 MB -> 1289 MB | 1582 MB -> 739 MB
Before: https://github.com/curl/curl/actions/runs/19765983026
After: https://github.com/curl/curl/actions/runs/19766373960?pr=19749
Ref: 02149b7e36Closes#19749
Install CMake from the Kitware GitHub release archive. To allow choosing
its version independently from the OS.
Switch to 3.7.0 (from 3.7.2) to test the earliest supported version.
Also tested OK with 3.18.4 and 3.7.2.
The download and install step takes 1-2 seconds.
Follow-up to c9e50e9e39#19737Closes#19738
GnuTLS 3.8.11 started requiring a nettle version new enough to be
missing from Ubuntu LTS released a year ago. To keep up testing it,
build nettle from source. Besides the necessary one time effort this
has the downside that nettle updates now need to be done manually
a couple of times per year when renovate detects one. (if I got the
renovate formula correct to catch the tag format).
Also:
- switch the local GnuTLS build to use the release tarball instead of
the Git repo and calling the script `bootstrap`. The script could
potentially download source code using the cleartext `git:` protocol.
It's also downloading lots of content, including a full OpenSSL repo.
Ref: 955f7a7fc2/NEWS (L41-L44)
Follow-up to 905b718de3#19642
Follow-up to a439fc0e37#19613Closes#19680
This reverts commit a439fc0e37.
It requires a version of libnettle that is not included in these Ubuntu
versions: "Libnettle 3.10 was not found"
Closes#19642
Usage:
- autotools: `--disable-typecheck` (or `--enable-typecheck` (default))
- cmake: `-DCURL_DISABLE_TYPECHECK=ON`.
To disable `curl_easy_setopt()`/`curl_easy_getinfo()` type checking with
supported (new) gcc and clang compilers. It is useful to improve build
performance for the `tests/libtest` target. In particular the CodeQL
analyzer may take above an hour to compile with type checking enabled,
and disabling it brings it down to seconds. On local machines it may
also cut build times in half when build testdeps, depending on platform
and compiler.
Other than these cases, we recommend leaving type checking enabled.
Ref: fdacf34aae#19632
Also:
- GHA/codeql: use it.
- test1165: check in `include/curl`.
- lib1912: delete stray todo comment.
- spelling and comment nits.
Closes#19637
macOS was chosen because xmllint comes preinstalled, saving the prereq
install step. But, macOS's xmllint jobs sometimes doesn't finish in 1m
(instead of under 1 second) and gets cancelled, causing flaky failures.
Go with Linux and an install phase (of 15s) instead.
Examples:
https://github.com/curl/curl/actions/runs/19558021722/job/56004334495Closes#19634
Turns out the cause of CodeQL hangs (or probably just extreme long
compile) is the header `curl/typecheck-gcc.h`. By accident I noticed
that the preprocessed output of libtests.c is 75 MB (megabytes). This
is much higher than the amounf of source code hinted, also compared to
e.g. units.c or other build targets. The reason for the extreme size
is each easy option call pulling in the large checker logic defined
in this header.
By compiling with `-DCURL_DISABLE_TYPECHECK`, preprocessed output drops
to 2.2 MB (34x), and the libtests target builds without issues.
Also build all tests and examples with the Linux HTTP/3 config, covering
3 more files.
With these, CodeQL C coverage is 893 out of 930 (96%) (was: 645 69%)
Follow-up to 71fc11e6bb#18695
Follow-up to a333fd4411#18557
Follow-up to b4922b1295#18564
Closes https://github.com/vszakats/curl/pull/11Closes#19632
Only the `test-ci` build target sets `--buildinfo` automatically,
since 985f39c0ce. It needs to be set
manually for other targets used in CI, such as `test-torture`,
to enable the `buildinfo.txt` dump in the runtests step.
For Test Clutch. In an attempt to re-sync `targetarch` with the rest of
macOS jobs on the feature matrix page:
https://testclutch.curl.se/static/reports/feature-matrix.html
Before this patch and possibly since the breaking update It's `aarch64e`
for torture jobs and `aarch64` for the rest
(stricly speaking `aarch64e` is the correct value for all macOS jobs, but
autotools and cmake report arm64/aarch64 without the `e`.)
Regression from 985f39c0ce#18147Closes#19601
There are 58 non-compliant files. Mark them with the `notxml` keyword.
Also include the compliant ones in the GHA/checksrc xmllint CI job.
Also:
- delete XML prolog from the 3 test data files that had them.
- FILEFORMAT.md: document the `notxml` keyword.
- FILEFORMAT.md: fix a typo.
Follow-up to de49cc89ab#19470
Follow-up to f3095f0dbd#19528
Follow-up to 87ba80a6dfCloses#19595
It also means that all supported OpenSSL versions and forks support
TLSv1.3 after this patch.
It reduces `openssl.c` size by more than 10%, or 400 LOC.
Ref: #18822Closes#18330
- badwords.pl: add `-a` option to check all lines in source code files.
Before this patch indented lines were skipped (to avoid Markdown code
fences.)
- GHA/checksrc: use `-a` when verifying the source code.
- GHA/checksrc: disable `So` and `But` rules for source code.
- GHA/checksrc: add docs/examples to the verified sources.
- badwords.txt: delete 4 duplicates.
- badwords.txt: group and sort contractions.
- badwords.txt: allow ` url = `, `DIR`, `<file name`.
Closes#19536
Fix SC2046: "Quote this to prevent word splitting."
Ref: https://www.shellcheck.net/wiki/SC2046
Also:
- shellcheck.sh: add `set -eu`.
- shellcheck.sh, yamlcheck.sh: always run from repo root.
- pass `--` before passing the list of files, where missing.
- badwords.pl, cleancmd.pl: rework to accept `git ls-files` arguments.
Requires Perl 5.22+ (2015-Jun-01) on Windows.
Ref: https://perldoc.perl.org/functions/open
- INTERNALS.md: require Perl 5.22 on Windows.
- spacecheck.pl: formatting.
- GHA/http3-linux: rework command to avoid SC2046.
- stop suppressing SC2046 warnings.
The yamlcheck.sh issue reported-by: Stanislav Fort (Aisle Research)
Ref: 20251109163515_6eb31da3-deb2-4f4d-8327-935904f27da5
Closes#19432
To debug the Windows CI fails further. Acting on the suspicions that
`taskkill` may sometimes be applied to the wrong process.
- log task info, and task child info before calling `taskkill` on a PID.
(on native Windows.)
One of the calls needs PowerShell.
- add env `CURL_TEST_NO_TASKKILL` to disable using `taskkill`.
- add env `CURL_TEST_NO_TASKKILL_TREE` to use `taskkill` without
`-t`, meaning to kill the process, but not child processes.
- GHA/windows: disable `taskkill` calls, to see what happens.
I'll revert or tweak this in a future commit depending on results.
Ref: https://github.com/curl/curl/discussions/14854#discussioncomment-13062859
Ref: https://github.com/curl/curl/discussions/14854#discussioncomment-14913014Closes#19421
They were disabled since these jobs ran in Zuul. The tests are 320, 321,
322, 323, 324. Of which, 323 runs in CI, the rest needs `gnutls-serv`
with SRP enabled, which is not available in current jobs and no longer
offered by Ubuntu's `gnutls-bin` package. 324 doesn't appear as
a skipped test, 323 seems to be running fine, the rest are logged as
skipped. This suggests it's safe to drop the exceptions.
Closes#19413
Build and cache LibreSSL locally with Fil-C and enable it in the Fil-C
job.
Also:
- disable test 776 in the Fil-C job. It fails consistently, and due to
flakiness seen earlier its result is disabled. In this job it seems to
be adding 1 to 9 minues to the test run step and fails consistently.
- include Fil-C version in the LibreSSL cache key to prepare for Fil-C
ABI changes.
- GHA/linux: fully quote `tflags` values to avoid breaking YAML.
Tested and confirmed working with OpenSSL too, but ended up with
LibreSSL for faster, smaller builds.
Closes#19407
Requirements for Fil-C:
- not to accidentally pick up system headers. E.g. from `/usr/include`
on Linux. It can happen when any dependency is auto-detected on this
header path. This makes Fil-C find the wrong system headers, which
in turn breaks the configuration step in subtle ways (with CMake) and
less subtle ways (autotools). Then CMake ends up running into an error
while compiling.
- build all dependencies with Fil-C too.
(this patch doesn't build any dependencies yet.)
- "unity" mode disabled. It should work, but needs a lot of memory and
slower than a standard compiler, or a Fil-C non-unity build.
- x86_64 Linux host platform when using the pre-built toolchain.
Observations on a minimal, static build made with no dependencies and
Fil-C 0.674 (based on clang 20.1.8).
- curl tool sizes:
- cmake, default, w/o -O: 30 MB (gcc 14.2.0: 1.7 MB)
- cmake, default, w/o -O, stripped: 29.6 MB (gcc: 1.4 MB)
- cmake, Release, -O3: 7.2 MB (gcc: 1 MB)
- cmake, Release, -O3, stripped: 6.8 MB (gcc: 0.93 MB)
- autotools, default, -O2: 7 MB
- libcurl.a size is 32 MB (cmake, default, w/o -O) (gcc: 2.7 MB)
- build times 3-3.5x longer (compared to system gcc 14.2.0):
- all runtests available pass OK.
- all pytests skipped due to missing features/dependencies.
- shared libcurl builds also work (cmake, default: 25 MB libcurl.so and
5.75 MB (5.6 stripped) curl tool)
- autotools works fine too, with dependencies disabled or set to avoid
`/usr/include`.
Closes#19391
It also does shellcheck on `run:` elements, overlapping with
the homegrown `shellcheck-ci.sh` with the same purpose. But it also does
more and perhaps could replace the script too, especially in curl
sub-repos.
Also:
- GHA/macos: delete potentially useful, but commented, and ultimately
unused, non-default Xcode-testing logic. It's causing unused matrix
exceptions, upsetting actionlint.
Ref: https://github.com/rhysd/actionlintCloses#19373
To add another, so far untested standalone toolchain variant to the mix.
This distro is a fairly compact, GCC mingw-w64.
Replacing an existing 15.0.1 snapshot toolchain build job.
Ref: https://github.com/skeeto/w64devkit/releasesCloses#19369
