Apparently regexps do not work as expected.
Also seems to pick up some implicit rule from the local machine,
but found no option to make the hidden magic visible.
zizmor 1.9.0 effectively bans using GHA macros within shell script
snippets. Rework them to pass GHA macro values via envs and use those
envs in shell snippets. `${{ env.* }}` macros could be converted
to native env references after making their names shell-compatible.
Envs and shell commands can no longer be used in GHA macro values. Most
cases could be fixed by using literals. Passing quoted values with
spaces combined with other args also doesn't work anymore. This was
replaced by passing them separately.
Despite the initial complications, avoiding GHA macros in scripts does
seems to make the CI code reasonable cleaner. It also makes it possible
to analyze the scripts with shellcheck, finding subtle issues that went
unnoticed till now.
Also:
- un-ignore and fix three existing zizmor `template-injection` issues.
- add script to extract and shellcheck all shell code stored within GHA
and Circle CI YAML files.
- add CI job to run this script.
- fix shellcheck issues discovered.
- fix minor differences between cmake and autotools FreeBSD jobs.
- merge cmake/autotools FreeBSD jobs to avoid developing unwanted
differences again.
- fix/sync quote use across shell code.
- replace `$HOME` with `~` or literal where it made sense.
- replace most `brew --prefix` with literals.
- move all curl install tests to the `curl-install*` prefix.
- add missing curl install tests to cygwin/msys/mingw/*bsd.
- pipe to `tar` instead of storing downloads on disk.
- drop unnecessary `PKG_CONFIG_PATH` when building nghttp3.
Ref: https://github.com/curl/curl/actions/runs/15461461371/job/43523546041
Ref: https://github.com/zizmorcore/zizmor/releases/tag/v1.9.0
Follow-up to e522f47986#17278Closes#17537
- replace ß (scharfes S) with links.
- replace § (section sign) with links.
- replace 🙏 emoji with `🙏`.
Supported by GitHub, Forgejo/Gitea and most likely GitLab.
- docs/libcurl/curl_mprintf.md: replace Unicode ± with `{+|-}`.
- docs/CIPHERS.md: URL encode Unicode in URLs.
- lib1560: use hex encoding in `räksmörgås.se`.
- unit1307: use hex encoding in `Lindmätarv`.
- drop LATIN SMALL LETTER A WITH ACUTE exception.
No longer appears in tests.
This leaves the single character exception: `ö`
And file exceptions holding contributor names.
Follow-up to 9243ed59b3#17329
Follow-up to 838dc53bb7#17247Closes#17335
- make the scanner not whitelist anything for test cases making
everything non-ascii forced to be hex encoded
- update all tests using non-ascii bytes to use %hex[] sequences
Closes#17331
Exclude test data files (4 of them) based on existing feature tags:
`codeset-utf8` and `Unicode`.
Add the new keyword `non-ascii` to mark remaining exceptions (9 files).
Follow-up to 838dc53bb7#17247Closes#17329
- update text on dependency feature detection variables, and move it
to its own section in `docs/INSTALL-CMAKE.md`.
Ref: #17032 (Discussion)
- tidy up descriptions/comments, alpha-sort.
- move comment to its own section in `docs/INSTALL-CMAKE.md`.
- split `HAVE_SSL_SET_QUIC_USE_LEGACY_CODEPOINT` to distinct names for
each TLS backend API. To make the names more stable and to sync them
with autotools.
Follow-up to 07cc50f8eb#17018
Follow-up to 342a654ef3#15873
- drop redundant condition while detecting QUICTLS API.
Follow-up to 07cc50f8eb#17018
- add config-comparison exception for `HAVE_SSL_SET_QUIC_TLS_CBS`.
Follow-up to 5eefdd71a3#17027
- detect `wolfSSL_get_peer_certificate` like autotools does.
- detect `wolfSSL_UseALPN` like autotools does.
Closes#17082
Build/install the man page. Install the script.
This an "import" rather than just adding the script into the release tarball
with maketgz to make sure the tarball build is completely reproducible.
wcurl development, issues etc is still to be done at
https://github.com/curl/wcurl
Assisted-by: Viktor Szakats
Assisted-by: Samuel Henrique
Closes#17035
Change multi's book keeping of transfers to no longer use lists, but a
special table and bitsets for unsigned int values.
`multi-xfers` is the `uint_tbl` where `multi_add_handle()` inserts a new
transfer which assigns it a unique identifier `mid`. Use bitsets to keep
track of transfers that are in state "process" or "pending" or
"msgsent".
Use sparse bitsets to replace `conn->easyq` and event handlings tracking
of transfers per socket. Instead of pointers, keep the mids involved.
Provide base data structures and document them in docs/internal:
* `uint_tbl`: a table of transfers with `mid` as lookup key,
handing out a mid for adds between 0 - capacity.
* `uint_bset`: a bitset keeping unsigned ints from 0 - capacity.
* `uint_spbset`: a sparse bitset for keeping a small number of
unsigned int values
* `uint_hash`: for associating `mid`s with a pointer.
This makes the `mid` the recommended way to refer to transfers inside
the same multi without risk of running into a UAF.
Modifying table and bitsets is safe while iterating over them. Overall
memory requirements are lower as with the double linked list apprach.
Closes#16761
Found by improving verify-examples.pl:
- Operate directly on markdown files to remove the need to render nroff files
first.
- Add -Wall as a compiler option to find more issues
Closes#17028
In the memory and address sanitizer builds.
Verify that nothing unexpected happens.
Starting out with 60 second runs.
The script does not set any seed so it runs with a new random every
time, meaning that if it fails in a single CI run it might not fail in a
subsequent one: but it should still show the full command that failed to
enable us to reproduce it locally. We can work on improving the seed
situation later if this script turns useful.
Closes#16884
Before this patch the curl repository and source tarball distribution
contained test certificates as binary blobs. Used by runtests.
Drop these certificates in favor of generating them dynamically as
part of the build process. Both via autotools and CMake.
As part of this, improve certificates, the generator script and process,
file layout, and fix any issue to make it work fast and smooth both in
CI and local builds.
Note, cert generator scripts require OpenSSL >=1.0.2
(or LibreSSL >=3.1.0). Generation requires POSIX shell, also with CMake.
Without a POSIX shell tests relying on TLS (and stunnel) will fail.
Details:
- build: generate certs as part of the test run process.
- build, tests: generate certs in the build directory.
- binarycheck: drop concept of known binary files with hashes.
- binarycheck: move binary check logic into spacecheck and drop this
separate checker tool.
- build: fix to clean all cert files.
- autotools: fix to not run leaf cert generators in parallel. To avoid
confusion when updating the revocation database and counter.
- scripts: drop `scripts` subdir, merge two scripts into one,
auto-generate root cert, allow generating multiple leafs at once.
- scripts: switch to EC-256 keys (was: RSA-2048). For key size and perf.
- scripts: drop `-x` echo, text dumps, most other output. To avoid log
noise and make it quicker in CI.
- scripts: make it non-RSA-specific.
- scripts: delete unused code.
- scripts: use POSIX shell shebang. Some envs don't have bash (Alpine).
- scripts: pass test pseudo-secrets via the command-line. To avoid:
```
+ openssl genrsa -out test-ca.key -passout fd:0 2048
Invalid password argument, starting with "fd:"
```
- cmake: fix to launch generator scripts via the detected POSIX shell.
- cmake: fix `build-certs` rule to not depend on `SRPFILES`
(`srp-verifier-*`).
- cmake: drop `EXCLUDE_FROM_ALL` for the cert subdir. It makes
the Visual Studio generator miss to create the `clean-certs`,
`build-certs` targets. No target depend on them, so they don't execute
implicitly anyway. Fixes:
```
MSBUILD : error MSB1009: Project file does not exist.
Switch: clean-certs.vcxproj
```
- cmake: add `VERBATIM USES_TERMINAL` to `build-certs` target.
- GHA/linux: install openssl on Alpine, for the cert generator scripts.
Follow-up to 556f722fe3#16593
Follow-up to fa461b4eff#14486Closes#16824
Describes using a package manager or pre-built binaries and adopts the
0.15 installation from source instructions.
Previously the rustls docs described installing rustls-ffi from source
using the GNU Makefile. The upstream project has switched to using
cargo-c as a cross-platform solution that works well with the rust
toolchain, pkg-config, and Windows and so this needs an update.
Similarly, for folks that want to avoid the extra cargo-c tool
requirement, rustls-ffi provides binary releases for common platforms,
and some Linux distributions/package managers offer pre-built packages.
The install instructions are expanded to cover these options since
they're generally better for end users than building the dep. from
source (no `rustc` required).
