Fail on errors from SSL_handle_events().
Force quit Caddy test instance that is left hanging longer with
openssl-quic tests for unknown reasons.
Reported in Joshua's sarif data
Closes#18720
Restructured the code in the following ways:
* add terminal states SUCCESS and FAILED
* split SOCK4 and SOCK5 states to be more clear
* use `bufq` for send/recv of SOCK messages
* reduce SOCKS4 states, more speaking names
* for most states, move code into static function
* reduce SOCKS5 states, more speaking names
* add helpers for traversing to FAILED state
* add helper to flush bufq
* add hepler to read minimum amount into bufq
Closes#18401
- requirements.txt: shorten copyright headers.
- requirements.txt: pin packages to versions.
- GHA/windows: use `tests/requirements.txt`.
Pick a `cryptography` package version that satifies both `impacket`
and pytests dependencies.
- GHA/checksrc: move pip deps into a new `requirements.txt`.
To make Dependabot detect and bump them.
- GHA/checksrc: replace apt packages for python test deps with pip
install `tests/**/requirements.txt` to a venv.
- GHA/checksrc: use venv and drop `--break-system-packages`.
- GHA/linux: fix to actually activate venvs.
Follow-up to 2638570241#15578
- GHA/linux: fixup (did not cause an issue)
Follow-up to d75785c7de#18660
- GHA: create venvs later, simplify commands.
- GHA: sync pip command-line options, e.g. drop progress-bar,
everywhere.
Assisted-by: Dan Fandrich
Closes#18708
The implementation was incomplete and lesser than the other backends. No
one ever reported a bug or requested enhancements for this, indicating
that this backend was never used.
Closes#18700
By appending `:all:[separator]` to the header name. The `[separator]` string
is output between each header value if there are more than one to output.
Test 764 and 765 verify
Idea-by: kapsiR on github
Ref: #18449Closes#18491
It was accidentally broken in commit 0f4c439fc7, shipped since
8.8.0 (May 2024) and yet not a single person has noticed or reported,
indicating that we might as well drop support for FTP Kerberos.
Krb5 support was added in 54967d2a3a (July 2007), and we have
been carrying the extra license information around since then for this
code. This commit removes the last traces of that code and thus we can
remove the extra copyright notices along with it.
Reported-by: Joshua Rogers
Closes#18577
Before this patch `accept4()`, `socket()`, `socketpair()`, `send()` and
`recv()` system symbols were remapped via macros, using the same name,
to local curl debug wrappers. This patch replaces these overrides by
introducing curl-namespaced macros that map either to the system symbols
or to their curl debug wrappers in `CURLDEBUG` (TrackMemory) builds.
This follows a patch that implemented the same for `accept()`.
The old method required tricks to make these redefines work in unity
builds, and avoid them interfering with system headers. These tricks
did not work for system symbols implemented as macros.
The new method allows to setup these mappings once, without interfering
with system headers, upstream macros, or unity builds. It makes builds
more robust.
Also:
- checksrc: ban all mapped functions.
- docs/examples: tidy up checksrc rules.
Follow-up to 9863599d69#18502
Follow-up to 3bb5e58c10#17827Closes#18503
- update Microsoft documentation links.
(also drop language designator where present.)
- checksrc: allow longer than 78 character lines if they
contain a https URL. To make these links easier to use and parse.
- merge links that were split into two lines.
Closes#18626
`-Weverything` is not enabled by curl, and not recommended by LLVM,
because it may enable experimental options, and will result in new
fallouts after toolchain upgrades. This patch aims to fix/silence as much
as possible as found with llvm/clang 21.1.0. It also permanently enables
warnings that were fixed in source and deemed manageable in the future.
`-Wformat` warnings are addressed separately via #18343.
Fix/silence warnings in the source:
- typecheck-gcc.h: fix `-Wreserved-identifier`.
- lib: silence `-Wcast-function-type-strict`.
For llvm 16+ or Apple clang 16+.
- asyn-ares: limit `HAPPY_EYEBALLS_DNS_TIMEOUT` to old c-ares versions.
- curl_trc: fix `-Wc++-hidden-decl`.
- doh: fix `-Wc++-keyword`.
- ftp: fix `-Wreserved-identifier`.
- ldap: fix `-Wreserved-identifier`.
- mqtt: comment unused macro to avoid warning.
- multi_ev: drop unused macros to avoid warnings.
- setopt: fix useless `break;` after `return;`.
- gtls, mbedtls, rustls: silence `-Wconditional-uninitialized`.
- socks_sspi, schannel, x509asn1: fix `-Wimplicit-int-enum-cast`.
- x509asn1: fix `-Wc++-keyword`.
- openssl: scope `OSSL_UI_METHOD_CAST` to avoid unused macro warning.
- libssh2, wolfssl: drop unused macros.
- curl_ngtcp2, curl_quiche, httpsrr, urlapi: drop/limit unused macros.
- tool_getparam: fix useless `break;` after `return;` or `break;`.
Not normally enabled because it doesn't work with unity.
https://github.com/llvm/llvm-project/issues/71046
- tool_operate: fix `-Wc++-keyword`.
- curlinfo: fix a `-Wunsafe-buffer-usage`.
- tests: silence `-Wformat-non-iso`.
- lib557: fix `-Wreserved-identifier`.
- lib1565: silence `-Wconditional-uninitialized`.
Enable the above clang warnings permanently in picky mode:
- `-Wc++-hidden-decl`
- `-Wc++-keyword` (except for Windows, where it collides with `wchar_t`)
- `-Wcast-function-type-strict`
- `-Wcast-function-type`
- `-Wconditional-uninitialized`
- `-Wformat-non-iso` (except for clang-cl)
- `-Wreserved-identifier`
- `-Wtentative-definition-compat`
Silence problematic `-Weverything` warnings globally (in picky mode):
- `-Wused-but-marked-unused` (88000+ hits) and
`-Wdisabled-macro-expansion` (2600+ hits).
Triggered by `typecheck-gcc.h` when building with clang 14+.
Maybe there exists a way to fix within that header?
Ref: https://discourse.llvm.org/t/removing-wused-but-marked-unused/55310
- `-Wunsafe-buffer-usage`. clang 16+. 7000+ hits.
May be useful in theory, but such high volume of hits makes it
impractical to review and possibly address. Meant for C++.
Ref: https://clang.llvm.org/docs/SafeBuffers.html
Ref: https://stackoverflow.com/questions/77017567/how-to-fix-code-to-avoid-warning-wunsafe-buffer-usage
Ref: https://discourse.llvm.org/t/rfc-c-buffer-hardening/65734
Ref: https://github.com/llvm/llvm-project/pull/111624
- `-Wimplicit-void-ptr-cast`. clang 21+. 1700+ hits.
C++ warning, deemed pure noise.
Ref: https://github.com/curl/curl/issues/18470#issuecomment-3253506266
- `-Wswitch-default` (180+ hits), `-Wswitch-enum` (190+ hits),
`-Wcovered-switch-default` (20+ hits).
Next to impossible to fix cleanly, esp. when the covered `case`
branches depend on compile-time options.
- `-Wdocumentation-unknown-command` (8+ hits).
Triggered in a few sources. Seems arbitrary and bogus.
- `-Wpadded` (550+ hits).
- `-Wc++-keyword` on Windows, where it collides with `wchar_t`.
(100+ hits)
Ref: https://github.com/llvm/llvm-project/issues/155988
- `-Wreserved-macro-identifier`. clang 13+. 5+ hits.
Sometimes it's necessary to set external macros that use
the reserved namespace. E.g. `_CRT_NONSTDC_NO_DEPRECATE`,
`__ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__`, `__NO_NET_API`,
possibly `_REENTRANT`, and more.
It's not worth trying to silence them individually.
- `-Wnonportable-system-include-path` with `clang-cl`.
It'd be broken by doing what the warning suggests.
- `-Wformat-non-iso` for clang-cl.
CMake `PICKY_COMPILER=ON` (the default) or `./configure`
`--enable-warnings` (not the default) is required to enable these
silencing rules.
Also:
- autotools, cmake: fix Apple clang and mainline llvm version translations.
Ref: https://en.wikipedia.org/wiki/Xcode#Toolchain_versions
- autotools, cmake: enable `-Warray-compare` for clang 20+.
Follow-up to 4b7accda5a#17196
- cmake: fix to enable `-Wmissing-variable-declarations` at an earlier
clang version.
- cmake: update internal logic to handle warning options with `+` in
them.
- cmake: fix internal logic to match the whole option when looking
into `CMAKE_C_FLAGS` for custom-disabled warnings.
Follow-up to b85cb8cb4e#18485Closes#18477
We used to treat 0 as "call strlen() to get the length" for
curlx_base64_encode, but it turns out this is rather fragile as we
easily do the mistake of passing in zero when the data is actually not
there and then calling strlen() is wrong.
Force the caller to pass in the correct size. A zero length input string
now returns a zero length output and a NULL pointer.
Closes#18617
Because parts of the cookie loading happens on transfer start the
in-memory cookie jar risks being incomplete and then a save might
wrongly truncate the target file.
Added test 1902 to verify.
Reported-by: divinity76 on github
Fixes#18621Closes#18622
- When an option name is used in text, this script no longer outputs the
short plus long version in the manpage output. It makes the text much
more readable.
This always showing both verions was previously done primarily to make
sure roffit would linkify it correctly, but since roffit 0.17 it
should link both long or short names correctly.
- When managen outputs generic text about options at the end of the
description it now highlights them properly so that they too get
linkified correctly in the HTML version. For consistency.
Closes#18580
Remove use of pthread_cancel in asnyc threaded resolving. While there
are system where this works, others might leak to resource leakage
(memory, files, etc.). The popular nsswitch is one example where resolve
code can be dragged in that is not prepared.
The overall promise and mechanism of pthread_cancel() is just too
brittle and the historcal design of getaddrinfo() continues to haunt us.
Fixes#18532
Reported-by: Javier Blazquez
Closes#18540
Access the error codes of ngtcp2 when a connect attempt failes. Trace
the information for analysis. Treat errors as permanent failure by
default, trigger retrying only when the server refused without
indicating an error.
Closes#18521
Tweaks around handling of --limit-rate:
* tracing: trace outstanding timeouts by name
* multi: do not mark transfer as dirty that have
an EXPIRE_TOOFAST set
* multi: have one static function to asses speed limits
* multi: when setting EXPIRE_TOOFAST remove the transfers
from the dirty set
* progress: rename vars and comment on how speed limit
timeouts are calculated, for clarity
* transfer: when speed limiting, exit the receive loop
after a quarter of the limit has been received, not
on the first chunk received.
* cf-ip-happy.c: clear EXPIRE_HAPPY_EYEBALLS on connect
* scorecard: add --limit-rate parameter to test with
speed limits in effect
Closes#18454
Only actually needed servers should be listed and none is then implied
if no servers are listed.
Outputs a warning if "none" is still set as a server.
Closes#18466
If a string argument is expected and the first two bytes are 0xe2 ex80
and the third has the 7th bit set, that's enough for curl to warn.
Previously we tried to detect and warn only for the unicode double
quote, but users might use single quotes, other quotes or even lead the
argument with one of the "zero widths" characters. This is an attempt to
detect many of those. Without triggering for "normal" IDN hostnames.
Closes#18459
Split out adding of individual request headers into a switch. Check
the connection http version only on fresh connections, use separate
methods.
Add TE: header directly without allocation. Add bit for indicating
Connection: header has been added and custom headers should not do
that again.
Closes#18444
test_10_08, uploading larger files for a h2 proxy, sporadically fails
with a decrpytion error on received data in AWS-LC. The frequency can
be increased by simulated network receive blocks.
Not setting a 4 * TLS record sized buffer, leaving AWS-LC at its
default buffer size seems to mitigate this problem.
Closes#18434
This allows --remote-time to set dates before 1970.
Due to a minor omission in the API, it will still avoid setting the time
if it is indeed exactly epoch 0 (jan 1 1970).
Verified by test 762
Fixes#18424
Reported-by: Terence Eden
Closes#18443
- using {} with single entries makes little sense
- when using {} sets with two entry lists, there can only be 64 to reach
maximum number of URLs
Verify the max check in test 761
Write out 9-length frames to client's WRITEFUNCTION
Read 0-length frames from READFUNCTION *if* the function
started a new frame via `curl_ws_start_frame()`.
Fixes#18286Closes#18332
Reported-by: Andriy Druk
Every time we first run runtests with -j and then again *without* -j,
this message was shown:
"Warning: $runnerid: cleardir(log) failed"
Not anymore.
Closes#18404
If --trace-config is used to set a level before -v is used, don't reset
the state on first -v (to "-all") as it otherwise does. This way,
--trace-config can be used to set specific trace items before -v on the
command line and it still works.
Previously, the first -v use would otherwise reset and undo the earlier
--trace-config items.
Fixes#18346Closes#18361
- schannel: apply BoringSSL workaround to AWS-LC too.
Affects Schannel + AWS-LC MultiSSL builds. (not tested in CI)
Ref: 274940d743#2643#2634
- curl_ntlm_core: deduplicate macro defines.
- curl_ntlm_core: document version thresholds for an AWS-LC-specific
workaround.
It was necessary between v1.2.0 2022-09-01 and v1.30.1 2024-06-21.
No longer necessary since v1.31.0 2024-07-01:
ba94617d99
Follow-up to 34ef4fab22#10320
- lib758: drop redundant OpenSSL version guards.
`OPENSSL_VERSION_NUMBER > 3` automatically guards against LibreSSL,
BoringSSL and AWS-LC.
Ref: https://github.com/curl/curl/pull/18288/commits/6ddd8f2c0bbfcb0847b1ee7f257fb772fa47310c
Follow-up to a5f0ab7995#18288
- dllmain, curl_sha512_256: formatting.
Closes#18387
To avoid NULL derferences with many outputs and --next.
Follow-up to 034612cd51
Add test 760 to verify
Reported-by: BobodevMm on github
Fixes#18375Closes#18376
... even if there are tests still pending. To help visualize tests that
might be hanging.
Attempts to set the limit at 10 seconds of silence.
Closes#18349
Changed strategy to start up and terminate resolver thread.
When starting up:
Start the thread with mutex acquired, wait for signal from thread that
it started and has incremented the ref counter. Thread set
pthread_cancel() to disabled before that and only enables cancelling
during resolving itself. This assure that the ref counter is correct and
the unlinking of the resolve context always happens.
When shutting down resolving:
If ref counting shows thread has finished, join it, free everything. If
thread has not finished, try pthread_cancel() (non Windows), but keep
the thread handle around.
When destroying resolving:
Shutdown first, then, if the thread is still there and 'quick_exit' is
not set, join it and free everything. This might occur a delay if
getaddrinfo() hangs and cannot be interrupted by pthread_cancel().
Destroying resolving happens when another resolve is started on an
easy handle or when the easy handle is closed.
Add test795 to check that connect timeout triggers correctly
when resolving is delayed. Add debug env var `CURL_DNS_DELAY_MS`
to simulate delays in resolving.
Fix test1557 to set `quick_exit` and use `xxx.invalid` as domain
instead of `nothing` that was leading to hangers in CI.
Closes#18263
