Before this patch curl used the C preprocessor to override standard
memory allocation symbols: malloc, calloc, strdup, realloc, free.
The goal of these is to replace them with curl's debug wrappers in
`CURLDEBUG` builds, another was to replace them with the wrappers
calling user-defined allocators in libcurl. This solution needed a bunch
of workarounds to avoid breaking external headers: it relied on include
order to do the overriding last. For "unity" builds it needed to reset
overrides before external includes. Also in test apps, which are always
built as single source files. It also needed the `(symbol)` trick
to avoid overrides in some places. This would still not fix cases where
the standard symbols were macros. It was also fragile and difficult
to figure out which was the actual function behind an alloc or free call
in a specific piece of code. This in turn caused bugs where the wrong
allocator was accidentally called.
To avoid these problems, this patch replaces this solution with
`curlx_`-prefixed allocator macros, and mapping them _once_ to either
the libcurl wrappers, the debug wrappers or the standard ones, matching
the rest of the code in libtests.
This concludes the long journey to avoid redefining standard functions
in the curl codebase.
Note: I did not update `packages/OS400/*.c` sources. They did not
`#include` `curl_setup.h`, `curl_memory.h` or `memdebug.h`, meaning
the overrides were never applied to them. This may or may not have been
correct. For now I suppressed the direct use of standard allocators
via a local `.checksrc`. Probably they (except for `curlcl.c`) should be
updated to include `curl_setup.h` and use the `curlx_` macros.
This patch changes mappings in two places:
- `lib/curl_threads.c` in libtests: Before this patch it mapped to
libcurl allocators. After, it maps to standard allocators, like
the rest of libtests code.
- `units`: before this patch it mapped to standard allocators. After, it
maps to libcurl allocators.
Also:
- drop all position-dependent `curl_memory.h` and `memdebug.h` includes,
and delete the now unnecessary headers.
- rename `Curl_tcsdup` macro to `curlx_tcsdup` and define like the other
allocators.
- map `curlx_strdup()` to `_strdup()` on Windows (was: `strdup()`).
To fix warnings silenced via `_CRT_NONSTDC_NO_DEPRECATE`.
- multibyte: map `curlx_convert_*()` to `_strdup()` on Windows
(was: `strdup()`).
- src: do not reuse the `strdup` name for the local replacement.
- lib509: call `_strdup()` on Windows (was: `strdup()`).
- test1132: delete test obsoleted by this patch.
- CHECKSRC.md: update text for `SNPRINTF`.
- checksrc: ban standard allocator symbols.
Follow-up to b12da22db1#18866
Follow-up to db98daab05#18844
Follow-up to 4deea9396b#18814
Follow-up to 9678ff5b1b#18776
Follow-up to 10bac43b87#18774
Follow-up to 20142f5d06#18634
Follow-up to bf7375ecc5#18503
Follow-up to 9863599d69#18502
Follow-up to 3bb5e58c10#17827Closes#19626
Change the send parameter from `const void *` to `const uint8_t *` and
adapt calling code. Several had already unsigned chars and were casting.
Closes#19729
Change `inputbuff` parameter from `const char *` to `const uint8_t *` to
reflect the binary nature of the input bytes. Half the code was casting
unsigned char to signed already in calling.
Closes#19722
Add protocol handler flag `PROTOPT_CONN_REUSE` to indicate that the
protocol allows reusing connections for other tranfers. Add that
to all handlers that support it.
Create connections with `conn->bits.close = FALSE` and remove all
the `connkeep()` calls in protocol handlers setup/connect implementations.
`PROTOPT_CONN_REUSE` assures that the default behaviour applies
at the end of a transfer without need to juggle the close bit.
`conn->bits.close` now serves as an additional indication that a
connection cannot be reused. Only protocol handles that allow
reuse need to set it to override the default behaviour.
Remove all `connclose()` and `connkeep()` calls from connection
filters. Filters should not modify connection flags. They are
supposed to run in eyeballing situations where a filter is just
one of many determining the outcome.
Fix http response header handling to only honour `Connection: close`
for HTTP/1.x versions.
Closes#19333
When OpenSSL fails to verify the peer certificate, we checked for
one specific reason code and did not ask Apple SecTrust for any
other failure.
Always ask Apple SecTrust after OpenSSL fails when the `native_ca_store`
is enabled. If the user configures a CAfile or CApath, the native store
is disabled, so this does not affect use cases where users asks curl
to use a specific set of trust anchors.
Do the same for GnuTLS
Fixes#19636
Reported-by: ffath-vo on github
Closes#19638
If verifypeer and verifyhost are disabled, to not generate
a failf() message for failed verifications.
Fixes#19615
Reported-by: ncaklovic on github
Closes#19625
It also means that all supported OpenSSL versions and forks support
TLSv1.3 after this patch.
It reduces `openssl.c` size by more than 10%, or 400 LOC.
Ref: #18822Closes#18330
- badwords.pl: add `-a` option to check all lines in source code files.
Before this patch indented lines were skipped (to avoid Markdown code
fences.)
- GHA/checksrc: use `-a` when verifying the source code.
- GHA/checksrc: disable `So` and `But` rules for source code.
- GHA/checksrc: add docs/examples to the verified sources.
- badwords.txt: delete 4 duplicates.
- badwords.txt: group and sort contractions.
- badwords.txt: allow ` url = `, `DIR`, `<file name`.
Closes#19536
Rename `Curl_timeleft()` to `Curl_timeleft_ms()` to make the units in
the returned `timediff_t` clear. (We used to always have ms there, but
with QUIC started to sometimes calc ns as well).
Rename some assigned vars without `_ms` suffix for clarity as well.
Closes#19486
Also:
- replace `manpage` with `man page`, add to `badwords.txt`.
- badwords.pl: import `-w` feature from curl-www, syncing the two
scripts fully.
- badwords.txt: import missing items from curl-www, syncing the two
files fully.
- pyspelling.words: drop `cURL` allowed word.
Closes#19468
find_ssl_filter used while(cf && cf->next) and skipped the last node.
If the SSL filter was last, channel binding lookup failed and we returned
CURLE_BAD_FUNCTION_ARGUMENT. Switch to while(cf) so the tail is examined.
This bug was found with ZeroPath.
Closes#19229
The definition of these constants does not give a numeric ordering
and MAX_DEFAULT needs to be checked in addition of ciphers and QUIC
checks to apply correctly.
Fixes#19340
Reported-by: Peter Piekarski
Closes#19341
When openssl does not verify the certificate, but apple sectrust
does, we also pass it the ocsp stapled response when configured and
available.
When openssl does not verify the cert chain, it will also not be able
to verify the ocsp stapling. Do not call it if sectrust is the
verifier of the cert chain.
Fixes#19307
Reported-by: Harry Sintonen
Closes#19308
Replace the check if a ssl session cache is configured with
a function checking if it is configured *and* if an ssl session
cache is available.
During normal operations, a session cache is always there, however
for "connect-only" transfers this might not be the case. When such
transfers receive new sessions/tickets, they need to silently
discard those and not fail.
Reported-by: Marc Aldorasi
Fixes https://github.com/curl/curl/issues/18983
Closes https://github.com/curl/curl/pull/19251
Curl is one of the last callers of PKCS12_PBE_add(). It has been a noop
since OpenSSL 0.9.8k (2006) stubbed it out when moving the built-in PBE
algorithms to a static table:
b8f702a0afCloses#19201
In providercheck(), when failing to open the "store", the exit path
would not previously free the created UI_METHOD and instead leak this
resource.
Pointed out by ZeroPath
Closes#19114
After this patch, the codebase no longer overrides system printf
functions. Instead it explicitly calls either the curl printf functions
`curl_m*printf()` or the system ones using their original names.
Also:
- drop unused `curl_printf.h` includes.
- checksrc: ban system printf functions, allow where necessary.
Follow-up to db98daab05#18844
Follow-up to 4deea9396b#18814Closes#18866
Also:
- tests/server: replace local `sstrerror()` with `curlx_strerror()`.
- tests/server: show the error code next to the string, where missing.
- curlx: use `curl_msnprintf()` when building for src and tests.
(units was already using it.)
- lib: drop unused includes found along the way.
- curlx_strerror(): avoid compiler warning (and another similar one):
```
In file included from servers.c:14:
../../lib/../../lib/curlx/strerr.c: In function ‘curlx_strerror’:
../../lib/../../lib/curlx/strerr.c:328:32: error: ‘snprintf’ output may be truncated before the last format character [-Werror=format-truncation=]
328 | SNPRINTF(buf, buflen, "%s", msg);
| ^
../../lib/../../lib/curlx/strerr.c:47:18: note: ‘snprintf’ output 1 or more bytes (assuming 2) into a destination of size 1
47 | #define SNPRINTF snprintf
| ^
../../lib/../../lib/curlx/strerr.c:328:7: note: in expansion of macro ‘SNPRINTF’
328 | SNPRINTF(buf, buflen, "%s", msg);
| ^~~~~~~~
```
Follow-up to 45438c8d6f#18823Closes#18840
```
lib/vtls/openssl.c: In function 'asn1_object_dump':
lib/vtls/openssl.c:299:42: error: passing argument 3 of 'i2t_ASN1_OBJECT' discards 'const' qualifier from pointer target type [-Werror=discarded-qualifiers]
299 | int i = i2t_ASN1_OBJECT(buf, (int)len, a);
| ^
In file included from /home/runner/djgpp/include/openssl/objects.h:965,
from /home/runner/djgpp/include/openssl/evp.h:94,
from /home/runner/djgpp/include/openssl/x509.h:73,
from /home/runner/djgpp/include/openssl/ssl.h:156,
from lib/curl_ntlm_core.c:71,
from bld/lib/CMakeFiles/libcurl_static.dir/Unity/unity_0_c.c:88:
/home/runner/djgpp/include/openssl/asn1.h:921:58: note: expected 'ASN1_OBJECT *' {aka 'struct asn1_object_st *'} but argument is of type 'const ASN1_OBJECT *' {aka 'const struct asn1_object_st *'}
921 | int i2t_ASN1_OBJECT(char *buf, int buf_len, ASN1_OBJECT *a);
| ~~~~~~~~~~~~~^
```
Ref: https://github.com/curl/curl/actions/runs/18236773678/job/51931937131?pr=18039
Follow-up to bb46d42407#18647Closes#18841
- configure/cmake support for enabling the option
- supported in OpenSSL and GnuTLS backends
- when configured, Apple SecTrust is the default trust store
for peer verification. When one of the CURLOPT_* for adding
certificates is used, that default does not apply.
- add documentation of build options and SSL use
Closes#18703
- it's just too random who got mentioned
- we can't mention all, so better consistently mention none
- make sure they all are mentioned in THANKS
- also remove some unnecessary comment ramblings
Closes#18803
- GHA/checkdocs: rename `spellcheck` job to `pyspelling` to say
the exact tool used.
- GHA/checkdocs: restore a comment.
- GHA/linux: add `-B .` to a cmake configure to avoid warning, and
future breakage.
- autotools: use correct casing for `Schannel`.
- doh: update RFC URL.
- drop redundant parenthesis.
- fix indentation, whitespace.
Closes#18756
Since it would indicate errors to the degree that continuing would just
risk hiding the earlier errors or make things weird.
Inspired by a report in Joshua's sarif data
Closes#18646
When OpenSSL reports SSL_ERROR_WANT_READ, set the io_need explicitly.
It should have already been set by the BIO, but be safe.
Reported in Joshua's sarif data
Closes#18733
- update Microsoft documentation links.
(also drop language designator where present.)
- checksrc: allow longer than 78 character lines if they
contain a https URL. To make these links easier to use and parse.
- merge links that were split into two lines.
Closes#18626
`-Weverything` is not enabled by curl, and not recommended by LLVM,
because it may enable experimental options, and will result in new
fallouts after toolchain upgrades. This patch aims to fix/silence as much
as possible as found with llvm/clang 21.1.0. It also permanently enables
warnings that were fixed in source and deemed manageable in the future.
`-Wformat` warnings are addressed separately via #18343.
Fix/silence warnings in the source:
- typecheck-gcc.h: fix `-Wreserved-identifier`.
- lib: silence `-Wcast-function-type-strict`.
For llvm 16+ or Apple clang 16+.
- asyn-ares: limit `HAPPY_EYEBALLS_DNS_TIMEOUT` to old c-ares versions.
- curl_trc: fix `-Wc++-hidden-decl`.
- doh: fix `-Wc++-keyword`.
- ftp: fix `-Wreserved-identifier`.
- ldap: fix `-Wreserved-identifier`.
- mqtt: comment unused macro to avoid warning.
- multi_ev: drop unused macros to avoid warnings.
- setopt: fix useless `break;` after `return;`.
- gtls, mbedtls, rustls: silence `-Wconditional-uninitialized`.
- socks_sspi, schannel, x509asn1: fix `-Wimplicit-int-enum-cast`.
- x509asn1: fix `-Wc++-keyword`.
- openssl: scope `OSSL_UI_METHOD_CAST` to avoid unused macro warning.
- libssh2, wolfssl: drop unused macros.
- curl_ngtcp2, curl_quiche, httpsrr, urlapi: drop/limit unused macros.
- tool_getparam: fix useless `break;` after `return;` or `break;`.
Not normally enabled because it doesn't work with unity.
https://github.com/llvm/llvm-project/issues/71046
- tool_operate: fix `-Wc++-keyword`.
- curlinfo: fix a `-Wunsafe-buffer-usage`.
- tests: silence `-Wformat-non-iso`.
- lib557: fix `-Wreserved-identifier`.
- lib1565: silence `-Wconditional-uninitialized`.
Enable the above clang warnings permanently in picky mode:
- `-Wc++-hidden-decl`
- `-Wc++-keyword` (except for Windows, where it collides with `wchar_t`)
- `-Wcast-function-type-strict`
- `-Wcast-function-type`
- `-Wconditional-uninitialized`
- `-Wformat-non-iso` (except for clang-cl)
- `-Wreserved-identifier`
- `-Wtentative-definition-compat`
Silence problematic `-Weverything` warnings globally (in picky mode):
- `-Wused-but-marked-unused` (88000+ hits) and
`-Wdisabled-macro-expansion` (2600+ hits).
Triggered by `typecheck-gcc.h` when building with clang 14+.
Maybe there exists a way to fix within that header?
Ref: https://discourse.llvm.org/t/removing-wused-but-marked-unused/55310
- `-Wunsafe-buffer-usage`. clang 16+. 7000+ hits.
May be useful in theory, but such high volume of hits makes it
impractical to review and possibly address. Meant for C++.
Ref: https://clang.llvm.org/docs/SafeBuffers.html
Ref: https://stackoverflow.com/questions/77017567/how-to-fix-code-to-avoid-warning-wunsafe-buffer-usage
Ref: https://discourse.llvm.org/t/rfc-c-buffer-hardening/65734
Ref: https://github.com/llvm/llvm-project/pull/111624
- `-Wimplicit-void-ptr-cast`. clang 21+. 1700+ hits.
C++ warning, deemed pure noise.
Ref: https://github.com/curl/curl/issues/18470#issuecomment-3253506266
- `-Wswitch-default` (180+ hits), `-Wswitch-enum` (190+ hits),
`-Wcovered-switch-default` (20+ hits).
Next to impossible to fix cleanly, esp. when the covered `case`
branches depend on compile-time options.
- `-Wdocumentation-unknown-command` (8+ hits).
Triggered in a few sources. Seems arbitrary and bogus.
- `-Wpadded` (550+ hits).
- `-Wc++-keyword` on Windows, where it collides with `wchar_t`.
(100+ hits)
Ref: https://github.com/llvm/llvm-project/issues/155988
- `-Wreserved-macro-identifier`. clang 13+. 5+ hits.
Sometimes it's necessary to set external macros that use
the reserved namespace. E.g. `_CRT_NONSTDC_NO_DEPRECATE`,
`__ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__`, `__NO_NET_API`,
possibly `_REENTRANT`, and more.
It's not worth trying to silence them individually.
- `-Wnonportable-system-include-path` with `clang-cl`.
It'd be broken by doing what the warning suggests.
- `-Wformat-non-iso` for clang-cl.
CMake `PICKY_COMPILER=ON` (the default) or `./configure`
`--enable-warnings` (not the default) is required to enable these
silencing rules.
Also:
- autotools, cmake: fix Apple clang and mainline llvm version translations.
Ref: https://en.wikipedia.org/wiki/Xcode#Toolchain_versions
- autotools, cmake: enable `-Warray-compare` for clang 20+.
Follow-up to 4b7accda5a#17196
- cmake: fix to enable `-Wmissing-variable-declarations` at an earlier
clang version.
- cmake: update internal logic to handle warning options with `+` in
them.
- cmake: fix internal logic to match the whole option when looking
into `CMAKE_C_FLAGS` for custom-disabled warnings.
Follow-up to b85cb8cb4e#18485Closes#18477
AWS-LC fixed a bug with large read ahead buffers in v1.61.0. Check a
define introduced in that version to enable the large read ahead again.
AWS-LC issue: https://github.com/aws/aws-lc/issues/2650Closes#18568
