To sync names for the same macro logic between lib and src, and to move
it to the curlx namespace, to match `curlx_free()` that it's calling.
Closes#21151
- Rename `Curl_resolv_unlink()` to `Curl_dns_entry_unlink()`.
- Change `Curl_dnscache_get()` to return CURLcode result. Returns
now `CURLE_COULDNT_RESOLVE_HOST` for "negative" cache entries.
- Add `Curl_dnscache_add_negative()` to put a "negative" entry
into the cache.
Closes#20864
Convert more `int port` to `uint16_t` port types. Reshuffle ports in
connectdata to save some bytes. Change `conn->destination` format to
- make it more readable and thus usable in tracing
- add the IPv6 scope_id only when not default (global)
and make it resemble more the textual format for IPv6
(e.g. suffix '%<scope_id>')
Closes#20918
Also:
- support per-directory and per-upper-directory whitelist entries.
- convert badlist input grep tweak into the above format.
(except for 'And' which had just a few hits.)
- fix many code exceptions, but do not enforce.
(there also remain about 350 'will' uses in lib)
- fix badwords in example code, drop exceptions.
- badwords-all: convert to Perl.
To make it usable from CMake.
- FAQ: reword to not use 'will'. Drop exception.
Closes#20886
Backtrack on previous change that aimed to solve the wrong `share.h`
being included. It turns out it did not fix this issue. At the same time
it introduced relative header filenames and the need to include the same
headers differently depending on the source files' location, reducing
readability and editability.
Replace this method by re-adding curl's lib source directory to the
header path and addressing headers by the their full, relative name to
that base directory. Aligning with this method already used in src and
tests.
With these advantages:
- makes includes easier to read, recognize, grep, sort, write, and copy
between sources,
- syncs the way these headers are included across curl components,
- avoids the ambiguity between system `schannel.h`, `rustls.h` vs.
local headers using the same names in `lib/vtls`,
- silences clang-tidy `readability-duplicate-include` checker, which
detects the above issue,
Ref: https://clang.llvm.org/extra/clang-tidy/checks/readability/duplicate-include.html
- possibly silences TIOBE coding standard warnings:
`6.10.2.a: Don't use relative paths in #include statements.`
- long shot: it works well with concatenated test sources, for
clang-tidy-friendly custom unity builds. Ref: #20667
Slight downside: it's not enforced.
If there happens to be a collision between a local `lib/*.h` header and
a system one, the solution is to rename (possibly with its `.c`
counterpart) into the `curl_` namespace. This is also the method used by
curl in the past.
Also:
- curlx/inet_pton: reduce scope of an include.
- toolx/tool_time: apply this to an include, and update VS project
files accordingly. Also dropping unnecessary lib/curlx header path.
- clang-tidy: enable `readability-duplicate-include`.
Follow-up to 3887069c66#19676
Follow-up to 625f2c1644#16991#16949Closes#20623
- For compatibility reasons send both ALPN ids http/1.0 and http/1.1 for
HTTP/1.0 requests.
Prior to this change for compatibility reasons curl would send ALPN
http/1.1 for HTTP/1.0 requests, since some servers do not recognize
ALPN http/1.0. However some servers may recognize only ALPN http/1.0 for
HTTP/1.0 requests. Therefore curl now sends both.
Reported-by: programmerlexi@users.noreply.github.com
Fixes https://github.com/curl/curl/issues/20487
Closes https://github.com/curl/curl/pull/20533
- de-dupe lib/src strdup/memdup functions into curlx.
- introduce `CURLX_STRDUP_LOW()` for mapping `strdup()`, and to do it at
one place within the code, in `curl_setup.h`.
- tests/server: use `curlx_strdup()`. (Also to fix building without
a system `strdup()`.)
- curlx/curlx.h: shorten and tidy up.
- adjust Windows build path to not need `HAVE_STRDUP`.
- build: stop detecting `HAVE_STRDUP` on Windows.
Closes#20497
And a few variables around.
There remain cases where the accepted pointer is const, yet the returned
pointer is written to.
Partly addressing (glibc 2.43):
```
* For ISO C23, the functions bsearch, memchr, strchr, strpbrk, strrchr,
strstr, wcschr, wcspbrk, wcsrchr, wcsstr and wmemchr that return
pointers into their input arrays now have definitions as macros that
return a pointer to a const-qualified type when the input argument is
a pointer to a const-qualified type.
```
Ref: https://lists.gnu.org/archive/html/info-gnu/2026-01/msg00005.html
Reported-by: Rudi Heitbaum
Ref: #20420Closes#20421
With the same semantics as Apple SecTrust, in both libcurl and the curl
tool, when using non-Schannel TLS backends. In practice it means that
it makes TLS work without manually or implicitly configuring a CA bundle
`.crt` file, such as `curl-ca-bundle.crt`.
To enable:
- autotools: `--enable-ca-native`
- cmake: `-DCURL_CA_NATIVE=ON`
- CPPFLAGS: `-DCURL_CA_NATIVE`
When enabled:
- enables `CURLSSLOPT_NATIVE_CA` (libcurl) / `--ca-native`
and `--proxy-ca-native` (curl tool) options by default.
- unsafe search for an on-disk CA bundle gets disabled by default.
Equivalent to `--disable-ca-search` with autotools,
`-DCURL_DISABLE_CA_SEARCH=ON` with CMake.
- build-time detection of CA bundle and CA path gets disabled. As with
Apple SecTrust. This was already the default for Windows.
- native CA can be disabled at run-time with the `--no-ca-native`
and/or `--no-proxy-ca-native` command-line options.
Rationale: This build option:
- has a repeat and active interest from packagers and users.
- helps integrating curl with Windows for those who need this.
- it also applies to macOS: #17525
Shipped in curl 8.17.0.
- makes it trivial to use custom certs configured on the OS.
- frees applications/packagers/users from the task of securely
distributing, and keeping up-to-date, a CA bundle.
- frees potentially many curl tool from configuring a CA bundle manually
to access HTTPS (and other TLS) URLs. This is traditionally difficult
on Windows because there is no concept of a universal, protected,
non-world-writable, location on the file system to securely store
a CA bundle.
- allows using modern features regardless of Windows version. Some of
these features are not supported with Schannel (e.g. HTTP/3, ECH) on
any Windows version.
- is necessary for HTTP/3 builds, where bootstrapping a CA bundle is not
possible with Schannel, because MultiSSL is not an option, and HTTP/3
is not supported with Schannel.
Ref: #16181 (previous attempt)
Ref: https://github.com/curl/curl/discussions/9348
Ref: https://github.com/curl/curl/issues/9350
Ref: https://github.com/curl/curl/pull/13111
Ref: https://github.com/microsoft/vcpkg/pull/46459#issuecomment-3162068701
Ref: 22652a5a4c#14582
Ref: eefd03c572#18703Closes#18279
Add casts to `bool`, or use `bit` type in local variables, where
neccessary to avoid MSVC compiler warnings C4242.
Note: There may remain places needing the above updates, where not
tested in CI, and missed in manual review.
Also:
- urldata: convert struct field `connect_only` to bitfield to match its
counterpart in another struct.
- rename curl-specific `bit` type to `curl_bit`.
Closes#20142
Note: This patch doesn't aim to add `timeval.h` includes missing from
local headers using `curltime` type. They remain relying on `urldata.h`
being included first. This patch also doesn't delete existing, used
includes already present in local headers (as internal users may rely
on them).
Ref: #20106Closes#20126
This function REQUIRES the size of the target buffer as well as the
length of the source string. Meant to make it harder to do a bad
strcpy().
Removes 23 calls to strcpy().
Closes#20067
- replace `sendf.h` with `curl_trc.h` where it was included just for it.
- drop unused `curl_trc.h` includes.
- easy: delete obsolete comment about `send.h` include reason.
Also:
- move out `curl_trc.h` include from `sendf.h` and include it directly
in users, where not done already. To flatten the include tree and
to less rely on indirect includes.
- stop including `sendf.h` from other headers, replace it with forward
declaration of `Curl_easy`, as done already elsewhere.
Verified with an all non-unity CI run.
Closes#20061
To make it available for all files. Drop includes from individual
sources. This header was already included from most sources and not
specific to any internal subsystem.
Also to ensure that two system symbol redefines on Windows (`read()` and
`write()`) get applied to all sources. Move them to `curl_setup.h`.
Closes#20056
Always use curlx_now() when calling Curl_pgrs_now(data). Tests with the
"manual" updates to now proved differ more then 100ms in parallel testing.
Add `curlx_nowp()` to set current time into a struct curltime.
Add `curlx_ptimediff_ms() and friends, passing pointers.
Update documentation.
Closes#19998
Use `data->progress.now` as the timestamp of proecssing a transfer.
Update it on significant events and refrain from calling `curlx_now()`
in many places.
The problem this addresses is
a) calling curlx_now() has costs, depending on platform. Calling it
every time results in 25% increase `./runtest` duration on macOS.
b) we used to pass a `struct curltime *` around to save on calls, but
when some method directly use `curx_now()` and some use the passed
pointer, the transfer experienes non-linear time. This results in
timeline checks to report events in the wrong order.
By keeping a timestamp in the easy handle and updating it there, no
longer invoking `curlx_now()` in the "lower" methods, the transfer
can observer a steady clock progression.
Add documentation in docs/internals/TIME-KEEPING.md
Reported-by: Viktor Szakats
Fixes#19935Closes#19961
When the Alt-Svc points to the same host and port, add the destination
ALPN to the `wanted` versions and set it also as the `preferred` version
in negotiations.
This allows Alt-Svc for h3 to point to h2 and have it tried first. Also,
this allows Alt-Svc to say http/1.1 is preferred and changes the ALPN
protocol ordering for the TLS handshake.
Add tests in various combination to verify this works.
Reported-by: yushicheng7788 on github
Fixes#19740Closes#19874
Before this patch curl used the C preprocessor to override standard
memory allocation symbols: malloc, calloc, strdup, realloc, free.
The goal of these is to replace them with curl's debug wrappers in
`CURLDEBUG` builds, another was to replace them with the wrappers
calling user-defined allocators in libcurl. This solution needed a bunch
of workarounds to avoid breaking external headers: it relied on include
order to do the overriding last. For "unity" builds it needed to reset
overrides before external includes. Also in test apps, which are always
built as single source files. It also needed the `(symbol)` trick
to avoid overrides in some places. This would still not fix cases where
the standard symbols were macros. It was also fragile and difficult
to figure out which was the actual function behind an alloc or free call
in a specific piece of code. This in turn caused bugs where the wrong
allocator was accidentally called.
To avoid these problems, this patch replaces this solution with
`curlx_`-prefixed allocator macros, and mapping them _once_ to either
the libcurl wrappers, the debug wrappers or the standard ones, matching
the rest of the code in libtests.
This concludes the long journey to avoid redefining standard functions
in the curl codebase.
Note: I did not update `packages/OS400/*.c` sources. They did not
`#include` `curl_setup.h`, `curl_memory.h` or `memdebug.h`, meaning
the overrides were never applied to them. This may or may not have been
correct. For now I suppressed the direct use of standard allocators
via a local `.checksrc`. Probably they (except for `curlcl.c`) should be
updated to include `curl_setup.h` and use the `curlx_` macros.
This patch changes mappings in two places:
- `lib/curl_threads.c` in libtests: Before this patch it mapped to
libcurl allocators. After, it maps to standard allocators, like
the rest of libtests code.
- `units`: before this patch it mapped to standard allocators. After, it
maps to libcurl allocators.
Also:
- drop all position-dependent `curl_memory.h` and `memdebug.h` includes,
and delete the now unnecessary headers.
- rename `Curl_tcsdup` macro to `curlx_tcsdup` and define like the other
allocators.
- map `curlx_strdup()` to `_strdup()` on Windows (was: `strdup()`).
To fix warnings silenced via `_CRT_NONSTDC_NO_DEPRECATE`.
- multibyte: map `curlx_convert_*()` to `_strdup()` on Windows
(was: `strdup()`).
- src: do not reuse the `strdup` name for the local replacement.
- lib509: call `_strdup()` on Windows (was: `strdup()`).
- test1132: delete test obsoleted by this patch.
- CHECKSRC.md: update text for `SNPRINTF`.
- checksrc: ban standard allocator symbols.
Follow-up to b12da22db1#18866
Follow-up to db98daab05#18844
Follow-up to 4deea9396b#18814
Follow-up to 9678ff5b1b#18776
Follow-up to 10bac43b87#18774
Follow-up to 20142f5d06#18634
Follow-up to bf7375ecc5#18503
Follow-up to 9863599d69#18502
Follow-up to 3bb5e58c10#17827Closes#19626
Change the send parameter from `const void *` to `const uint8_t *` and
adapt calling code. Several had already unsigned chars and were casting.
Closes#19729
Change `inputbuff` parameter from `const char *` to `const uint8_t *` to
reflect the binary nature of the input bytes. Half the code was casting
unsigned char to signed already in calling.
Closes#19722
Windows CRTs have a `share.h`. Before this patch when trying to
`#include <share.h>` it, the compiler picked up curl's internal
`lib/share.h` instead. Rename it to avoid this issue.
CRT `share.h` has constants necessary for using safe open CRT functions.
Also rename `lib/share.c` to keep matching the header.
Ref: https://learn.microsoft.com/cpp/c-runtime-library/sharing-constants
Ref: 625f2c1644#16949#16991
Cherry-picked from #19643Closes#19676
- badwords.pl: add `-a` option to check all lines in source code files.
Before this patch indented lines were skipped (to avoid Markdown code
fences.)
- GHA/checksrc: use `-a` when verifying the source code.
- GHA/checksrc: disable `So` and `But` rules for source code.
- GHA/checksrc: add docs/examples to the verified sources.
- badwords.txt: delete 4 duplicates.
- badwords.txt: group and sort contractions.
- badwords.txt: allow ` url = `, `DIR`, `<file name`.
Closes#19536
Cleanup the vtls pinned key matching somewhat. Add a DEBUGF
for pinned key hashes that do not match, so we can see in
traces what was going on.
Ref #19489Closes#19529
A regression in curl 8.17.0 led to a customer CAPATH set by the
application (or the curl command) to be ignored unless licurl was built
with a default CAPATH.
Add test cases using `--capath` on the custom pytest CA, generated with
the help of the openssl command when available.
Fixes#19401
Reported-by: Brad King
Closes#19308
socks_gssapi: the malloc + memcpy was superflous and can be skipped
cleartext: avoid malloc + three memcpy with aprintf()
digest_sspi: use memdup0 instead of malloc + memcpy
vtls: use memdup0 instead of malloc + memcpy
Closes#19282
After this patch, the codebase no longer overrides system printf
functions. Instead it explicitly calls either the curl printf functions
`curl_m*printf()` or the system ones using their original names.
Also:
- drop unused `curl_printf.h` includes.
- checksrc: ban system printf functions, allow where necessary.
Follow-up to db98daab05#18844
Follow-up to 4deea9396b#18814Closes#18866
- configure/cmake support for enabling the option
- supported in OpenSSL and GnuTLS backends
- when configured, Apple SecTrust is the default trust store
for peer verification. When one of the CURLOPT_* for adding
certificates is used, that default does not apply.
- add documentation of build options and SSL use
Closes#18703
By introducing wrappers for them in the curlx namespace:
`curlx_fopen()`, `curlx_fdopen()`, `curlx_fclose()`.
The undefine/redefine/`(function)()` methods broke on systems
implementing these functions as macros. E.g. AIX 32-bit's `fopen()`.
Also:
- rename `lib/fopen.*` to `lib/curl_fopen.*` (for `Curl_fopen()`)
to make room for the newly added `curlx/fopen.h`.
- curlx: move file-related functions from `multibyte.c` to `fopen.c`.
- tests/server: stop using the curl-specific `fopen()` implementation
on Windows. Unicode isn't used by runtests, and it isn't critical to
run tests on longs path. It can be re-enabled if this becomes
necessary, or if the wrapper receives a feature that's critical for
test servers.
Reported-by: Andrew Kirillov
Bug: https://github.com/curl/curl/issues/18510#issuecomment-3274393640
Follow-up to bf7375ecc5#18503
Follow-up to 9863599d69#18502
Follow-up to 3bb5e58c10#17827Closes#18634
Rewrite the code that removes a filter from the connection and discards
it. Always look at the connection, otherwise it will not work of the
filter is at the top of the chain.
Change QUIC filter setup code to always tear down the chain in
construction when an error occured.
HTTP proxy, do not remove the h1/h2 sub filter on close. Leave it to be
discarded with the connection. Avoids keeping an additional pointer that
might become dangling.
Triggered by a reported on a code bug in discard method.
Reported-by: Joshua Rogers
Closes#18596
When a HTTP version has been negotiated via ALPN, set the member
`conn->httpversion_seen` accordingly. This allows pending transfers to
reuse multiplexed http connections before the response to the first
transfer has arrived.
Fixes#18177
Reported-by: IoannisGS on github
Closes#18181
`getsock()` calls operated on a global limit that could
not be configure beyond 16 sockets. This is no longer adequate
with the new happy eyeballing strategy.
Instead, do the following:
- make `struct easy_pollset` dynamic. Starting with
a minimal room for two sockets, the very common case,
allow it to grow on demand.
- replace all protocol handler getsock() calls with pollsets
and a CURLcode to return failures
- add CURLcode return for all connection filter `adjust_pollset()`
callbacks, since they too can now fail.
- use appropriately in multi.c and multi_ev.c
- fix unit2600 to trigger pollset growth
Closes#18164
