When a connection is tested for reuse in a transfer that *may* upgrade
to TLS (commonly via STARTTLS), the SSL configuration must match the
existing connection.
Reported-by: Andrew Nesbit
Closes#21522
The CURLOPT_SSH_HOST_PUBLIC_KEY_* options are documented to verify the
host at connect time and not for connection reuse. Once the SSH host has
been deemed okay, it remains okay as long as the connection survives.
In addition: this function currently always returned TRUE since the
pointers have been NULLed in the SSH backend code before this function
is called.
Follow-up to c31fcf2dec
Reported-by: Andrew Nesbit
Closes#21519
`struct Curl_peer` keeps information about a communication endpoint
together. It will replace `conn->host` and `conn->conn_to_host` and
proxyinfo host. It will also become part of `struct ssl_peer`.
It has a reference counter, so an instance can be shared between
connections and filters.
Elminiates `conn->host` and `conn->connect_to_host`, used in the
proxyinfo structures. Passed to DNS resolution and socks filters, etc.
Pass peer to http proxy and socks tunnel filters. Use peer in dns filter
and resolving. Make `Curl_peer` a member in the `struct ssl_peer`.
Add `docs/internals/PEERS.md` for documentation.
Closes#21472
So that CURLcode failure is correct and not figured out later via
`conn->bits.proxy`. Add the flag to the async struct.
`for_proxy` is figured out by the caller of Curl_resolv() when it
figures out which host/proxy name it wants the addresses for.
Remove CONN_IS_PROXIED macro as no longer needed.
Closes#21423
- make sure all UNITTEST prototypes mark in which unit test they are used,
with "@unittest" markup
- make sure all UNITTEST functions do not use Curl_ prefix, as that is a
prefix we use for global private functions and these functions are static
and therefore not global and the prefix is wrong
- drop UNITTEST for functions not used in unit tests
- make the extract-unit-protos script highlight the above issues if found
- extract-unit-protos now also outputs the unit test number for all the
generated protos in lib/unitprotos.h to aid readers. It also adds the source
file and line number where the proto originates from.
- extract-unit-protos now exits with a non-zero value if any of the above
warnings are triggered
- cf-dns: Curl_cf_dns_result => static cf_dns_result
- hostip: Curl_ipv6works => static ipv6works
- url: remove Curl_setup_conn() - not used anymore
- connect: Curl_timeleft_now_ms => UNITTEST timeleft_now_ms
Closes#21330
To make it scoped for the single request appropriately.
Reported-by: Muhamad Arga Reksapati
Verify with libtest 2504: a custom Host *disabled* on reused handle
Closes#21312
- examples: sync debug output printf masks.
- INSTALL-CMAKE.md: tidy up section for some options.
- curl_sha512_256: delete comment suggesting an optimization.
- vtls/keylog: scope a variable.
- vtls/openssl: make a source code URL a permalink.
- vtls/schannel: drop redundant parentheses.
- test1119.pl: robustify `$1` -> `$s`.
- sync arg names in comments to match the code.
- tidy up and minor fixes to comments.
- fix formatting/indenting/comment/newline/include nits.
- move `UNITTEST` protos next to definitions, sync their argument names.
- make variables static.
- add parentheses to Perl `open()` calls.
- drop unnecessary double quotes in Perl.
- clang-format.
Closes#21000
req.no_body was only initialized in Curl_connect, while HTTP/2 server
push adds a duplicated handle via Curl_multi_add_perform and calls
Curl_init_do with conn==NULL, never invoking Curl_connect.
Verify it by amending test 1620
Found by Codex Security
Closes#21194
To sync names for the same macro logic between lib and src, and to move
it to the curlx namespace, to match `curlx_free()` that it's calling.
Closes#21151
- allow to specify when they are wanted on starting a resolve
- match dns cache entries accordingly. An entry which never
tried to get HTTPS-RRs is no answer for a resolve that wants
it.
- fix late arrivals of resolve answers to match the "async"
records that started them - if it still exists.
- provide for multiple "async" resolves in a transfer at the
same time. We may need to resolve an IP interface while the
main connection resolve has not finished yet.
- allow lookup of HTTPS-RR information as soon as it is
available, even if A/AAAA queries are still ongoing.
For this, the "async" infrastructure is changed:
- Defined bits for DNS queries `CURL_DNSQ_A`, `CURL_DNSQ_AAAA`
and `CURL_DNSQ_HTTPS`. These replace `ip_version` which says
nothing about HTTPS.
Use them in dns cache entries for matching.
- enhance the `async->id` to be a unique `uint32_t` for
resolves inside one multi. This is weak, as the id may
wrap around. However it is combined with the `mid` of
the easy handle, making collisions highly unlikely.
`data->state.async` is only accessed in few places where
the mid/async-id match is performed.
- vtls: for ECH supporting TLS backends (openssl, rustls, wolfssl),
retrieve the HTTPS-RR information from the dns connection filter.
Delay the connect if the HTTPS-RR is needed, but has not
been resolved yet.
The implementation of all this is complete for the threaded
resolver. c-ares resolver and DoH do not take advantage of
all new async features yet. To be done in separate PRs.
Details:
c-ares: cleanup settings and initialisation. Any ares channel
is only being created on starting a resolve and propagating
operations in setopt.c to the channel are not helpful.
Changed threaded+ares pollset handling so that they do not
overwrite each others `ASYNC_NAME` timeouts.
Add trace name 'threads' for tracing thread queue and
pool used by threaded resolver.
Closes#21175
New connection filter `cf-dns` that manages DNS queries. If hands
out addresses and HTTPS-RR records to anyone interested. Used by
HTTPS and IP happy eyeballing.
Information may become available *before* the libcurl "dns entry"
is complete, e.g. all queries have been answered. The cf-ip-happy
filter uses this information to start connection attempts as soon
as the first address is available.
The multi MSTATE_RESOLVING was removed. A new connection always
goes to MSTATE_CONNECTING. The connectdata bit `dns_resolved`
indicates when DNS information is complete. This is used for
error reporting and starting the progress meter.
Removed dns entries `data->state.dns[i]`, as the `cf-dns` filter
now keeps the reference now.
Many minor tweaks for making this work and pass address information
around safely.
Closes#21027
Use a thread queue and pool for asnyc threaded DNS resolves.
Add pytest test_21_* for verification.
Add `CURLMOPT_RESOLVE_THREADS_MAX` to allow applications to
resize the thread pool used.
Add `CURLMOPT_QUICK_EXIT` to allow applications to skip thread
joins when cleaning up a multi handle. Multi handles in
`curl_easy_perform()` inherit this from `CURLOPT_QUICK_EXIT`.
Add several debug environment variables for testing.
Closes#20936
- librtmp has no test cases, makes no proper releases and has not had a
single commit within the last year
- librtmp parses the URL itself and requires non-compliant URLs for this
- we have no RTMP tests
- RTMP was used by 2.2% of curl users (self-identified in the 2025
survey)
Closes#20673
Replace the `volatile int dirty` with a reference counter
protected by a mutex when available.
Solve the problem of when to call application's lock function
by adding a volatile flag that indicates a share has been added
to easy handles in its lifetime. That flag ever goes from
FALSE to TRUE, so volatile might work (in the absence of a mutex).
(The problem is that the lock/unlock functions need 2-3
`curl_share_setopt()` invocations to become usable and there
is no way of telling if the third will ever happen. Calling
the lock function before the 3rd setopt may crash the
application.)
When removing a share from an easy handle (or replacing it with
another share), detach the easy connection on a share with a
connection pool.
When cleaning up a share, allow this even if it is still used in
easy handles. It will be destroyed when the reference count
drops to 0.
Closes#20870
- Rename `Curl_resolv_unlink()` to `Curl_dns_entry_unlink()`.
- Change `Curl_dnscache_get()` to return CURLcode result. Returns
now `CURLE_COULDNT_RESOLVE_HOST` for "negative" cache entries.
- Add `Curl_dnscache_add_negative()` to put a "negative" entry
into the cache.
Closes#20864
Curl_1st_err() does not return the second error if the first result is
CURLE_AGAIN. This may cause errors to not become noticeable when they
should be.
Replace all use of Curl_1st_err() with Curl_1st_fatal(), which handles
CURLE_AGAIN as a not-a-real-error case.
Closes#20980
Add protocol.h and protocol.c containing all about libcurl's
known URI schemes and their protocol handlers (so they exist).
Moves the scheme definitions from the various sources files into
protocol.c. Schemes are known and used, even of the protocol
handler is not build or just not implemented at all.
Closes#20906
Convert more `int port` to `uint16_t` port types. Reshuffle ports in
connectdata to save some bytes. Change `conn->destination` format to
- make it more readable and thus usable in tracing
- add the IPv6 scope_id only when not default (global)
and make it resemble more the textual format for IPv6
(e.g. suffix '%<scope_id>')
Closes#20918
Eliminate `conn->bits.ipv6_ip`
The bit was only correct for the first transfer using a connection. Use
`data->state.up.hostname` instead in places that need the URL hostname
in its original form.
Fix parseurlandfillconn() to not modify `data->state.up.hostname` before
copying the connection's hostname, but modify the copy instead, leaving
the URL hostname intact.
Closes#20919
Also:
- support per-directory and per-upper-directory whitelist entries.
- convert badlist input grep tweak into the above format.
(except for 'And' which had just a few hits.)
- fix many code exceptions, but do not enforce.
(there also remain about 350 'will' uses in lib)
- fix badwords in example code, drop exceptions.
- badwords-all: convert to Perl.
To make it usable from CMake.
- FAQ: reword to not use 'will'. Drop exception.
Closes#20886
The protocol handler method `connection_check` allowed to variable
operations to trigger with variable result bits. Only the `CONNCHECK_ISDEAD`
and `CONNRESULT_DEAD` were in use. Transform the function into
`connection_is_dead` without extra parameter and a bool result.
- Remove defines for `CONNCHECK_*` and `CONNRESULT_*`
- Rename protocol function in handler comments
- Change RTSP implementation (only protocol that uses this)
Closes#20890
The strdup() of the hostname to resolve is unnecessary as the sync
resolve code does not keep the string and the async code makes copies
already.
Remove the member from `connectdata`.
Closes#20833
- the value collided with the internal one used for WebSocket
- we add all new ones as internal bits
- bump to 64-bit internal type
- introduce new define for setting all protocols (CURLPROTO_64ALL)
Reported by Codex Security
Closes#20798
- vms/curlmsg_vms.h: delete unused/commented code.
- vtls/schannel_verify: sort includes.
- typecheck-gcc.h: fix indent and alignment.
- lib/config-win32.h: drop idle `#undef`.
- spacecheck: check for stray empty lines before after curly braces.
- make literals more readable: 1048576 -> 1024 * 1024
- scope variables.
- use ISO date in a comment.
- drop redundant parentheses.
- drop empty comments.
- unfold lines.
- duplicate/stray spaces in comments.
- fix indent, whitespace, minor typos.
Closes#20690
- fix internal macro `AN_APPLE_OS` reused between sources without
resetting it. It may potentially have left the system sha256
function unused.
- fix to define `WOLFSSL_OPTIONS_IGNORE_SYS` so that it always applies
to wolfSSL headers, also during feature detection.
- md4, md5, sha256: simplify fallback logic.
- delete 20+ unused macros.
- scope or move macros to avoid `-Wunused-macros` warnings.
- examples: delete unused code.
The warning detects macros defined but not used within the same C
source. It does not warn for macros defined in headers. It also works
with unity builds, but to a lesser extent.
Closes#20593
