Use a dns connection filter to resolve hostnames where their addresses
are locally resolved and forwarded to the SOCKS proxy.
This makes all improvements, like in #21295 for example, also apply
to socks connections. Curl_resolv() is now only called from a DNS
filter.
(ftp still calls Curl_resolv_blocking() in two places, one of which
may be replaceable with a DNS filter as well to remove the block, tbd)
Closes#21297
- examples: sync debug output printf masks.
- INSTALL-CMAKE.md: tidy up section for some options.
- curl_sha512_256: delete comment suggesting an optimization.
- vtls/keylog: scope a variable.
- vtls/openssl: make a source code URL a permalink.
- vtls/schannel: drop redundant parentheses.
- test1119.pl: robustify `$1` -> `$s`.
- sync arg names in comments to match the code.
- tidy up and minor fixes to comments.
- fix formatting/indenting/comment/newline/include nits.
- move `UNITTEST` protos next to definitions, sync their argument names.
- make variables static.
- add parentheses to Perl `open()` calls.
- drop unnecessary double quotes in Perl.
- clang-format.
Closes#21000
When a user supplied an ip address in a URL as hostname, use that even
when address family restrictions like -4 or -6 are set.
Add test_10_15/16 to verify with a local proxy server.
Fixes#21146
Reported-by: Terrance Wong
How:
- cf-dns: on see the hostname is an ip(v6) address, add the respective
A/AAAA to the dns query bits
- cf-dns/hostip: only hand out addrinfos for a family if that family
is part of the DNS queries. That prevents for example ipv6 addresses
to show up from dns cache entries
- change cf-ip-happy to no longer check for "ip_version" and instead
use all addresses that cf-dns hands out
Closes#21295
Connection filters can now carry the flag CF_TYPE_SETUP, indicating that
they are only needed during connection setup, e.g. connect.
Once the connection is fully established, those filter are removed
again. This frees resources and also makes the filter (call) chains
shorter.
Closes#21269
Make cf-https-connect work async correctly:
- only start first baller when at least one A/AAAA address
is available
- select first connect attempt after that with HTTPS-RR info
there or not.
- select second connect attempt only when HTTPS-RR is resolved
(may have resolved to "not known") and select possible ALPN
from things known by then. May not select any second attempt
when first already covers everything.
This means when the HTTPS-RR is known at/before the first address
is resolved, everything behaves as before. When the HTTPS-RR is
late, a first connection attempt will have been started. Any
ALPN preference from the HTTPS-RR that is not already ongoing will
then start the second attempt.
For HTTPS-RRs that recommend 2 or more ALPNs, the first will always
be attempted: either it is already ongong or it will be the ALPN
for the second attempt. The 2nd ALPN recommendation from HTTPS-RR
*may* be honored or not, depending on what is already selected.
The difference in behaviour between early/late HTTPS-RR resolve
cannot be helped - unless we do not perform any attempts before
it arrives. Trade offs.
Closes#21267
- allow to specify when they are wanted on starting a resolve
- match dns cache entries accordingly. An entry which never
tried to get HTTPS-RRs is no answer for a resolve that wants
it.
- fix late arrivals of resolve answers to match the "async"
records that started them - if it still exists.
- provide for multiple "async" resolves in a transfer at the
same time. We may need to resolve an IP interface while the
main connection resolve has not finished yet.
- allow lookup of HTTPS-RR information as soon as it is
available, even if A/AAAA queries are still ongoing.
For this, the "async" infrastructure is changed:
- Defined bits for DNS queries `CURL_DNSQ_A`, `CURL_DNSQ_AAAA`
and `CURL_DNSQ_HTTPS`. These replace `ip_version` which says
nothing about HTTPS.
Use them in dns cache entries for matching.
- enhance the `async->id` to be a unique `uint32_t` for
resolves inside one multi. This is weak, as the id may
wrap around. However it is combined with the `mid` of
the easy handle, making collisions highly unlikely.
`data->state.async` is only accessed in few places where
the mid/async-id match is performed.
- vtls: for ECH supporting TLS backends (openssl, rustls, wolfssl),
retrieve the HTTPS-RR information from the dns connection filter.
Delay the connect if the HTTPS-RR is needed, but has not
been resolved yet.
The implementation of all this is complete for the threaded
resolver. c-ares resolver and DoH do not take advantage of
all new async features yet. To be done in separate PRs.
Details:
c-ares: cleanup settings and initialisation. Any ares channel
is only being created on starting a resolve and propagating
operations in setopt.c to the channel are not helpful.
Changed threaded+ares pollset handling so that they do not
overwrite each others `ASYNC_NAME` timeouts.
Add trace name 'threads' for tracing thread queue and
pool used by threaded resolver.
Closes#21175
New connection filter `cf-dns` that manages DNS queries. If hands
out addresses and HTTPS-RR records to anyone interested. Used by
HTTPS and IP happy eyeballing.
Information may become available *before* the libcurl "dns entry"
is complete, e.g. all queries have been answered. The cf-ip-happy
filter uses this information to start connection attempts as soon
as the first address is available.
The multi MSTATE_RESOLVING was removed. A new connection always
goes to MSTATE_CONNECTING. The connectdata bit `dns_resolved`
indicates when DNS information is complete. This is used for
error reporting and starting the progress meter.
Removed dns entries `data->state.dns[i]`, as the `cf-dns` filter
now keeps the reference now.
Many minor tweaks for making this work and pass address information
around safely.
Closes#21027
