curl/lib
Stefan Eissing 2b3dfb4ad4
lib: make resolving HTTPS DNS records reliable:
- allow to specify when they are wanted on starting a resolve
- match dns cache entries accordingly. An entry which never
  tried to get HTTPS-RRs is no answer for a resolve that wants
  it.
- fix late arrivals of resolve answers to match the "async"
  records that started them - if it still exists.
- provide for multiple "async" resolves in a transfer at the
  same time. We may need to resolve an IP interface while the
  main connection resolve has not finished yet.
- allow lookup of HTTPS-RR information as soon as it is
  available, even if A/AAAA queries are still ongoing.

For this, the "async" infrastructure is changed:

- Defined bits for DNS queries `CURL_DNSQ_A`, `CURL_DNSQ_AAAA`
  and `CURL_DNSQ_HTTPS`. These replace `ip_version` which says
  nothing about HTTPS.
  Use them in dns cache entries for matching.
- enhance the `async->id` to be a unique `uint32_t` for
  resolves inside one multi. This is weak, as the id may
  wrap around. However it is combined with the `mid` of
  the easy handle, making collisions highly unlikely.
  `data->state.async` is only accessed in few places where
  the mid/async-id match is performed.
- vtls: for ECH supporting TLS backends (openssl, rustls, wolfssl),
  retrieve the HTTPS-RR information from the dns connection filter.
  Delay the connect if the HTTPS-RR is needed, but has not
  been resolved yet.

The implementation of all this is complete for the threaded
resolver. c-ares resolver and DoH do not take advantage of
all new async features yet. To be done in separate PRs.

Details:

c-ares: cleanup settings and initialisation. Any ares channel
is only being created on starting a resolve and propagating
operations in setopt.c to the channel are not helpful.

Changed threaded+ares pollset handling so that they do not
overwrite each others `ASYNC_NAME` timeouts.

Add trace name 'threads' for tracing thread queue and
pool used by threaded resolver.

Closes #21175
2026-04-01 15:36:31 +02:00
..
curlx
vauth
vquic cf-dns: connection filter for DNS queries 2026-03-30 13:28:38 +02:00
vssh
vtls lib: make resolving HTTPS DNS records reliable: 2026-04-01 15:36:31 +02:00
.gitignore
altsvc.c altsvc: cap the list at 5,000 entries 2026-04-01 12:05:47 +02:00
altsvc.h altsvc: cap the list at 5,000 entries 2026-04-01 12:05:47 +02:00
amigaos.c
amigaos.h
arpa_telnet.h
asyn-ares.c lib: make resolving HTTPS DNS records reliable: 2026-04-01 15:36:31 +02:00
asyn-base.c lib: make resolving HTTPS DNS records reliable: 2026-04-01 15:36:31 +02:00
asyn-thrdd.c lib: make resolving HTTPS DNS records reliable: 2026-04-01 15:36:31 +02:00
asyn.h lib: make resolving HTTPS DNS records reliable: 2026-04-01 15:36:31 +02:00
bufq.c
bufq.h
bufref.c
bufref.h
cf-dns.c lib: make resolving HTTPS DNS records reliable: 2026-04-01 15:36:31 +02:00
cf-dns.h lib: make resolving HTTPS DNS records reliable: 2026-04-01 15:36:31 +02:00
cf-h1-proxy.c
cf-h1-proxy.h
cf-h2-proxy.c
cf-h2-proxy.h
cf-haproxy.c
cf-haproxy.h
cf-https-connect.c lib: make resolving HTTPS DNS records reliable: 2026-04-01 15:36:31 +02:00
cf-https-connect.h cf-dns: connection filter for DNS queries 2026-03-30 13:28:38 +02:00
cf-ip-happy.c cf-dns: connection filter for DNS queries 2026-03-30 13:28:38 +02:00
cf-ip-happy.h cf-dns: connection filter for DNS queries 2026-03-30 13:28:38 +02:00
cf-socket.c lib: make resolving HTTPS DNS records reliable: 2026-04-01 15:36:31 +02:00
cf-socket.h cf-dns: connection filter for DNS queries 2026-03-30 13:28:38 +02:00
cfilters.c cf-dns: connection filter for DNS queries 2026-03-30 13:28:38 +02:00
cfilters.h cf-dns: connection filter for DNS queries 2026-03-30 13:28:38 +02:00
CMakeLists.txt cmake: use AIX built-in variable (with CMake 4.0+) 2026-03-30 13:05:17 +02:00
config-mac.h src: drop detecting and redefining system symbol ftruncate 2026-03-30 13:46:14 +02:00
config-os400.h build: drop redundant HAVE_PTHREAD_H guard 2026-03-30 18:57:07 +02:00
config-riscos.h src: drop detecting and redefining system symbol ftruncate 2026-03-30 13:46:14 +02:00
config-win32.h src: drop detecting and redefining system symbol ftruncate 2026-03-30 13:46:14 +02:00
conncache.c
conncache.h
connect.c lib: make resolving HTTPS DNS records reliable: 2026-04-01 15:36:31 +02:00
connect.h cf-dns: connection filter for DNS queries 2026-03-30 13:28:38 +02:00
content_encoding.c
content_encoding.h
cookie.c cookie: fix rejection when tabs in value 2026-04-01 10:45:05 +02:00
cookie.h
cshutdn.c
cshutdn.h
curl_addrinfo.c cf-dns: connection filter for DNS queries 2026-03-30 13:28:38 +02:00
curl_addrinfo.h cf-dns: connection filter for DNS queries 2026-03-30 13:28:38 +02:00
curl_config-cmake.h.in build: drop redundant HAVE_PTHREAD_H guard 2026-03-30 18:57:07 +02:00
curl_ctype.h
curl_endian.c
curl_endian.h
curl_fnmatch.c
curl_fnmatch.h
curl_fopen.c fopen: for temp files, inherit permissions only for owner 2026-03-30 18:39:12 +02:00
curl_fopen.h
curl_get_line.c
curl_get_line.h
curl_gethostname.c
curl_gethostname.h
curl_gssapi.c
curl_gssapi.h
curl_hmac.h lib: accept larger input to md5/hmac/sha256/sha512 functions 2026-03-31 17:19:56 +02:00
curl_ldap.h
curl_md4.h
curl_md5.h lib: accept larger input to md5/hmac/sha256/sha512 functions 2026-03-31 17:19:56 +02:00
curl_memrchr.c
curl_memrchr.h
curl_ntlm_core.c
curl_ntlm_core.h
curl_printf.h
curl_range.c
curl_range.h
curl_sasl.c
curl_sasl.h
curl_setup.h build: drop redundant HAVE_PTHREAD_H guard 2026-03-30 18:57:07 +02:00
curl_sha256.h lib: accept larger input to md5/hmac/sha256/sha512 functions 2026-03-31 17:19:56 +02:00
curl_sha512_256.c lib: accept larger input to md5/hmac/sha256/sha512 functions 2026-03-31 17:19:56 +02:00
curl_sha512_256.h
curl_share.c cf-dns: connection filter for DNS queries 2026-03-30 13:28:38 +02:00
curl_share.h
curl_sspi.c
curl_sspi.h
curl_threads.c
curl_threads.h build: drop duplicate pthread.h includes 2026-03-30 13:46:14 +02:00
curl_trc.c lib: make resolving HTTPS DNS records reliable: 2026-04-01 15:36:31 +02:00
curl_trc.h lib: make resolving HTTPS DNS records reliable: 2026-04-01 15:36:31 +02:00
cw-out.c
cw-out.h
cw-pause.c
cw-pause.h
dict.c
dict.h
dllmain.c
dnscache.c lib: make resolving HTTPS DNS records reliable: 2026-04-01 15:36:31 +02:00
dnscache.h lib: make resolving HTTPS DNS records reliable: 2026-04-01 15:36:31 +02:00
doh.c lib: make resolving HTTPS DNS records reliable: 2026-04-01 15:36:31 +02:00
doh.h lib: make resolving HTTPS DNS records reliable: 2026-04-01 15:36:31 +02:00
dynhds.c
dynhds.h
easy.c lib: make resolving HTTPS DNS records reliable: 2026-04-01 15:36:31 +02:00
easy_lock.h build: drop redundant HAVE_PTHREAD_H guard 2026-03-30 18:57:07 +02:00
easygetopt.c
easyif.h
easyoptions.c
easyoptions.h
escape.c
escape.h
fake_addrinfo.c
fake_addrinfo.h
file.c
file.h
fileinfo.c
fileinfo.h
formdata.c
formdata.h
ftp-int.h
ftp.c lib: make resolving HTTPS DNS records reliable: 2026-04-01 15:36:31 +02:00
ftp.h
ftplistparser.c
ftplistparser.h
functypes.h
getenv.c
getinfo.c
getinfo.h
gopher.c
gopher.h
hash.c
hash.h
headers.c
headers.h
hmac.c lib: accept larger input to md5/hmac/sha256/sha512 functions 2026-03-31 17:19:56 +02:00
hostip.c lib: make resolving HTTPS DNS records reliable: 2026-04-01 15:36:31 +02:00
hostip.h lib: make resolving HTTPS DNS records reliable: 2026-04-01 15:36:31 +02:00
hostip4.c lib: make resolving HTTPS DNS records reliable: 2026-04-01 15:36:31 +02:00
hostip6.c lib: make resolving HTTPS DNS records reliable: 2026-04-01 15:36:31 +02:00
hsts.c HSTS: cap the list at 1,000 entries 2026-04-01 14:38:41 +02:00
hsts.h HSTS: cap the list at 1,000 entries 2026-04-01 14:38:41 +02:00
http.c http: refactor http_on_response into sub functions 2026-03-31 23:53:45 +02:00
http.h
http1.c
http1.h
http2.c http2: return error on OOM in push headers 2026-03-31 08:59:07 +02:00
http2.h http: refactor http_on_response into sub functions 2026-03-31 23:53:45 +02:00
http_aws_sigv4.c
http_aws_sigv4.h
http_chunks.c
http_chunks.h
http_digest.c
http_digest.h
http_negotiate.c
http_negotiate.h
http_ntlm.c
http_ntlm.h
http_proxy.c
http_proxy.h
httpsrr.c lib: make resolving HTTPS DNS records reliable: 2026-04-01 15:36:31 +02:00
httpsrr.h lib: make resolving HTTPS DNS records reliable: 2026-04-01 15:36:31 +02:00
idn.c
idn.h
if2ip.c
if2ip.h
imap.c
imap.h
ldap.c
libcurl.def
libcurl.rc
libcurl.vers.in
llist.c
llist.h
macos.c
macos.h
Makefile.am
Makefile.inc cf-dns: connection filter for DNS queries 2026-03-30 13:28:38 +02:00
Makefile.soname
md4.c
md5.c md5: fix large size handling 2026-03-31 17:53:03 +02:00
memdebug.c
mime.c
mime.h
mprintf.c
mqtt.c
mqtt.h
multi.c lib: make resolving HTTPS DNS records reliable: 2026-04-01 15:36:31 +02:00
multi_ev.c
multi_ev.h
multi_ntfy.c
multi_ntfy.h
multihandle.h lib: make resolving HTTPS DNS records reliable: 2026-04-01 15:36:31 +02:00
multiif.h
netrc.c
netrc.h
noproxy.c
noproxy.h
openldap.c
optiontable.pl
parsedate.c
parsedate.h
pingpong.c
pingpong.h
pop3.c
pop3.h
progress.c
progress.h
protocol.c
protocol.h
psl.c
psl.h
rand.c
rand.h
ratelimit.c
ratelimit.h
request.c lib: make resolving HTTPS DNS records reliable: 2026-04-01 15:36:31 +02:00
request.h
rtsp.c
rtsp.h
select.c
select.h
sendf.c
sendf.h
setopt.c lib: make resolving HTTPS DNS records reliable: 2026-04-01 15:36:31 +02:00
setopt.h
setup-os400.h
setup-vms.h
setup-win32.h
sha256.c lib: accept larger input to md5/hmac/sha256/sha512 functions 2026-03-31 17:19:56 +02:00
sigpipe.h
slist.c
slist.h
smb.c
smb.h
smtp.c
smtp.h
sockaddr.h
socketpair.c
socketpair.h
socks.c lib: make resolving HTTPS DNS records reliable: 2026-04-01 15:36:31 +02:00
socks.h
socks_gssapi.c socks: reject zero-length GSSAPI/SSPI tokens from proxy 2026-03-30 16:07:06 +02:00
socks_sspi.c socks: reject zero-length GSSAPI/SSPI tokens from proxy 2026-03-30 16:07:06 +02:00
splay.c stop using the word 'just' 2026-03-03 15:30:22 +01:00
splay.h
strcase.c
strcase.h
strequal.c
strerror.c
strerror.h
system_win32.c
system_win32.h
telnet.c
telnet.h
tftp.c protocol source, all about protocols and uri schemes 2026-03-16 08:39:02 +01:00
tftp.h
thrdpool.c lib: make resolving HTTPS DNS records reliable: 2026-04-01 15:36:31 +02:00
thrdpool.h lib: make resolving HTTPS DNS records reliable: 2026-04-01 15:36:31 +02:00
thrdqueue.c lib: make resolving HTTPS DNS records reliable: 2026-04-01 15:36:31 +02:00
thrdqueue.h lib: make resolving HTTPS DNS records reliable: 2026-04-01 15:36:31 +02:00
transfer.c
transfer.h
uint-bset.c
uint-bset.h
uint-hash.c
uint-hash.h
uint-spbset.c
uint-spbset.h
uint-table.c
uint-table.h
url.c lib: make resolving HTTPS DNS records reliable: 2026-04-01 15:36:31 +02:00
url.h cf-dns: connection filter for DNS queries 2026-03-30 13:28:38 +02:00
urlapi-int.h
urlapi.c
urldata.h lib: make resolving HTTPS DNS records reliable: 2026-04-01 15:36:31 +02:00
version.c
ws.c
ws.h