This patch adds two major proxy capabilities to curl (ngtcp2 QUIC):
- HTTP/3 Proxy CONNECT: Tunnel HTTP/1.1 or HTTP/2 traffic through an
HTTPS proxy that speaks HTTP/3 (QUIC) using the standard CONNECT
method over an HTTP/3 connection.
- MASQUE CONNECT-UDP: Tunnel HTTP/3 (QUIC) traffic through an HTTP
proxy (speaking HTTP/1.1, HTTP/2, or HTTP/3) using the extended
CONNECT method with the CONNECT-UDP protocol (RFC9297 & RFC9298).
Public API additions:
- `CURLPROXY_HTTPS3`: new proxy type constant for HTTP/3 proxy
- `--proxy-http3`: new CLI flag to negotiate HTTP/3 with HTTPS proxy
The implementation adds two new filters:
- `H3-PROXY` - enables negotiating HTTP/3 (QUIC) to the proxy and
running CONNECT/CONNECT-UDP through that proxy transport.
- `CAPSULE` - dedicated filter inserted between QUIC transport and
HTTP-PROXY to handle datagram capsule encapsulation/decapsulation.
Here is how the curl filter chaining looks in different scenarios:
- HTTP/3 Proxy CONNECT (tunneling TCP protocols over QUIC proxy):
conn -> HTTP/1.1 or HTTP/2 -> SSL -> HTTP-PROXY ->
H3-PROXY -> HAPPY-EYEBALLS -> UDP
- MASQUE CONNECT-UDP (tunneling QUIC over any proxy):
conn -> HTTP/3 -> CAPSULE -> HTTP-PROXY -> H3-PROXY ->
HAPPY-EYEBALLS -> UDP
conn -> HTTP/3 -> CAPSULE -> HTTP-PROXY -> H1-PROXY or H2-PROXY ->
SSL -> HAPPY-EYEBALLS -> TCP
- Both features currently require the ngtcp2 QUIC backend.
- Both features are experimental (disabled by default). Enable with
`--enable-proxy-http3`(autotools) or `-DUSE_PROXY_HTTP3=ON`(CMake).
Tests:
- tests/unit/unit3400.c: Unit tests for capsule protocol encode/decode
- tests/http/test_60_h3_proxy.py: Comprehensive pytest integration suite
- tests/http/testenv/h2o.py: Managing h2o instances with HTTP/1.1, HTTP/2,
and HTTP/3 (QUIC) listeners, proxy.connect and proxy.connect-udp enabled.
References:
RFC 9297 - HTTP Datagrams and the Capsule Protocol
RFC 9298 - Proxying UDP in HTTP
RFC 9000 §16 — Variable-Length Integer Encoding
Signed-off-by: Aritra Basu <aritrbas+gh@cisco.com>
Closes#21153
The HTTP/2 feature is deprecated, few servers implement it and our
implementation is complicated by its state management. Make the two
CURLOPT_* involved a nop and deprecate them.
Closes#21723
In AppleSecTrust or NativeCA-enabled builds, make sure override it when
setting a custom `CURLOPT_PROXY_CAINFO_BLOB`.
Reported-by: Joshua Rogers (Aisle Research)
Follow-up to 1730407b74#18279
Follow-up to eefd03c572#18703Closes#21631
Make CURLOPT_GSSAPI_DELEGATION effective on Windows builds that use SSPI
(instead of a native GSS-API implementation), so Kerberos delegation can
be requested during SPNEGO/Negotiate authentication.
Closes#21528
This bit was used to do Digest authentication like Internet Explorer
before version 7 (released on October 18, 2006). Presumably no one uses
this anymore and since it is hard to use and does broken auth, starting
in 8.21.0 this bit does nothing (except setting the actual Digest bit).
Closes#21486
- passing an unknown string to CURLOPT_ECH now returns error
To properly allow applications to spot if they pass in a typo or
something to libcurl.
- CURLECH_DISABLE is now a plain zero internally, not a dedicated bit which
simplifies checks for when ECH is enabled
- Dropped the CURLECH_CLA_CFG bit, and just check STRING_ECH_CONFIG
- Turn grease/enable/hard into three different numerical values, no bitmask
needed
- Convert the struct field 'tls_ech' from an int to a byte.
Closes#21532
- enable three checks:
- bugprone-invalid-enum-default-initialization
- bugprone-sizeof-expression
- readability-inconsistent-declaration-parameter-name (strict)
- fix remaining discrepancies with arg names in prototypes
and implementation, in strict mode.
- document reason for some checks tested but not enabled.
Closes#20794
To sync names for the same macro logic between lib and src, and to move
it to the curlx namespace, to match `curlx_free()` that it's calling.
Closes#21151
- allow to specify when they are wanted on starting a resolve
- match dns cache entries accordingly. An entry which never
tried to get HTTPS-RRs is no answer for a resolve that wants
it.
- fix late arrivals of resolve answers to match the "async"
records that started them - if it still exists.
- provide for multiple "async" resolves in a transfer at the
same time. We may need to resolve an IP interface while the
main connection resolve has not finished yet.
- allow lookup of HTTPS-RR information as soon as it is
available, even if A/AAAA queries are still ongoing.
For this, the "async" infrastructure is changed:
- Defined bits for DNS queries `CURL_DNSQ_A`, `CURL_DNSQ_AAAA`
and `CURL_DNSQ_HTTPS`. These replace `ip_version` which says
nothing about HTTPS.
Use them in dns cache entries for matching.
- enhance the `async->id` to be a unique `uint32_t` for
resolves inside one multi. This is weak, as the id may
wrap around. However it is combined with the `mid` of
the easy handle, making collisions highly unlikely.
`data->state.async` is only accessed in few places where
the mid/async-id match is performed.
- vtls: for ECH supporting TLS backends (openssl, rustls, wolfssl),
retrieve the HTTPS-RR information from the dns connection filter.
Delay the connect if the HTTPS-RR is needed, but has not
been resolved yet.
The implementation of all this is complete for the threaded
resolver. c-ares resolver and DoH do not take advantage of
all new async features yet. To be done in separate PRs.
Details:
c-ares: cleanup settings and initialisation. Any ares channel
is only being created on starting a resolve and propagating
operations in setopt.c to the channel are not helpful.
Changed threaded+ares pollset handling so that they do not
overwrite each others `ASYNC_NAME` timeouts.
Add trace name 'threads' for tracing thread queue and
pool used by threaded resolver.
Closes#21175
Replace the `volatile int dirty` with a reference counter
protected by a mutex when available.
Solve the problem of when to call application's lock function
by adding a volatile flag that indicates a share has been added
to easy handles in its lifetime. That flag ever goes from
FALSE to TRUE, so volatile might work (in the absence of a mutex).
(The problem is that the lock/unlock functions need 2-3
`curl_share_setopt()` invocations to become usable and there
is no way of telling if the third will ever happen. Calling
the lock function before the 3rd setopt may crash the
application.)
When removing a share from an easy handle (or replacing it with
another share), detach the easy connection on a share with a
connection pool.
When cleaning up a share, allow this even if it is still used in
easy handles. It will be destroyed when the reference count
drops to 0.
Closes#20870
- Rename `Curl_resolv_unlink()` to `Curl_dns_entry_unlink()`.
- Change `Curl_dnscache_get()` to return CURLcode result. Returns
now `CURLE_COULDNT_RESOLVE_HOST` for "negative" cache entries.
- Add `Curl_dnscache_add_negative()` to put a "negative" entry
into the cache.
Closes#20864
- the value collided with the internal one used for WebSocket
- we add all new ones as internal bits
- bump to 64-bit internal type
- introduce new define for setting all protocols (CURLPROTO_64ALL)
Reported by Codex Security
Closes#20798
- fix internal macro `AN_APPLE_OS` reused between sources without
resetting it. It may potentially have left the system sha256
function unused.
- fix to define `WOLFSSL_OPTIONS_IGNORE_SYS` so that it always applies
to wolfSSL headers, also during feature detection.
- md4, md5, sha256: simplify fallback logic.
- delete 20+ unused macros.
- scope or move macros to avoid `-Wunused-macros` warnings.
- examples: delete unused code.
The warning detects macros defined but not used within the same C
source. It does not warn for macros defined in headers. It also works
with unity builds, but to a lesser extent.
Closes#20593
- de-dupe lib/src strdup/memdup functions into curlx.
- introduce `CURLX_STRDUP_LOW()` for mapping `strdup()`, and to do it at
one place within the code, in `curl_setup.h`.
- tests/server: use `curlx_strdup()`. (Also to fix building without
a system `strdup()`.)
- curlx/curlx.h: shorten and tidy up.
- adjust Windows build path to not need `HAVE_STRDUP`.
- build: stop detecting `HAVE_STRDUP` on Windows.
Closes#20497
GCC 15.2 warns when assigning computed "all" bitmask macros to 32-bit
flag types because negated masks expand to the full width of unsigned
long/long on 64-bit platforms.
Mask these macros to a 32-bit domain so they do not set high bits and
avoid -Woverflow/-Wconversion warnings in callers.
Reported-by: Patrick Monnerat
Fixes#20242Closes#20416
- Use upper limit INT_MAX instead of UINT_MAX.
UINT_MAX doesn't work as the max value for the variable since it is
passed as a long and becomes -1 on platforms that have same sized
int and long, like Windows.
Closes https://github.com/curl/curl/pull/20414
This allows builds know about all schemes - but only have the protocol
implementations for those actually built-in.
It further allows multiple protocols to reuse the same protocol setup
and functions for both TLS and non-TLS implementations instead of
needing two (or more) structs.
The scheme information is now in 'struct Curl_scheme' and all the
function pointers for each scheme/protocol implementation are in struct
Curl_protocol.
The URL API now always work with all known protocols.
Closes#20351
- Split setopt_long() into multiple functions
(setopt_long_bool/net/http/proxy/ssl/proto/misc) and dispatch them
using a small function table
- Extract proxy string option handling into setopt_cptr_proxy() and call
it early from setopt_cptr()
- Reorder options to simplify #ifdef blocks and reduce per-function size
- Updates code paths to return CURLE_UNKNOWN_OPTION from helper defaults
so dispatching can continue cleanly
Closes#20280
To simplify the directory layout.
- OS400 and vms support move from `packages` to `projects`.
- Windows README and `generate.bat` files move from `projects`
to `projects/Windows`.
Closes#20271
This makes the easy handle 432 bytes smaller (totally 5352 bytes on my
rather maximized Linux 64 bit build). The 440 byte mimepost struct is
now allocated only when needed.
Closes#20260
Makes sure they work identically cross-platform, as long varies in size
between Windows vs non-Windows. Makes Curl_easy 16 bytes smaller on 64
bit Linux.
This reduces support for the RTSP cseq counters to 32 bit (down from 63
bit previously on 64 bit non-Windows), but it is probably safe.
Implementations probably rarely support anything above 32 bits anyway
and this is how curl has worked on Windows since always.
There is now only one 'long' left in urldata.h (in the ssl_config_data
struct). That field, certverifyresult, is used to store the response
code from TLS backend code and in the OpenSSL case that function returns
an actual 'long'.
Closes#20227
In particular, it turns 'unsigned long' into 'uint32_t' since the code
needs to build and run just as fine on Windows which has 32 bit longs,
so we know the code works with 32 bit field versions.
This makes Curl_easy 56 bytes smaller on my 64 bit Linux (maximized
build).
Closes#20209
- asyn-thrdd.c: scope an include.
- apply more clang-format suggestions.
- tidy-up PP guard comments.
- delete empty line from the top of headers.
- add empty line after `curl_setup.h` include where missing.
- fix indent.
- CODE_STYLE.md: add `strcpy`.
Follow-up to 8636ad55df#20088
- lib1901.c: drop unnecessary line.
Follow-up to 436e67f65b#20076Closes#20070
- drop unused `http.h` includes.
- drop unused `http1.h` include.
- drop unused `http2.h` includes.
- vssh/ssh.h: drop unused `vssh.h` include.
- urldata.h: drop unused protocol includes.
- url: include `smtp.h` directly.
- rtsp.h: include directly where used.
- imap, smtp: drop redundant include, move another from .h to .c.
Verified with an all non-unity CI run.
Closes#20093
- replace `sendf.h` with `curl_trc.h` where it was included just for it.
- drop unused `curl_trc.h` includes.
- easy: delete obsolete comment about `send.h` include reason.
Also:
- move out `curl_trc.h` include from `sendf.h` and include it directly
in users, where not done already. To flatten the include tree and
to less rely on indirect includes.
- stop including `sendf.h` from other headers, replace it with forward
declaration of `Curl_easy`, as done already elsewhere.
Verified with an all non-unity CI run.
Closes#20061
To make it available for all files. Drop includes from individual
sources. This header was already included from most sources and not
specific to any internal subsystem.
Also to ensure that two system symbol redefines on Windows (`read()` and
`write()`) get applied to all sources. Move them to `curl_setup.h`.
Closes#20056
- curl_range: replace `sendf.h` with direct header dependency
`curl_trc.h`.
- drop `curl/curl.h` includes from internal sourcees in favor of the
include made from `curl_setup.h`. Replace it with the latter where
it's the only include.
- include `curl_setup.h` before using macros, where missing.
- drop redundant `stdlib.h`, `string.h` includes, in favor of
`curl_setup_once.h` including them.
- drop redundant `limits.h` in favor of `curl_setup.h` including it.
- fake_addrinfo.h: fix typo in comment.
- curl_setup_once.h: drop `stdio.h` in favor of earlier include in
`curl_setup.h`.
- drop stray, unused, `stddef.h` includes.
- memdebug.h: add missing `stddef.h` include. (relying on accidental
includes via other headers before this patch.)
- stddef.h: document why it's included.
- strerr: drop `curl/mprintf.h` in favor of `curl/curl.h` including it
via `curl_setup.h`.
Closes#20027
Always use curlx_now() when calling Curl_pgrs_now(data). Tests with the
"manual" updates to now proved differ more then 100ms in parallel testing.
Add `curlx_nowp()` to set current time into a struct curltime.
Add `curlx_ptimediff_ms() and friends, passing pointers.
Update documentation.
Closes#19998
