Seen in the 'curl-for-win / Windows gcc zlib-classic (x64)' CI job,
after it got an upstream upgrade from gcc-15 to gcc-16:
```
lib/http.c:206:6: error: 'out.str' may be used uninitialized [-Werror=maybe-uninitialized]
206 | if(header_has_value(&header, &out)) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
lib/api.h:188:3: error: 'guard.data' may be used uninitialized [-Werror=maybe-uninitialized]
188 | Curl_mapi_enter((g), (m), CURL_MAPI_FN_##fn, (r)) && (m)
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
lib/multi.c:1695:6: note: in expansion of macro 'CURL_MAPI_ENTER'
1695 | if(CURL_MAPI_ENTER(&guard, m, multi_poll, &mresult)) {
| ^~~~~~~~~~~~~~~
```
Ref: https://github.com/curl/curl/actions/runs/32626356576/job/97162205296#step:3:4906Closes#22651
Correct grammar and spelling in documentation and source comments. Also
fix the Aspell configuration key and one user-facing OpenSSL error
message.
Closes#22647
Change the storage of easy handle option strings from an array sized for
all possible options to a hash set to reduce memory footprint.
Give the hash set initially room for 4 strings, with first allocation
happening when it goes beyond that. Measurements without test suite
and a forced fail on growing the set gives:
Size Result
2 1261 tests out of 1951 reported OK: 64%
4 1792 tests out of 1951 reported OK: 91%
8 1944 tests out of 1951 reported OK: 99%
16 1949 tests out of 1951 reported OK: 99%
32 single fail of 3211, unit test for u8_strset
Add u8_strset that keeps the tuples (uint8_t id, char *str)
and allows set/unset by `id`. Add that as data->set.strings.
Define MACROS
* CURL_EASY_STR(data, id) for access
* CURL_EASY_STR_SET(data, id, s) for setting, making a copy
* CURL_EASY_STR_SETN(data, id, s) for setting, no copy
* CURL_EASY_STR_CLEAR(data, id) for unsetting
* CURL_EASY_STR_CLEAR0(data, id) for unsetting and zero-ing value
Add `data->set.str_copypostfields` to handle former `STRING_COPYPOSTFIELDS`
string that was not always a string and could carry NUL bytes.
Add unit tests to test3211.
Closes#22628
As documented and as it does for other methods.
Difficult to test since it needs a successful kerberos ticket.
Reported-by: sdgh179 on github
Fixes#22610Closes#22622
Move the two remaining fields into the state struct instead, within the
HTTP ifdef.
Also: make the AWS sigv4 code always rely on the http_host value. It
was previously conditionally trying to also work if it was not set, but
that logic was not creating an identical copy so it would fail.
Closes#22620
- 'rtsp_transport'
There is no need to keep this data around once the RTSP request has been
issued.
- 'accept_encoding'
Only needed when creating the RTSP request.
- 'uagent'
The header is generated on demand from set.str[STRING_USERAGENT]
- 'ref'
Not necessary to keep around
Closes#22603
- Move expire timeout code from multi into splay.c
- keep a "time_base" timestamp to calculate timediff_t for
actual timeout values. Unfortunately this means our
timeouts will go wrong after ~500,000 years of continuous
operations...
- use timediff_t as key in splay instead of curltime
- use timediff_t in transfers expire times instead of curltime
- re-comment splay.c for better understanding how it works
- replace splay nodes double-linked "same" list with a single link,
we almost never have duplicate keys
- keep transfer `mid` in splay nodes instead of the transfer pointer
- keep registered bit in splay node for tracking instead of separate
bit in transfer
- adapt unit1309.c to changes in timediff_t and mid
Closes#22584
- make `const` data `static`, where missing and possible.
- replace `strlen()` on literal or const strings with `sizeof()`.
While the latter is optimized by popular C compiler, e.g. MSVC only
does it with `/O2`.
- replace magic numbers with `sizeof()`, where missing.
- introduce `CURL_CSTRLEN()` macro for `sizeof(char[]) - 1`.
- use `CURL_CSTRLEN()` macro.
- move `const` before integer types, where missing.
- replace `char *var` with `var[]`, where missing and possible.
- use double const, where missing.
`static const char *` -> `static const char * const`.
- lib1514: constify pointers.
- unit3205: drop redundant cast, avoid another one.
- unit1666: map `OID()` macro to identical `STRCONST()`.
Closes#22406
Do not let unrelated credentials from a redirected URL bypass the
cross-host auth boundary
Verified by test 5023 to 5025
Follow-up to a55731050eCloses#22395
Add support for signing outgoing HTTP requests per RFC 9421 using
Ed25519 or HMAC-SHA256 algorithms.
New libcurl options:
- CURLOPT_HTTPSIG: signing algorithm ("ed25519" or "hmac-sha256")
- CURLOPT_HTTPSIG_KEY: path to hex-encoded key file
- CURLOPT_HTTPSIG_KEYID: key identifier for Signature-Input
- CURLOPT_HTTPSIG_HEADERS: space-separated components to sign
New CLI flags: --httpsig, --httpsig-key, --httpsig-keyid,
--httpsig-headers
The crypto layer follows the sha256.c multi-backend pattern with
implementations for OpenSSL (EVP_DigestSign) and wolfSSL
(wc_ed25519_sign_msg). HMAC-SHA256 uses the existing Curl_hmacit()
infrastructure which works on all backends.
Verified by test 5000 to 5021
Assisted-by: Daniel Stenberg
Signed-off-by: Sameeh Jubran <sameeh@wolfssl.com>
Closes#22386Closes#21239
- connclose/streamclose/connkeep() remove description string that was
never used anywhere. Add trace statements where reasons for closing
were not already traced and maybe not obvious.
- multi_remove_handle: only lookup former connection in pool when
transfer is set to connect only
- test1554: adapt expectations now that pool is less often locked
Closes#22379
Make sure hostname used in URL to proxy is IDN decoded form, but keep
the original hostname in case it was ipv6.
Fixes#22382
Reported-by: RMMoreton on github
Closes#22385
* format `data->state.aptr.host` without crlf, so signatures do not need
to strip it again. Add the crlf when adding the header to the request
dynbuf
* check `connect-to` strings on normalized hostname and user supplied
hostname (when those differ)
* libssh: always use the peer for setting ssh option SSH_OPTIONS_HOST,
preserve ipv6 [] enclosure, use IDN converted hostname otherwise. This
is the libssh documented expectation.
Do NOT use strings from URL parsing.
Closes#22128
A custom header name padded with blanks (`Authorization :`) missed the
exact-length compare and slipped past the cross-host Authorization and
Cookie drop, forwarding the header to the redirect target. Trim the
parsed name in both the request and proxy CONNECT header builders.
Closes#22178
Remove members of `struct urlpieces` that now live in peers/creds.
Change all remaining uses of those urlpieces.
When comparing schemes, use the protocol constants.
Closes#22171
Since we have the quirky of empty credentials (the empty string for
username and password) for Negotiate reactivated, we need to check for
this when considering Basic and Digest auth.
Verify a redirect to blank user+password in test 2208
Closes#22060
`Curl_secure_context()` checked `conn->scheme` to determine if Secure
cookies may be sent. Since 73daec6, `conn->scheme` is set to the proxy's
scheme when using an HTTPS forwarding proxy, causing the function to
return TRUE for HTTP origins. This leaked Secure cookies over the
plaintext connection between proxy and origin.
Use `data->state.origin->scheme` instead, which always reflects the
origin's scheme regardless of proxy configuration.
Not an approved vulnerability because the regression was introduced
after the last release and is not present in any released version.
Verified by test 3401
Follow-up to 73daec6620
Reported-by: daviey on hackerone
URL: https://hackerone.com/reports/3803415Closes#22024
Now that `conn->origin` can be the proxy, we need to change how
Curl_xfer_is_secure() and some other places work.
Pointed out by Codex Security
Closes#22015
Add `data->state.origin` as the origin the transfer is sending the
current request to/gets the response from. Use it for request specific
properties like authentication, hsts and cookie handling, etc.
Unless talking to a forwarding HTTP proxy (e.g. not tunneling),
`data->state.origin` and `conn->origin` are the same.
With a forwarding HTTP proxy in play, `conn->origin` is set to
`conn->http_proxy.peer` and `conn->bits.origin_is_proxy` (a new bit) is
set.
Remove the connection bits, now replaced with:
* `conn->bits.socksproxy` -> `conn->socks_proy.peer`
* `conn->bits.httpproxy` -> `conn->http_proy.peer`
* `conn->bits.proxy` -> `(conn->socks_proy.peer || conn->http_proy.peer`)
* `conn->bits.tunnel_proxy` -> (`conn->http_proy.peer && !conn->bits.origin_is_proxy`)
* `(conn->bits.httpproxy && !conn->bits.tunnel_proxy)` -> `conn->bits.origin_is_proxy`
Rename `noproxy.[ch]` to `proxy.[ch]`. Move the connection proxy setup
code from `url.c` to `proxy.c`.
Remove `data->info.conn_remote_port` as no one uses it.
Add test_40_02b for a SOCKS connection to a forwarding HTTPS proxy.
Update internal documentation about peers and creds.
Closes#21967
Trailers are delivered to the application as headers via
CLIENTWRITE_TRAILER, but unlike regular response headers they skipped
the verify_header() checks, so a server could smuggle a nul byte (or
stray CR) into a header reaching CURLOPT_HEADERFUNCTION and
curl_easy_header().
Run each assembled trailer line through Curl_verify_header(), the same
validation used for normal headers.
Covered by the new test 2106.
Closes#21896
- H3 proxy: re-sync code with original source `curl_ngtcp2.c` to reduce
differences, and to apply missed minor fixes. Also apply clang-format.
Drop redundant `#undef`s, casts, `#endif` comments, includes, drop
intermediate variables, sync include and macro order.
Follow-up to e78b1b3ecc#21153
- INSTALL-CMAKE.md: move `CURL_ENABLE_SMB` to the enable section.
- tests/http/env: rename `tcpdmp` to `tcpdump` to match object variable.
- mbedtls: drop incorrect `mbedTLS 4+` comments.
(features are also supported by 3+, meaning it's always supported.)
- lib1648: rename a variable to match purpose.
- CIPHERS.md: alpha-sort link list.
- replace rare `X''` hex markup with `0x`.
- `IP v4/6` -> `IPv4/6`.
- 'version X.Y' -> 'vX.Y', where sensible.
- 'VX.Y' -> 'vX.Y', where sensible.
- fix indents, casing, newlines, typos.
Closes#21772
This patch adds two major proxy capabilities to curl (ngtcp2 QUIC):
- HTTP/3 Proxy CONNECT: Tunnel HTTP/1.1 or HTTP/2 traffic through an
HTTPS proxy that speaks HTTP/3 (QUIC) using the standard CONNECT
method over an HTTP/3 connection.
- MASQUE CONNECT-UDP: Tunnel HTTP/3 (QUIC) traffic through an HTTP
proxy (speaking HTTP/1.1, HTTP/2, or HTTP/3) using the extended
CONNECT method with the CONNECT-UDP protocol (RFC9297 & RFC9298).
Public API additions:
- `CURLPROXY_HTTPS3`: new proxy type constant for HTTP/3 proxy
- `--proxy-http3`: new CLI flag to negotiate HTTP/3 with HTTPS proxy
The implementation adds two new filters:
- `H3-PROXY` - enables negotiating HTTP/3 (QUIC) to the proxy and
running CONNECT/CONNECT-UDP through that proxy transport.
- `CAPSULE` - dedicated filter inserted between QUIC transport and
HTTP-PROXY to handle datagram capsule encapsulation/decapsulation.
Here is how the curl filter chaining looks in different scenarios:
- HTTP/3 Proxy CONNECT (tunneling TCP protocols over QUIC proxy):
conn -> HTTP/1.1 or HTTP/2 -> SSL -> HTTP-PROXY ->
H3-PROXY -> HAPPY-EYEBALLS -> UDP
- MASQUE CONNECT-UDP (tunneling QUIC over any proxy):
conn -> HTTP/3 -> CAPSULE -> HTTP-PROXY -> H3-PROXY ->
HAPPY-EYEBALLS -> UDP
conn -> HTTP/3 -> CAPSULE -> HTTP-PROXY -> H1-PROXY or H2-PROXY ->
SSL -> HAPPY-EYEBALLS -> TCP
- Both features currently require the ngtcp2 QUIC backend.
- Both features are experimental (disabled by default). Enable with
`--enable-proxy-http3`(autotools) or `-DUSE_PROXY_HTTP3=ON`(CMake).
Tests:
- tests/unit/unit3400.c: Unit tests for capsule protocol encode/decode
- tests/http/test_60_h3_proxy.py: Comprehensive pytest integration suite
- tests/http/testenv/h2o.py: Managing h2o instances with HTTP/1.1, HTTP/2,
and HTTP/3 (QUIC) listeners, proxy.connect and proxy.connect-udp enabled.
References:
RFC 9297 - HTTP Datagrams and the Capsule Protocol
RFC 9298 - Proxying UDP in HTTP
RFC 9000 §16 — Variable-Length Integer Encoding
Signed-off-by: Aritra Basu <aritrbas+gh@cisco.com>
Closes#21153
Authorizdation credentials are kept in `struct Curl_creds`. This contains:
* `user`: the username, maybe the empty string
* `passwd`: the password, maybe the empty string
* `sasl_authzid`: the SASL authz value, maybe the empty string
* `oauth_bearer`: the OAUTH bearer token, maybe the empty string
* `source`: where the credentials from from
* `refcount`: a reference counter to link/unkink creds
A `creds` with all values empty is equivalent to NULL, e.g. no `creds`
instance. With reference counting, `creds` can be linked/unlinked
in several places.
See docs/internals/CREDENTIALS.md for use.
Closes#21548
- rename `req->proxyuserpwd` to `req->hd_proxy_auth`
- rename `req->userpwd` to `req->hd_auth`
- rename parameter `proxytunnel` to `is_connect` for Curl_http_output_auth()
- move path+query concatenation into Curl_http_output_auth(), saving an alloc when no auth is in play
- rename `H1_HD_USER_AUTH` into `H1_HD_AUTH`
Closes#21513
`struct Curl_peer` keeps information about a communication endpoint
together. It will replace `conn->host` and `conn->conn_to_host` and
proxyinfo host. It will also become part of `struct ssl_peer`.
It has a reference counter, so an instance can be shared between
connections and filters.
Elminiates `conn->host` and `conn->connect_to_host`, used in the
proxyinfo structures. Passed to DNS resolution and socks filters, etc.
Pass peer to http proxy and socks tunnel filters. Use peer in dns filter
and resolving. Make `Curl_peer` a member in the `struct ssl_peer`.
Add `docs/internals/PEERS.md` for documentation.
Closes#21472
To make it scoped for the single request appropriately.
Reported-by: Muhamad Arga Reksapati
Verify with libtest 2504: a custom Host *disabled* on reused handle
Closes#21312
- examples: sync debug output printf masks.
- INSTALL-CMAKE.md: tidy up section for some options.
- curl_sha512_256: delete comment suggesting an optimization.
- vtls/keylog: scope a variable.
- vtls/openssl: make a source code URL a permalink.
- vtls/schannel: drop redundant parentheses.
- test1119.pl: robustify `$1` -> `$s`.
- sync arg names in comments to match the code.
- tidy up and minor fixes to comments.
- fix formatting/indenting/comment/newline/include nits.
- move `UNITTEST` protos next to definitions, sync their argument names.
- make variables static.
- add parentheses to Perl `open()` calls.
- drop unnecessary double quotes in Perl.
- clang-format.
Closes#21000
Fix http auth to set the `picked` auth when sending the corresponding
headers. Fix reporting Digest as avail.
Fixes#21274
Reported-by: sergio-nsk on github
Closes#21284
... unless it is a POST and the user explicitly asked to keep doing
POST.
Add test1983/1984: verify --follow with 303 and PUT + custom GET
Fixes#20715
Reported-by: Dan Arnfield
Closes#21280
To sync names for the same macro logic between lib and src, and to move
it to the curlx namespace, to match `curlx_free()` that it's calling.
Closes#21151
