Add `data->state.origin` as the origin the transfer is sending the
current request to/gets the response from. Use it for request specific
properties like authentication, hsts and cookie handling, etc.
Unless talking to a forwarding HTTP proxy (e.g. not tunneling),
`data->state.origin` and `conn->origin` are the same.
With a forwarding HTTP proxy in play, `conn->origin` is set to
`conn->http_proxy.peer` and `conn->bits.origin_is_proxy` (a new bit) is
set.
Remove the connection bits, now replaced with:
* `conn->bits.socksproxy` -> `conn->socks_proy.peer`
* `conn->bits.httpproxy` -> `conn->http_proy.peer`
* `conn->bits.proxy` -> `(conn->socks_proy.peer || conn->http_proy.peer`)
* `conn->bits.tunnel_proxy` -> (`conn->http_proy.peer && !conn->bits.origin_is_proxy`)
* `(conn->bits.httpproxy && !conn->bits.tunnel_proxy)` -> `conn->bits.origin_is_proxy`
Rename `noproxy.[ch]` to `proxy.[ch]`. Move the connection proxy setup
code from `url.c` to `proxy.c`.
Remove `data->info.conn_remote_port` as no one uses it.
Add test_40_02b for a SOCKS connection to a forwarding HTTPS proxy.
Update internal documentation about peers and creds.
Closes#21967
Move as sockaddr2string() into cf-socket.c where its only callers are.
Mark as UNITTEST for unit1609.
Move "struct Curl_sockaddr_ex" into sockaddr.h, so connect.h and
cf-socket.h can be included without all the system headers needed.
Closes#21946
Adjust code to avoid `-Wformat-signedness` warnings, while making sure
that enums are always cast to a known type when passing them to `printf`
functions, to support compilers and compiler settings where enums are
not default-size signed ints.
- cast integers printed as hex to `unsigned`. (63 times, 20 of them in
`mbedtls.c`)
- cast misc enums to `int` for printing. (31 times)
- cast `CURL_LOCK_DATA_*` enums to `int`. (4 times)
- cast `CURL_FORMADD_*` enums to `int`. (13 times)
- cast `CURLSHE_*` enums to `int`. (3 times)
- cast `CURLUE_*` enums to `int`. (33 times)
- cast `CURLMSG_*` enums to `int`. (6 times)
- cast `CURLE_*` enums to `int`. (~380 times)
- unit1675: fix mask.
Follow-up to 7c34365cce#21879
Ref: #18343 (initial attempt)
Closes#20848
Share common functions/structs between ngtcp2 HTTP/3 and the proxy
version.
Fix bugs in proxy implementation when it comes to stream and pollset
handling and transfer lifetimes.
Curl_multi_xfer_sockbuf_borrow: work without multi
When a connection gets shutdown by a share, the easy handle used is
share->admin and it does not have a multi handle. In that case let
Curl_multi_xfer_sockbuf_borrow() allocate a buffer to be freed on
release.
This happens when a TLS filter sends its last notify through a HTTP/3
proxy tunnel.
Closes#21871
Whack the times reported for a transfer (see
https://curl.se/libcurl/c/curl_easy_getinfo.html#TIMES) into order for
all variations of up-/download, http/ftp etc. Make sure they are
reported in the documented order.
There is still the *possibility* of PRETRANSFER being longer then
POSTTRANSFER, if a server sends a response before an upload is done.
POST is the time the first response byte is received, and PRE is the
time the last byte was sent by curl.
This may happen with more likelihood on HTTP/2 and 3 for a server
rejected upload. But for successful uploads, the answer will almost over
come afterwards.
Undo the previous twists in lib500.c tests, adjust pytest timeline
checks.
Fixes#21828
Reported-by: BazaarAcc32 on github
Closes#21843
closing a filter chain and reconnecting it again is a complication
that only the HTTP/1.x proxy filter used. Remove it from all filters.
Instead, a filter can return CURLE_AGAIN during the connect phase
and the cf-setup filter will tear down all "sub filters" and restart
over.
With this, a filter never resets to the initial phase but progresses
through connect -> connected -> shutdown -> destroy once.
Closes#21831
All users of the `verify_memory()` macro used a fixed-length buffer for
the test output, which then needed a workaround to silence GCC
`-Waddress` warnings.
```
tests/unit/unit1615.c: In function 'test_unit1615':
tests/libtest/unitcheck.h:51:8: error: the address of 'output_buf' will always evaluate as 'true' [-Werror=address]
51 | if((dynamic) && memcmp(dynamic, check, len)) { \
| ^
tests/unit/unit1615.c:114:3: note: in expansion of macro 'verify_memory'
114 | verify_memory(output_buf, precomp_hash1, CURL_SHA512_256_DIGEST_LENGTH);
| ^~~~~~~~~~~~~
```
Drop redundant address check and the workarounds with it.
Closes#21833
- drop more uses of the word "just". (not enforced here)
- drop some uses of the "will" word.
- "then" -> "than".
- tests/http/testenv/curl.py: fix copy-paste typo in error message.
- pytest: replace `shutdownh` with `shutdown` in test names.
Spotted by GitHub Code Quality.
- comment typos.
- whitespace and newlines fixes.
Closes#21830
When a transfer goes against another origin than the initial one, do not
add the following to the ssl configuration: client cert, client key, srp
user/pass, pinned key.
Closes#21695
code:
- less exception handling in existing code
- true ip happy eyeballing
- enable certificate verification
- cf-h2-proxy: abort connection when server closed connection
tests:
- remove all --insecure and --proxy-insecure args
- make session reuse test_60_12 a working one
- resolve port conflicts between h2o and nghttpx
- use proxy args better
- make test_60_06 run shorter
- kill h2o at the end of tests, normal stop takes too long
Ref: 59213f8248#21789
Follow-up to e78b1b3ecc#21153Closes#21798
This patch adds two major proxy capabilities to curl (ngtcp2 QUIC):
- HTTP/3 Proxy CONNECT: Tunnel HTTP/1.1 or HTTP/2 traffic through an
HTTPS proxy that speaks HTTP/3 (QUIC) using the standard CONNECT
method over an HTTP/3 connection.
- MASQUE CONNECT-UDP: Tunnel HTTP/3 (QUIC) traffic through an HTTP
proxy (speaking HTTP/1.1, HTTP/2, or HTTP/3) using the extended
CONNECT method with the CONNECT-UDP protocol (RFC9297 & RFC9298).
Public API additions:
- `CURLPROXY_HTTPS3`: new proxy type constant for HTTP/3 proxy
- `--proxy-http3`: new CLI flag to negotiate HTTP/3 with HTTPS proxy
The implementation adds two new filters:
- `H3-PROXY` - enables negotiating HTTP/3 (QUIC) to the proxy and
running CONNECT/CONNECT-UDP through that proxy transport.
- `CAPSULE` - dedicated filter inserted between QUIC transport and
HTTP-PROXY to handle datagram capsule encapsulation/decapsulation.
Here is how the curl filter chaining looks in different scenarios:
- HTTP/3 Proxy CONNECT (tunneling TCP protocols over QUIC proxy):
conn -> HTTP/1.1 or HTTP/2 -> SSL -> HTTP-PROXY ->
H3-PROXY -> HAPPY-EYEBALLS -> UDP
- MASQUE CONNECT-UDP (tunneling QUIC over any proxy):
conn -> HTTP/3 -> CAPSULE -> HTTP-PROXY -> H3-PROXY ->
HAPPY-EYEBALLS -> UDP
conn -> HTTP/3 -> CAPSULE -> HTTP-PROXY -> H1-PROXY or H2-PROXY ->
SSL -> HAPPY-EYEBALLS -> TCP
- Both features currently require the ngtcp2 QUIC backend.
- Both features are experimental (disabled by default). Enable with
`--enable-proxy-http3`(autotools) or `-DUSE_PROXY_HTTP3=ON`(CMake).
Tests:
- tests/unit/unit3400.c: Unit tests for capsule protocol encode/decode
- tests/http/test_60_h3_proxy.py: Comprehensive pytest integration suite
- tests/http/testenv/h2o.py: Managing h2o instances with HTTP/1.1, HTTP/2,
and HTTP/3 (QUIC) listeners, proxy.connect and proxy.connect-udp enabled.
References:
RFC 9297 - HTTP Datagrams and the Capsule Protocol
RFC 9298 - Proxying UDP in HTTP
RFC 9000 §16 — Variable-Length Integer Encoding
Signed-off-by: Aritra Basu <aritrbas+gh@cisco.com>
Closes#21153
- use `UNITTEST_BEGIN_SIMPLE`/`UNITTEST_END_SIMPLE` where missing.
- drop redundant `(void)arg;` where using `UNITTEST_BEGIN*`.
- unit1636: drop redundant block after `UNITTEST_BEGIN*`.
- unit1609: fix typo in comment.
- unit1627: merge to `if`s.
Closes#21715
cert_type, key, key_type, key_passwd and key_blob lived in
ssl_config_data but not in ssl_primary_config, so they were invisible to
match_ssl_primary_config() and to the TLS session cache peer key.
Two easy handles sharing a connection pool could reuse each other's
authenticated connections when they differed only on SSLKEY, SSLKEYTYPE,
KEYPASSWD, SSLCERTTYPE or SSLKEYBLOB. The second handle would silently
inherit the first handle's authenticated identity.
Promote all five fields into ssl_primary_config so the conn-reuse
predicate and session cache key cover the complete client credential
set. Also replace the fixed ":CCERT" session cache marker with the
actual clientcert path so sessions are not shared across different
client certificates.
Verified by test 3303 and 3304
Reported-By: Joshua Rogers (AISLE Research)
Closes#21667
Refactor the netrc scanner. Add test case for checking that the last
matched machine with unmatched login does not return the password as
success (unit1304).
Closes#21624
Also fix fallouts found.
Windows clang-tidy CI job is a little pickier than I'd prefer due to the
`_CURL_TESTS_CONCAT=ON` option used there, and all macros considered
local, thus checked by the compiler. Upside: it revealed macro usage
dynamics in tests. If too annoying, `first.h` may be opted-out from the
concat logic. Some macros may also be deleted instead of `#if 0`-ing.
Follow-up to e0e56e9ae4#21550
Follow-up to 5fa5cb3825#20593Closes#21554
The dh(g) parameter was read from param->beg instead of from the
cursor p returned by parsing dh(p). This caused dh(g) to always
report the same value as dh(p) when inspecting DH certificates
via CURLOPT_CERTINFO on non-OpenSSL backends.
The DSA branch correctly advances the cursor; the DH branch lost
this during what appears to be a copy-paste.
Add unit1676 to verify that dh(p) and dh(g) report distinct values
using a hand-crafted minimal DER certificate.
Assisted by: Claude Opus 4.6
Signed-off-by: Sergio Correia <scorreia@redhat.com>
Closes#21595
The SASL service name, used in authentication, is part of curl's credentials
when authenticating to a server/proxy. Make it part of `struct Curl_creds`.
Change code to use `creds` to obtain a service name. By tying creds used
to the connection, connection reuse is also only allowed when the service
name matches.
Closes#21585
Authorizdation credentials are kept in `struct Curl_creds`. This contains:
* `user`: the username, maybe the empty string
* `passwd`: the password, maybe the empty string
* `sasl_authzid`: the SASL authz value, maybe the empty string
* `oauth_bearer`: the OAUTH bearer token, maybe the empty string
* `source`: where the credentials from from
* `refcount`: a reference counter to link/unkink creds
A `creds` with all values empty is equivalent to NULL, e.g. no `creds`
instance. With reference counting, `creds` can be linked/unlinked
in several places.
See docs/internals/CREDENTIALS.md for use.
Closes#21548
Make CURLOPT_GSSAPI_DELEGATION effective on Windows builds that use SSPI
(instead of a native GSS-API implementation), so Kerberos delegation can
be requested during SPNEGO/Negotiate authentication.
Closes#21528
Add new internal `curl_url_same_origin()` to check if a href has the
same origin as a base URL. Add test cases in test1675 and use this in
http2 push handling.
Closes#21328
- make sure all UNITTEST prototypes mark in which unit test they are used,
with "@unittest" markup
- make sure all UNITTEST functions do not use Curl_ prefix, as that is a
prefix we use for global private functions and these functions are static
and therefore not global and the prefix is wrong
- drop UNITTEST for functions not used in unit tests
- make the extract-unit-protos script highlight the above issues if found
- extract-unit-protos now also outputs the unit test number for all the
generated protos in lib/unitprotos.h to aid readers. It also adds the source
file and line number where the proto originates from.
- extract-unit-protos now exits with a non-zero value if any of the above
warnings are triggered
- cf-dns: Curl_cf_dns_result => static cf_dns_result
- hostip: Curl_ipv6works => static ipv6works
- url: remove Curl_setup_conn() - not used anymore
- connect: Curl_timeleft_now_ms => UNITTEST timeleft_now_ms
Closes#21330
There is no reason we should treat this part different on Windows. Noe
anything except blank, localhost or 127.0.0.1 cause error there as well.
Also: fix query handling in urlencode_str
Closes#21296
- examples: sync debug output printf masks.
- INSTALL-CMAKE.md: tidy up section for some options.
- curl_sha512_256: delete comment suggesting an optimization.
- vtls/keylog: scope a variable.
- vtls/openssl: make a source code URL a permalink.
- vtls/schannel: drop redundant parentheses.
- test1119.pl: robustify `$1` -> `$s`.
- sync arg names in comments to match the code.
- tidy up and minor fixes to comments.
- fix formatting/indenting/comment/newline/include nits.
- move `UNITTEST` protos next to definitions, sync their argument names.
- make variables static.
- add parentheses to Perl `open()` calls.
- drop unnecessary double quotes in Perl.
- clang-format.
Closes#21000
