Add `data->state.origin` as the origin the transfer is sending the
current request to/gets the response from. Use it for request specific
properties like authentication, hsts and cookie handling, etc.
Unless talking to a forwarding HTTP proxy (e.g. not tunneling),
`data->state.origin` and `conn->origin` are the same.
With a forwarding HTTP proxy in play, `conn->origin` is set to
`conn->http_proxy.peer` and `conn->bits.origin_is_proxy` (a new bit) is
set.
Remove the connection bits, now replaced with:
* `conn->bits.socksproxy` -> `conn->socks_proy.peer`
* `conn->bits.httpproxy` -> `conn->http_proy.peer`
* `conn->bits.proxy` -> `(conn->socks_proy.peer || conn->http_proy.peer`)
* `conn->bits.tunnel_proxy` -> (`conn->http_proy.peer && !conn->bits.origin_is_proxy`)
* `(conn->bits.httpproxy && !conn->bits.tunnel_proxy)` -> `conn->bits.origin_is_proxy`
Rename `noproxy.[ch]` to `proxy.[ch]`. Move the connection proxy setup
code from `url.c` to `proxy.c`.
Remove `data->info.conn_remote_port` as no one uses it.
Add test_40_02b for a SOCKS connection to a forwarding HTTPS proxy.
Update internal documentation about peers and creds.
Closes#21967
Share common functions/structs between ngtcp2 HTTP/3 and the proxy
version.
Fix bugs in proxy implementation when it comes to stream and pollset
handling and transfer lifetimes.
Curl_multi_xfer_sockbuf_borrow: work without multi
When a connection gets shutdown by a share, the easy handle used is
share->admin and it does not have a multi handle. In that case let
Curl_multi_xfer_sockbuf_borrow() allocate a buffer to be freed on
release.
This happens when a TLS filter sends its last notify through a HTTP/3
proxy tunnel.
Closes#21871
Do not send PONG frames unless there is sufficient space left in the
websocket send buffer. A server might be lazy in reading our data and
intermediary PONG frames can be skipped by a client (RFC 6455, ch.
5.5.3).
Add test case measuring no real RSS increase on a server blasting with
PING frames.
Closes#21911
- dante.py, dnsd.py, sshd.py: drop redundant conditions.
Spotted in sshd by GitHub Code Quality.
- curl.py: comment out `if` to silence CodeQL warning.
Reported by GitHub CodeQL
Follow-up to 8145476d5d#21916Closes#21917
Also:
- drop two unreachable return statements.
- test_17_ssl_use: avoid implicit string concatenations in lists.
Reported by GitHub CodeQL
Closes#21916
Whack the times reported for a transfer (see
https://curl.se/libcurl/c/curl_easy_getinfo.html#TIMES) into order for
all variations of up-/download, http/ftp etc. Make sure they are
reported in the documented order.
There is still the *possibility* of PRETRANSFER being longer then
POSTTRANSFER, if a server sends a response before an upload is done.
POST is the time the first response byte is received, and PRE is the
time the last byte was sent by curl.
This may happen with more likelihood on HTTP/2 and 3 for a server
rejected upload. But for successful uploads, the answer will almost over
come afterwards.
Undo the previous twists in lib500.c tests, adjust pytest timeline
checks.
Fixes#21828
Reported-by: BazaarAcc32 on github
Closes#21843
- drop more uses of the word "just". (not enforced here)
- drop some uses of the "will" word.
- "then" -> "than".
- tests/http/testenv/curl.py: fix copy-paste typo in error message.
- pytest: replace `shutdownh` with `shutdown` in test names.
Spotted by GitHub Code Quality.
- comment typos.
- whitespace and newlines fixes.
Closes#21830
- merge tests into a single class.
For shorter names, to fix sort order by test number, and to align with
other tests.
- fix preconditions to make `test_60_04_guard_proxy_http3_unsupported`
actually run.
- replace local precondition with constant of the same effect.
- drop redundant non-`ngtcp2` requirement for
`test_60_04_guard_proxy_http3_unsupported`.
(seemed relevant for no longer supported openssl-quic builds.)
- drop unused `NGTCP2_ONLY_MSG` constant.
Follow-up to e4139a73c8#21798
- avoid creating unnecessary test data blobs, and minimize their scopes.
Follow-up to 91facd7bb3#21791
Follow-up to e78b1b3ecc#21153Closes#21811
Add test_31_13 to check connection reuse on mixed --ssl-reqd setting.
For that add debug env var CURL_DBG_NO_USE_SSL_ON_FIRST to disable
--ssl-reqd for the first url. Check that the connection without SSL
from the first url is not reused on the second URL that requires it.
Tweak special ftp: protocol check to fail a DEBUGASSERT on mismatched
`use_ssl` settings as that should have been caught before in the
connection reuse matching (imap/smtp etc. do not have this extra check
and rely on the general part doing its job).
Closes#21665
- test_60_02_connect_tunnel_fail[fail_h1_over_h3_proxytunnel]
- test_60_02_connect_tunnel_fail[fail_h3_over_h2_proxytunnel]
- test_60_02_connect_tunnel_fail[fail_h3_over_h3_proxytunnel]
- test_60_03_h3_target_auto_connect_udp[proxy_h3]
- test_60_15_connect_timeout
Further flaky ones may be disabled in future commits.
All to be re-enabled after stabilizing them.
Follow-up to 59213f8248#21789
Follow-up to e78b1b3ecc#21153Closes#21803
- replace literal -1 with `NGHTTP3_ERR_CALLBACK_FAILURE` in nghttp3
callback.
- replace `NGHTTP3_ERR_CALLBACK_FAILURE` with
`NGTCP2_ERR_CALLBACK_FAILURE` in ngtcp2 callbacks.
- test_60_h3_proxy: fix non-critical typo in symbol.
Spotted by GitHub Code Quality
Follow-up to e78b1b3ecc#21153Closes#21802
code:
- less exception handling in existing code
- true ip happy eyeballing
- enable certificate verification
- cf-h2-proxy: abort connection when server closed connection
tests:
- remove all --insecure and --proxy-insecure args
- make session reuse test_60_12 a working one
- resolve port conflicts between h2o and nghttpx
- use proxy args better
- make test_60_06 run shorter
- kill h2o at the end of tests, normal stop takes too long
Ref: 59213f8248#21789
Follow-up to e78b1b3ecc#21153Closes#21798
- H3 proxy: re-sync code with original source `curl_ngtcp2.c` to reduce
differences, and to apply missed minor fixes. Also apply clang-format.
Drop redundant `#undef`s, casts, `#endif` comments, includes, drop
intermediate variables, sync include and macro order.
Follow-up to e78b1b3ecc#21153
- INSTALL-CMAKE.md: move `CURL_ENABLE_SMB` to the enable section.
- tests/http/env: rename `tcpdmp` to `tcpdump` to match object variable.
- mbedtls: drop incorrect `mbedTLS 4+` comments.
(features are also supported by 3+, meaning it's always supported.)
- lib1648: rename a variable to match purpose.
- CIPHERS.md: alpha-sort link list.
- replace rare `X''` hex markup with `0x`.
- `IP v4/6` -> `IPv4/6`.
- 'version X.Y' -> 'vX.Y', where sensible.
- 'VX.Y' -> 'vX.Y', where sensible.
- fix indents, casing, newlines, typos.
Closes#21772
This patch adds two major proxy capabilities to curl (ngtcp2 QUIC):
- HTTP/3 Proxy CONNECT: Tunnel HTTP/1.1 or HTTP/2 traffic through an
HTTPS proxy that speaks HTTP/3 (QUIC) using the standard CONNECT
method over an HTTP/3 connection.
- MASQUE CONNECT-UDP: Tunnel HTTP/3 (QUIC) traffic through an HTTP
proxy (speaking HTTP/1.1, HTTP/2, or HTTP/3) using the extended
CONNECT method with the CONNECT-UDP protocol (RFC9297 & RFC9298).
Public API additions:
- `CURLPROXY_HTTPS3`: new proxy type constant for HTTP/3 proxy
- `--proxy-http3`: new CLI flag to negotiate HTTP/3 with HTTPS proxy
The implementation adds two new filters:
- `H3-PROXY` - enables negotiating HTTP/3 (QUIC) to the proxy and
running CONNECT/CONNECT-UDP through that proxy transport.
- `CAPSULE` - dedicated filter inserted between QUIC transport and
HTTP-PROXY to handle datagram capsule encapsulation/decapsulation.
Here is how the curl filter chaining looks in different scenarios:
- HTTP/3 Proxy CONNECT (tunneling TCP protocols over QUIC proxy):
conn -> HTTP/1.1 or HTTP/2 -> SSL -> HTTP-PROXY ->
H3-PROXY -> HAPPY-EYEBALLS -> UDP
- MASQUE CONNECT-UDP (tunneling QUIC over any proxy):
conn -> HTTP/3 -> CAPSULE -> HTTP-PROXY -> H3-PROXY ->
HAPPY-EYEBALLS -> UDP
conn -> HTTP/3 -> CAPSULE -> HTTP-PROXY -> H1-PROXY or H2-PROXY ->
SSL -> HAPPY-EYEBALLS -> TCP
- Both features currently require the ngtcp2 QUIC backend.
- Both features are experimental (disabled by default). Enable with
`--enable-proxy-http3`(autotools) or `-DUSE_PROXY_HTTP3=ON`(CMake).
Tests:
- tests/unit/unit3400.c: Unit tests for capsule protocol encode/decode
- tests/http/test_60_h3_proxy.py: Comprehensive pytest integration suite
- tests/http/testenv/h2o.py: Managing h2o instances with HTTP/1.1, HTTP/2,
and HTTP/3 (QUIC) listeners, proxy.connect and proxy.connect-udp enabled.
References:
RFC 9297 - HTTP Datagrams and the Capsule Protocol
RFC 9298 - Proxying UDP in HTTP
RFC 9000 §16 — Variable-Length Integer Encoding
Signed-off-by: Aritra Basu <aritrbas+gh@cisco.com>
Closes#21153
Since all API features we need for TLSv1.3 earlydata support do exist
only from version 3.7.2 onwards, make that the minimal version required.
Fixes#21750
Reported-by: Johannes Schlatow
Closes#21751
Fix a recent regression: when a unix-socket is configured, all proxy
settings must be ignored. The `via_peer` had been checked correctly,
but the connections proxy bits were not cleared.
Add test_11_04 to verify.
Reported-by: Fabian Keil (libcurl mailing list)
Closes#21630
Or consisting of just a single dot.
Such names cannot be resolved with DNS.
While they *can* still be resolved with /etc/hosts or --resolve tricks,
they easily cause internal problems because their trailing dots.
Let's not allow them anymore.
Closes#21622
- update action `actions/cache` from 5.0.4 to 5.0.5
- update action `actions/upload-artifact` from 7.0.0 to 7.0.1
- update action `github/codeql-action` from 4.32.4 to 4.35.2
- update action `msys2/setup-msys2` from 2.31.0 to 2.31.1
- update pip `filelock` from 3.25.2 to 3.29.0
- update pip `impacket` to 0.13.0
- update pip `ruff` from 0.15.10 to 0.15.12
Closes#21483Closes#21482
The combination of a misspelled directory name and failing to check the
result of cd leads to infinite recursion as `make check` simply invokes
itself over and over.
Follow-up to 00887aee8c#18079Closes#21378
Add a "limit" handler to httpd that responds 429 after 5 requests and
then 429s all requests for 2 seconds. After which another 4 requests are
served before 429 strikes again.
Closes#21357
Support HTTPS records in dnsd that have ALPN preferences.
Add pytest test_22_* to verify that HTTPS record ALPN
preferencces take effect if the HTTPS resolve arrives in time.
Fix HTTPS eyeballing use of timeouts for second attempt. Also,
make an initial HTTP/1.1 attempt switch off HTTP/2.
Closes#21329
Test has been flaky due to not waiting for the test server to be
listening. On slow CI systems, the started thread is late and the curl
command gets a refused connection.
Closes#21337
Add "Delay-A: ms", "Delay-AAAA: ms" and "Delay-HTTPS: ms" to the test
dnsd config and implement delayed response handling.
Add test_21_09 and test_21_10 to check that delayed responses connect
using the undelayed address family.
Closes#21299
When a user supplied an ip address in a URL as hostname, use that even
when address family restrictions like -4 or -6 are set.
Add test_10_15/16 to verify with a local proxy server.
Fixes#21146
Reported-by: Terrance Wong
How:
- cf-dns: on see the hostname is an ip(v6) address, add the respective
A/AAAA to the dns query bits
- cf-dns/hostip: only hand out addrinfos for a family if that family
is part of the DNS queries. That prevents for example ipv6 addresses
to show up from dns cache entries
- change cf-ip-happy to no longer check for "ip_version" and instead
use all addresses that cf-dns hands out
Closes#21295
Add `resolv-threaded` to curlinfo to detect use of the threaded resolver
correctly even with c-ares linked to https-rr.
Run test_21_05 exactly when threaded resolver is built.
Closes#21287
