Trailers are delivered to the application as headers via
CLIENTWRITE_TRAILER, but unlike regular response headers they skipped
the verify_header() checks, so a server could smuggle a nul byte (or
stray CR) into a header reaching CURLOPT_HEADERFUNCTION and
curl_easy_header().
Run each assembled trailer line through Curl_verify_header(), the same
validation used for normal headers.
Covered by the new test 2106.
Closes#21896
Some tests may take a long time in torture mode. Make it possible
to skip individual tests when runtests in running in torture mode.
Also:
- skip test 357 for the reason above.
Saved 1-3 minutes for the Linux CI torture job, 1-1.5m on Windows.
No savings on macOS.
Reported-by: Stefan Eissing
Fixes#21873Closes#21906
As the 'c' pointer might actually get modified before it is time to free
the memory.
Verify in test 2310
Reported-by: Eunsoo Kim
Fixes#21898Closes#21900
Whack the times reported for a transfer (see
https://curl.se/libcurl/c/curl_easy_getinfo.html#TIMES) into order for
all variations of up-/download, http/ftp etc. Make sure they are
reported in the documented order.
There is still the *possibility* of PRETRANSFER being longer then
POSTTRANSFER, if a server sends a response before an upload is done.
POST is the time the first response byte is received, and PRE is the
time the last byte was sent by curl.
This may happen with more likelihood on HTTP/2 and 3 for a server
rejected upload. But for successful uploads, the answer will almost over
come afterwards.
Undo the previous twists in lib500.c tests, adjust pytest timeline
checks.
Fixes#21828
Reported-by: BazaarAcc32 on github
Closes#21843
closing a filter chain and reconnecting it again is a complication
that only the HTTP/1.x proxy filter used. Remove it from all filters.
Instead, a filter can return CURLE_AGAIN during the connect phase
and the cf-setup filter will tear down all "sub filters" and restart
over.
With this, a filter never resets to the initial phase but progresses
through connect -> connected -> shutdown -> destroy once.
Closes#21831
All users of the `verify_memory()` macro used a fixed-length buffer for
the test output, which then needed a workaround to silence GCC
`-Waddress` warnings.
```
tests/unit/unit1615.c: In function 'test_unit1615':
tests/libtest/unitcheck.h:51:8: error: the address of 'output_buf' will always evaluate as 'true' [-Werror=address]
51 | if((dynamic) && memcmp(dynamic, check, len)) { \
| ^
tests/unit/unit1615.c:114:3: note: in expansion of macro 'verify_memory'
114 | verify_memory(output_buf, precomp_hash1, CURL_SHA512_256_DIGEST_LENGTH);
| ^~~~~~~~~~~~~
```
Drop redundant address check and the workarounds with it.
Closes#21833
- drop more uses of the word "just". (not enforced here)
- drop some uses of the "will" word.
- "then" -> "than".
- tests/http/testenv/curl.py: fix copy-paste typo in error message.
- pytest: replace `shutdownh` with `shutdown` in test names.
Spotted by GitHub Code Quality.
- comment typos.
- whitespace and newlines fixes.
Closes#21830
Not documented and experimental, example:
`-D_CURL_PYTEST=/test_60_h3_proxy.py`
Ideally, this should be an env like `TFLAGS` and it should allow
selecting any test ID or a group of them, but so far could not figure
out how even a basic env could work.
Closes#21818
- merge tests into a single class.
For shorter names, to fix sort order by test number, and to align with
other tests.
- fix preconditions to make `test_60_04_guard_proxy_http3_unsupported`
actually run.
- replace local precondition with constant of the same effect.
- drop redundant non-`ngtcp2` requirement for
`test_60_04_guard_proxy_http3_unsupported`.
(seemed relevant for no longer supported openssl-quic builds.)
- drop unused `NGTCP2_ONLY_MSG` constant.
Follow-up to e4139a73c8#21798
- avoid creating unnecessary test data blobs, and minimize their scopes.
Follow-up to 91facd7bb3#21791
Follow-up to e78b1b3ecc#21153Closes#21811
When a transfer goes against another origin than the initial one, do not
add the following to the ssl configuration: client cert, client key, srp
user/pass, pinned key.
Closes#21695
Add test_31_13 to check connection reuse on mixed --ssl-reqd setting.
For that add debug env var CURL_DBG_NO_USE_SSL_ON_FIRST to disable
--ssl-reqd for the first url. Check that the connection without SSL
from the first url is not reused on the second URL that requires it.
Tweak special ftp: protocol check to fail a DEBUGASSERT on mismatched
`use_ssl` settings as that should have been caught before in the
connection reuse matching (imap/smtp etc. do not have this extra check
and rely on the general part doing its job).
Closes#21665
- test_60_02_connect_tunnel_fail[fail_h1_over_h3_proxytunnel]
- test_60_02_connect_tunnel_fail[fail_h3_over_h2_proxytunnel]
- test_60_02_connect_tunnel_fail[fail_h3_over_h3_proxytunnel]
- test_60_03_h3_target_auto_connect_udp[proxy_h3]
- test_60_15_connect_timeout
Further flaky ones may be disabled in future commits.
All to be re-enabled after stabilizing them.
Follow-up to 59213f8248#21789
Follow-up to e78b1b3ecc#21153Closes#21803
- replace literal -1 with `NGHTTP3_ERR_CALLBACK_FAILURE` in nghttp3
callback.
- replace `NGHTTP3_ERR_CALLBACK_FAILURE` with
`NGTCP2_ERR_CALLBACK_FAILURE` in ngtcp2 callbacks.
- test_60_h3_proxy: fix non-critical typo in symbol.
Spotted by GitHub Code Quality
Follow-up to e78b1b3ecc#21153Closes#21802
code:
- less exception handling in existing code
- true ip happy eyeballing
- enable certificate verification
- cf-h2-proxy: abort connection when server closed connection
tests:
- remove all --insecure and --proxy-insecure args
- make session reuse test_60_12 a working one
- resolve port conflicts between h2o and nghttpx
- use proxy args better
- make test_60_06 run shorter
- kill h2o at the end of tests, normal stop takes too long
Ref: 59213f8248#21789
Follow-up to e78b1b3ecc#21153Closes#21798
Also:
- check `curl_url()` for NULL where missing.
- free memory `curl_url_get()` return pointer where missing.
- propagate `curl_url_set()` errors in `clear_url()`, where missing.
- add missing NULL-check before `strcmp()` in `clear_url()`.
Closes#21780
- H3 proxy: re-sync code with original source `curl_ngtcp2.c` to reduce
differences, and to apply missed minor fixes. Also apply clang-format.
Drop redundant `#undef`s, casts, `#endif` comments, includes, drop
intermediate variables, sync include and macro order.
Follow-up to e78b1b3ecc#21153
- INSTALL-CMAKE.md: move `CURL_ENABLE_SMB` to the enable section.
- tests/http/env: rename `tcpdmp` to `tcpdump` to match object variable.
- mbedtls: drop incorrect `mbedTLS 4+` comments.
(features are also supported by 3+, meaning it's always supported.)
- lib1648: rename a variable to match purpose.
- CIPHERS.md: alpha-sort link list.
- replace rare `X''` hex markup with `0x`.
- `IP v4/6` -> `IPv4/6`.
- 'version X.Y' -> 'vX.Y', where sensible.
- 'VX.Y' -> 'vX.Y', where sensible.
- fix indents, casing, newlines, typos.
Closes#21772
The message triggered earlier than intended and did not take the
transfer/content type into account.
Ref #21603
Reported-by: Joshua Rogers
Closes#21756
This patch adds two major proxy capabilities to curl (ngtcp2 QUIC):
- HTTP/3 Proxy CONNECT: Tunnel HTTP/1.1 or HTTP/2 traffic through an
HTTPS proxy that speaks HTTP/3 (QUIC) using the standard CONNECT
method over an HTTP/3 connection.
- MASQUE CONNECT-UDP: Tunnel HTTP/3 (QUIC) traffic through an HTTP
proxy (speaking HTTP/1.1, HTTP/2, or HTTP/3) using the extended
CONNECT method with the CONNECT-UDP protocol (RFC9297 & RFC9298).
Public API additions:
- `CURLPROXY_HTTPS3`: new proxy type constant for HTTP/3 proxy
- `--proxy-http3`: new CLI flag to negotiate HTTP/3 with HTTPS proxy
The implementation adds two new filters:
- `H3-PROXY` - enables negotiating HTTP/3 (QUIC) to the proxy and
running CONNECT/CONNECT-UDP through that proxy transport.
- `CAPSULE` - dedicated filter inserted between QUIC transport and
HTTP-PROXY to handle datagram capsule encapsulation/decapsulation.
Here is how the curl filter chaining looks in different scenarios:
- HTTP/3 Proxy CONNECT (tunneling TCP protocols over QUIC proxy):
conn -> HTTP/1.1 or HTTP/2 -> SSL -> HTTP-PROXY ->
H3-PROXY -> HAPPY-EYEBALLS -> UDP
- MASQUE CONNECT-UDP (tunneling QUIC over any proxy):
conn -> HTTP/3 -> CAPSULE -> HTTP-PROXY -> H3-PROXY ->
HAPPY-EYEBALLS -> UDP
conn -> HTTP/3 -> CAPSULE -> HTTP-PROXY -> H1-PROXY or H2-PROXY ->
SSL -> HAPPY-EYEBALLS -> TCP
- Both features currently require the ngtcp2 QUIC backend.
- Both features are experimental (disabled by default). Enable with
`--enable-proxy-http3`(autotools) or `-DUSE_PROXY_HTTP3=ON`(CMake).
Tests:
- tests/unit/unit3400.c: Unit tests for capsule protocol encode/decode
- tests/http/test_60_h3_proxy.py: Comprehensive pytest integration suite
- tests/http/testenv/h2o.py: Managing h2o instances with HTTP/1.1, HTTP/2,
and HTTP/3 (QUIC) listeners, proxy.connect and proxy.connect-udp enabled.
References:
RFC 9297 - HTTP Datagrams and the Capsule Protocol
RFC 9298 - Proxying UDP in HTTP
RFC 9000 §16 — Variable-Length Integer Encoding
Signed-off-by: Aritra Basu <aritrbas+gh@cisco.com>
Closes#21153
Since all API features we need for TLSv1.3 earlydata support do exist
only from version 3.7.2 onwards, make that the minimal version required.
Fixes#21750
Reported-by: Johannes Schlatow
Closes#21751
