* move the TIMER_POSTQUEUE to the time a connection is chosen,
so that TIMER_NAMELOOKUP always happens afterwards
* client writer: do not trigger TIMER_STARTTRANSFER on CLIENTWRITE_INFO
as ftp and other pingpong protocols write that before starting anything
that is the tranfer itself
* Elimnating debug trancing of "closed stream/connection - bailing"
as confusing, as connection is not really closed on most cases.
* Setting 'data->req.upload_done` correctly, so that no "abort upload"
is happening at the end of a perfectly fine download.
* Adding test cases with up-/download of 0-length files.
* pytest: add a "timeline" of timer value checks to Resulst in curl.py,
so that this can be used in several test cases, replacing the local
stuff in test_16
* add timeline checks to ftp test cases
Closes#19269
Description of how this works in `docs/internal/RATELIMITS.ms`.
Notable implementation changes:
- KEEP_SEND_PAUSE/KEEP_SEND_HOLD and KEEP_RECV_PAUSE/KEEP_RECV_HOLD
no longer exist. Pausing is down via blocked the new rlimits.
- KEEP_SEND_TIMED no longer exists. Pausing "100-continue" transfers
is done in the new `Curl_http_perform_pollset()` method.
- HTTP/2 rate limiting implemented via window updates. When
transfer initiaiting connection has a ratelimit, adjust the
initial window size
- HTTP/3 ngtcp2 rate limitin implemnented via ack updates
- HTTP/3 quiche does not seem to support this via its API
- the default progress-meter has been improved for accuracy
in "current speed" results.
pytest speed tests have been improved.
Closes#19384
The parsing of the HTTP/1.1 formatted request into the h2/h3 header
structures should detect CURLOPT_CUSTOMREQUEST methods and forward them
correctly.
Add test_01_20 to verify
Fixes#19543
Reported-by: Omdahake on github
Closes#19563
- badwords.pl: add `-a` option to check all lines in source code files.
Before this patch indented lines were skipped (to avoid Markdown code
fences.)
- GHA/checksrc: use `-a` when verifying the source code.
- GHA/checksrc: disable `So` and `But` rules for source code.
- GHA/checksrc: add docs/examples to the verified sources.
- badwords.txt: delete 4 duplicates.
- badwords.txt: group and sort contractions.
- badwords.txt: allow ` url = `, `DIR`, `<file name`.
Closes#19536
Add new functions in `curlx/warnless.h` for controlled type
conversions:
* curlx_uitouz, convert unsigned into to size_t (should always work)
* curlx_uztoso, convert size_t to curl_off_t, capping at CURL_OFF_T_MAX
* curlx_sztouz, convert ssize_t to size_t, return TRUE when ok
* curlx_sotouz_range, convert curl_off_t to size_t interval, capping
values to interval bounds
Remove some unnecesary casts, convert some internal recv functions
to the "return result, have size_t* arg" pattern.
Closes#19495
Rename `Curl_timeleft()` to `Curl_timeleft_ms()` to make the units in
the returned `timediff_t` clear. (We used to always have ms there, but
with QUIC started to sometimes calc ns as well).
Rename some assigned vars without `_ms` suffix for clarity as well.
Closes#19486
In pytests test_08 with the Caddy server, the new rate-limiting in
ngtcp2 did close the connection because it found "too many" stream data
packet repeats.
It is unclear if this is some Caddy issue or if the ngtcp2 implementaton
is wrong. Or if curl can do anything here.
Reported as https://github.com/ngtcp2/ngtcp2/issues/1850
This PR overwrites the ratelimit defaults in ngtcp2 with ten times
increased values. This makes the errors disappear on macOS.
Enable test_08_04/05 in CI again to see if there are any issues
to be found there. (We had those disabled before having parallel
pytests.)
Closes#19274
Replace the check if a ssl session cache is configured with
a function checking if it is configured *and* if an ssl session
cache is available.
During normal operations, a session cache is always there, however
for "connect-only" transfers this might not be the case. When such
transfers receive new sessions/tickets, they need to silently
discard those and not fail.
Reported-by: Marc Aldorasi
Fixes https://github.com/curl/curl/issues/18983
Closes https://github.com/curl/curl/pull/19251
Change the test certificate to carry a altname 'dns:127.0.0.1' which
should *not* match in test_17_05_bad_ip_addr.
wolfSSL: since `wolfSSL_check_domain_name()` does not differentiate
between DNS and IP names, use if only for DNS names. For IP addresses,
get the peer certificate after the handshake and check that using
wolfSSL_X509_check_ip_asc().
Unfortunately, this succeeds where it should not, as wolfSSL internally
used the same check code for both cases. So, skip the test case until
wolfSSL fixes that.
Reported-by: Joshua Rogers
Closes#19252
The parameter `max_pkts` was not checked in the recvmsg() implementation
of vquic_recv_packets() as the packter counter was never increased. This
led to the loop running until an EAGAIN was encountered. Which, in any
real case scenario, does no harm as long as libcurl is ingesting packets
faster than a server is able to send them.
However on a slow device and a fast network this could happen and allow
a denial of serice.
Not a real regression as the vulnerable code has never been released.
libcurl 8.16.0 does not have this bug.
Closes#19186
Adopt ngtcp2_conn_get_stream_user_data which has been available since
ngtcp2 v1.17.0. This improves the time complexity of searching
h3_stream_ctx from O(n) to O(1) where n is the number of stream.
Closes#19132
The choice to continue processing incoming data although the
writeout of the headers/data failed is not obvious. Add a comment
explaining why this is done.
Closes#19093
The transfer loop used to check the socket and if no poll events
were seen, triggered a "DATA_IDLE" event into the filters to let
them schedule times/do things anyway.
Since we no longer check the socket, the filters have been called
already and the DATA_IDLE event is unnecessary work. Remove it.
Closes#19060
quiche receive may report a TLS failure after a verified handshake. That
needs to lead to a transfer receive error.
Reported-by: Joshua Rogers
Closes#18934
The stream blocking might not be the one of the current easy handle.
Look up the stream to be marked as blocking via its stream_id in the
internal hash. Theoretically, this does not have to be one of the h3
streams, so not finding it is not an error.
Fixes#18905
Reported-by: Joshua Rogers
Closes#18906
Similar to our ngtcp2 backend, set our idle timeout for the connection
to 0, meaning we have no such timeout from our side. The effective idle
timeout is then the one announced by the peer.
Closes#18907
In both send/recv functions of the ngtcp2 filter, when TLS verification
has failed, jump out by skipping ingress/egress handling.
Reported-by: Joshua Rogers
Closes#18881
After this patch, the codebase no longer overrides system printf
functions. Instead it explicitly calls either the curl printf functions
`curl_m*printf()` or the system ones using their original names.
Also:
- drop unused `curl_printf.h` includes.
- checksrc: ban system printf functions, allow where necessary.
Follow-up to db98daab05#18844
Follow-up to 4deea9396b#18814Closes#18866
When the close of the quiche filter was never called, the destroy function
did not release all allicated resources.
When closing a quiche filter, set the connected flag to FALSE.
Reported-by: Joshua Rogers
Closes#18880
Also:
- tests/server: replace local `sstrerror()` with `curlx_strerror()`.
- tests/server: show the error code next to the string, where missing.
- curlx: use `curl_msnprintf()` when building for src and tests.
(units was already using it.)
- lib: drop unused includes found along the way.
- curlx_strerror(): avoid compiler warning (and another similar one):
```
In file included from servers.c:14:
../../lib/../../lib/curlx/strerr.c: In function ‘curlx_strerror’:
../../lib/../../lib/curlx/strerr.c:328:32: error: ‘snprintf’ output may be truncated before the last format character [-Werror=format-truncation=]
328 | SNPRINTF(buf, buflen, "%s", msg);
| ^
../../lib/../../lib/curlx/strerr.c:47:18: note: ‘snprintf’ output 1 or more bytes (assuming 2) into a destination of size 1
47 | #define SNPRINTF snprintf
| ^
../../lib/../../lib/curlx/strerr.c:328:7: note: in expansion of macro ‘SNPRINTF’
328 | SNPRINTF(buf, buflen, "%s", msg);
| ^~~~~~~~
```
Follow-up to 45438c8d6f#18823Closes#18840
By making them defaults, then fixing and/or reshuffling remaining
exceptions as necessary.
- checksrc: ban by default: `snprintf`, `vsnprintf`, `sscanf`, `strtol`.
- examples: replace `strtol` with `atoi` to avoid a checksrc exception.
- tests/libtest: replace `strtol` with `atol`.
- tests/server: replace most `strtol` with `atol`.
- tests/server: replace most `strtoul` with `atol`/`atoi`.
- tests/server: drop no longer used `util_ultous`.
- fix typo in checksrc rules: `vsnprint` -> `vsnprintf`.
- update local exceptions.
Also:
- examples: ban curl printf functions. They're discouraged in user code.
- examples: replace curl printf with system printf.
Add `snprintf` workaround for <VS2015.
- examples/synctime: fix `-Wfloat-equal`.
- examples/synctime: exclude for non-Windows and non-UWP Windows.
- examples/synctime: build by default.
Closes#18823
- better tracing of what system call is used and how often
- ngtcp2: combine vquic_send into larger chunks
- ngtcp2: define own PMTU values and enable MTU probing
- ngtcp2: trace interesting remote transport parameters
Closes#18812
- configure/cmake support for enabling the option
- supported in OpenSSL and GnuTLS backends
- when configured, Apple SecTrust is the default trust store
for peer verification. When one of the CURLOPT_* for adding
certificates is used, that default does not apply.
- add documentation of build options and SSL use
Closes#18703
HTTP/3 defines "reserved stream types" that are intended to be ignored
by a receiver. This is part of the "greasing" effort that flexes parts
of the protocol that are needed for future extensions.
curl's OpenSSL-QUIC implementation treated all unexpected streams as
an error. Which seems the right thing to do *but* for these reserved types.
However OpenSSL does not expose this type and thus, curl needs to silently
discard all unexpected streams opened by the server to allow interop
with servers that flex the GREASE parts.
Fixes#18780
Reported-by: Pocs Norbert
Closes#18791
Replace them by `curlx_open()` and `curlx_stat()`.
To make it obvious in the source code what is being executed.
Also:
- tests/server: stop overriding `open()` for test servers.
This is critical for the call made from the signal handler.
For other calls, it's an option to use `curlx_open()`, but
doesn't look important enough to do it, following the path
taken with `fopen()`.
Follow-up to 10bac43b87#18774
Follow-up to 20142f5d06#18634
Follow-up to bf7375ecc5#18503Closes#18776
