The cookie flushing (saving to a cookie jar) should only be done if a
transfer has been started. This is now done by checking the
cookies->running field, which is not reset in curl_easy_reset() so the
saving works correctly even after a call to that.
Follow-up to fd6eb8d6e7
Verified by test 1920
Reported-by: Alexander Batischev
Fixes#20090Closes#20094
test 798 - incoming cookie header in a folded line
test 1665 - verify HTTP headers without final CRLF. Make sure all complete
headers are delivered even if the reponse is partial
Only the TLS 1.2 structure for now since it's simpler, and only has a
single label type. This has the bonus of also testing libressl that only
supports logging keys in TLS 1.2
Closes#19816
To make the test files XML-compliant, and the expected results
possibly easier to manage by keeping them in `.md`, `.html`, `.1`
and `.txt` files.
Non-XML-compliant files are down to 31 (1.6%) after this patch.
Closes#19882
To make the test files XML-compliant, and the expected results
possibly easier to manage by keeping them in `.c` files.
Non-XML-compliant files are down to 36 after this patch.
Also:
- make all macro expansions apply to `%includetext` contents.
Closes#19799
Before this patch curl used the C preprocessor to override standard
memory allocation symbols: malloc, calloc, strdup, realloc, free.
The goal of these is to replace them with curl's debug wrappers in
`CURLDEBUG` builds, another was to replace them with the wrappers
calling user-defined allocators in libcurl. This solution needed a bunch
of workarounds to avoid breaking external headers: it relied on include
order to do the overriding last. For "unity" builds it needed to reset
overrides before external includes. Also in test apps, which are always
built as single source files. It also needed the `(symbol)` trick
to avoid overrides in some places. This would still not fix cases where
the standard symbols were macros. It was also fragile and difficult
to figure out which was the actual function behind an alloc or free call
in a specific piece of code. This in turn caused bugs where the wrong
allocator was accidentally called.
To avoid these problems, this patch replaces this solution with
`curlx_`-prefixed allocator macros, and mapping them _once_ to either
the libcurl wrappers, the debug wrappers or the standard ones, matching
the rest of the code in libtests.
This concludes the long journey to avoid redefining standard functions
in the curl codebase.
Note: I did not update `packages/OS400/*.c` sources. They did not
`#include` `curl_setup.h`, `curl_memory.h` or `memdebug.h`, meaning
the overrides were never applied to them. This may or may not have been
correct. For now I suppressed the direct use of standard allocators
via a local `.checksrc`. Probably they (except for `curlcl.c`) should be
updated to include `curl_setup.h` and use the `curlx_` macros.
This patch changes mappings in two places:
- `lib/curl_threads.c` in libtests: Before this patch it mapped to
libcurl allocators. After, it maps to standard allocators, like
the rest of libtests code.
- `units`: before this patch it mapped to standard allocators. After, it
maps to libcurl allocators.
Also:
- drop all position-dependent `curl_memory.h` and `memdebug.h` includes,
and delete the now unnecessary headers.
- rename `Curl_tcsdup` macro to `curlx_tcsdup` and define like the other
allocators.
- map `curlx_strdup()` to `_strdup()` on Windows (was: `strdup()`).
To fix warnings silenced via `_CRT_NONSTDC_NO_DEPRECATE`.
- multibyte: map `curlx_convert_*()` to `_strdup()` on Windows
(was: `strdup()`).
- src: do not reuse the `strdup` name for the local replacement.
- lib509: call `_strdup()` on Windows (was: `strdup()`).
- test1132: delete test obsoleted by this patch.
- CHECKSRC.md: update text for `SNPRINTF`.
- checksrc: ban standard allocator symbols.
Follow-up to b12da22db1#18866
Follow-up to db98daab05#18844
Follow-up to 4deea9396b#18814
Follow-up to 9678ff5b1b#18776
Follow-up to 10bac43b87#18774
Follow-up to 20142f5d06#18634
Follow-up to bf7375ecc5#18503
Follow-up to 9863599d69#18502
Follow-up to 3bb5e58c10#17827Closes#19626
Description of how this works in `docs/internal/RATELIMITS.ms`.
Notable implementation changes:
- KEEP_SEND_PAUSE/KEEP_SEND_HOLD and KEEP_RECV_PAUSE/KEEP_RECV_HOLD
no longer exist. Pausing is down via blocked the new rlimits.
- KEEP_SEND_TIMED no longer exists. Pausing "100-continue" transfers
is done in the new `Curl_http_perform_pollset()` method.
- HTTP/2 rate limiting implemented via window updates. When
transfer initiaiting connection has a ratelimit, adjust the
initial window size
- HTTP/3 ngtcp2 rate limitin implemnented via ack updates
- HTTP/3 quiche does not seem to support this via its API
- the default progress-meter has been improved for accuracy
in "current speed" results.
pytest speed tests have been improved.
Closes#19384
To untangle the different curl/server requirements of these tests.
Also to make this test run for non-H2 builds. Searching the cause of
the flakiness documented in #19481.
Also:
- fix the callback function prototype. Detected by ASAN with this patch,
though the issue was pre-existing.
```
lib/cw-out.c:211:14: runtime error: call to function emptyWriteFunc
through pointer to incorrect function type
'unsigned long (*)(char *, unsigned long, unsigned long, void *)'
tests/libtest/lib2405.c:72: note: emptyWriteFunc defined here
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior lib/cw-out.c:211:14
```
Ref: https://github.com/curl/curl/actions/runs/19296686908/job/55180334364?pr=19487#step:44:3768
Follow-up to 96a5ce5a82#19481Closes#19487
When asking for the last N bytes of a file, and that size was larger
than the file size, it would miss the first byte due to a logic error.
The fixed range parser is now made a common function in the file now
renamed to vssh.c (from curl_path.c) - used by both backends.
Unit test 2605 verifies the parser.
Reported-by: Stanislav Fort (Aisle Research)
Closes#19460
Custom IMAP commands using -X (e.g. 'FETCH 123 BODY[1]') were only
returning the first line of responses containing literals, instead of
the full multi-line body data.
The issue was that custom commands route through imap_perform_list()
and imap_state_listsearch_resp(), which didn't detect or handle IMAP
literal syntax {size}.
This commit adds literal detection to imap_state_listsearch_resp():
- Detects literal syntax {size} in untagged responses
- Writes the response header line containing the literal marker
- Handles any literal body data already in the pingpong buffer
- Sets up transfer layer to read remaining literal data from socket
- Configures maxdownload and transfer size to include header + body
- Initializes pp->overflow to 0 when no buffered data present
- Modifies imap_done() to transition to FETCH_FINAL for custom
commands that set up downloads
Test 841 and 3206 verify.
Fixes#18847
Reported-by: BohwaZ
Bug: https://github.com/curl/curl/issues/18847Closes#19246
When a reused connection did transfer 0 bytes, it assumed the transfer
had failed and needed a retry. Add a check for data->red.done, so we can
successfully accept the transfer of a 0-length file via SFTP.
Add test case 1583 to verfiy.
Fix SFTP disconnect debug trace when there was nothing to
disconnect (like when reusing a connection).
Fixes#19165
Reported-by: Alexander Blach
Closes#19189
The config file parser now has a maximum level of inclusions allowed (5)
to detect and prevent recursive inclusions of itself leading to badness.
Bonus: clean up return code handling from the config parser.
Test 774 verifies
Closes#19168
Unless it is identical to the previous one.
Follow-up to dbcaa00657
Adjusted test 580, added test 772 and 773
Fixes#19130
Reported-by: Jakub Stasiak
Closes#19134
- Keep data from a failed download instead of discarding it on retry in
some limited cases when we know it's ok (currently only HTTP 200/206).
Prior to this change on failed transfer the tool truncated any outfile
data written before retrying the transfer. This change adds an exception
for HTTP downloads when the user requested auto-resume, because in that
case we can keep the outfile data and resume from the new position.
Reported-by: tkzv@users.noreply.github.com
Fixes https://github.com/curl/curl/issues/18035
Closes https://github.com/curl/curl/pull/18665
Return error if there is something after the number other than
whitespace and newline.
Allow comma separated numbers and repeated headers as long as the new value is
the same as was set before.
Add test 767 to 771 to verify.
Reported-by: Ignat Loskutov
Fixes#18921Closes#18925
This test does active FTP with a socketopt callback that returns error
for the CURLSOCKTYPE_ACCEPT "purpose" to make sure we test and exercise
this error path - without leaks.
Closes#18879
When there is more than one user-supplied 'Connection: ' header, add
values that curl needs internally to the first one and emit all
subsequent ones thereafter.
Fixes#18662
Reported-by: Evgeny Grin (Karlson2k)
Closes#18686
By appending `:all:[separator]` to the header name. The `[separator]` string
is output between each header value if there are more than one to output.
Test 764 and 765 verify
Idea-by: kapsiR on github
Ref: #18449Closes#18491
Because parts of the cookie loading happens on transfer start the
in-memory cookie jar risks being incomplete and then a save might
wrongly truncate the target file.
Added test 1902 to verify.
Reported-by: divinity76 on github
Fixes#18621Closes#18622
Remove use of pthread_cancel in asnyc threaded resolving. While there
are system where this works, others might leak to resource leakage
(memory, files, etc.). The popular nsswitch is one example where resolve
code can be dragged in that is not prepared.
The overall promise and mechanism of pthread_cancel() is just too
brittle and the historcal design of getaddrinfo() continues to haunt us.
Fixes#18532
Reported-by: Javier Blazquez
Closes#18540
This allows --remote-time to set dates before 1970.
Due to a minor omission in the API, it will still avoid setting the time
if it is indeed exactly epoch 0 (jan 1 1970).
Verified by test 762
Fixes#18424
Reported-by: Terence Eden
Closes#18443
- using {} with single entries makes little sense
- when using {} sets with two entry lists, there can only be 64 to reach
maximum number of URLs
Verify the max check in test 761
To avoid NULL derferences with many outputs and --next.
Follow-up to 034612cd51
Add test 760 to verify
Reported-by: BobodevMm on github
Fixes#18375Closes#18376
Changed strategy to start up and terminate resolver thread.
When starting up:
Start the thread with mutex acquired, wait for signal from thread that
it started and has incremented the ref counter. Thread set
pthread_cancel() to disabled before that and only enables cancelling
during resolving itself. This assure that the ref counter is correct and
the unlinking of the resolve context always happens.
When shutting down resolving:
If ref counting shows thread has finished, join it, free everything. If
thread has not finished, try pthread_cancel() (non Windows), but keep
the thread handle around.
When destroying resolving:
Shutdown first, then, if the thread is still there and 'quick_exit' is
not set, join it and free everything. This might occur a delay if
getaddrinfo() hangs and cannot be interrupted by pthread_cancel().
Destroying resolving happens when another resolve is started on an
easy handle or when the easy handle is closed.
Add test795 to check that connect timeout triggers correctly
when resolving is delayed. Add debug env var `CURL_DNS_DELAY_MS`
to simulate delays in resolving.
Fix test1557 to set `quick_exit` and use `xxx.invalid` as domain
instead of `nothing` that was leading to hangers in CI.
Closes#18263
When an application install its own OpenSSL verify callback and that
callback invokes `SSL_set_retry_verify()`, the transfer is automatically
paused and does not progress the connect attempt any further until
unpaused via `curl_easy_pause().
Added test758 to verify.
Ref: #18284
Original PR by @Natris
Bug: https://curl.se/mail/lib-2025-08/0012.htmlCloses#18288
Easily triggered when doing an FTP ranged download with the end range
larger than the file size.
A regression since 30de937bda (shipped in curl 8.9.0)
Add test 2307 to verify.
Reported-by: kupavcevdenis on github
Fixes#18283Closes#18295
curl_easy_reset() did not reset the `rewind_read` flag. This caused any
handles that previously had a CURLE_SEND_FAIL_REWIND error to get stuck
with that error, failing any subsequent requests, even if they didn't
have any body at all.
Verified in test 3034
Fixes#18206Closes#18207
Hold them for half the normal lifetime. Helps when told to transfer N
URLs in quick succession that all use the same non-resolving hostname.
Done by storing a DNS entry with a NULL pointer for 'addr'.
Previously an attempt was made in #12406 by Björn Stenberg that was
ultimately never merged.
Closes#18157
Output the current UTC time using strftime format. %f is an extra curl
specific flag to output the microsecond fraction of the current second.
Verified by test 1981
Closes#18119
Makes curl follow redirects an act on the response code and change a
custom method accordingly, contrary to --location.
Potential future command line to send QUERY and following a redirect
according to the status code:
curl -d "request-body" -X QUERY --follow https://example.com
add test 794,796,797
Assisted-by: Daniel Böhmer <post@daniel-boehmer.de>
Closes#16543
New multi option CURLMOPT_NETWORK_CHANGED with a long bitmask value:
- CURLM_NWCOPT_CLEAR_CONNS: do not reuse existing connections, close all
idle connections.
- CURLM_NWCOPT_CLEAR_DNS: clear the multi's DNS cache.
All other bits reserved for future extensions.
Fixes#17225
Reported-by: ウさん
Closes#17613
Add a new commandline option --out-null that discards all
response bytes into the void. Replaces non-portable use of
'-o /dev/null' with more efficiency.
Feature added in 8.16.0
Closes#17800
When transfers read client input without an upload file, the check if
upload file is '.' strcompared a NULL.
Add test 1548 to reproduce and verify fix.
Reported-by: d1r3ct0r
Fixes#17978Closes#17987
This test makes sure that a number of internal and public structs are
within their maximum allowed size limits.
The public structs can only grow in controlled ways, while the internal
ones may be allowed to grow if deemed right.
The idea here is to control, to know and make sure all important struct
growth is intentional.
Closes#17823
