Commit graph

15012 commits

Author SHA1 Message Date
Viktor Szakats
2c27a67daa
tests: always make bundles, adapt build and tests
Make test bundles the default. Drop non-bundle build mode.
Also do all the optimizations and tidy-ups this allows, simpler builds,
less bundle exceptions, streamlined build mechanics.

Also rework the init/deinit macro magic for unit tests. The new method
allows using unique init/deinit function names, and calling them with
arguments. This is in turn makes it possible to reduce the use of global
variables.

Note this drop existing build options `-DCURL_TEST_BUNDLES=` from cmake
and `--enable-test-bundles` / `--disable-test-bundles` from autotools.

Also:
- rename test entry functions to have unique names: `test_<testname>`
  This removes the last exception that was handled in the generator.
- fix `make dist` to not miss test sources with test bundles enabled.
- sync and merge `tests/mk-bundle.pl` into `scripts/mk-unity.pl`.
- mk-unity.pl: add `--embed` option and use it when `CURL_CLANG_TIDY=ON`
  to ensure that `clang-tidy` does not miss external test C sources.
  (because `clang-tidy` ignores code that's #included.)
- tests/unit: drop no-op setup/stop functions.
- tests: reduce symbol scopes, global macros, other fixes and tidy-ups.
- tool1621: fix to run, also fix it to pass.
- sockfilt: fix Windows compiler warning in certain unity include order,
  by explicitly including `warnless.h`.

Follow-up to 6897aeb105 #17468

Closes #17590
2025-06-14 21:08:23 +02:00
Stefan Eissing
1cdac95e2e
openssl: fix handling of buffered data
`SSL_pending()` only checks if the *current* TLS packet has more data.
There might be more data in SSL's buffers.

`SSL_has_pending()` only checks if there is data in buffers, but does
*not* check if there is a complete TLS packet that can be decoded.

If we only check the first, we will poll on socket events without having
processed all data and may stall. If we only check the second, we would
busy loop without SSL_read() ever giving something.

Add the flag `connssl->input_pending` that is set on incoming data in
the BIO receive. Clear the flag when encountering a CURLE_AGAIN on
the filters receive (via SSL_read()) or see an EOF.

Ref: #17596
Closes #17601
2025-06-13 00:32:06 +02:00
Daniel Stenberg
cbc062a7b8
http: split http_header into sub functions
Closes #17603
2025-06-12 13:13:46 +02:00
Daniel Stenberg
c257753ee9
asyn-thrdd: free the previous name before strdup'ing the new
Fixes #17602
Reported-by: hiimmat on github
Closes #17604
2025-06-12 12:53:57 +02:00
Daniel McCarney
1e2e808def
tls: remove Curl_ssl false_start
The secure transport vTLS backend was the only Curl_ssl struct instance
that populated the false_start field. Since its removed, we can now
remove that field entirely. This was a protocol feature specific to TLS
1.2 that has been replaced by the more widely adopted TLS 1.3 early data
mechanisms.

--false-start is now deprecated

Closes #17595
2025-06-12 08:29:15 +02:00
Daniel Stenberg
b950c8c03f
http_ntlm: remove unreachable code
Since the code now checks for NULL and returns before this point, 'ntlm'
cannot be NULL at this point in the code. Pointed out by Coverity.

Follow-up to 5586520745

Closes #17587
2025-06-11 23:06:06 +02:00
Edwin Török
aea336aa23
openssl: enable readahead
Speeds up TLS operations up to ~%13.

Closes #17548

Signed-off-by: Edwin Török <edwin.torok@cloud.com>
2025-06-11 13:26:53 +02:00
Stefan Eissing
20c90ba298
lib: unify recv/send function signatures
cfilter/conn: change send/recv function signatures. Unify the
calling/return conventions in our send/receive handling.

Curl_conn_recv(), adjust pnread type

Parameter `pnread` was a `ssize_t *`, but `size_t *` is better since the
function returns any error in its `CURLcode` return value.

Closes #17546
2025-06-11 11:21:10 +02:00
Stefan Eissing
14b42c89db
openssl: error on SSL_ERROR_SYSCALL
Convert the debug-only handling of SSL_ERROR_SYSCALL so that it is
enabled in all builds with openssl. This should not make a difference in
supported OpenSSL versions, but if whatever version or fork we link
against *does* return SSL_ERROR_SYSCALL, handle this as a fatal error.

Fixes #17471
Reported-by: Michael Kaufmann
Closes #17531
2025-06-11 10:44:55 +02:00
Daniel Stenberg
5586520745
http_ntlm: protect against null deref
Closes #17585
2025-06-11 10:15:32 +02:00
Stefan Eissing
e39b8c4819
url: fix connection lifetime checks
The checks for a connection being "too long idle" or "too old" where
rounding down the elapsed time to seconds before comparing to the
configured max values. This caused connections to be reused for up to
999ms longer than intended.

Change the compares to scale the configured seconds up to ms, so
connection will properly be "too old" 1 ms after the coonfigured values.

Fixes sporadic failures of test1542 on platforms where "sleep(2)"
returnes before 2 full seconds on the internal clock where passed.

Reported-by: Christian Weisgerber
URL: https://curl.se/mail/lib-2025-06/0004.html
Closes #17571
2025-06-11 08:07:57 +02:00
Daniel Stenberg
f9d8ed63ed
urlapi: simplify and split into sub functions
Closes #17565
2025-06-11 07:58:52 +02:00
Stefan Eissing
3c26e6a896
libssh: de-complex myssh_statemach_act()
Make static functions for the largest state handling code
with a common signature and naming scheme.

Closes #17573
2025-06-11 07:58:09 +02:00
Daniel Stenberg
08a3e8e19a
TLS: remove support for Secure Transport and BearSSL
These libraries do not support TLS 1.3 and have been marked for removal
for over a year. We want to help users select a TLS dependency that is
future-proof and reliable, and not supporting TLS 1.3 in 2025 does not
infer confidence. Users who build libcurl are likely to be served better
and get something more future-proof with a TLS library that supports
1.3.

Closes #16677
2025-06-11 07:54:19 +02:00
Viktor Szakats
3c9d16f4b8
cmake: drop never propagated C macros
Delete macros from `curl_config.h.cmake` that were never set by
the CMake script: `_LARGE_FILES`, `_THREAD_SAFE`, `const`, `size_t`.

Also:
- lib/config-riscos.h: drop `#undef _LARGE_FILES`. This is an
  IBM-specific macro, no need to unset it on other platforms.

Cherry-picked from #17576

Closes #17580
2025-06-11 06:32:22 +02:00
Viktor Szakats
9ed34cc45b
schannel: drop Windows 2000 compatibility logic
curl requires Windows XP as a minimum.

Co-authored-by: Jay Satiro

Closes #17447
2025-06-11 05:39:14 +02:00
Viktor Szakats
0d71b18153
windows: reduce/stop loading DLLs at runtime
- replace dynamic `InitSecurityInterface()` call with early binding and
  link `secur32` system DLL.
  The library and function are available in all supported curl Windows
  targets, meaning WinXP or newer.  Add small hack for mingw32ce to
  make it build.

- detect and use `if_nametoindex()` on Windows when available. Link
  `iphlpapi` system DLL. Requires targeting Vista or newer.
  Replacing the dynamic call and the pre-load optimization for lib3026.

Suggested-by: Jay Satiro

Closes #17413
2025-06-11 05:39:09 +02:00
Stefan Eissing
ab650379a8
vauth: move auth structs to conn meta data
Remove structs for negotiate, krb5, ntlm and gsasl from connectdata and
store them as connection meta data with auto cleanup.

De-complexify sasl mech selection by moving code into static functions.

Closes #17557
2025-06-10 15:57:44 +02:00
Daniel Stenberg
b530c11bfd
curl.h: remove the "RESERVED" error codes
Return codes for tests should be kept private.

Follow-up to 9465327084

Closes #17563
2025-06-09 23:57:23 +02:00
Daniel Stenberg
1886260a95
lib: make curlx_inet_ntop()
move function to curlx/, change all callers

Closes #17560
2025-06-09 13:16:01 +02:00
Carlos Henrique Lima Melara
0ede81dcc6
curl_path: make SFTP handle a path like /~ properly.
... without a trailing slash.

Fixes #17534
Closes #17542
2025-06-09 11:23:16 +02:00
Stefan Eissing
c314759c4c
pingpong: on disconnect, check for unflushed pingpong state
When a pingpong based protocol tries to perform a connection disconnect,
it sends a sort of "logout" command to the server, unless the connection
is deemed dead.

But the disconnect might happen before pingpong data has been completely
sent, in which case sending the "logout" will not work. Check the
pingpong state and do not "logout" when data is pending.

This was detected as a condition in fuzzing that triggered a debug
assert in the pingpong sending.

Closes #17555
2025-06-09 11:22:19 +02:00
Daniel Stenberg
614313f12f
system.h: remove some macros
Since curl_off_t is always 64 bit these days, we can simplify and avoid
using some macros.

Closes #17498
2025-06-05 10:56:31 +02:00
Stefan Eissing
34d694f89c
SCP/SFTP: avoid busy loop after EAGAIN
The ssh libraries do not reveal if they still have data buffered from
the peer. Only when their buffers are read empty can curl be sure that
it is safe to rely on socket polling.

This change adds detection of EGAIN on receive in the transfer loop and
allows SFTP/SCP transfers to avoid a busy loop in such a case (which
should happen often when CPU exceeds network bandwidth).

Closes #17533
2025-06-05 09:11:29 +02:00
Jeremy Drake
7a30481760 dllmain: exclude from Cygwin builds
On Cygwin, it is unsafe to call POSIX functions from DllMain, which
OPENSSL_thread_stop does.  Additionally, it should be unnecessary as
OpenSSL uses pthread_key_create to register a thread destructor to do
thread cleanup in a POSIX way.

Reported-by: Yuyi Wang

Ref: https://cygwin.com/pipermail/cygwin/2025-June/258235.html

Fixes #17262
Closes https://github.com/curl/curl/pull/17528
2025-06-03 14:30:31 -04:00
Stefan Eissing
f1e1c8b98a
tls BIOs: handle BIO_CTRL_EOF correctly
Needs to return 1 if EOF from underlying filter has been seen.

Fixes #17471
Reported-by: Michael Kaufmann
Closes #17526
2025-06-03 16:52:01 +02:00
Ruocco, Calvin
d3594be653
ws: tests and fixes
This started out as regression tests for the `curl_ws_recv()` and
`curl_ws_send()` implementation and ended up with a bugfix, additional
protocol validation and minor logging improvements.

- Fix reset of fragmented message decoder state when a PING/PONG is
  received in between message fragments.

- Fix undefined behavior (applying zero offset to null pointer) in
  curl_ws_send() when the given buffer is NULL.

- Detect invalid overlong PING/PONG/CLOSE frames.
- Detect invalid fragmented PING/PONG/CLOSE frames.
- Detect invalid sequences of fragmented frames.

  - a) A continuation frame (0x80...) is received without any ongoing
    fragmented message.
  - b) A new fragmented message is started (0x81/0x01/0x82/0x02...)
    before the ongoing fragmented message has terminated.

- Made logs for invalid opcodes easier to understand.
- Moved noisy logs to the `CURL_TRC_WS` log level.
- Unified the prefixes for WebSocket log messages: `[WS] ...`

- Add env var `CURL_WS_FORCE_ZERO_MASK` in debug builds.

  - If set, it forces the bit mask applied to outgoing payloads to
    0x00000000, which effectively means the payload is not masked at
    all. This drastically simplifies defining the expected `<protocol>`
    data in test cases.

- 2700: Frame types
- 2701: Invalid opcode 0x3
- 2702: Invalid opcode 0xB
- 2703: Invalid reserved bit RSV1 _(replaces 2310)_
- 2704: Invalid reserved bit RSV2
- 2705: Invalid reserved bit RSV3
- 2706: Invalid masked server message
- 2707: Peculiar frame sizes _(part. replaces 2311)_
- 2708: Automatic PONG
- 2709: No automatic PONG _(replaces 2312)_
- 2710: Unsolicited PONG
- 2711: Empty PING/PONG/CLOSE
- 2712: Max sized PING/PONG/CLOSE
- 2713: Invalid oversized PING _(replaces 2307)_
- 2714: Invalid oversized PONG
- 2715: Invalid oversized CLOSE
- 2716: Invalid fragmented PING
- 2717: Invalid fragmented PONG
- 2718: Invalid fragmented CLOSE
- 2719: Fragmented messages _(part. replaces 2311)_
- 2720: Fragmented messages with empty fragments
- 2721: Fragmented messages with interleaved pong
- 2722: Invalid fragmented message without initial frame
- 2723: Invalid fragmented message without final frame

- 2305: curl_ws_recv() loop reading three larger frames
  - This test involuntarily sent an invalid sequence of opcodes (0x01...,0x01...,0x81...) , but neither libcurl nor the test caught this! The correct sequence was tested in 2311 (0x01...,0x00...,0x80...). See below for 2311.
  - Validation of the opcode sequence was added to libcurl and is now tested in 2723.
  - Superseded by 2719 (fragmented message) and 2707 (large frames).
- 2307: overlong PING payload
  - The tested PING payload length check was actually missing, but the test didn't catch this since it involuntarily sent an invalid opcode (0x19... instead of 0x89...) so that the expected error occurred, but for the wrong reason.
  - Superseded by 2713.
- 2310: unknown reserved bit set in frame header
  - Superseded by 2703 and extended by 2704 and 2705.
- 2311: curl_ws_recv() read fragmented message
  - Superseded by 2719 (fragmented message) and 2707 (large frames).
- 2312: WebSockets no auto ping
  - Superseded by 2709.

- No tests for `CURLOPT_WRITEFUNCTION`.
- No tests for sending of invalid frames/fragments.

Closes #17136
2025-06-02 11:15:38 +02:00
Daniel Stenberg
f738093896
asyn-thrdd: fix cleanup when RR fails due to OOM
Fixes #17507
Closes #17514
2025-06-02 09:08:04 +02:00
Yedaya Katsman
8645e4f6ee
license: update some copyright links to curl.se
Instead of curl.haxx.se

Also widen the .gitignore for libtest, since it missed libtest751,
so ignore three digit tests that start with 5-9 instead of just 5-6.

Closes #17502
2025-06-01 13:44:15 +02:00
z2_
d1145df24d
ws: handle blocked sends better
Closes #17496
2025-05-31 17:04:18 +02:00
Stefan Eissing
4977747f0d
ftp: fix teardown of DATA connection in done
When ftp_done() is called to terminate the transfer, it needs to tear
down any open SECONDARY filter chain. The condition on when to do that
was relying on there to be a valid socket. This is not sufficient as the
socket is only set *after* happy eyeballing has decided on one.

Instead of checking for a valid conn->sock, check if any connection
filter is installed.

Fixes #17482
Reported-by: Rasmus Melchior Jacobsen
Closes #17491
2025-05-31 15:15:05 +02:00
Daniel Stenberg
fe81a80ae7
spelling: call it null-terminate consistently
With a dash, using two Ls. Also for different forms of the word.

Use NULL in all uppercase if it means a zero pointer.

Follow-up to 307b7543ea

Closes #17489
2025-05-30 17:29:45 +02:00
Stefan Eissing
54834b4ad3
wolfssl: fix sending of early data
Early data was reported as being sent, but was not. While everything was
aligned with the Gods and early data was reported as accepted, the
actual sending required another call to wolfSSL.

Fixes #17481
Reported-by: Ethan Everett
Closes #17488
2025-05-30 11:44:20 +02:00
Daniel Stenberg
bdb7d8b004
spelling: 'a' vs 'an'
Closes #17487
2025-05-30 11:38:35 +02:00
Viktor Szakats
94d2ae6205
libssh: adjust indentation
Follow-up to 54d04e2536 #17478
2025-05-30 09:58:19 +02:00
Daniel Stenberg
b13a7a3ec4
misc: we write *an* IPv6 address
Add it as a "bad word"

Closes #17484
2025-05-30 07:53:31 +02:00
John Bampton
217fd5b424
misc: fix spelling
Closes #17479
2025-05-29 10:21:05 +02:00
John Bampton
54d04e2536
misc: fix spelling
Closes #17478
2025-05-28 22:47:12 +02:00
Stefan Eissing
71bb004c29
http: fail early when rewind of input failed when following redirects
When inspecting a possible follow HTTP request, the result of a rewind
of the upload data was ignored as it was not clear at that point in time
if the request would become a GET.

This initiated the followup, rewound again, which failed again and
terminated the follow up.

This was confusing to users as it was not clear of the follow up was
done or not.

Fix: fail the early rewind when the request is not converted to GET.

Fixes #17472
Closes #17474
Reported-by: Jeroen Ooms
2025-05-28 14:53:02 +02:00
Stefan Eissing
d16ccbd55d
multi: fix add_handle resizing
Due to someone being stupid, the resizing of the multi's transfer
table was actually shrinking it. Oh my.

Add test751 to reproduce, add code assertion.

Fixes #17473
Reported-by: Jeroen Ooms
Closes #17475
2025-05-28 14:48:44 +02:00
Kai Pastor
ac7e2c3dc6
asyn-ares: remove obsolete reference to local_ip6
7bf576064c moved local_ip6 from the parameter list to the actual
implementation of Curl_async_ares_set_dns_local_ip6. The no-op code for
!( defined(HAVE_CARES_SET_LOCAL) && defined(USE_IPV6) ) still had an
reference which is removed by this change.

Closes #17450
2025-05-26 10:12:28 +02:00
Stefan Eissing
745c3519a5
ftp: fix race in upload handling
When TYPE was skipped for an immediate STORE command and the server
replied fast and the EPRT data connection was not ready, the transfer
was not initated, leading to no upload.

Fixes #17394
Closes #17428
Reported-by: JoelAtWisetech on github
2025-05-25 11:56:16 +02:00
Jacob Mealey
8ab73bb1ff
urlapi: move complex switch case to separate func
Move complex switch statement cases in curl_url_set
to their own functions and call them from switch.

Closes #17435
2025-05-24 13:39:34 +02:00
Daniel Stenberg
129719c29c
lib: change progress bitwise ops to bitfields
Since they are mostly independent, using them as bitfelds makes the code
easier.

- remove the unused struct field 'width'.

- convert 'speeder_c' to an unsigned char from int

Closes #17431
2025-05-23 23:13:03 +02:00
Stefan Eissing
37b25f7bb0
sasl: give help when unable to select AUTH
When SASL is unable to select an AUTH mechanism, give user help
in info message why no AUTH could be selected.

Fixes #17420
Closes #17427
Reported-by: Aditya Garg
2025-05-23 22:56:05 +02:00
Daniel Stenberg
b445031e94
urldata: remove the unused struct field 'hide_progress'
It was only set, never read.

Closes #17430
2025-05-23 17:15:32 +02:00
Daniel Stenberg
1fef013b81
setopt: create set_ssl_options()
Used for both CURLOPT_SSL_OPTIONS and CURLOPT_PROXY_SSL_OPTIONS

Also: make the DoH code use the full original argument value instead of
each individual flag. Makes it easier to keep all of these in synk.

Closes #17429
2025-05-23 17:15:00 +02:00
Daniel Stenberg
9eaaa0748d
http_aws_sigv4: simplify, avoid many gotos
Closes #17422
2025-05-23 09:09:26 +02:00
Daniel Stenberg
6ad9230c2d
http_aws_sigv4: return error on too many query components
Instead of just silently continuing.

Closes #17421
2025-05-23 09:08:36 +02:00
Daniel Stenberg
4a2bd8e458
mprintf: move outputs into separate functions
To make everything easier to read and follow.

Closes #17409
2025-05-21 22:56:02 +02:00