Commit graph

15201 commits

Author SHA1 Message Date
Oxan van Leeuwen
a93113b5b4
libcurl: reset rewind flag in curl_easy_reset()
curl_easy_reset() did not reset the `rewind_read` flag. This caused any
handles that previously had a CURLE_SEND_FAIL_REWIND error to get stuck
with that error, failing any subsequent requests, even if they didn't
have any body at all.

Verified in test 3034

Fixes #18206
Closes #18207
2025-08-06 17:58:21 +02:00
Daniel Stenberg
019991c25e
multi: fix bad splay management
The splay tree is a tree where each easy handle can be added *once*. The
expire time for that node is the closest expire time for that easy
handle.

Easy handles can however have more expire times queued up, so when the
node is removed from the splay tree because it is the next in line to
take care of, we must check if there is another expire time in the queue
and then add the node back into the splay.

Failing to do the later part, the calling of add_next_timeout after
Curl_splaygetbest, would leave the state.expiretime on the previous time
stamp, which when could make the next call to Curl_splaygetbest use the
wrong time stamp and get a wrong node out, causing trouble.

Reported-by: letshack9707 on hackerone
Closes #18201
2025-08-06 14:13:36 +02:00
Stefan Eissing
6cebd35b4c
lib: xfer_setup simplify
Make variants for transfers that send/receive or do both with just the
parameters they need. Split out the shutdown setting into a separate
function. Only FTP bothers with that.

Closes #18203
2025-08-06 14:12:46 +02:00
Stefan Eissing
6d5570c4dd
lib: replace conn [write]sockfd members by index
The `connectdata` members `sockfd` and `writesockfd` needed to by either
CURL_SOCKET_BAD or a copy of one of `conn->sock[2]`. When equal to one,
that index was used to send/recv/poll the proper socket or connection
filter chain.

Replace those with `send_idx` and `recv_idx` which are either -1, 0 or 1
to indicate which socket/filter to send/receive on.

Closes #18179
2025-08-06 08:47:12 +02:00
Daniel Stenberg
aec28f3ad3
splay: add another assert for detected usage problem
Closes #18199
2025-08-06 08:42:39 +02:00
Daniel Stenberg
bf5265ad11
dynbuf: assert on curlx_dyn_addf use with just "%s"
Because it is wasteful and should be done better.

Closes #18194
2025-08-06 08:32:07 +02:00
Stefan Eissing
41fe621ae1
vtls: set seen http version on successful ALPN
When a HTTP version has been negotiated via ALPN, set the member
`conn->httpversion_seen` accordingly. This allows pending transfers to
reuse multiplexed http connections before the response to the first
transfer has arrived.

Fixes #18177
Reported-by: IoannisGS on github
Closes #18181
2025-08-05 16:01:39 +02:00
Daniel Stenberg
b059f7deaf
setopt: add helper functions to setopt_long()
- Consistently keep options within ranges
- Reduce the maximum maxredirs value to fit a signed short
- Removed comments as the place to document the options is not here

Closes #18174
2025-08-05 13:47:47 +02:00
David Zhuang
60587049f7
vquic: use curl_getenv
getenv isn't defined on all platforms, which prevents vquic from
building. I specifically ran into this issue building on PlayStation.

Closes #18170
2025-08-05 13:12:00 +02:00
Roberto Hidalgo
4442e06b1f
openssl: output unescaped utf8 x509 issuer/subject DNs
Closes #18171
2025-08-05 13:08:40 +02:00
Daniel Stenberg
3689ef21bd
multi: change prefix for the network change bits to CURLMNWC_
Because "CURLM_" is used for curl multi error codes and it is convenient
to use the prefix to identify the number family.

Closes #18176
2025-08-05 13:05:59 +02:00
Anthony Hu
2340a60070
wolfssl: rename ML-KEM hybrids to match IETF draft
Closes #18123
2025-08-05 08:58:19 +02:00
Daniel Stenberg
1f34125141
urldata: reduce two long struct fields to unsigned short
Closes #18173
2025-08-05 08:48:07 +02:00
Daniel Stenberg
df2b4ccc22
hostip: cache negative name resolves
Hold them for half the normal lifetime. Helps when told to transfer N
URLs in quick succession that all use the same non-resolving hostname.

Done by storing a DNS entry with a NULL pointer for 'addr'.

Previously an attempt was made in #12406 by Björn Stenberg that was
ultimately never merged.

Closes #18157
2025-08-05 08:05:31 +02:00
Viktor Szakats
06c12cc08b
build: disable TCP_NODELAY for emscripten
In WebAssembly, using `TCP_NODELAY` fails with:
```
* Could not set TCP_NODELAY: Protocol not available
```

Add a new feature macro in `curl_setup.h` telling whether `TCP_NODELAY`
is known to be supported at runtime, when defined at compile-time.

Keep `TCP_NODELAY` guards at their current positions to ensure the
necessary headers (e.g. `netinet/tcp.h` and `netinet/in.h`) define it.

Reported-by: Jeroen Ooms
Fixes #17974
Closes #18155
2025-08-05 00:34:24 +02:00
Stefan Eissing
1ad2009ad6
multi: add new information extraction method
Adds `curl_off_t curl_multi_get_offt(CURLM *multi_handle, CURLMinfo_offt
info)` to the multi interface with enums:

* CURLMINFO_XFERS_CURRENT: current number of transfers
* CURLMINFO_XFERS_RUNNING: number of running transfers
* CURLMINFO_XFERS_PENDING: number of pending transfers
* CURLMINFO_XFERS_DONE: number of finished transfers to read
* CURLMINFO_XFERS_ADDED: total number of transfers added, ever

Add documentation for functions and info enums.

Add use in the curl command line tool to replace two static
variables counting the same "from the outside".

refs #17870
Closes #17992
2025-08-04 23:48:57 +02:00
Stefan Eissing
5b80b4c012
lib: replace getsock() logic with pollsets
`getsock()` calls operated on a global limit that could
not be configure beyond 16 sockets. This is no longer adequate
with the new happy eyeballing strategy.

Instead, do the following:
- make `struct easy_pollset` dynamic. Starting with
  a minimal room for two sockets, the very common case,
  allow it to grow on demand.
- replace all protocol handler getsock() calls with pollsets
  and a CURLcode to return failures
- add CURLcode return for all connection filter `adjust_pollset()`
  callbacks, since they too can now fail.
- use appropriately in multi.c and multi_ev.c
- fix unit2600 to trigger pollset growth

Closes #18164
2025-08-04 23:43:13 +02:00
Daniel Stenberg
29a6e15b27
setopt: split out cookielist() and cookiefile()
into their own sub functions

Closes #18162
2025-08-04 16:31:54 +02:00
Daniel Stenberg
854b0e230c
hostip: do DNS cache pruning in milliseconds
Instead of using integer seconds. Also: if the cache contains over
30,000 entries after first pruning, it makes anoter round and removes
all entries that are older than half the age of the oldest entry until
it goes below 30,000.

Closes #18160
2025-08-04 16:20:50 +02:00
Daniel Stenberg
40caca581f
splay: rename KEY_NOTUSED TO SPLAY_SUBNODE
- explains its purpose better
- make it global static const
- added an assert for a condition that should never happen (that we
  also catch run-time)

Closes #18152
2025-08-03 22:06:26 +02:00
Daniel Stenberg
2bad773b30
urlapi: allow more path characters "raw" when asked to URL encode
Setting the path component to contain the letters:

    ! $ & ' ( ) { } [ ] * + , ; = : @

now leaves them un-encoded when CURLU_URLENCODE is used.

Amended test 1560 to verify.

Reported-by: Jeroen Ooms
Fixes #17977
Closes #18024
2025-08-01 23:22:32 +02:00
Viktor Szakats
b131db5410
CURLOPT: bump CURLPROTO_* macros to long
This patch bumps the size of these macros from `int` to `long`, while
keeping their actual values the same. It may cause incompatibilities in
user code, requiring the bump of holder variables and/or adding casts:

- CURLPROTO_HTTP
- CURLPROTO_HTTPS
- CURLPROTO_FTP
- CURLPROTO_FTPS
- CURLPROTO_SCP
- CURLPROTO_SFTP
- CURLPROTO_TELNET
- CURLPROTO_LDAP
- CURLPROTO_LDAPS
- CURLPROTO_DICT
- CURLPROTO_FILE
- CURLPROTO_TFTP
- CURLPROTO_IMAP
- CURLPROTO_IMAPS
- CURLPROTO_POP3
- CURLPROTO_POP3S
- CURLPROTO_SMTP
- CURLPROTO_SMTPS
- CURLPROTO_RTSP
- CURLPROTO_RTMP
- CURLPROTO_RTMPT
- CURLPROTO_RTMPE
- CURLPROTO_RTMPTE
- CURLPROTO_RTMPS
- CURLPROTO_RTMPTS
- CURLPROTO_GOPHER
- CURLPROTO_SMB
- CURLPROTO_SMBS
- CURLPROTO_MQTT
- CURLPROTO_GOPHERS
- CURLPROTO_ALL

Also:
- keep existing cast within the documentation to make sure it applies
  to older curl versions as well.

Closes #18136
2025-08-01 18:49:23 +02:00
Stefan Eissing
ec4c559104
openssl: check SSL_write() length on retries
When an SSL_write() blocks we need to retry it with the
same length as before or stupid OpenSSL freaks out. Remember
it, limit any longer sends and fail shorter ones.

Fixes #18121
Reported-by: adamse on github
Closes #18132
2025-08-01 17:54:05 +02:00
Stefan Eissing
7f5ad2028d
pytest: test very long urls
test_02_36 tests h1/h2/h3 with urls longer than 1/16/32/64K.

Protocols behave the same until the size exceed 64k when h2 frame limits
bite and h3 exhibits a different http status.

Failed attempt to reproduce #18121
Closes #18129
2025-08-01 14:35:37 +02:00
Stefan Eissing
ba9ddb9357
tls: CURLINFO_TLS_SSL_PTR testing
Add tests of CURLINFO_TLS_SSL_PTR and its returned value in test client
'hx-download'. Use obtained pointer to look up the negotiated TLS
version.

Update manpage of CURLINFO_TLS_SSL_PTR to also describe the behaviour of
wolfSSL similar to OpenSSL. Fix the wolfSSl implementation for TCP to
behave like that. Update the QUIC queries.

Fix rustls `get_internals()` to return the rustls_connection* and not
the address of the pointer.

Assisted-by: Viktor Szakats
Closes #18066
2025-08-01 09:37:36 +02:00
Stefan Eissing
af69c9d636
ip happy eyeballing: keep attempts running
When `CURLOPT_HAPPY_EYEBALLS_TIMEOUT_MS` expires, start the next ip
connect attempt, but keep all ongoing attempts alive.

Separate happy-eyeballs connection filter into own source files.

Closes #18105
2025-08-01 09:30:59 +02:00
Viktor Szakats
902262b166
windows: document toolchain support for CERT_NAME_SEARCH_ALL_NAMES_FLAG
Follow-up to 5f99b45693 #18113
Follow-up to 3bfcfe82b9 #18085
2025-07-31 20:43:21 +02:00
Viktor Szakats
923db3515d
schannel: not supported with UWP, drop redundant code
Schannel is not supported by UWP. SSPI is also required by Schannel in
curl, and SSPI also isn't supported by UWP.

mingw-w64 is able to create such build regardless (my guess: due to API
parts not accurately marked as UWP-only), but the binary is unlikely
to work. With MSVC the failure happens at build-time.

Ref: https://learn.microsoft.com/windows/win32/api/sspi/nf-sspi-initsecurityinterfacea#requirements
Ref: https://learn.microsoft.com/windows/win32/secauthn/initializesecuritycontext--schannel#requirements

Drop all UWP-related logic, including two related feature checks, that
can now be permanently enabled.

Also:
- build: show fatal error for Schannel in UWP mode.
- build: do not allow enabling SSPI in UWP mode.
- drop undocumented option `DISABLE_SCHANNEL_CLIENT_CERT`. Added without
  mention in an unrelated commit. The PR text says to save size. On x64
  this is 0.3%, or 4KB out of 1.3MB. The tiny gain doesn't justify
  an extra build variant. Ref: 8beff43559
- move `MPROTO_SCHANNEL_CERT_SHARE_KEY` closer to its use.
- replace commented block with `#if 0`.

Reviewed-by: Jay Satiro
Follow-up to cd0ec4784c #17089
Closes #18116
2025-07-31 20:05:32 +02:00
Viktor Szakats
b5c245045e
windows: drop two interim, single-use macros
Follow-up to e77d867068

Closes #18114
2025-07-31 11:54:18 +02:00
Cole Leavitt
4108d11008
bufq: add integer overflow checks before chunk allocations
Closes #18112
2025-07-31 10:07:11 +02:00
Viktor Szakats
5f99b45693
windows: document toolchain support for some macros (cont.)
Follow-up to 3bfcfe82b9 #18085

Closes #18113
2025-07-31 09:29:48 +02:00
Viktor Szakats
3bfcfe82b9
windows: document toolchain support for some macros
The oldest MS SDK I checked is 6.0A (VS2008). Versions are approximate
beyond 7.1A. I only have two Win10 SDKs to verify:
10.0.16299.0 (VS2017-15.4) and 10.0.22621.0 (VS2022).

Ref: https://en.wikipedia.org/wiki/Microsoft_Windows_SDK
Ref: https://developer.microsoft.com/windows/downloads/sdk-archive/index-legacy (recent versions mostly)

Closes #18085
2025-07-31 00:43:57 +02:00
Viktor Szakats
08acef2aab
schannel: assume CERT_CHAIN_REVOCATION_CHECK_CHAIN
Always available in supported mingw-w64 and MSVC compilers, except
in UWP mode. For mingw32ce this macro is defined later in the code.

Also available in OpenWatcom 2.
ce6c37eb29/bld/w32api/include/winerror.mh

Closes #18108
2025-07-31 00:41:36 +02:00
Viktor Szakats
d413ed2d40
windows: drop CRYPT_E_* macro fallbacks, limit one to mingw32ce
They are defined by all mingw-w64 versions and all supported MSVC
versions (VS2008 and up).

Also by OpenWatcom 2:
ce6c37eb29/bld/w32api/include/winerror.mh

mingw32ce misses `CRYPT_E_NOT_IN_REVOCATION_DATABASE`.

Closes #18092
2025-07-30 15:28:28 +02:00
Viktor Szakats
e32a6dc510
schannel: drop fallbacks for 4 macros
They are defined by all mingw-w64 versions and all supported MSVC
versions (SDK 7.1A+).

Also by OpenWatcom 2:
ce6c37eb29/bld/w32api/include/wincrypt.mh

These aren't defined by mingw32ce. And likely defined by MS WinCE SDK,
but curl code doesn't use them in WinCE builds.

Closes #18093
2025-07-30 12:24:37 +02:00
Viktor Szakats
dfe3844c56
schannel: drop fallbacks for unused BCRYPT_* macros
In case they are used again in the future, the fallbacks are not
necessary for the supported mingw-w64 (v3+) and MSVC versions.

Follow-up to 6238888ca7 #15621

Closes #18091
2025-07-30 12:24:37 +02:00
Viktor Szakats
58906838c5
schannel: fix recent update for mingw32ce
Follow-up to 952c929bdf #18084
Closes #18097
2025-07-30 11:55:54 +02:00
Viktor Szakats
217ae24a94
build: fix mingw-w64 version guard for mingw32ce
Follow-up to a28f5f68b9 #18010
Closes #18095
2025-07-30 11:55:54 +02:00
Daniel Stenberg
8d00527e9e
schannel: use if(result) like the code style says
instead of comparing != CURLE_OK

Closes #18094
2025-07-30 11:26:04 +02:00
Stefan Eissing
83da4d9d3b
connectdata: remove primary+secondary ip_quadruple
Since the content varies during connection setup and while doing it
(eyeballing), remove these strcut from `connectdata` and replace use
with querying the connection filters. Those keep that information
already.

Change the info logging of established connections to also give the
local address and port.

Closes #17960
2025-07-30 10:56:19 +02:00
Dominik Tomecki
450c00f983
smtp: allow suffix behind a mail address for RFC 3461
Verified in test 3215

Closes #16643
2025-07-30 09:52:34 +02:00
Daniel Stenberg
d7918861d1
openssl: split cert_stuff into smaller sub functions
- rename it client_cert
- make it return CURLcode

Closes #18081
2025-07-30 09:12:10 +02:00
Daniel Stenberg
7ed349de4c
unit-tests: build the unitprotos.h from here
Make the bundle depend on the header in the lib dir and built it now if
not present.

Reported-by: Todd Gamblin
Fixes #18088
Closes #18089
2025-07-30 09:06:35 +02:00
Viktor Szakats
58e0ff809f
tidy-up: whitespace, indent, #if 0
Closes #18090
2025-07-30 03:46:21 +02:00
Viktor Szakats
daa5b044b8
cmake: defer building unitprotos.h till a test target needs it
Follow-up to c9bb9cd165 #17750
Ref: https://github.com/curl/curl/pull/17750#issuecomment-3133749477
Closes #18086
2025-07-30 02:38:14 +02:00
Viktor Szakats
952c929bdf
schannel: drop old-mingw special case
mingw-w64 always defines `CERT_CHAIN_REVOCATION_CHECK_CHAIN`.

Also confirmed in CI.

Follow-up to 38029101e2 #11625
Ref: a28f5f68b9 #18010
Closes #18084
2025-07-29 22:35:06 +02:00
Viktor Szakats
cd586149d5
tests: constify command-line arguments
For libtests, tunits, units.

Also:
- lib3033: tidy up headers.
- lib/netrc: constify an arg in `Curl_parsenetrc()`.

Closes #18076
2025-07-29 13:44:50 +02:00
Daniel Stenberg
3407bee8c8
setopt: refactor out the booleans from setopt_long to setopt_bool
- add a message if a boolean is set to something not 1 or 0 (as it might be
  made to mean something else in a future)

- use 's->' in all setopt_* functions

Closes #17887
2025-07-29 13:07:22 +02:00
Stefan Eissing
55c045c863
multi: add CURLMOPT_NETWORK_CHANGED to signal network changed
New multi option CURLMOPT_NETWORK_CHANGED with a long bitmask value:

- CURLM_NWCOPT_CLEAR_CONNS: do not reuse existing connections, close all
  idle connections.

- CURLM_NWCOPT_CLEAR_DNS: clear the multi's DNS cache.

All other bits reserved for future extensions.

Fixes #17225
Reported-by: ウさん
Closes #17613
2025-07-29 11:18:26 +02:00
Viktor Szakats
c4ed28aebb
windows: assume ADDRESS_FAMILY, drop feature checks
Early mingw-w64 releases missed it, but by requiring v3.0, this is no
longer an issue. Supported Visual Studio SDKs also offer it.

Follow-up to a28f5f68b9 #18010
Closes #18057
2025-07-29 02:06:35 +02:00