Commit graph

15227 commits

Author SHA1 Message Date
sunriseL
c294d0abc5
libssh2: fix sftp_statemachine busyloop when *block=true
Closes #18311
2025-08-19 08:07:33 +02:00
Daniel Stenberg
b9e3ea4edb
ftp: store dir components as start+len instead of memdup'ing
- Avoids allocating every path segment separately
- Improved directory handling in connection reuse

Closes #18312
2025-08-19 07:59:50 +02:00
Daniel Stenberg
79eb66f2a6
ftp: keep the "raw" URL decoded version of the path around
Instead of doing temporary decodes in three different places.

Closes #18312
2025-08-19 07:59:50 +02:00
Marcel Raad
a559f94321
asyn-thrdd: silence unused parameter warning
Closes https://github.com/curl/curl/pull/18310
2025-08-18 20:04:53 +02:00
David Zhuang
7dafe10db2
ngtcp2: use custom mem funcs
Pass curl's memory functions to the nghttp3 and ngtcp2 functions that
allow them. This allows custom memory functions passed by the curl user
to be used in nghttp3 and ngtcp2.

Closes #18196
2025-08-18 13:25:52 +02:00
Daniel Stenberg
fc4ae23cc2
asyn-thrdd: fix Curl_async_pollset without socketpair
- remove a superfluous if()
- edit the #ifdef to only do the timeout checks when socketpair support
  is missing, as that code flow could not be reached otherways anyway

CodeSonar pointed out possible issue that led me to see this.

Closes #18306
2025-08-18 13:03:51 +02:00
Daniel Stenberg
294ebba565
bufq: simplify condition
'result' is always CURLE_AGAIN here

Pointed out by CodeSonar

Closes #18305
2025-08-18 09:58:31 +02:00
Daniel Stenberg
e440686034
ftp: use 'conn' instead of 'data->conn'
In the ftp_state_use_port function, as we already use that local
variable for all other conn accesses.

Closes #18304
2025-08-18 09:57:45 +02:00
xfangfang
de89b86046
cookie: remove expired cookies before listing
Closes #18299
2025-08-17 22:25:25 +02:00
Stefan Eissing
a5f0ab7995
openssl: auto-pause on verify callback retry
When an application install its own OpenSSL verify callback and that
callback invokes `SSL_set_retry_verify()`, the transfer is automatically
paused and does not progress the connect attempt any further until
unpaused via `curl_easy_pause().

Added test758 to verify.

Ref: #18284
Original PR by @Natris
Bug: https://curl.se/mail/lib-2025-08/0012.html
Closes #18288
2025-08-15 13:50:28 +02:00
Daniel Stenberg
c5f76b8096
sendf: getting less data than "max allowed" is okay
Easily triggered when doing an FTP ranged download with the end range
larger than the file size.

A regression since 30de937bda (shipped in curl 8.9.0)

Add test 2307 to verify.

Reported-by: kupavcevdenis on github
Fixes #18283
Closes #18295
2025-08-15 13:35:04 +02:00
Viktor Szakats
d63e40f8e6
mbedtls: check for feature macros instead of version
Drop three interim macros and mbedTLS version checks in favor of feature
macros defined by mbedTLS itself.

These mbedTLS features require mbedTLS 3.6.0/3.6.1 for production.
Earlier versions may have partial/experimental support for them,
disabled by default and (AFAICS) without documented build options
to enable them.

One feature guard already used the native macro instead of the interim
one before this patch. (`MBEDTLS_SSL_SESSION_TICKETS`)

Ref: https://github.com/curl/curl/pull/18254#issuecomment-3182668817

Closes #18271
2025-08-14 12:25:14 +02:00
Daniel Stenberg
16ddcd6712
wolfssh: move the sftp upload init out of the state machine
Into its own function to reduce complexity. Takes complexity down from
72 to 50.

Closes #18278
2025-08-14 08:58:59 +02:00
Daniel Stenberg
adb0fa737d
socks5: split out two functions from the SOCKS5 state machine
Complexity down from 74 to 59

Closes #18277
2025-08-14 08:57:07 +02:00
Viktor Szakats
3eb00fa795
openssl: save and restore OpenSSL error queue in two functions
After merging #18228, I reviewed whether the clearing of the error queue
may interfere with preceding code. Turns out there may be a preceding
`SSL_Connect()` call.

This patch replaces the previous fix of clearing the error queue with
saving and restoring it in two functions which may be called between
the connect call and the `SSL_get_error()` call following it:
- `ossl_log_tls12_secret()`
- `Curl_ssl_setup_x509_store()`

The `ERR_set_mark()`, `ERR_pop_to_mark()` functions are present in all
supported OpenSSL and LibreSSL versions. Also in BoringSSL since its
initial commit.

OpenSSL may modify its error queue in all API calls that can fail.

Thanks-to: Viktor Dukhovni
Ref: https://github.com/curl/curl/issues/18190#issuecomment-3167702142
Ref: https://github.com/curl/curl/issues/18190#issuecomment-3169211739
Ref: https://github.com/curl/curl/issues/18190#issuecomment-3169988050

Follow-up to 8ec241bc99 #18228 #18190
Ref: e8b00fcd6a #10432 #10389
Fixes #18190
Closes #18234
2025-08-13 18:54:08 +02:00
Viktor Szakats
01a2308236
mbedtls: bump minimum version required to 3.2.0
3.2.0 was released on July 11, 2022.

Ref: #18161
Closes #18254
2025-08-13 09:09:45 +02:00
Daniel Stenberg
c6ae07c6a5
cookie: don't treat the leading slash as trailing
If there is only a leading slash in the path, keep that. Also add an
assert to make sure the path is never blank.

Reported-by: Google Big Sleep
Closes #18266
2025-08-13 01:00:28 +02:00
Stefan Eissing
37cecfc7b9
websocket: support CURLOPT_READFUNCTION
Add support for CURLOPT_READFUNCTION with WebSocket urls when *not* in
connect-only mode, e.g. when using curl_multi_perform.

Install the callback function and set CURLOPT_UPLOAD. Return
CURL_READFUNC_PAUSE when having nothing more to send and unpause the
transfer when more data is ready.

This will send the read bytes in a WebSocket BINARY frame.

Add support for this mode in the pytest "ws_data" client and have all
tests run in 'curl_ws_send/recv' and 'peform' mode as well.

Add `curl_ws_start_frame()`. Document, cover in libcurl-ws.md and
explain the READFUNCTION mode for websockets.

Add example `websocket-updown` for this.

Closes #17683
2025-08-11 23:28:54 +02:00
Stefan Eissing
9cc4e24ad9
resolving: dns error tracing
* Add more tracing information to c-ares errors.
* remove CURL_ASYNC_SUCCESS, rename `ares->last_status` to
  `ares->ares_status`. Give trace explanation for "common"
  errors
* add ares "csv" information to tracing on failure
* DoH: invoke `Curl_resolver_error()` on failure to populate
  error buf

Closes #18247
2025-08-11 14:35:07 +02:00
Viktor Szakats
8ec241bc99
openssl: clear errors after a failed d2i_X509()
Without it, subsequent OpenSSL API calls may fail with an error caught
within the OpenSSL `d2i_X509()` (decode) call.

It was seen to happen when importing from the Windows certificate store
(e.g. with `--ca-native`), and any one of the certificates failed while
decoding, then skipped.

Behind the scene (and undocumented), the failed decode call is adding
an error to an internal OpenSSL error queue. This error is picked up
later, at the connect phase, by another OpenSSL API call, which happens
to check the error queue, without clearing it first. It made the connect
fail with the error collected earlier, while decoding the malformed and
discarded certificate.

Fix by explicitly clearing the error queue if the decode call fails.

Ref: https://docs.openssl.org/3.5/man3/d2i_X509/

`-vvvv` output before this patch:
```
[0-0] == Info: successfully imported Windows ROOT store
[0-0] == Info: successfully imported Windows CA store
[0-0] == Info: [SSL] SSL_connect() -> err=-1, detail=1
[0-0] == Info: TLS connect error: error:068000DD:asn1 encoding routines::illegal padding
[0-0] == Info: [SSL] cf_connect() -> 35, done=0
```

Mainline OpenSSL (as of 3.5.2) and quictls (as of 3.3.0) are affected.

LibreSSL is not affected. (I did not test BoringSSL and AWS-LC)

Assisted-by: Stefan Eissing
Reported-by: Michał Petryka
Fixes #18190

Closes #18228
2025-08-08 20:08:31 +02:00
Jay Satiro
673b6c8cb8 schannel: add an error message for client cert not found
Prior to this change curl would show the generic error message
"curl: (58) Problem with the local SSL certificate".

Closes https://github.com/curl/curl/pull/18124
2025-08-08 03:43:54 -04:00
Stefan Eissing
10e60e825c
cfilters: make Curl_conn_get_socket simpler
Since it is only used for the first socket anyway, simplify the
function.

Closes #18219
2025-08-07 14:44:52 +02:00
Stefan Eissing
fd9429cc29
request: eliminate request getheader bool, reverse header default
Deduce that the transfer response expects headers by the protocol
handler implementing `write_resp_hd` callback. This eleminates the
`getheader` parameter in the `Curl_xfer_setup_*()` methods.

Add an implementation to RTSP for `write_resp_hd`, joining the HTTP
protocol in the only handlers having it.

Reverse the default of request's `header` bit that signals that headers
are expected. Default is now FALSE, set to TRUE when setting up the
transfer by presence of `write_resp_hd` in the protocol handler.

Closes #18218
2025-08-07 13:39:50 +02:00
Stefan Eissing
c1372df2e2
ares: destroy channel on shutdown
When we cancel async resolv operations, we have kept an existing ares
channel open. This seems unreliable as reported in #18216.

To get reliable behaviour, always destroy the ares channel on async
shutdown and create a new one on demand.

Fixes #18216
Reported-by: devgs on github
Closes #18217
2025-08-07 11:35:35 +02:00
Stefan Eissing
2d9f24bf24
http: resurrect addition from 41fe621
For some reason, these lines were unintentionally deleted in 6d5570c.
Thanks to IoannisGS for noticing.

Bug: https://github.com/curl/curl/pull/18179#issuecomment-3162876532
Reported-by: IoannisGS on github
Closes #18215
2025-08-07 10:40:47 +02:00
Karthik Das
d79269166e
curl_quiche.c: add include
Fixes #18211
Closes #18212
2025-08-07 09:54:09 +02:00
Oxan van Leeuwen
a93113b5b4
libcurl: reset rewind flag in curl_easy_reset()
curl_easy_reset() did not reset the `rewind_read` flag. This caused any
handles that previously had a CURLE_SEND_FAIL_REWIND error to get stuck
with that error, failing any subsequent requests, even if they didn't
have any body at all.

Verified in test 3034

Fixes #18206
Closes #18207
2025-08-06 17:58:21 +02:00
Daniel Stenberg
019991c25e
multi: fix bad splay management
The splay tree is a tree where each easy handle can be added *once*. The
expire time for that node is the closest expire time for that easy
handle.

Easy handles can however have more expire times queued up, so when the
node is removed from the splay tree because it is the next in line to
take care of, we must check if there is another expire time in the queue
and then add the node back into the splay.

Failing to do the later part, the calling of add_next_timeout after
Curl_splaygetbest, would leave the state.expiretime on the previous time
stamp, which when could make the next call to Curl_splaygetbest use the
wrong time stamp and get a wrong node out, causing trouble.

Reported-by: letshack9707 on hackerone
Closes #18201
2025-08-06 14:13:36 +02:00
Stefan Eissing
6cebd35b4c
lib: xfer_setup simplify
Make variants for transfers that send/receive or do both with just the
parameters they need. Split out the shutdown setting into a separate
function. Only FTP bothers with that.

Closes #18203
2025-08-06 14:12:46 +02:00
Stefan Eissing
6d5570c4dd
lib: replace conn [write]sockfd members by index
The `connectdata` members `sockfd` and `writesockfd` needed to by either
CURL_SOCKET_BAD or a copy of one of `conn->sock[2]`. When equal to one,
that index was used to send/recv/poll the proper socket or connection
filter chain.

Replace those with `send_idx` and `recv_idx` which are either -1, 0 or 1
to indicate which socket/filter to send/receive on.

Closes #18179
2025-08-06 08:47:12 +02:00
Daniel Stenberg
aec28f3ad3
splay: add another assert for detected usage problem
Closes #18199
2025-08-06 08:42:39 +02:00
Daniel Stenberg
bf5265ad11
dynbuf: assert on curlx_dyn_addf use with just "%s"
Because it is wasteful and should be done better.

Closes #18194
2025-08-06 08:32:07 +02:00
Stefan Eissing
41fe621ae1
vtls: set seen http version on successful ALPN
When a HTTP version has been negotiated via ALPN, set the member
`conn->httpversion_seen` accordingly. This allows pending transfers to
reuse multiplexed http connections before the response to the first
transfer has arrived.

Fixes #18177
Reported-by: IoannisGS on github
Closes #18181
2025-08-05 16:01:39 +02:00
Daniel Stenberg
b059f7deaf
setopt: add helper functions to setopt_long()
- Consistently keep options within ranges
- Reduce the maximum maxredirs value to fit a signed short
- Removed comments as the place to document the options is not here

Closes #18174
2025-08-05 13:47:47 +02:00
David Zhuang
60587049f7
vquic: use curl_getenv
getenv isn't defined on all platforms, which prevents vquic from
building. I specifically ran into this issue building on PlayStation.

Closes #18170
2025-08-05 13:12:00 +02:00
Roberto Hidalgo
4442e06b1f
openssl: output unescaped utf8 x509 issuer/subject DNs
Closes #18171
2025-08-05 13:08:40 +02:00
Daniel Stenberg
3689ef21bd
multi: change prefix for the network change bits to CURLMNWC_
Because "CURLM_" is used for curl multi error codes and it is convenient
to use the prefix to identify the number family.

Closes #18176
2025-08-05 13:05:59 +02:00
Anthony Hu
2340a60070
wolfssl: rename ML-KEM hybrids to match IETF draft
Closes #18123
2025-08-05 08:58:19 +02:00
Daniel Stenberg
1f34125141
urldata: reduce two long struct fields to unsigned short
Closes #18173
2025-08-05 08:48:07 +02:00
Daniel Stenberg
df2b4ccc22
hostip: cache negative name resolves
Hold them for half the normal lifetime. Helps when told to transfer N
URLs in quick succession that all use the same non-resolving hostname.

Done by storing a DNS entry with a NULL pointer for 'addr'.

Previously an attempt was made in #12406 by Björn Stenberg that was
ultimately never merged.

Closes #18157
2025-08-05 08:05:31 +02:00
Viktor Szakats
06c12cc08b
build: disable TCP_NODELAY for emscripten
In WebAssembly, using `TCP_NODELAY` fails with:
```
* Could not set TCP_NODELAY: Protocol not available
```

Add a new feature macro in `curl_setup.h` telling whether `TCP_NODELAY`
is known to be supported at runtime, when defined at compile-time.

Keep `TCP_NODELAY` guards at their current positions to ensure the
necessary headers (e.g. `netinet/tcp.h` and `netinet/in.h`) define it.

Reported-by: Jeroen Ooms
Fixes #17974
Closes #18155
2025-08-05 00:34:24 +02:00
Stefan Eissing
1ad2009ad6
multi: add new information extraction method
Adds `curl_off_t curl_multi_get_offt(CURLM *multi_handle, CURLMinfo_offt
info)` to the multi interface with enums:

* CURLMINFO_XFERS_CURRENT: current number of transfers
* CURLMINFO_XFERS_RUNNING: number of running transfers
* CURLMINFO_XFERS_PENDING: number of pending transfers
* CURLMINFO_XFERS_DONE: number of finished transfers to read
* CURLMINFO_XFERS_ADDED: total number of transfers added, ever

Add documentation for functions and info enums.

Add use in the curl command line tool to replace two static
variables counting the same "from the outside".

refs #17870
Closes #17992
2025-08-04 23:48:57 +02:00
Stefan Eissing
5b80b4c012
lib: replace getsock() logic with pollsets
`getsock()` calls operated on a global limit that could
not be configure beyond 16 sockets. This is no longer adequate
with the new happy eyeballing strategy.

Instead, do the following:
- make `struct easy_pollset` dynamic. Starting with
  a minimal room for two sockets, the very common case,
  allow it to grow on demand.
- replace all protocol handler getsock() calls with pollsets
  and a CURLcode to return failures
- add CURLcode return for all connection filter `adjust_pollset()`
  callbacks, since they too can now fail.
- use appropriately in multi.c and multi_ev.c
- fix unit2600 to trigger pollset growth

Closes #18164
2025-08-04 23:43:13 +02:00
Daniel Stenberg
29a6e15b27
setopt: split out cookielist() and cookiefile()
into their own sub functions

Closes #18162
2025-08-04 16:31:54 +02:00
Daniel Stenberg
854b0e230c
hostip: do DNS cache pruning in milliseconds
Instead of using integer seconds. Also: if the cache contains over
30,000 entries after first pruning, it makes anoter round and removes
all entries that are older than half the age of the oldest entry until
it goes below 30,000.

Closes #18160
2025-08-04 16:20:50 +02:00
Daniel Stenberg
40caca581f
splay: rename KEY_NOTUSED TO SPLAY_SUBNODE
- explains its purpose better
- make it global static const
- added an assert for a condition that should never happen (that we
  also catch run-time)

Closes #18152
2025-08-03 22:06:26 +02:00
Daniel Stenberg
2bad773b30
urlapi: allow more path characters "raw" when asked to URL encode
Setting the path component to contain the letters:

    ! $ & ' ( ) { } [ ] * + , ; = : @

now leaves them un-encoded when CURLU_URLENCODE is used.

Amended test 1560 to verify.

Reported-by: Jeroen Ooms
Fixes #17977
Closes #18024
2025-08-01 23:22:32 +02:00
Viktor Szakats
b131db5410
CURLOPT: bump CURLPROTO_* macros to long
This patch bumps the size of these macros from `int` to `long`, while
keeping their actual values the same. It may cause incompatibilities in
user code, requiring the bump of holder variables and/or adding casts:

- CURLPROTO_HTTP
- CURLPROTO_HTTPS
- CURLPROTO_FTP
- CURLPROTO_FTPS
- CURLPROTO_SCP
- CURLPROTO_SFTP
- CURLPROTO_TELNET
- CURLPROTO_LDAP
- CURLPROTO_LDAPS
- CURLPROTO_DICT
- CURLPROTO_FILE
- CURLPROTO_TFTP
- CURLPROTO_IMAP
- CURLPROTO_IMAPS
- CURLPROTO_POP3
- CURLPROTO_POP3S
- CURLPROTO_SMTP
- CURLPROTO_SMTPS
- CURLPROTO_RTSP
- CURLPROTO_RTMP
- CURLPROTO_RTMPT
- CURLPROTO_RTMPE
- CURLPROTO_RTMPTE
- CURLPROTO_RTMPS
- CURLPROTO_RTMPTS
- CURLPROTO_GOPHER
- CURLPROTO_SMB
- CURLPROTO_SMBS
- CURLPROTO_MQTT
- CURLPROTO_GOPHERS
- CURLPROTO_ALL

Also:
- keep existing cast within the documentation to make sure it applies
  to older curl versions as well.

Closes #18136
2025-08-01 18:49:23 +02:00
Stefan Eissing
ec4c559104
openssl: check SSL_write() length on retries
When an SSL_write() blocks we need to retry it with the
same length as before or stupid OpenSSL freaks out. Remember
it, limit any longer sends and fail shorter ones.

Fixes #18121
Reported-by: adamse on github
Closes #18132
2025-08-01 17:54:05 +02:00
Stefan Eissing
7f5ad2028d
pytest: test very long urls
test_02_36 tests h1/h2/h3 with urls longer than 1/16/32/64K.

Protocols behave the same until the size exceed 64k when h2 frame limits
bite and h3 exhibits a different http status.

Failed attempt to reproduce #18121
Closes #18129
2025-08-01 14:35:37 +02:00