Commit graph

6867 commits

Author SHA1 Message Date
Viktor Szakats
d65436d9a0
openssl: BoringSSL / AWS-LC tidy-ups
- schannel: apply BoringSSL workaround to AWS-LC too.
  Affects Schannel + AWS-LC MultiSSL builds. (not tested in CI)
  Ref: 274940d743 #2643 #2634

- curl_ntlm_core: deduplicate macro defines.

- curl_ntlm_core: document version thresholds for an AWS-LC-specific
  workaround.
  It was necessary between v1.2.0 2022-09-01 and v1.30.1 2024-06-21.
  No longer necessary since v1.31.0 2024-07-01:
  ba94617d99
  Follow-up to 34ef4fab22 #10320

- lib758: drop redundant OpenSSL version guards.
  `OPENSSL_VERSION_NUMBER > 3` automatically guards against LibreSSL,
  BoringSSL and AWS-LC.
  Ref: https://github.com/curl/curl/pull/18288/commits/6ddd8f2c0bbfcb0847b1ee7f257fb772fa47310c
  Follow-up to a5f0ab7995 #18288

- dllmain, curl_sha512_256: formatting.

Closes #18387
2025-08-24 00:30:06 +02:00
Viktor Szakats
cff4c16b83
pytest: fix test_17_09_ssl_min_max for BoringSSL
Ref: https://github.com/curl/curl/actions/runs/17179514833/job/48740057095

Also fix indent.

Cherry-picked from #18384
Closes #18385
2025-08-23 23:04:02 +02:00
Daniel Stenberg
29e4a124a3
tool_operate: bail out better on no URL
To avoid NULL derferences with many outputs and --next.

Follow-up to 034612cd51

Add test 760 to verify

Reported-by: BobodevMm on github
Fixes #18375
Closes #18376
2025-08-23 13:43:46 +02:00
Viktor Szakats
4f0e530c77
tidy-up: formatting
Closes #18373
2025-08-23 01:12:27 +02:00
Daniel Stenberg
844c385812
typecheck-gcc: add type checks for curl_multi_setopt()
Test 745 verifies that all options are tested.

Closes #18357
2025-08-22 17:12:58 +02:00
Daniel Stenberg
b3570b36d7
runtests: show still running tests when nothing has happened for a while
... even if there are tests still pending. To help visualize tests that
might be hanging.

Attempts to set the limit at 10 seconds of silence.

Closes #18349
2025-08-22 16:28:03 +02:00
Daniel Stenberg
33572b851e
tool_urlglob: handle a zero length glob piece
Follow-up to 034612cd51

Add test 759 to verify

Reported-by: Stanislav Osipov
Fixes #18352
Closes #18353
2025-08-22 16:22:07 +02:00
Viktor Szakats
3298a43133
spelling: file system
Closes #18348
2025-08-22 15:53:41 +02:00
Viktor Szakats
f6c2907172
test1557: pass long type to multi_setopt()
A wrong type here has seen to manifest in CI failures with gcc-12 macOS.

Ref: https://github.com/curl/curl/pull/18348#issuecomment-3213881790
Ref: https://github.com/curl/curl/actions/runs/17153761944/job/48665734013?pr=18349

Follow-up to b63cce7fee #18339
Follow-up to 88fc6c491f #18263

Closes #18355
2025-08-22 14:16:45 +02:00
Viktor Szakats
7956a2a756
gnutls: fix building with older supported GnuTLS versions
Also:
- GHA/linux-old: switch jobs from OpenSSL 1.0.2 to GnuTLS 3.5.8.

Ref: https://gitlab.com/gnutls/gnutls/blob/master/NEWS
Follow-up to fa0ccd9f1f #15774
Follow-up to 68bd759c2b #15667
Cherry-picked from #18330
Closes #18335
2025-08-21 18:04:25 +02:00
Viktor Szakats
696150b752
runtests: replace --ci with --buidinfo, show OS/Perl version again
I figure OS/Perl version may be useful to see in local runs,
and also in CI runs where `--ci` was not set:
https://curl.se/dev/log.cgi?id=20250820041228-707387#prob1

Also, only show a message if the `diff` tool is missing.

Follow-up to 985f39c0ce #18147
Closes #18329
2025-08-21 13:11:32 +02:00
Viktor Szakats
8f9d73bcaf
tests: drop unused BLANK envs, unset CURL_NOT_SET
Closes #18328
2025-08-21 13:11:32 +02:00
Viktor Szakats
8fac255fc1
test: add cygwin feature and use it (test 1056, 1517)
To replace custom checks with the same effect.

Closes #18327
2025-08-21 13:11:32 +02:00
Stefan Eissing
88fc6c491f
threaded-resolver: fix shutdown
Changed strategy to start up and terminate resolver thread.

When starting up:

Start the thread with mutex acquired, wait for signal from thread that
it started and has incremented the ref counter. Thread set
pthread_cancel() to disabled before that and only enables cancelling
during resolving itself. This assure that the ref counter is correct and
the unlinking of the resolve context always happens.

When shutting down resolving:

If ref counting shows thread has finished, join it, free everything. If
thread has not finished, try pthread_cancel() (non Windows), but keep
the thread handle around.

When destroying resolving:

Shutdown first, then, if the thread is still there and 'quick_exit' is
not set, join it and free everything. This might occur a delay if
getaddrinfo() hangs and cannot be interrupted by pthread_cancel().

Destroying resolving happens when another resolve is started on an
easy handle or when the easy handle is closed.

Add test795 to check that connect timeout triggers correctly
when resolving is delayed. Add debug env var `CURL_DNS_DELAY_MS`
to simulate delays in resolving.

Fix test1557 to set `quick_exit` and use `xxx.invalid` as domain
instead of `nothing` that was leading to hangers in CI.

Closes #18263
2025-08-21 09:26:49 +02:00
Viktor Szakats
2b2d8b7756
tests: delete unused commands
`runtests.pl` defaults to `-` if a command is not set, since
c43ad0f972.

Also:
- drop a stray `</file>`.
- replace a `nothing` with a guaranteed invalid hostname.
  Ref: https://github.com/curl/curl/pull/18263/commits/4334033b43acca9b01ba43e08e611c8654feb84d
- replace unused URLs with `-`.

Closes #18319
2025-08-20 15:31:22 +02:00
Daniel Stenberg
b9e3ea4edb
ftp: store dir components as start+len instead of memdup'ing
- Avoids allocating every path segment separately
- Improved directory handling in connection reuse

Closes #18312
2025-08-19 07:59:50 +02:00
Viktor Szakats
2388b0e587
processhelp.pm: use Win32::Process* perl modules if available
`Win32::Process::List` and `Win32::Process`.

To replace external calls to `tasklist.exe` and `taskkill.exe`.

The perl modules are wrappers/binding to Win32 API calls. They avoid
launching external processes with a command shell (including MSYS2),
the external tool, and creating command-lines and parsing tool output.

According to local tests and the CI, one test session calls
`tasklist.exe` 350-400 times. `taskkill.exe` is rarely called:
https://github.com/curl/curl/actions/runs/17012376726?pr=18296

It's hard to predict any possible side-effect of dynamically loading
the two necessary, module DLLs into the Perl process. The MSYS2 runtime
if prone to fail when doing this in fork operations, as seen earlier
with the Win32.DLL module. But, is looks like a symptom, not the root
cause for these failures, because the failures are present with or
without perl.exe loading the Win32.DLL.
Ref: be01b60ce5 #18287

Cherry-picked from #18296
Closes #18308
2025-08-18 23:46:56 +02:00
Viktor Szakats
a19bd43210
cmake: honor CMAKE_C_FLAGS in test 1119 and 1167
`CMAKE_C_FLAGS` is not set by curl, but may contain custom options
required for a successful compiler run, when invoked by these tests.

One such case is when configuring Visual Studio or clang-cl via compiler
options, instead of envs.

Cherry-picked from #18301
Closes #18307
2025-08-18 11:08:53 +02:00
Daniel Stenberg
0ba8e7f9db
test1549: verify CURLOPT_COOKIEFILE after expired cookies
Verifies #18299 (de89b86046)
Closes #18303
2025-08-17 22:52:18 +02:00
Stefan Eissing
a5f0ab7995
openssl: auto-pause on verify callback retry
When an application install its own OpenSSL verify callback and that
callback invokes `SSL_set_retry_verify()`, the transfer is automatically
paused and does not progress the connect attempt any further until
unpaused via `curl_easy_pause().

Added test758 to verify.

Ref: #18284
Original PR by @Natris
Bug: https://curl.se/mail/lib-2025-08/0012.html
Closes #18288
2025-08-15 13:50:28 +02:00
Daniel Stenberg
c5f76b8096
sendf: getting less data than "max allowed" is okay
Easily triggered when doing an FTP ranged download with the end range
larger than the file size.

A regression since 30de937bda (shipped in curl 8.9.0)

Add test 2307 to verify.

Reported-by: kupavcevdenis on github
Fixes #18283
Closes #18295
2025-08-15 13:35:04 +02:00
Viktor Szakats
be01b60ce5
runtests: assume Time::HiRes, drop Perl Win32 dependency
`Time::HiRes` was already used unconditionally before this patch in
`servers.pm`. This package, and functions used by runtests (`sleep` and
`gettimeofday`) are supported by the minimum Perl version required for
curl:

https://perldoc.perl.org/5.8.0/Time::HiRes

- Drop the `portable_sleep()` wrapper in favor of `Time::HiRes::sleep()`.
- Use `Time::HiRes` unconditionally in `serverhelp.pm`.
- Stop using the `Win32` package where available. It was included
  to provide a Windows fallback for `Time::HiRes::sleep()`. It was never
  actually called, but the dependency may have loaded `Win32.dll`, which
  often appears in failed fork operations in GHA logs.
  Ref: a6fed41f6f #5054 #5034
  Ref: https://github.com/curl/curl/discussions/14854

Closes #18287
2025-08-14 16:34:06 +02:00
Viktor Szakats
c24d4be057
processhelp.pm: fix to use the correct null device on Windows
To not create a file named `nul` in the `tests` directory when running
tests with MSYS2/Cygwin Perl (the only supported Perl for Windows).

I imagine this may cause issues when being written in parallel. Also
`nul` is a reserved filename on Windows.

Closes #18282
2025-08-14 12:25:15 +02:00
Stefan Eissing
37cecfc7b9
websocket: support CURLOPT_READFUNCTION
Add support for CURLOPT_READFUNCTION with WebSocket urls when *not* in
connect-only mode, e.g. when using curl_multi_perform.

Install the callback function and set CURLOPT_UPLOAD. Return
CURL_READFUNC_PAUSE when having nothing more to send and unpause the
transfer when more data is ready.

This will send the read bytes in a WebSocket BINARY frame.

Add support for this mode in the pytest "ws_data" client and have all
tests run in 'curl_ws_send/recv' and 'peform' mode as well.

Add `curl_ws_start_frame()`. Document, cover in libcurl-ws.md and
explain the READFUNCTION mode for websockets.

Add example `websocket-updown` for this.

Closes #17683
2025-08-11 23:28:54 +02:00
Viktor Szakats
626051cd8b
test436: fix running on Windows with _curlrc present
in the user home directory.

Before this patch, the curl tool found the system curlrc first, ignoring
the custom one set by the test via `CURL_HOME`.

Closes #18242
2025-08-10 00:33:45 +02:00
Viktor Szakats
7fd7d561c4
cmake: set CURL_DIRSUFFIX automatically in multi-config builds
To make it easier to run tests when using the Visual Studio generator,
also perhaps Xcode.

Also drop manual settings from CI.

Closes #18241
2025-08-10 00:33:45 +02:00
Daniel Stenberg
b823966d66
tool_parsecfg: use dynbuf for quoted arguments
Instead of doing malloc/free every time and "manual" buffer population.
This reuses the dynbuf for this purpose for the duration of parsing the
file.

Closes #18230
2025-08-08 18:27:03 +02:00
Daniel Stenberg
3b40128b0f
curl: make global truly global
The GlobalConfig only exists in a single instance and it has worked like
this since the dawn of time. It is about time we stop passing around
pointers to what was already essentially a global object and instead
just use a... global.

It simplifies things.

Closes #18213
2025-08-07 10:43:06 +02:00
Viktor Szakats
17bf8809ef
cmake: ExternalProject test tidy-ups
Follow-up to b8296d367a #18208

Closes #18214
2025-08-07 10:37:10 +02:00
Viktor Szakats
b8296d367a
cmake: make the ExternalProject test work
By micromanaging the project dependency and its inclusion into the test
project. It feels like an awkward construct, but perhaps better than
nothing.

It's also fragile because it's a static build with no assistance from
the external project (curl in this case). Mitigated in test by disabling
all dependencies and some features.

Since there is no special core cmake logic to be tested here, in CI
the test is tested really. To keep CI jobs at minimum, only add 3 of
them, taking 42s in total. (All 6 would take 270s.)

Follow-up to e2a23d5d0d #17203

Closes #18208
2025-08-07 09:07:46 +02:00
Oxan van Leeuwen
a93113b5b4
libcurl: reset rewind flag in curl_easy_reset()
curl_easy_reset() did not reset the `rewind_read` flag. This caused any
handles that previously had a CURLE_SEND_FAIL_REWIND error to get stuck
with that error, failing any subsequent requests, even if they didn't
have any body at all.

Verified in test 3034

Fixes #18206
Closes #18207
2025-08-06 17:58:21 +02:00
Viktor Szakats
4e5459c8e1
tests/ech_tests.sh: indent, if/for style, inline ifs
Sync with rest of shell scripts.

Follow-up to 2485a2d100 #17001
Closes #18187
2025-08-06 15:07:04 +02:00
Daniel Stenberg
3689ef21bd
multi: change prefix for the network change bits to CURLMNWC_
Because "CURLM_" is used for curl multi error codes and it is convenient
to use the prefix to identify the number family.

Closes #18176
2025-08-05 13:05:59 +02:00
Daniel Stenberg
df2b4ccc22
hostip: cache negative name resolves
Hold them for half the normal lifetime. Helps when told to transfer N
URLs in quick succession that all use the same non-resolving hostname.

Done by storing a DNS entry with a NULL pointer for 'addr'.

Previously an attempt was made in #12406 by Björn Stenberg that was
ultimately never merged.

Closes #18157
2025-08-05 08:05:31 +02:00
Viktor Szakats
06c12cc08b
build: disable TCP_NODELAY for emscripten
In WebAssembly, using `TCP_NODELAY` fails with:
```
* Could not set TCP_NODELAY: Protocol not available
```

Add a new feature macro in `curl_setup.h` telling whether `TCP_NODELAY`
is known to be supported at runtime, when defined at compile-time.

Keep `TCP_NODELAY` guards at their current positions to ensure the
necessary headers (e.g. `netinet/tcp.h` and `netinet/in.h`) define it.

Reported-by: Jeroen Ooms
Fixes #17974
Closes #18155
2025-08-05 00:34:24 +02:00
Stefan Eissing
1ad2009ad6
multi: add new information extraction method
Adds `curl_off_t curl_multi_get_offt(CURLM *multi_handle, CURLMinfo_offt
info)` to the multi interface with enums:

* CURLMINFO_XFERS_CURRENT: current number of transfers
* CURLMINFO_XFERS_RUNNING: number of running transfers
* CURLMINFO_XFERS_PENDING: number of pending transfers
* CURLMINFO_XFERS_DONE: number of finished transfers to read
* CURLMINFO_XFERS_ADDED: total number of transfers added, ever

Add documentation for functions and info enums.

Add use in the curl command line tool to replace two static
variables counting the same "from the outside".

refs #17870
Closes #17992
2025-08-04 23:48:57 +02:00
Daniel Stenberg
fadc487567
writeout: add %time{}
Output the current UTC time using strftime format. %f is an extra curl
specific flag to output the microsecond fraction of the current second.

Verified by test 1981

Closes #18119
2025-08-04 23:45:48 +02:00
Stefan Eissing
5b80b4c012
lib: replace getsock() logic with pollsets
`getsock()` calls operated on a global limit that could
not be configure beyond 16 sockets. This is no longer adequate
with the new happy eyeballing strategy.

Instead, do the following:
- make `struct easy_pollset` dynamic. Starting with
  a minimal room for two sockets, the very common case,
  allow it to grow on demand.
- replace all protocol handler getsock() calls with pollsets
  and a CURLcode to return failures
- add CURLcode return for all connection filter `adjust_pollset()`
  callbacks, since they too can now fail.
- use appropriately in multi.c and multi_ev.c
- fix unit2600 to trigger pollset growth

Closes #18164
2025-08-04 23:43:13 +02:00
Daniel Stenberg
13cbabf05f
curl: add --follow
Makes curl follow redirects an act on the response code and change a
custom method accordingly, contrary to --location.

Potential future command line to send QUERY and following a redirect
according to the status code:

    curl -d "request-body" -X QUERY --follow https://example.com

add test 794,796,797

Assisted-by: Daniel Böhmer <post@daniel-boehmer.de>

Closes #16543
2025-08-04 17:12:14 +02:00
Daniel Stenberg
854b0e230c
hostip: do DNS cache pruning in milliseconds
Instead of using integer seconds. Also: if the cache contains over
30,000 entries after first pruning, it makes anoter round and removes
all entries that are older than half the age of the oldest entry until
it goes below 30,000.

Closes #18160
2025-08-04 16:20:50 +02:00
Viktor Szakats
985f39c0ce
runtests: add --ci option, show Env: only when non-empty
To reduce log noise in local test runs:
- move the `buildinfo.txt` dump and header info lines
  `OS:`, `Perl:`, `diff:` behind a `--ci` `runtests.pl` option.
- enable this option for the CI test targets.
- hide `Env:` header info line if empty.
- merge `Env:` output into a single `logmsg()` call.

Closes #18147
2025-08-04 14:55:10 +02:00
Viktor Szakats
37ac4498ee
unit2604: avoid UNCONST()
Closes #18143
2025-08-04 14:55:10 +02:00
Viktor Szakats
c2fed8c669
libtests: use FMT_SOCKET_T, drop more casts
Follow-up to 37913c01a5 #18106

Closes #18142
2025-08-04 14:55:10 +02:00
Viktor Szakats
1644a49ab8
configure: if no perl, disable unity and shell completion, related tidy ups
Also:
- GHA/linux, macos: test `install` with autotools too.
- GHA/linux, macos: enable fish and zsh completion in an autotools job.
- scripts: drop dynamic perl checks redundant after this patch.
- scripts: drop two interim variables.
- autotools: make `build-certs` target check perl first.
- autotools: replace `$(PERL)` with `@PERL@` to match the pattern used
  in most automake scripts. For consistency. This makes `PERL` be
  defined at configure-time, as opposed to make-time, for these cases
  now.

Closes #18141
2025-08-04 14:55:10 +02:00
Stefan Eissing
d07504aa8d
unit2600: add another case
Add a case with 1 ipv4 and 3 ipv6 and check that all are attempted with
the correct minimum duration before failures. To check that more ipv6
than ipv4 lead to the correct behaviour.

Closes #18144
2025-08-03 22:03:42 +02:00
Viktor Szakats
51bc836c42
cmake: re-add simple test target, and name it tests
Before this patch there was no (easy) way to run tests without various
verbose options, hindering readability of the output due its length.

Unfortunately CMake reserves `test` as a target name, which was
the reason for its removal earlier. Re-add it with the name `tests`,
to have the functionality without the name collision.

Also add a `tests` target as an alias to `test` for autotools.

Ref: https://cmake.org/cmake/help/v4.1/policy/CMP0037.html
Follow-up to cfea4f2f4d #6258
Closes #18145
2025-08-02 15:03:54 +02:00
Viktor Szakats
d01d2ec9f1
docs: add CURLOPT type change history, drop casts where present
Some CURLOPT constants defined in the curl public headers were initially
enums (= ints), or macros with bare numeric values. Recent curl releases
upgraded them to `long` constants, to make them pass correctly to
`curl_easy_setop()` by default, i.e. without requiring a `(long)` cast.

This patch drops such casts from the examples embedded in the docs. At
the same time it documents which curl release made them `long` types,
to keep them useful when working with previous libcurl versions.

Also:
- drop a `(long)` cast that was never necessary.
- CURLOPT_ALTSVC_CTRL.md: bump local copy of macros to long.
- test1119: make it ignore symbols ending with an underscore, to skip
  wildcard, e.g. `**CURLAUTH_***`.

Closes #18130
2025-08-02 00:05:33 +02:00
Daniel Stenberg
18e00c8ed0
managen: reset text mode at end of table marker
It previously mostly ignored it which lead to bad format after a table
has ended with "##".

Closes #18139
2025-08-01 23:24:32 +02:00
Daniel Stenberg
2bad773b30
urlapi: allow more path characters "raw" when asked to URL encode
Setting the path component to contain the letters:

    ! $ & ' ( ) { } [ ] * + , ; = : @

now leaves them un-encoded when CURLU_URLENCODE is used.

Amended test 1560 to verify.

Reported-by: Jeroen Ooms
Fixes #17977
Closes #18024
2025-08-01 23:22:32 +02:00
Viktor Szakats
b131db5410
CURLOPT: bump CURLPROTO_* macros to long
This patch bumps the size of these macros from `int` to `long`, while
keeping their actual values the same. It may cause incompatibilities in
user code, requiring the bump of holder variables and/or adding casts:

- CURLPROTO_HTTP
- CURLPROTO_HTTPS
- CURLPROTO_FTP
- CURLPROTO_FTPS
- CURLPROTO_SCP
- CURLPROTO_SFTP
- CURLPROTO_TELNET
- CURLPROTO_LDAP
- CURLPROTO_LDAPS
- CURLPROTO_DICT
- CURLPROTO_FILE
- CURLPROTO_TFTP
- CURLPROTO_IMAP
- CURLPROTO_IMAPS
- CURLPROTO_POP3
- CURLPROTO_POP3S
- CURLPROTO_SMTP
- CURLPROTO_SMTPS
- CURLPROTO_RTSP
- CURLPROTO_RTMP
- CURLPROTO_RTMPT
- CURLPROTO_RTMPE
- CURLPROTO_RTMPTE
- CURLPROTO_RTMPS
- CURLPROTO_RTMPTS
- CURLPROTO_GOPHER
- CURLPROTO_SMB
- CURLPROTO_SMBS
- CURLPROTO_MQTT
- CURLPROTO_GOPHERS
- CURLPROTO_ALL

Also:
- keep existing cast within the documentation to make sure it applies
  to older curl versions as well.

Closes #18136
2025-08-01 18:49:23 +02:00