Commit graph

3201 commits

Author SHA1 Message Date
Viktor Szakats
49145249be
tidy-up: drop stray "unused" comments
Closes #18453
2025-09-03 16:31:16 +02:00
Daniel Stenberg
765b5ab0cd
tool_getparam: warn on more unicode prefixes
If a string argument is expected and the first two bytes are 0xe2 ex80
and the third has the 7th bit set, that's enough for curl to warn.

Previously we tried to detect and warn only for the unicode double
quote, but users might use single quotes, other quotes or even lead the
argument with one of the "zero widths" characters. This is an attempt to
detect many of those. Without triggering for "normal" IDN hostnames.

Closes #18459
2025-09-03 11:35:10 +02:00
Daniel Stenberg
9598ccee59
tool_filetime: correct the conditions
The libcurl API for CURLINFO_FILETIME_T clearly says it contains -1 if
not set. Everything else is a valid time stamp so use that.

Follow-up to 54f1ef05d6
Closes #18446
2025-09-02 07:50:11 +02:00
Daniel Stenberg
84c92f7113
tool: move the error buffer to the per transfer struct
To avoid having to alloc or manage it separately.

Closes #18442
2025-09-01 08:52:57 +02:00
Daniel Stenberg
54f1ef05d6
tool_filetime: accept setting negative filetime
This allows --remote-time to set dates before 1970.

Due to a minor omission in the API, it will still avoid setting the time
if it is indeed exactly epoch 0 (jan 1 1970).

Verified by test 762

Fixes #18424
Reported-by: Terence Eden
Closes #18443
2025-09-01 08:51:36 +02:00
Daniel Stenberg
4a3ed6fc16
urlglob: only accept 255 globs
- using {} with single entries makes little sense
- when using {} sets with two entry lists, there can only be 64 to reach
  maximum number of URLs

Verify the max check in test 761
2025-08-30 22:27:28 +02:00
Daniel Stenberg
93f333c18f
curl_setup: use SIZE_MAX instead of SIZE_T_MAX
As SIZE_MAX exists in C99

Assisted-by: Stefan Eissing
Assisted-by: Jay Satiro

Ref: #18406
Closes #18426
2025-08-29 09:45:06 +02:00
Viktor Szakats
7ceb9c54aa
clang-tidy: disable clang-analyzer-security.ArrayBound
It's causing false-positives with clang-tidy v21, in cases in system
headers (seen in `FD_ISSET()` with macOS SDK). In some cases in
tests/server, there was no distinct source line that was triggering it.

Example:
```
/Applications/Xcode_16.4.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX15.5.sdk/usr/include/sys/_types/_fd_def.h:83:10: error: Potential out of bound access to 'fds_read.fds_bits' with tainted index [clang-analyzer-security.ArrayBound,-warnings-as-errors]
   83 |                 return _p->fds_bits[(unsigned long)_fd / __DARWIN_NFDBITS] & ((__int32_t)(((unsigned long)1) << ((unsigned long)_fd % __DARWIN_NFDBITS)));
      |                        ^
[...]
/Users/runner/work/curl/curl/tests/server/socksd.c:679:5: note: Taking false branch
  679 |     if(rc < 0) {
      |     ^
```

Closes #18422
2025-08-29 03:07:42 +02:00
Daniel Stenberg
ebff00f1af
tool_getparam: let --trace-config override -v
If --trace-config is used to set a level before -v is used, don't reset
the state on first -v (to "-all") as it otherwise does. This way,
--trace-config can be used to set specific trace items before -v on the
command line and it still works.

Previously, the first -v use would otherwise reset and undo the earlier
--trace-config items.

Fixes #18346
Closes #18361
2025-08-26 14:59:59 +02:00
Daniel Stenberg
fef318553b
tool_urlglob: add integer overflow protection
It is most likely impossible to actually overflow, but this makes it
certain.

Closes #18398
2025-08-26 09:50:00 +02:00
Viktor Szakats
1611e87d66
docs: point two broken links to archive.org
Closes #18393
2025-08-25 19:43:03 +02:00
Daniel Stenberg
29e4a124a3
tool_operate: bail out better on no URL
To avoid NULL derferences with many outputs and --next.

Follow-up to 034612cd51

Add test 760 to verify

Reported-by: BobodevMm on github
Fixes #18375
Closes #18376
2025-08-23 13:43:46 +02:00
Viktor Szakats
4f0e530c77
tidy-up: formatting
Closes #18373
2025-08-23 01:12:27 +02:00
Daniel Stenberg
bfbee798ca
tool_operate: use stricter curl_multi_setopt() arguments
To please the curl_multi_setopt() typechecker

- set multi callback protypes to match exactylu
- set CURLMOPT_MAX_HOST_CONNECTIONS as a long

Closes #18357
2025-08-22 17:13:07 +02:00
Daniel Stenberg
33572b851e
tool_urlglob: handle a zero length glob piece
Follow-up to 034612cd51

Add test 759 to verify

Reported-by: Stanislav Osipov
Fixes #18352
Closes #18353
2025-08-22 16:22:07 +02:00
Viktor Szakats
3298a43133
spelling: file system
Closes #18348
2025-08-22 15:53:41 +02:00
Daniel Stenberg
a46944a599
tool_cb_wrt: stop alloc/free for every chunk windows console output
Instead realloc to the largest buffer and keep that for reuse during the
entire lifetime.

Co-authored-by: Jay Satiro <raysatiro@yahoo.com>
Closes #18233
2025-08-11 09:37:29 +02:00
Viktor Szakats
c04292648f
cmake: drop redundant unity mode for curlinfo
Unity mode adds nothing besides some overhead and log noise for
targets built from a single source file.

I wish cmake disabled unity automatically in this case.

Closes #18238
2025-08-09 14:25:29 +02:00
Daniel Stenberg
b823966d66
tool_parsecfg: use dynbuf for quoted arguments
Instead of doing malloc/free every time and "manual" buffer population.
This reuses the dynbuf for this purpose for the duration of parsing the
file.

Closes #18230
2025-08-08 18:27:03 +02:00
Daniel Stenberg
da27db068f
tool_operate: cleanups
- move the state struct from config to global. It is used as a single
  instance anyway so might as well be a single one to save memory.
- simplify and combine several conditions
- set default retry delay inititally
- use better struct field names to make it easier to understand their
  purposes
- remove the state->outfiles field as it was not necessary
- remove superfluous glob cleanup call
- move conditions around to remove an indent level
- move the ->url NULL check

Takes single_transfer()'s complexity score down from 78 to 68.

Closes #18226
2025-08-08 11:43:28 +02:00
Daniel Stenberg
065a653158
tool_writeout: check gmtime return code too
If the unlikely event happen that it fails, it returns NULL.
CodeSonar is not happy unless we check for it.

Closes #18225
2025-08-08 11:42:33 +02:00
Daniel Stenberg
02b22ee4ea
curl: remove first argument from helpf()
It was always the same and it is a global already!

Closes #18221
2025-08-07 16:55:39 +02:00
Daniel Stenberg
16eac53dc9
tool_writeout: check strftime() return code
Because if it fails, the contents of the output buffer is undefined.

Pointed out by CodeSonar

Also polished the documentation

Follow-up to fadc487567

Closes #18220
2025-08-07 16:19:22 +02:00
Daniel Stenberg
3b40128b0f
curl: make global truly global
The GlobalConfig only exists in a single instance and it has worked like
this since the dawn of time. It is about time we stop passing around
pointers to what was already essentially a global object and instead
just use a... global.

It simplifies things.

Closes #18213
2025-08-07 10:43:06 +02:00
Daniel Stenberg
4b61e44e3d
tool_operate: use the correct config pointer
Triggered when doing 100+ parallel globbed upload transfers.

Reported-by: letshack9707 on hackerone

Closes #18200
2025-08-06 16:47:52 +02:00
Daniel Stenberg
034612cd51
tool_urlglob: polish, cleanups, improvements
- assert instead of printing "internal error" for unlikely events
- avoid allocating the main struct
- convert globerror() from macro to function
- renames to shorter and clearer names
- malloc + copy => memdup0
- change buffer handling to dynbuf
- realloc to handle more globs, but use less memory for few

Closes #18198
2025-08-06 08:47:53 +02:00
Daniel Stenberg
e1a0e2b72c
config2setopts: avoid curlx_dyn_addf(..., "%s"
Closes #18194
2025-08-06 08:32:07 +02:00
Daniel Stenberg
952117c032
tool_cb_hrd: remove global pointer from 'struct HdrCbData'
It is not necessary

Closes #18184
2025-08-05 15:47:56 +02:00
Daniel Stenberg
b3b7d65239
tool: replace three malloc + copy with memdup0
The function already existed for private use in var.c

Closes #18185
2025-08-05 15:46:51 +02:00
Daniel Stenberg
2978d33089
tool_cb_wrt: use dynbuf instead of "manual" malloc
When creating new file names for no-clobber

Closes #18182
2025-08-05 14:26:16 +02:00
Daniel Stenberg
fe4ade53b7
tool_cb_wrt: split out win_console() from tool_write_cb()
Closes #18180
2025-08-05 14:05:40 +02:00
Stefan Eissing
1ad2009ad6
multi: add new information extraction method
Adds `curl_off_t curl_multi_get_offt(CURLM *multi_handle, CURLMinfo_offt
info)` to the multi interface with enums:

* CURLMINFO_XFERS_CURRENT: current number of transfers
* CURLMINFO_XFERS_RUNNING: number of running transfers
* CURLMINFO_XFERS_PENDING: number of pending transfers
* CURLMINFO_XFERS_DONE: number of finished transfers to read
* CURLMINFO_XFERS_ADDED: total number of transfers added, ever

Add documentation for functions and info enums.

Add use in the curl command line tool to replace two static
variables counting the same "from the outside".

refs #17870
Closes #17992
2025-08-04 23:48:57 +02:00
Daniel Stenberg
fadc487567
writeout: add %time{}
Output the current UTC time using strftime format. %f is an extra curl
specific flag to output the microsecond fraction of the current second.

Verified by test 1981

Closes #18119
2025-08-04 23:45:48 +02:00
Petar Popovic
c85c2b7be7
tool_paramhlp: fix secs2ms()
- remove one zero from digs[5]
- remove size of size

Closes #18167
2025-08-04 23:26:32 +02:00
Daniel Stenberg
13cbabf05f
curl: add --follow
Makes curl follow redirects an act on the response code and change a
custom method accordingly, contrary to --location.

Potential future command line to send QUERY and following a redirect
according to the status code:

    curl -d "request-body" -X QUERY --follow https://example.com

add test 794,796,797

Assisted-by: Daniel Böhmer <post@daniel-boehmer.de>

Closes #16543
2025-08-04 17:12:14 +02:00
Viktor Szakats
1644a49ab8
configure: if no perl, disable unity and shell completion, related tidy ups
Also:
- GHA/linux, macos: test `install` with autotools too.
- GHA/linux, macos: enable fish and zsh completion in an autotools job.
- scripts: drop dynamic perl checks redundant after this patch.
- scripts: drop two interim variables.
- autotools: make `build-certs` target check perl first.
- autotools: replace `$(PERL)` with `@PERL@` to match the pattern used
  in most automake scripts. For consistency. This makes `PERL` be
  defined at configure-time, as opposed to make-time, for these cases
  now.

Closes #18141
2025-08-04 14:55:10 +02:00
Viktor Szakats
54da6f5a32
build: if no perl, fix to use the pre-built hugehelp, if present
- cmake: Before this patch a missing perl disabled the curl manual.
  After this patch, it automatically picks up a pre-built hugehelp,
  if present (= when building from a release tarball).
  Follow-up to 0035ff45c5 #16081

- autotools: Rework behavior when perl is missing. Before this patch
  it caused a hard error when docs/manual/ca-embed were enabled.
  Of these, docs were enabled by default. After this patch, doc
  generation is automatically skipped, with a warning. Manual generation
  falls back to using a pre-built hugehelp, or a stub if that's missing.
  CA-embed is automatically skipped, with a warning.
  Slight difference from cmake: When built with no perl and no pre-built
  hugehelp, the manual is enabled, but the content is empty; with cmake
  it's disabled proper.
  Follow-up to 137aecfbf1 #13514
  Follow-up to 541321507e #12857

Also:
- GHA/distcheck: verify if the pre-built hugehelp makes it into curl.
- GHA/distcheck: streamline disabling perl with autotools.
- GHA/distcheck: `--disable-docs` no longer required to build without perl.
  Passing it also implicitly disables the curl manual, which is
  undesired in these tests.
- cmake: drop redundant `HAVE_MANUAL_TOOLS` interim variable.
- cmake: move two `set()` lines close to their counterparts.

Follow-up to 25119fbaaa #18104

Closes #18118
2025-08-01 23:54:12 +02:00
Daniel Stenberg
4654493fed
curl: add --parallel-max-host to limit concurrent connections per host
Where 'host' is protocol + hostname + portnumber.

Closes #18052
2025-07-31 23:30:40 +02:00
Daniel Stenberg
4d025fd912
curl: make --retry-delay and --retry-max-time accept decimal seconds
Like other time options already do.

Reported-by: Alice Lee Poetics
Fixes #18109
Closes #18111
2025-07-31 08:55:58 +02:00
Daniel Stenberg
98bcb885fe
curl: make the URL indexes 64 bit
Otherwise we could misbehave already at 2 billion URLs and we can't have
that. A few of the counters are already correctly using the right type.

Closes #18096
2025-07-30 23:13:45 +02:00
Daniel Stenberg
cc27bb07a8
config2setopts: set more options unconditionally
This function would set some options to NULL and some not. This change
now more consistently set the value, even if NULL/0/default.

Also removes the 'new in [version]' comments as they were not
consistently used and as they have moved around probably aren't correct
anymore.

Closes #18078
2025-07-29 16:18:58 +02:00
Daniel Stenberg
b2ccfbf2fb
tool_operate: simplify single_transfer
- let the caller do the cleanup on fail
- avoid gotos and use direct returns more
- use while() loop

Closes #18077
2025-07-29 16:15:05 +02:00
Daniel Stenberg
a7e6c78bfa
parallel-max: bump the max value to 65535
When doing HTTP/2 and HTTP/3, it is possible to achieve quite a massive
parallelism so limiting this to 300 seems restrictive.

With other protocols, going beyond 300-400 might not be recommended but
curl does not have to enforce the limit.

Closes #18068
2025-07-29 09:11:19 +02:00
Stefan Eissing
6845533e24
curl: add long option '--out-null'
Add a new commandline option --out-null that discards all
response bytes into the void. Replaces non-portable use of
'-o /dev/null' with more efficiency.

Feature added in 8.16.0

Closes #17800
2025-07-28 14:57:38 +02:00
Viktor Szakats
430f9b03fd
CURLOPT: bump CURLFTP* enums to long, drop casts
This patch bumps the size of these constants from `int` to `long`, while
keeping their actual values the same. It may cause incompatibilities in
user code, requiring the bump of holder variables and/or adding casts:

- CURLFTP_CREATE_DIR
- CURLFTP_CREATE_DIR_NONE
- CURLFTP_CREATE_DIR_RETRY
- CURLFTPAUTH_DEFAULT
- CURLFTPAUTH_SSL
- CURLFTPAUTH_TLS
- CURLFTPMETHOD_DEFAULT
- CURLFTPMETHOD_MULTICWD
- CURLFTPMETHOD_NOCWD
- CURLFTPMETHOD_SINGLECWD
- CURLFTPSSL_CCC_ACTIVE
- CURLFTPSSL_CCC_NONE
- CURLFTPSSL_CCC_PASSIVE

Also:
- keep existing casts within the documentation to make sure it applies
  to older curl versions as well.

Closes #17797
2025-07-28 10:32:13 +02:00
Viktor Szakats
5debe7cb34
CURLOPT: drop redundant long casts
Also:
- CURLOPT_HSTS_CTRL.md: sync macro definitions with `curl/curl.h`.
  Perhaps it'd be better to delete copies like this?
- keep existing casts within the documentation to make sure it applies
  to older curl versions as well.
- CURLOPT_IPRESOLVE.md: re-add a long cast to man page, for consistency
  with the above.

Closes #17791
2025-07-28 10:32:13 +02:00
Viktor Szakats
fd2ca2399e
build: extend GNU C guards to clang where applicable, fix fallouts
Some GNU C version guards implicitly include the clang compiler, because
clang reports itself as GCC 4.2.1.

This implicit inclusion doesn't happen if the guard requires a GCC
version above 4.2.1.

Fix two such guards to explicitly include clang where it does support
the guarded feature:

- curl/curl.h: use `typecheck-gcc.h` with clang.
  llvm clang v14+ supports this. The corresponding Apple clang version
  is also v14.
  Ref: https://en.wikipedia.org/wiki/Xcode#Toolchain_versions
  Apple clang v14 tested OK in CI:
  https://github.com/curl/curl/actions/runs/16353901480/job/46207437204

- tool_urlglib: use `__builtin_mul_overflow()` with clang v8+.
  llvm clang v3.8+ supports this, but to accommodate for Apple clang,
  start with v8, the Apple version having the mainline v3.8 feature set.

Also fix compile warnings triggered by the above:
- lib1912: fix duplicate `;`:
  ```
  tests/libtest/lib1912.c:44:57: error: empty expression statement has no effect; remove unnecessary ';' to silence this warning [-Werror,-Wextra-semi-stmt]
   44 |       print_err(o->name, "CURLOT_LONG or CURLOT_VALUES");
      |                                                         ^
  [...]
  ```
  Ref: https://github.com/curl/curl/actions/runs/16351302841/job/46198524880?pr=17955#step:12:61

- lib2032: silence typcheck warning with a cast:
  ```
  tests/libtest/lib2032.c:145:29: error: sizeof on pointer operation will return size of 'CURL **' (aka 'void **') instead of 'CURL *[3]' (aka 'void *[3]') [-Werror,-Wsizeof-array-decay]
    145 |                   ntlm_easy + num_handles);
        |                   ~~~~~~~~~ ^
  ```
  Ref: https://github.com/curl/curl/actions/runs/16351302841/job/46198524880?pr=17955#step:12:86

Closes #17955
2025-07-27 23:02:05 +02:00
Viktor Szakats
2ec54556d4
scripts: enable strict warnings in Perl where missing, fix fallouts
- add 'use warnings' and 'use strict' where missing from Perl scripts.
- fix 'Use of uninitialized value'.
- fix missing declarations.
- test1140.pl: fix 'Possible precedence issue with control flow operator'.
- fix other misc issues.

Most actual errors found during this PR were fixed and merged via
separate PRs.

Likely there are remaining warnings not found and fixed in this PR.

Closes #17877
2025-07-27 22:35:18 +02:00
Viktor Szakats
89771d19d5
tidy-up: prefer ifdef/ifndef for single checks
Closes #18018
2025-07-27 22:35:17 +02:00
Viktor Szakats
b2bccdc257
tidy-up: move literal to the right side of comparisons
Closes #17876
2025-07-27 22:35:17 +02:00