Commit graph

278 commits

Author SHA1 Message Date
Daniel Stenberg
e2452cf60e
tool_getparam: error if --next is used without a prior URL
Reported-by: 積丹尼 Dan Jacobson
Ref: https://github.com/curl/curl/pull/10769#discussion_r1137895629

Closes #10782
2023-03-17 14:07:10 +01:00
Jay Satiro
2f17a9b654 tool: improve --stderr handling
- freopen stderr with the user-specified file (--stderr file) instead of
  using a separate 'errors' stream.

- In tool_setup.h override stdio.h's stderr macro as global variable
  tool_stderr.

Both freopen and overriding the stderr macro are necessary because if
the user-specified filename is "-" then stdout is assigned to
tool_stderr and no freopen takes place. See the PR for more information.

Ref: https://github.com/curl/curl/issues/10491

Closes https://github.com/curl/curl/pull/10673
2023-03-12 00:58:40 -05:00
Daniel Stenberg
780ff2c95c
tool_getparam: don't add a new node for just --no-remote-name
Unless --remote-name-all is used.

Fixes #10564
Reported-by: u20221022 on github
Closes #10582
2023-02-22 08:43:13 +01:00
Daniel Stenberg
6841f2ed5f
curl: make --silent work stand-alone
- renamed the struct field to 'silent' to match the cmdline option
- make --show-error toggle independently of --silent
- make --silent independent of ->noprogress as well

By doing this, the three options --silent, --no-progress-meter and
--show-error should work independently of each other and also work with
and without '--no-' prefix as documented.

Reported-by: u20221022 on github
Fixes #10535
Closes #10536
2023-02-17 09:22:49 +01:00
Daniel Stenberg
ce24ea373f
tool_getparam: make --get a true boolean
To match how it is documented in the man page.

Fixes #10527
Reported-by: u20221022 on github
Closes #10531
2023-02-16 10:53:59 +01:00
Daniel Stenberg
e6b72644c2
tool_getparam: fix compiler warning when !HAVE_WRITABLE_ARGV
Follow-up to 2ed0e1f70e

Closes #10286
2023-01-13 10:39:23 +01:00
Daniel Stenberg
13991d60ee
src: add --http3-only
Warning: --http3 and --http3-only are subject to change again (or be
removed) before HTTP/3 support goes non-experimental.

Closes #10264
2023-01-12 16:18:05 +01:00
Harry Sintonen
2ed0e1f70e
tool_getparam: fix hiding of command line secrets
Closes #10276
2023-01-12 14:09:23 +01:00
Daniel Stenberg
2bc1d775f5
copyright: update all copyright lines and remove year ranges
- they are mostly pointless in all major jurisdictions
- many big corporations and projects already don't use them
- saves us from pointless churn
- git keeps history for us
- the year range is kept in COPYING

checksrc is updated to allow non-year using copyright statements

Closes #10205
2023-01-03 09:19:21 +01:00
Daniel Stenberg
e9c580de4e
lib: parse numbers with fixed known base 10
... instead of using 0 argument that allows decimal, hex or octal when
the number is documented and assumed to use base 10.

Closes #9933
2022-11-17 23:15:37 +01:00
Patrick Monnerat
184fc6f07a
tool: use feature names instead of bit mask, when possible
If the run-time libcurl is too old to support feature names, the name
array is created locally from the bit masks. This is the only sequence
left that uses feature bit masks.

Closes #9583
2022-11-14 09:19:10 +01:00
Daniel Stenberg
0f860da4b6
tool_getparam: make --no-get work as the opposite of --get
... as documented.

Closes #9891
2022-11-12 00:13:29 +01:00
Daniel Stenberg
b6e1afd069
curl: add --url-query
This option adds a piece of data, usually a name + value pair, to the
end of the URL query part. The syntax is identical to that used for
--data-urlencode with one extension:

If the argument starts with a '+' (plus), the rest of the string is
provided as-is unencoded.

This allows users to "build" query parts with options and URL encoding
even when not doing GET requests, which the already provided option -G
(--get) is limited to.

This idea was born in a Twitter thread.

Closes #9691
2022-11-08 14:37:34 +01:00
Daniel Stenberg
52cc4a85fd
style: use space after comment start and before comment end
/* like this */

/*not this*/

checksrc is updated accordingly

Closes #9828
2022-10-30 22:31:29 +01:00
Daniel Stenberg
a55256cfb2
curl: timeout in the read callback
The read callback can timeout if there's nothing to read within the
given maximum period. Example use case is when doing "curl -m 3
telnet://example.com" or anything else that expects input on stdin or
similar that otherwise would "hang" until something happens and then not
respect the timeout.

This fixes KNOWN_BUG 8.1, first filed in July 2009.

Bug: https://sourceforge.net/p/curl/bugs/846/

Closes #9815
2022-10-28 17:57:14 +02:00
Rickard Hallerbäck
703efb3379
tool_paramhlp: make the max argument a 'double'
To fix compiler warnings "Implicit conversion from 'long' to 'double'
may lose precision"

Closes #9700
2022-10-16 00:46:52 +02:00
Daniel Stenberg
1527dd07f1
tool_getparam: pass in the snprintf("%.*s") string length as 'int'
Reported by Coverity CID 1515928

Closes #9679
2022-10-10 08:42:48 +02:00
Daniel Stenberg
b90f857fab
tool_getparam: split out data_urlencode() into its own function
Closes #9673
2022-10-08 11:48:20 +02:00
Patrick Monnerat
677266c769
tool: remove protocol count limitation
Replace bit mask protocol sets by null-terminated arrays of protocol
tokens. These are the addresses of the protocol names returned by
curl_version_info().

Protocol names are sorted case-insensitively before output to satisfy CI
tests matches consistency.

The protocol list returned by curl_version_info() is augmented with all
RTMP protocol variants.

Test 1401 adjusted for new alpha ordered output.

Closes #9546
2022-09-22 13:49:10 +02:00
Patrick Monnerat
dd2a024323
cli tool: do not use disabled protocols
As they are now rejected by the library, take care of not passing
disabled protocol names to CURLOPT_PROTOCOLS_STR and
CURLOPT_REDIR_PROTOCOLS_STR.

Rather than using the CURLPROTO_* constants, dynamically assign protocol
numbers based on the order they are listed by curl_version_info().

New type proto_set_t implements prototype bit masks: it should therefore
be large enough to accomodate all library-enabled protocols. If not,
protocol numbers beyond the bit count of proto_set_t are recognized but
"inaccessible": when used, a warning is displayed and the value is
ignored. Should proto_set_t overflows, enabled protocols are reordered to
force those having a public CURLPROTO_* representation to be accessible.

Code has been added to subordinate RTMP?* protocols to the presence of
RTMP in the enabled protocol list, being returned by curl_version_info()
or not.
2022-09-16 23:29:08 +02:00
Daniel Stenberg
79e404d039
curl: warn for --ssl use, considered insecure
Closes #9519
2022-09-16 21:27:20 +02:00
Daniel Stenberg
a850b9edb9
getparameter: return PARAM_MANUAL_REQUESTED for -M even when disabled
... to improve the output in this situation. Now it doesn't say "option
unknown" anymore.

Closes #9485
2022-09-14 08:04:39 +02:00
Daniel Stenberg
472f1cbe7e
NPN: remove support for and use of
Next Protocol Negotiation is a TLS extension that was created and used
for agreeing to use the SPDY protocol (the precursor to HTTP/2) for
HTTPS. In the early days of HTTP/2, before the spec was finalized and
shipped, the protocol could be enabled using this extension with some
servers.

curl supports the NPN extension with some TLS backends since then, with
a command line option `--npn` and in libcurl with
`CURLOPT_SSL_ENABLE_NPN`.

HTTP/2 proper is made to use the ALPN (Application-Layer Protocol
Negotiation) extension and the NPN extension has no purposes
anymore. The HTTP/2 spec was published in May 2015.

Today, use of NPN in the wild should be extremely rare and most likely
totally extinct. Chrome removed NPN support in Chrome 51, shipped in
June 2016. Removed in Firefox 53, April 2017.

Closes #9307
2022-09-05 07:39:02 +02:00
Daniel Stenberg
5b059ba895
getparam: correctly clean args
Follow-up to bf7e887b24

The previous fix for #9128 was incomplete and caused #9397.

Fixes #9397
Closes #9399
2022-08-31 01:03:36 +02:00
Daniel Stenberg
f7e14fee68
tool_getparam: make --doh-url "" switch it off
A possible future addition could be to parse the URL first too to verify
that it is valid before trying to use it.

Assisted-by: Jay Satiro
Closes #9207
2022-07-27 10:56:13 +02:00
Daniel Stenberg
206550a9c2
tool_getparam: fix cleanarg() for unicode builds
Use the correct type, and make cleanarg an empty macro if the cleaning
ability is absent.

Fixes #9195
Closes #9196

Reviewed-by: Jay Satiro
Reviewed-by: Marcel Raad
2022-07-25 23:58:43 +02:00
Daniel Stenberg
bf7e887b24
tool_getparam: repair cleanarg
Regression since 9e5669f.

Make sure the "cleaning" of command line arguments is done on the
original argv[] pointers. As a bonus, it also exits better on out of
memory error.

Reported-by: Litter White
Fixes #9128
Closes #9130
2022-07-10 15:49:14 +02:00
Daniel Stenberg
e6f8445ede
setopt: add CURLOPT_PROTOCOLS_STR and CURLOPT_REDIR_PROTOCOLS_STR
... as replacements for deprecated CURLOPT_PROTOCOLS and
CURLOPT_REDIR_PROTOCOLS as these new ones do not risk running into the
32 bit limit the old ones are facing.

CURLINFO_PROTCOOL is now deprecated.

The curl tool is updated to use the new options.

Added test 1597 to verify the libcurl protocol parser.

Closes #8992
2022-07-04 08:35:09 +02:00
max.mehl
ad9bc5976d
copyright: make repository REUSE compliant
Add licensing and copyright information for all files in this repository. This
either happens in the file itself as a comment header or in the file
`.reuse/dep5`.

This commit also adds a Github workflow to check pull requests and adapts
copyright.pl to the changes.

Closes #8869
2022-06-13 09:13:00 +02:00
Boris Verkhovskiy
83ee5c428d
curl: re-enable --no-remote-name
Closes #8931
2022-05-31 13:23:22 +02:00
JustAnotherArchivist
10cd69623a tool_getparam: fix --parallel-max maximum value constraint
- Clamp --parallel-max to MAX_PARALLEL (300) instead of resetting to
  default value.

Previously, --parallel-max 300 would use 300 concurrent transfers, but
--parallel-max 301 would unexpectedly use only 50. This change clamps
higher values to the maximum (ie --parallel-max 301 would use 300).

Closes https://github.com/curl/curl/pull/8930
2022-05-28 03:23:57 -04:00
Daniel Stenberg
8f48b5d783
curl: add --rate to set max request rate per time unit
--rate "12/m" - for 12 per minute or
--rate "5/h" - for 5 per hour

Removed from TODO

Closes #8671
2022-05-23 17:59:56 +02:00
Daniel Stenberg
84052154f7
curl: deprecate --random-file and --egd-file
As libcurl no longer has any functionality for them, the tool now does
nothing with them.

Closes #8670
2022-05-17 11:06:07 +02:00
Daniel Gustafsson
79bf349a50 tool_getparam: simplify conditional statement
param_place cannot be NULL here since we immediately efter this block
perform arithmetic on it (and use it in order to get here) so there is
little reason to check.

Closes: #8786
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
2022-05-02 14:04:59 +02:00
Daniel Stenberg
a09a12806e
tool_getparam: error out on missing -K file
Add test 411 to verify.

Reported-by: Median Median Stride
Bug: https://hackerone.com/reports/1542881
Closes #8731
2022-04-25 11:38:50 +02:00
Ian Blanes
9e5bd9ba19
curl: fix segmentation fault for empty output file names.
Function glob_match_url set *result to NULL when called with filename =
"", producing an indirect NULL pointer dereference.

Closes #8606
2022-03-28 09:39:09 +02:00
HexTheDragon
1831a6e7f1
curl: add --no-clobber
Does not overwrite output files if they already exist

Closes #7708
Co-authored-by: Daniel Stenberg
2022-03-11 08:38:01 +01:00
Daniel Stenberg
08a96c6e4e
curl: add --remove-on-error
If a transfer returns an error, using this option makes curl remove the
leftover downloded (partial) local file before exiting.

Added test 376 to verify

Closes #8503
2022-03-10 19:57:55 +01:00
Daniel Stenberg
2610142139
lib: remove support for CURL_DOES_CONVERSIONS
TPF was the only user and support for that was dropped.

Closes #8378
2022-02-04 08:05:35 +01:00
Daniel Stenberg
32160cae84
tool_getparam: initial --json support
Adds these test cases:

 383 - simple single command line option
 384 - reading it from stdin
 385 - getting two --json options on command line
 386 - --next works after --json

Closes #8314
2022-02-01 10:39:55 +01:00
Daniel Stenberg
0041fe7ef5
tool_getparam: DNS options that need c-ares now fail without it
Just silently accepting the options and then not having any effect is
not good.

Ref: #8283
Closes #8285
2022-01-17 08:44:41 +01:00
Patrick Monnerat
b6acbdeb63
curl tool: erase some more sensitive command line arguments
As the ps command may reveal sensitive command line info, obfuscate
options --tlsuser, --tlspasswd, --proxy-tlsuser, --proxy-tlspassword and
--oauth2-bearer arguments.

Reported-by: Stephen Boost <s.booth@epcc.ed.ac.uk>

Closes #7964
2022-01-10 15:49:25 +01:00
Patrick Monnerat
b20b364764
mime: use percent-escaping for multipart form field and file names
Until now, form field and file names where escaped using the
backslash-escaping algorithm defined for multipart mails. This commit
replaces this with the percent-escaping method for URLs.

As this may introduce incompatibilities with server-side applications, a
new libcurl option CURLOPT_MIME_OPTIONS with bitmask
CURLMIMEOPT_FORMESCAPE is introduced to revert to legacy use of
backslash-escaping. This is controlled by new cli tool option
--form-escape.

New tests and documentation are provided for this feature.

Reported by: Ryan Sleevi
Fixes #7789
Closes #7805
2021-11-15 10:40:03 +01:00
Борис Верховский
8758a26f88
curl: actually append "-" to --range without number only
Closes #7837
2021-10-11 22:51:13 +02:00
Mats Lindestam
d1e7d9197b
libssh2: add SHA256 fingerprint support
Added support for SHA256 fingerprint in command line curl and in
libcurl.

Closes #7646
2021-09-26 23:20:53 +02:00
a1346054
7cf5e8e70e
misc: fix typos in docs and comments
No user facing output from curl/libcurl is changed by this, just
comments.

Closes #7747
2021-09-23 12:57:55 +02:00
Daniel Stenberg
2b99f5e02c
parse_args: redo the warnings for --remote-header-name combos
... to avoid the memory leak risk pointed out by scan-build.

Follow-up from 7a3e981781

Closes #7698
2021-09-10 17:02:21 +02:00
Cao ZhenXiang
7a3e981781
curl: add warning for incompatible parameters usage
--continue-at - and --remote-header-name are known incompatible parameters

Closes #7674
2021-09-06 16:59:08 +02:00
Daniel Stenberg
5ceb83ff6c
getparameter: fix the --local-port number parser
It could previously get tricked into parsing the uninitialized stack
based buffer.

Reported-by: Brian Carpenter
Closes #7582
2021-08-17 10:58:22 +02:00
Daniel Stenberg
265b14d6b3
metalink: remove
Warning: this will make existing curl command lines that use metalink to
stop working.

Reasons for removal:

1. We've found several security problems and issues involving the
   metalink support in curl. The issues are not detailed here. When
   working on those, it become apparent to the team that several of the
   problems are due to the system design, metalink library API and what
   the metalink RFC says. They are very hard to fix on the curl side
   only.

2. The metalink usage with curl was only very briefly documented and was
   not following the "normal" curl usage pattern in several ways, making
   it surprising and non-intuitive which could lead to further security
   issues.

3. The metalink library was last updated 6 years ago and wasn't so
   active the years before that either. An unmaintained library means
   there's a security problem waiting to happen. This is probably reason
   enough.

4. Metalink requires an XML parsing library, which is complex code (even
   the smaller alternatives) and to this day often gets security
   updates.

5. Metalink is not a widely used curl feature. In the 2020 curl user
   survey, only 1.4% of the responders said that they'd are using it. In
   2021 that number was 1.2%. Searching the web also show very few
   traces of it being used, even with other tools.

6. The torrent format and associated technology clearly won for
   downloading large files from multiple sources in parallel.

Cloes #7176
2021-06-07 08:14:25 +02:00