Replace bit mask protocol sets by null-terminated arrays of protocol
tokens. These are the addresses of the protocol names returned by
curl_version_info().
Protocol names are sorted case-insensitively before output to satisfy CI
tests matches consistency.
The protocol list returned by curl_version_info() is augmented with all
RTMP protocol variants.
Test 1401 adjusted for new alpha ordered output.
Closes#9546
As they are now rejected by the library, take care of not passing
disabled protocol names to CURLOPT_PROTOCOLS_STR and
CURLOPT_REDIR_PROTOCOLS_STR.
Rather than using the CURLPROTO_* constants, dynamically assign protocol
numbers based on the order they are listed by curl_version_info().
New type proto_set_t implements prototype bit masks: it should therefore
be large enough to accomodate all library-enabled protocols. If not,
protocol numbers beyond the bit count of proto_set_t are recognized but
"inaccessible": when used, a warning is displayed and the value is
ignored. Should proto_set_t overflows, enabled protocols are reordered to
force those having a public CURLPROTO_* representation to be accessible.
Code has been added to subordinate RTMP?* protocols to the presence of
RTMP in the enabled protocol list, being returned by curl_version_info()
or not.
Next Protocol Negotiation is a TLS extension that was created and used
for agreeing to use the SPDY protocol (the precursor to HTTP/2) for
HTTPS. In the early days of HTTP/2, before the spec was finalized and
shipped, the protocol could be enabled using this extension with some
servers.
curl supports the NPN extension with some TLS backends since then, with
a command line option `--npn` and in libcurl with
`CURLOPT_SSL_ENABLE_NPN`.
HTTP/2 proper is made to use the ALPN (Application-Layer Protocol
Negotiation) extension and the NPN extension has no purposes
anymore. The HTTP/2 spec was published in May 2015.
Today, use of NPN in the wild should be extremely rare and most likely
totally extinct. Chrome removed NPN support in Chrome 51, shipped in
June 2016. Removed in Firefox 53, April 2017.
Closes#9307
Use the correct type, and make cleanarg an empty macro if the cleaning
ability is absent.
Fixes#9195Closes#9196
Reviewed-by: Jay Satiro
Reviewed-by: Marcel Raad
Regression since 9e5669f.
Make sure the "cleaning" of command line arguments is done on the
original argv[] pointers. As a bonus, it also exits better on out of
memory error.
Reported-by: Litter White
Fixes#9128Closes#9130
... as replacements for deprecated CURLOPT_PROTOCOLS and
CURLOPT_REDIR_PROTOCOLS as these new ones do not risk running into the
32 bit limit the old ones are facing.
CURLINFO_PROTCOOL is now deprecated.
The curl tool is updated to use the new options.
Added test 1597 to verify the libcurl protocol parser.
Closes#8992
Add licensing and copyright information for all files in this repository. This
either happens in the file itself as a comment header or in the file
`.reuse/dep5`.
This commit also adds a Github workflow to check pull requests and adapts
copyright.pl to the changes.
Closes#8869
- Clamp --parallel-max to MAX_PARALLEL (300) instead of resetting to
default value.
Previously, --parallel-max 300 would use 300 concurrent transfers, but
--parallel-max 301 would unexpectedly use only 50. This change clamps
higher values to the maximum (ie --parallel-max 301 would use 300).
Closes https://github.com/curl/curl/pull/8930
param_place cannot be NULL here since we immediately efter this block
perform arithmetic on it (and use it in order to get here) so there is
little reason to check.
Closes: #8786
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
If a transfer returns an error, using this option makes curl remove the
leftover downloded (partial) local file before exiting.
Added test 376 to verify
Closes#8503
Adds these test cases:
383 - simple single command line option
384 - reading it from stdin
385 - getting two --json options on command line
386 - --next works after --json
Closes#8314
As the ps command may reveal sensitive command line info, obfuscate
options --tlsuser, --tlspasswd, --proxy-tlsuser, --proxy-tlspassword and
--oauth2-bearer arguments.
Reported-by: Stephen Boost <s.booth@epcc.ed.ac.uk>
Closes#7964
Until now, form field and file names where escaped using the
backslash-escaping algorithm defined for multipart mails. This commit
replaces this with the percent-escaping method for URLs.
As this may introduce incompatibilities with server-side applications, a
new libcurl option CURLOPT_MIME_OPTIONS with bitmask
CURLMIMEOPT_FORMESCAPE is introduced to revert to legacy use of
backslash-escaping. This is controlled by new cli tool option
--form-escape.
New tests and documentation are provided for this feature.
Reported by: Ryan Sleevi
Fixes#7789Closes#7805
Warning: this will make existing curl command lines that use metalink to
stop working.
Reasons for removal:
1. We've found several security problems and issues involving the
metalink support in curl. The issues are not detailed here. When
working on those, it become apparent to the team that several of the
problems are due to the system design, metalink library API and what
the metalink RFC says. They are very hard to fix on the curl side
only.
2. The metalink usage with curl was only very briefly documented and was
not following the "normal" curl usage pattern in several ways, making
it surprising and non-intuitive which could lead to further security
issues.
3. The metalink library was last updated 6 years ago and wasn't so
active the years before that either. An unmaintained library means
there's a security problem waiting to happen. This is probably reason
enough.
4. Metalink requires an XML parsing library, which is complex code (even
the smaller alternatives) and to this day often gets security
updates.
5. Metalink is not a widely used curl feature. In the 2020 curl user
survey, only 1.4% of the responders said that they'd are using it. In
2021 that number was 1.2%. Searching the web also show very few
traces of it being used, even with other tools.
6. The torrent format and associated technology clearly won for
downloading large files from multiple sources in parallel.
Cloes #7176
Reported by GCC analyzer:
Error: GCC_ANALYZER_WARNING (CWE-476):
src/tool_getparam.c: scope_hint: In function 'parse_args'
src/tool_getparam.c:2318:38: warning[-Wanalyzer-possible-null-dereference]: dereference of possibly-NULL 'orig_opt'
lib/curlx.h:56: included_from: Included from here.
src/tool_getparam.c:28: included_from: Included from here.
lib/curl_multibyte.h:70:51: note: in definition of macro 'curlx_convert_tchar_to_UTF8'
src/tool_getparam.c:2316:16: note: in expansion of macro 'curlx_convert_tchar_to_UTF8'
Reviewed-by: Marcel Raad
Reviewed-by: Daniel Stenberg
Closes#7023
- Disable auto credentials by default. This is a breaking change
for clients that are using it, wittingly or not.
- New libcurl ssl option value CURLSSLOPT_AUTO_CLIENT_CERT tells libcurl
to automatically locate and use a client certificate for
authentication, when requested by the server.
- New curl tool options --ssl-auto-client-cert and
--proxy-ssl-auto-client-cert map to CURLSSLOPT_AUTO_CLIENT_CERT.
This option is only supported for Schannel (the native Windows SSL
library). Prior to this change Schannel would, with no notification to
the client, attempt to locate a client certificate and send it to the
server, when requested by the server. Since the server can request any
certificate that supports client authentication in the OS certificate
store it could be a privacy violation and unexpected.
Fixes https://github.com/curl/curl/issues/2262
Reported-by: Jeroen Ooms
Assisted-by: Wes Hinsley
Assisted-by: Rich FitzJohn
Ref: https://curl.se/mail/lib-2021-02/0066.html
Reported-by: Morten Minde Neergaard
Closes https://github.com/curl/curl/pull/6673
Instead output a warning about it and continue with the defaults.
These SSL versions are typically not supported by the TLS libraries since a
long time back already since they are inherently insecure and broken. Asking
for them to be used will just cause an error to be returned slightly later.
In the unlikely event that a user's TLS library actually still supports these
protocol versions, this change might make the request a little less insecure.
Closes#6772
Previously only a single -b cookie parameter was supported with the last
one winning. This adds support for supplying multiple -b params to have
them serialized semicolon separated. Both cookiefiles and cookies can be
entered multiple times.
Closes#6649
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Previously an empty 'Referer:' header was added to the HTTP request when
passing `--referer ';auto'` or `--referer ''` on the command-line. This
patch makes `--referer` work like `--header 'Referer:'` and will only add
the header if it has a non-zero length value.
Reviewed-by: Jay Satiro
Closes#6610
- New libcurl options CURLOPT_DOH_SSL_VERIFYHOST,
CURLOPT_DOH_SSL_VERIFYPEER and CURLOPT_DOH_SSL_VERIFYSTATUS do the
same as their respective counterparts.
- New curl tool options --doh-insecure and --doh-cert-status do the same
as their respective counterparts.
Prior to this change DOH SSL certificate verification settings for
verifyhost and verifypeer were supposed to be inherited respectively
from CURLOPT_SSL_VERIFYHOST and CURLOPT_SSL_VERIFYPEER, but due to a bug
were not. As a result DOH verification remained at the default, ie
enabled, and it was not possible to disable. This commit changes
behavior so that the DOH verification settings are independent and not
inherited.
Ref: https://github.com/curl/curl/pull/4579#issuecomment-554723676
Fixes https://github.com/curl/curl/issues/4578
Closes https://github.com/curl/curl/pull/6597
- Add support services without region and service prefixes in
the URL endpoint (ex. Min.IO, GCP, Yandex Cloud, Mail.Ru Cloud Solutions, etc)
by providing region and service parameters via aws-sigv4 option.
- Add [:region[:service]] suffix to aws-sigv4 option;
- Fix memory allocation errors.
- Refactor memory management.
- Use Curl_http_method instead() STRING_CUSTOMREQUEST.
- Refactor canonical headers generating.
- Remove repeated sha256_to_hex() usage.
- Add some docs fixes.
- Add some codestyle fixes.
- Add overloaded strndup() for debug - curl_dbg_strndup().
- Update tests.
Closes#6524
We currently use both spellings the british "behaviour" and the american
"behavior". However "behavior" is more used in the project so I think
it's worth dropping the british name.
Closes#6395
This option sets the (octal) mode to use for the remote file when one is
created, using the SFTP, SCP or FILE protocols. When not set, the
default is 0644.
Closes#6244
- enable in the build (configure)
- header parsing
- host name lookup
- unit tests for the above
- CI build
- CURL_VERSION_HSTS bit
- curl_version_info support
- curl -V output
- curl-config --features
- CURLOPT_HSTS_CTRL
- man page for CURLOPT_HSTS_CTRL
- curl --hsts (sets CURLOPT_HSTS_CTRL and works with --libcurl)
- man page for --hsts
- save cache to disk
- load cache from disk
- CURLOPT_HSTS
- man page for CURLOPT_HSTS
- added docs/HSTS.md
- fixed --version docs
- adjusted curl_easy_duphandle
Closes#5896
Previously, it would pass on a define (int) which could make libcurl
read junk as a value - which prevented the CURLOPT_IPRESOLVE option to
"take". This could then make test 2100 do two DoH requests instead of
one!
Fixes#6042Closes#6043
... since it is converted and the original pointer is freed on Windows
unicode handling.
Follow-up to aa8777f63fFixes#5977Closes#5978
Reported-by: xwxbug on github
