Commit graph

657 commits

Author SHA1 Message Date
Daniel Stenberg
a300879b63
curl: return error if etag options are used with multiple URLs
And document it.

Add tests 484 and 485

Fixes #15729
Reported-by: Tamir Duberstein
Closes #15731
2024-12-13 10:11:26 +01:00
Daniel Stenberg
d5c738c608
strparse: string parsing helper functions
Designed to aid converting off from sscanf parsers. sscanf is hard to
use right, easy to mess up and often makes for sloppy error checking.

The new parsers allow more exact and pedandic parsing.

This new set of functions should be possible to use (and extend) and
switch over other libcurl parser code to use going forward.

Adapts the following to use the new functions:

- altsvc.c
- hsts.c
- http_aws_sigv4.c

Bonus: fewer memory copies, fewer stack buffers.

Test: Unit test1664

Docs: docs/internals/STRPARSE.md

Closes #15692
2024-12-12 16:00:52 +01:00
Daniel Stenberg
a8c852b9a5
cookie: parse only the exact expire date
The date parser function is very forgiving and skips most "irrelevant"
characters in its hunt for a date to figure out. Therefore it is
important to make sure the date string is properly null terminated so
that it does not accidentally parse a piece of whatever text follows
after the date.

Add test483: test (overly) long expire dates in cookies

Closes #15709
2024-12-11 09:31:53 +01:00
Daniel Stenberg
0169b80e5c
curl: --continue-at is mutually exclusive with --remove-on-error
Test 482 verifies

Fixes #15645
Reported-by: Harry Sintonen
Closes #15668
2024-12-02 21:22:57 +01:00
Daniel Stenberg
ffbcde0022
curl: --continue-at is mutually exclusive with --no-clobber
Test 481 verifies

Fixes #15645
Reported-by: Harry Sintonen
Closes #15668
2024-12-02 21:22:47 +01:00
Daniel Stenberg
e9b9bbac22
netrc: address several netrc parser flaws
- make sure that a match that returns a username also returns a
  password, that should be blank if no password is found

- fix handling of multiple logins for same host where the password/login
  order might be reversed.

- reject credentials provided in the .netrc if they contain ASCII control
  codes - if the used protocol does not support such (like HTTP and WS do)

Reported-by: Harry Sintonen

Add test 478, 479 and 480 to verify. Updated unit 1304.

Closes #15586
2024-11-17 11:33:56 +01:00
Daniel Stenberg
f5c616930b
duphandle: also init netrc
The netrc init was only done in the Curl_open, meaning that a duplicated
handle would not get inited properly.

Added test 2309 to verify. It does netrc auth with a duplicated handle.

Regression from 3b43a05e00

Reported-by: tranzystorekk on github
Fixes #15496
Closes #15503
2024-11-07 09:47:53 +01:00
Daniel Stenberg
9bee39bfed
url: use same credentials on redirect
Previously it could lose the username and only use the password.

Added test 998 and 999 to verify.

Reported-by: Tobias Bora
Fixes #15262
Closes #15282
2024-10-13 23:18:42 +02:00
Daniel Stenberg
5ee43bb82e
tests: 780 - 783, new HSTS tests
test780: verify updated HSTS data in response header

test781: HSTS update expiry, with parent includeSubDomains domain
present

test782: HSTS update expiry, with two includeSubDomains domains present

test783: HSTS update expiry, removing includesubdomains in update
2024-10-09 13:48:14 +02:00
Daniel Stenberg
b0c82239c2
tool: support --show-headers AND --remote-header-name
By keeping the headers in memory until we know the target file name,
then output them all.

Previously this option combination would cause an error.

Add test 1310 and 1492 to verify. Adjusted test 1460 to work in the new
conditions.

Closes #15110
2024-10-02 08:04:33 +02:00
Dan Fandrich
8e9a8dd978 tests: add file: tests with existing files
Windows sometimes has issues when opening the same file twice, so these
test two situations where that could potentially occur.

Reported-by: ralfjunker on github
Ref: #15043
Closes #15045
2024-09-26 10:10:36 -07:00
Daniel Stenberg
b4f7ec71ce
tool_operate: let --create-dirs work for --dump-header as well
Add test 3011 to verify

Proposed-by: Montg0mery on github
Fixes #14941
Closes #14965
2024-09-22 21:27:29 +02:00
Daniel Stenberg
0cfc7fcca1
tool_cb_wrt: use "curl_response" if no file name in URL
Use the same fallback for content-disposition cases as for regular -O

Add test692: verify -JO with URL without a file name

Reported-by: Brian Inglis
Fixes #14939
Closes #14940
2024-09-17 13:39:51 +02:00
Daniel Stenberg
aef384a7df
http: make max-filesize check not count ignored bodies
Add test 477 to verify

Reported-by: MasterInQuestion on github
Fixes #14899
Closes #14900
2024-09-14 22:51:57 +02:00
Viktor Szakats
805bbf7c55
NTLM_WB: delete remains in tests, docs updates
Also update more docs to reflect the removal of NTLM_WB.

Follow-up to 50def7c881 #13249
Closes #14832
2024-09-10 14:13:36 +02:00
Aki
a2bcec0ee0
openssl: fix the data race when sharing an SSL session between threads
The SSL_Session object is mutated during connection inside openssl,
and it might not be thread-safe. Besides, according to documentation
of openssl:

```
SSL_SESSION objects keep internal link information about the session
cache list, when being inserted into one SSL_CTX object's session
cache. One SSL_SESSION object, regardless of its reference count,
must therefore only be used with one SSL_CTX object (and the SSL
objects created from this SSL_CTX object).
```
If I understand correctly, it is not safe to share it even in a
single thread.

Instead, serialize the SSL_SESSION before adding it to the cache,
and deserialize it after retrieving it from the cache, so that no
concurrent write to the same object is infeasible.

Also
 - add a ci test for thread sanitizer
 - add a test for sharing ssl sessions concurrently
 - avoid redefining memory functions when not building libcurl, but
   including the soruce in libtest
 - increase the concurrent connections limit in sws

Notice that there are fix for a global data race for openssl which
is not yet release. The fix is cherry pick for the ci test with
thread sanitizer.
d8def79838

Closes #14751
2024-09-02 23:35:44 +02:00
Stefan Eissing
4cd10ee28b
POP3: fix multi-line responses
Some POP3 commands are multi-line, e.g. have responses terminated by a
last line with '.', but some are not. Define the known command
properties and fix response handling.

Add test case for STAT.

Fixes #14677
Reported-by: ralfjunker on github
Closes #14707
2024-08-30 15:38:25 +02:00
Daniel Stenberg
55672d0aa3
test476: test ASCII FTP upload where file already uses CRLF
Closes #14717
2024-08-29 20:24:34 +02:00
Daniel Stenberg
ee17f35d43
test475: verify a 72K ASCII FTP upload
Extended the test format and runtest.pl so that the verify/upload part
can be marked using crlf newlines even when the client/file does not
have it.

Closes #14717
2024-08-29 20:24:23 +02:00
Daniel Stenberg
5603204448
curl: fix the -w urle.* variables
urle.scheme, urle.user, urle.password and urle.options mistakenly
operated on the original URL instead of the *effective* (last) URL.

Add test 474 to verify.

Reported-by: Gruber Glass
Fixes #14550
Closes #14560
2024-08-15 14:01:24 +02:00
Austin Moore
cf3e3d93d1
aws_sigv4: fix canon order for headers with same prefix
If a request containing two headers that have equivalent prefixes (ex.
"x-amz-meta-test:test" and "x-amz-meta-test-two:test2") AWS expects the
header with the shorter name to come first. The previous implementation
used `strcmp` on the full header. Using the example, this would result
in a comparison between the ':' and '-' chars and sort
"x-amz-meta-test-two" before "x-amz-meta-test", which produces a
different "StringToSign" than the one calculated by AWS.

Test 1976 verifies

Closes #14370
2024-08-05 23:32:10 +02:00
Daniel Stenberg
732cb15b97
curl: add --skip-existing
With this option, the entire download is skipped if the selected target
filename already exists when the opertion is about to begin.

Test 994, 995 and 996 verify.

Ref: #11012
Closes #13993
2024-08-04 23:28:09 +02:00
Daniel Stenberg
9a0cf56471
curl: --help [option] displays documentation for given cmdline option
Since the documentation text blob might be gzipped, it needs to search
for what to output in a streaming manner. It then first searches for
"\nALL OPTIONS".

Then, it looks for the start to display at "\n    -[option]" and stops
again at "\n    -". Except for the last option in the man page, which
ends at "\nFILES" - the subtitle for the section following all options
in the manpage.

Test 1707 to 1710 verify

Closes #13997
2024-08-04 16:06:17 +02:00
Daniel Stenberg
9b1e4b4632
tool_operate: support --dump-header % to direct to stderr
Similar to how --trace and --trace-ascii already do it.

Added test 1489 to verify

Closes #13992
2024-08-04 16:02:24 +02:00
Daniel Stenberg
e26eefd9ce
tool_operate: for -O, use "default" as filename when the URL has none
... or pick the last directory part from the path if available.

Instead of returning error.

Add test 690 and 691 to verify. Test 76 and 2036 no longer apply.

Closes #13988
2024-08-04 15:59:51 +02:00
Viktor Szakats
f5b826532f
cmake: drop no-op tests/data/CMakeLists.txt
It's a no-op since
d162fca69a #9333 (2022-08-18).

Also revert 476499c75c that is
no longer necessary: move `Makefile.inc` back into `Makefile.am`.

Closes #14357
2024-08-03 01:15:45 +02:00
Viktor Szakats
4d34fd26db
autotools: fix typo in tests/data target
Ref: c1b369fd4c (2002-12-12)
Closes #14355
2024-08-02 14:36:16 +02:00
Daniel Stenberg
2bc1d775f5
copyright: update all copyright lines and remove year ranges
- they are mostly pointless in all major jurisdictions
- many big corporations and projects already don't use them
- saves us from pointless churn
- git keeps history for us
- the year range is kept in COPYING

checksrc is updated to allow non-year using copyright statements

Closes #10205
2023-01-03 09:19:21 +01:00
Daniel Stenberg
aedd0a7c9b
tests/data/Makefile: remove 'filecheck' target
No practical use anymore since 3c0f4622cd

Closes #9332
2022-08-18 11:39:12 +02:00
max.mehl
ad9bc5976d
copyright: make repository REUSE compliant
Add licensing and copyright information for all files in this repository. This
either happens in the file itself as a comment header or in the file
`.reuse/dep5`.

This commit also adds a Github workflow to check pull requests and adapts
copyright.pl to the changes.

Closes #8869
2022-06-13 09:13:00 +02:00
Daniel Stenberg
3c0f4622cd
runtests: parse data/Makefile.inc instead of using make
The warning about missing entries in that file then doesn't require that
the Makefile has been regenerated which was confusing.

The scan for the test num is a little more error prone than before
(since now it doesn't actually verify that it is legitimate Makefile
syntax), but I think it is good enough.

Closes #7177
2021-06-02 23:21:18 +02:00
Harry Sintonen
e4db6fb02c
filecheck: quietly remove test-place/*~
Closes #7179
2021-06-02 23:17:44 +02:00
Daniel Stenberg
4d2f800677
curl.se: new home
Closes #6172
2020-11-04 23:59:47 +01:00
Daniel Stenberg
9a8b3b3e13
copyright: fix out-of-date copyright ranges and missing headers
Reported by the new script 'scripts/copyright.pl'. The script has a
regex whitelist for the files that don't need copyright headers.

Removed three (mostly usesless) README files from docs/

Closes #5141
2020-03-24 15:05:59 +01:00
Roger Leigh
e3e06e1aee tests: Distribute CMakeLists.txt files in subdirectories 2015-06-25 07:48:50 -07:00
Peter Wu
476499c75c tests: move TESTCASES to Makefile.inc, add show for cmake
This change allows runtests.pl to be run from the CMake builddir:

    export srcdir=/tmp/curl/tests;
    perl -I$srcdir $srcdir/runtests.pl -l

In order to make this possible, all test cases have been moved from
Makefile.am to Makefile.inc.

Signed-off-by: Peter Wu <peter@lekensteyn.nl>
2014-10-13 11:50:21 +02:00
Daniel Stenberg
bf7023d165 test2036: verify -O with no slash at all in the URL
Similar to test 76 but that test's URL has a slash just no file name
part.
2014-10-09 16:50:26 +02:00
moparisthebest
93e450793c SSL: implement public key pinning
Option --pinnedpubkey takes a path to a public key in DER format and
only connect if it matches (currently only implemented with OpenSSL).

Provides CURLOPT_PINNEDPUBLICKEY for curl_easy_setopt().

Extract a public RSA key from a website like so:
openssl s_client -connect google.com:443 2>&1 < /dev/null | \
sed -n '/-----BEGIN/,/-----END/p' | openssl x509 -noout -pubkey \
| openssl rsa -pubin -outform DER > google.com.der
2014-10-07 14:44:19 +02:00
Steve Holme
8acbb074f8 tests: Added test1420 to the makefile 2014-09-07 12:13:34 +01:00
Dan Fandrich
ce6c691da1 test1398: Added test to Makefile.am 2014-06-15 12:28:43 +02:00
Daniel Stenberg
313b274b9d test1134: verify CREDSPERREQUEST for HTTP
Verifies that the change in 68f0166a92 works as intended and that
different HTTP auth credentials to the same host still re-uses the
connection properly.
2014-05-18 16:55:03 +02:00
Daniel Stenberg
2e57c7e0fc test325: verify --proto-redir https=>http 2014-04-23 22:36:01 +02:00
Daniel Stenberg
74851340bd PROXYHEADER: send these headers in "normal" proxy requests too
Updated the docs to clarify and the code accordingly, with test 1528 to
verify:

When CURLHEADER_SEPARATE is set and libcurl is asked to send a request
to a proxy but it isn't CONNECT, then _both_ header lists
(CURLOPT_HTTPHEADER and CURLOPT_PROXYHEADER) will be used since the
single request is then made for both the proxy and the server.
2014-04-04 17:03:43 +02:00
Daniel Stenberg
d3d27551e7 test1428: verify --proxy-header 2014-04-04 17:03:43 +02:00
Daniel Stenberg
ac887eedbc CURLOPT_PROXYHEADER: set headers for proxy-only
Includes docs and new test cases: 1525, 1526 and 1527

Co-written-by: Vijay Panghal
2014-04-04 17:03:43 +02:00
Daniel Stenberg
42937f87e6 HTTP: don't send Content-Length: 0 _and_ Expect: 100-continue
Without request body there's no point in asking for 100-continue.

Bug: http://curl.haxx.se/bug/view.cgi?id=1349
Reported-by: JimS
2014-04-04 15:46:35 +02:00
Steve Holme
b201d9adb8 tests: Added SMTP with credientials test when not supported by server 2014-04-01 07:19:29 +01:00
Steve Holme
dd07e79023 tests: Added SMTP with credentials test for RFC-821 based server
Added SMTP (RFC-821 only) based test case as a reference for the fix
provided by commit fe260b75e7.
2014-03-31 20:55:58 +01:00
Steve Holme
afbcfc4f46 tests: Added email unit tests to verify login credential connection re-use 2014-03-30 15:13:31 +01:00
Richard J. Moore
4d06b27921 test1397: unit test for certificate name wildcard handling 2014-03-25 23:01:37 +01:00