With this change, the argument passed to the CURLOPT_FOLLOWLOCATION
option is now instead a "mode" instead of just a boolean. Documentation
is extended to describe the two new modes.
Test 1571 to 1581 verify.
Closes#16473
Further testing with timeouts in event based processing revealed that
our current shutdown handling in the connection pool was not clear
enough. Graceful shutdowns can only happen inside a multi handle and it
was confusing to track in the code which situation actually applies. It
seems better to split the shutdown handling off and have that code
always be part of a multi handle.
Add `cshutdn.[ch]` with its own struct to maintain connections being
shut down. A `cshutdn` always belongs to a multi handle and uses that
for socket/timeout monitoring.
The `cpool`, which can be part of a multi or share, either passes
connections to a `cshutdn` or terminates them with a one-time, best
effort.
Add an `admin` easy handle to each multi and share. This is used to
perform all maintenance operations where no "real" easy handle is
available. This solves the problem that the multi admin handle requires
some additional initialisation (e.g. timeout list).
The share needs its admin handle as it is often cleaned up when no other
transfer or multi handle exists any more. But we need a `data` in almost
every call.
Fix file:// handling of errors when adding a new connection to the pool.
Changes in `curl` itself:
- for parallel transfers, do not set a connection pool in the share,
rely on the multi's connection pool instead. While not a requirement
for the new `cshutdn` to work, this is
a) helpful in testing to trigger graceful shutdowns
b) a broader code coverage of libcurl via the curl tool
- on test_event with uv, cleanup the multi handle before returning from
parallel_event(). The uv struct is on the stack, cleanup of the multi
later will crash when it tries to register sockets. This is a "eat
your own dogfood" related fix.
Closes#16508
This allows you to use the `certs` and `num_certs` writeout variables in
the curl tool, and getting information about the server certificates
using CURLINFO_CERTINFO.
Closes#16459
Support multi-target cmake builds via `CURL_DIRSUFFIX` env. For example:
`export CURL_DIRSUFFIX=Debug/`.
Multi-target generators place their output to `src/<subdir>/`,
`lib/<subdir>/`, `tests/server/<subdir>`, `tests/libtest/<subdir>` and
`tests/unit/<subdir>/` by default. Before this patch, `runtests.pl`
couldn't run on such builds because it expected the binaries under the
their `<subdir>`-less directories. This patch allows to set such subdir
and make `runtests.pl` find the binaries. In CI we use multi-target
builds with tests for MSVC. It also helps Xcode-generator builds, though
in CI we don't have such job running tests.
There may be better solutions to configure this, but passing a custom
value to `runtests.pl` including its subprocesses is somewhat tricky.
The reason the configuration value expects the slash at the end is
because MSYS is automagically expanding the env to a (wrong) absolute
path if the slash is in the front.
Also:
- drop the `-DCMAKE_RUNTIME_OUTPUT_DIRECTORY_*=` workaround from CI.
- replace `resolve` references in tests with a new `%RESOLVE` variable.
It didn't use a filename extension before. After this patch it uses
`exe_ext('TOOL')`. I'm not sure if this is the correct choice vs.
`exe_ext('SRV')`.
- fix `-c` option format in manual.
- fix some whitespace.
Note, in CI we still tweak `CMAKE_RUNTIME_OUTPUT_DIRECTORY_*` in jobs
which share steps between `./configure` and cmake. It's easier that way.
Ref: #15000
Cherry-picked from #16394Closes#16452
This now insists on using a server auth option unless --insecure is
provided. As an added bonus, it now also only checks for the knownhosts
file once (if found).
Ref: #16197Closes#16205
- make it only accept version 1.0, as that is the version curl supports
- convert the parser to use strparse
- the status code max is now 999, but it does allow != 3 digits
Closes#16435
Verified in test 455 and 487.
If the provided string cannot be base64-decoded, it will instead use
"[64dec-fail]" (without the quotes).
Documented
Ref: #16288Closes#16330
Around 2.3x speed-up parsing many large hexadecimal numbers. The decimal and
octal parser get marginally faster.
Still very readable, compact and easy to follow code.
Tweaks
- combine the max and the overflow check, gains 3ns/num (use a separate
check outside of the loop instead for max < base)
- one less indirection in the pointer, gains 3ns/num
- using the table lookup for hex nums, gains 5ns/num
- unfold the num_digit() macro, gains 3s/num
- use the hexasciitable unconditionally, gains 2ns/num
- use post-increment pointer in the table lookup, gains 1ns/num
- improved valid_digit() using the table for the hex case,
gains 26 ns/num
- use "max char" in valid_digit(), gains 3ns/num
Behavior changes:
- no longer returns STRE_TOO_BIG - only STRE_OVERFLOW
- does not move the char ** on error, which is probably better
Updated and extended test 1664 (significantly).
Closes#16374
- add hex and octal parsers to the Curl_str_* family
- make curlx_strtoofft use these parsers
- remove all use of strtol() and strtoul() in library code
- generally use Curl_str_* more than strtoofft, for stricter parsing
- supports 64-bit universally, instead of 'long' which differs in size
between platforms
Extended the unit test 1664 to verify hex and octal parsing.
Closes#16336
Instead of strtoul() and strtol() calls.
Easier API with better integer overflow detection and built-in max check
that now comes automatic everywhere this is used.
Closes#16319
Extend output filter to pick up backslashes. This makes them pass in CI
when run in the vcpkg MSVC job, for example.
Also:
- cmake: allow TrackMemory, aka `ENABLE_CURLDEBUG` again. Drop FIXME.
- cmake: drop stale TODO.
- runtests: include the word 'TrackMemory' in the message disabling it.
Follow-up to 9f23c8f201#14541
Follow-up to 94c596bbc5#16283Closes#16289
Keeping the relevant 'ssl_scache' in 'data->state' leads to problems
when the owner of the cache is cleaned up and this reference is left
dangling.
Remove the ref entirely and always find the ssl_scache at the current
share or multi.
Folded in #16260 (test 3208) to verify this fixes the bug with a
dangling reference when an easy handle is used with easy_perform first
and in a multi_perform after.
Ref: #16236Closes#16261
Sync the test path with test1515. If fixes the test when the perl tool
resides on a path with spaces in it. E.g. when using the perl from Git
for Windows. This is a workaround, there may be a better fix this
type of issue.
Similar fix for test1515: 38b055446a
Failure:
```
test 1516...[caching of manual libcurl DNS entries after dead connection]
lib1515.exe returned 3, when expecting 0
1516: exit FAILED
=== Start of file stderr1516
Test: lib1515
[...]
17:59:32.390000 == Info: Expire cleared
request http://testserver.example.com:63621/C:/Program Files/Git/path/15160001 failed with 3
Test ended with result 3
=== End of file stderr1516
```
Ref: https://github.com/curl/curl/actions/runs/13184790755/job/36804217128?pr=16217#step:13:2805Closes#16223
Made the netrc parser return a more consistent set of error codes and
error messages, and also return error properly if the .netrc file is
missing.
Add test 697 to verify error on missing netrc file.
Fixes#16163
Reported-by: arlt on github
Closes#16165
For ASN.1 tags with indefinite length, curl's own parser for TLS
backends that do not support certificate inspection calls itself
recursively. A malicious server certificate can then lead to high
recursion level exhausting the stack space.
This PR limits the recursion level to 16 which should be safe on all
architectures.
Added unit test 1657 to verify behaviour.
Fixes#16135
Reported-by: z2_
Closes#16137
RFC 6455 Section 5.2 notes that for bits RSV1, RSV2, and RSV3 of the
framing header, a non-zero value that is not defined by a negotiated
extension MUST Fail the WebSocket connection.
Test 2310 verifies
Closes#16069
- Detach and disconnect an attached connection before performing.
Prior to this change it was not possible to safely reuse an easy handle
with an attached connection in a second call to curl_easy_perform. The
only known case of this is a connect-only type handle where the
connection was detached when curl_easy_perform returned, only to be
reattached by either curl_easy_send/recv.
This commit effectively reverts 2f8ecd5d and be82a360, the latter of
which treated the reuse as an error. Prior to that change undefined
behavior may occur in such a case.
Bug: https://curl.se/mail/lib-2025-01/0044.html
Reported-by: Aleksander Mazur
Closes https://github.com/curl/curl/pull/16008
- Change the swsbounce keyword to override the part number on a
subsequent request to the previous part number + 1.
Note the previous part number in this case is the part number that
was returned as a response to the previous request and contained
the swsbounce keyword.
Prior to this change swsbounce incremented the part number of the
subsequent request instead of overriding it, and did so in a more
limited fashion that prevented chaining swsbounce in multiple responses.
For example, if the test makes a request that causes the sws server to
return `<data>` as a response and that response contains `swsbounce`
then for the next response the sws server returns `<data1>`. If
`<data1>` also contains `swsbounce` then for the next response the sws
server now returns `<data2>` instead of the requested part.
Fixes https://github.com/curl/curl/discussions/16074
Closes https://github.com/curl/curl/pull/16085
Allow building with c-ares and yet use threaded resolver for the main
host A/AAAA resolving:
`--with-ares` provides the c-ares install path and defaults to use
c-ares for name resolving
`--with-threaded-resolver` still uses c-ares in the build (for HTTPS)
but uses the threaded resolver for "normal" resolves.
It works similarly for cmake: ENABLE_ARES enables ares, and if
ENABLE_THREADED_RESOLVER also is set, c-ares is used for HTTPS RR and
the threaded resolver for "normal" resolves.
HTTPSRR and c-ares-rr are new features return by curl_version_info() and
thus shown by curl -V.
The c-ares-rr feature bit is there to make it possible to distinguish
between builds using c-ares for all name resolves and builds that use
the threaded resolves for the regular name resolves and c-ares for
HTTPSRR only. "c-ares-rr" means it does not use c-ares for "plain" name
resolves.
HTTPSRR support is EXPERIMENTAL only.
Closes#16054
The variable `conn->httpversion` was used for several purposes and it
was unclear at which time the value represents what.
- rename `conn->httpversion` to `conn->httpversion_seen`
This makes clear that the variable only records the last
HTTP version seen on the connection - if any. And that it
no longer is an indication of what version to use.
- Change Alt-Svc handling to no longer modify `conn->httpversion`
but set `data->state.httpwant` for influencing the HTTP version
to use on a transfer.
- Add `data->req.httpversion_sent` to have a record of what
HTTP version was sent in a request
- Add connection filter type CF_TYPE_HTTP
- Add filter query `CF_QUERY_HTTP_VERSION` to ask what HTTP
filter version is in place
- Lookup filters HTTP version instead of using `conn->httpversion`
Test test_12_05 now switches to HTTP/1.1 correctly and the
expectations have been fixed.
Removed the connection fitler "is_httpN()" checks and using
the version query instead.
Closes#16073
With tests. (Most SFTP tests fail with wolfSSH.)
Also:
- disable pseudo-terminal in wolfSSH builds (also in CircleCI).
Not needed for curl.
- test642: add "compressed" to the description.
To avoid being the same as test600.
Follow-up to b7b4dc0d49#1735Closes#16025
When enabled in the build.
Update test2100: verify with HTTPS RR included
Adjust runtests and server/disabled.c to include "HTTPSRR" as a feature
in the test suite.
Also, decode the ALPN list in HTTPS records straight into IDs. There's
no point in storing everything in string format. Skip ALPNs we do not
support.
Closes#16007
This typically happens if CURL_CONNECT_ONLY is used and a second
curl_easy_perform() is attempted.
A connection "taken over" with CURL_CONNECT_ONLY cannot be ended any
other way than a curl_easy_cleanup() on the easy handle that holds it.
Add test 696 to verify.
Closes#16003
- Change setopt and pretransfer to always reset URL related variables
for a CURLU handle set CURLOPT_CURLU.
This change is to ensure we are in compliance with the doc which says
CURLU handles must be able to override a URL set via CURLOPT_URL and
that if the contents of the CURLU handle changes between transfers then
the updated contents must be used.
Prior to this change, although subsequent transfers appear to be
performed correctly in those cases, the work URL `data->state.url` was
not updated. CURLINFO_EFFECTIVE_URL returns data->state.url to the user
so it would return the URL from the initial transfer which was the wrong
URL. It's likely there are other cases as well.
Ref: https://curl.se/libcurl/c/CURLOPT_CURLU.html
Reported-by: Nicolás San Martín
Fixes https://github.com/curl/curl/issues/15984
Closes https://github.com/curl/curl/pull/15985
The pending cookie RFC update (currently known as 6265bis draft-19) says
Let cookie-age-limit be the maximum age of the cookie (which name of
Max-Age and an attribute-value of expiry-time. SHOULD be 400 days or
less.
This change makes received cookies over the wire get capped to 400 days.
It does not cap the expiry date of cookies loaded from file.
It does this by rounding the expire time to a even minute. This, to
allow the test suite to do the same and have a chance to get the same
number for stable testing without requiring a debug build.
The test script generates TWO numbers in the output file for each
%days[] used in the input test file, and the function that subsequently
compares and verifies output is fine with *either* of the two numbers.
This is done so that if the test case is generated the second
immediately before curl runs, that updated expiry number is also deemed
okay. It still checks for an exact match of either number.
Closes#15937
Adds the experimental feature `ssls-export` to libcurl and curl for
importing and exporting SSL sessions from/to a file.
* add functions to libcurl API
* add command line option `--ssl-sessions <filename>` to curl
* add documenation
* add support in configure
* add support in cmake
+ add pytest case
Closes#15924
Subparts may have been previously used as a top-level mime structure and
thus not rewound.
New test 695 checks the proper functioning in these particular conditions.
Reported-by: Qriist on github
Fixes#15842Closes#15911
- Treat negative Retry-After date-based times as 0.
- Treat Retry-After times greater than 6 hours as 6 hours.
Prior to this change Retry-After did not have a limited range and the
server could have set a time greater than 6 hours or a date in the past
that would result in a negative time, either of which may be unexpected
by the user.
The 6 hour limit is purposely not documented so that it can be changed
in the future if necessary.
Closes https://github.com/curl/curl/pull/15833
Follow-up to 40c264db61 after discussions on IRC.
The new style is
name[0-99]=contents
and
name[0-99]@filename
A) This does not cause the same problems with old curl versions trying
the new syntax as this way will cause old curls just fail with syntax
error and not risk using the wrong file.
B) Adds the same byte range support for "normal" assigns, which the
previous syntax did not. Thus lets a user get a partial content of a
variable etc.
Added test 790 and 791 to verify non-file assigns with ranges.
Closes#15862
Use 'banfunc' and 'allowfunc' in .checksrc to specify which functions to
ban or allow to be used. This saves us from having to edit the script
going forward when we want to ban or allow specific functions.
This replaces a set of previous rules and all banned functions are now
checked with the BANNEDFUNC rule.
There is a set of default banned functions, shown by invoking
./checksrc.
Also, -a and -b options are added to specify allowed or banned functions
on the command line.
Closes#15835
The redirect logic was broken when the redirect-to URL was a relative
URL only as a fragment or query (starting with '#' or '?').
Extended test 1560 to reproduce, then verify.
Reported-by: Jeroen Ooms
Fixes#15836Closes#15848
- Parse the input string without modifying it.
Prior to this change a segfault could occur if the input string was
const because the tokenizer modified the input string. For example if
the user set CURLOPT_COOKIELIST to a const string then libcurl would
likely cause a crash when modifying that string. Even if the string was
not const or a crash did not occur there was still the incorrect and
unexpected modification of the user's input string.
This issue was caused by 30da1f59 (precedes 8.11.0) which refactored
some options parsing and eliminated the copy of the input string. Also,
an earlier commit f88cc654 incorrectly cast the input pointer when
passing it to strtok.
Co-authored-by: Daniel Stenberg
Closes https://github.com/curl/curl/pull/15826
Let CURLINFO_QUEUE_TIME_T count only the time a transfer spends queued,
including possible redirect requests.
Add var 'time_queue' for reporting the time in write outs.
Add test for verifying correct reporting.
Closes#15512
Allowing --variable read a portion of provided files, makes curl work on
partial files for any options that accepts strings. Like --data and others.
The byte offset is provided within brackets, with a semicolon separator
like: --variable name@file;[100-200]"
Inspired by #14479
Assisted-by: Manuel Einfalt
Test 784 - 789. Documentation update provided.
Closes#15739
lib : remove all hyper code
configure: stop detecting hyper
docs: no more mention of hyper
tests: mo more special-handling of hyper builds
CI: no jobs using hyper
Closes#15120
