Maintenance tasks like connection upkeep, liveliness checks and cache
eviction must only run on an admin handle, not the application transfer
from the calling context.
Then application transfers will get only attached to the connections
they actually work on. This makes `data->state.recent_conn_id`
meaningful for checks if a subsequent address is finding the previous
connection again.
One thing led to another:
- add Curl_get_admin(data), to obtain an admin handle for
an application handle, inheriting some properties for
connection operation.
- rename `Curl_easy *data` parameter to `Curl_easy *admin`
where only admin handles should be passed. Add DEBUGASSERTs
in the called function.
- rename `Curl_conn_terminate()` to `Curl_conn_close()`
because this makes connections enter the shutdown close
where possible.
- rename `Curl_cshutdn_terminate()` to `Curl_conn_terminate()`
since this definitely kills the connection and does not
involve a `cshutdn` instance.
- add Curl_share_lock_share(), Curl_share_unlock_share()
so it possible to operate on a share without the passed
easy handle having the share set.
This catches a case where a shared needed to be locked
but was not before. Adjust test1554 results.
Fixes#22567
Reported-by: cybertron10 on github
Closes#22572
- decode results when individual requests are done
- makes happy eyeballing start asap
- remove doh_resp structures as no longer needed
- remove CURL_DNS_TYPE_NS, CURL_DNS_TYPE_CNAME and
CURL_DNS_TYPE_DNAME from DoH
- DoH: do not set PIPEWAIT and SSL OPTS when url starts with http:
- mark Doh master handle as dirty after every sub-request, not only the last
- Doh: start probe on AAAA before A (was the other way).
cf-dns: set EXPIRE_HAPPY_EYEBALLS timer when waiting 50ms on AAAA
result or progress will not be triggered in time.
Add debug env var CURL_DBG_HE_AAAA_AWAIT_MS to override the
default 50ms on waiting for the AAAA result to arrive.
test2100: set CURL_DBG_HE_AAAA_AWAIT_MS to 60 seconds to
provide enough time for slow CI runs to sent all DoH requests.
closes#22514
- makes test names less complicated
- makes them less likely to wrap lines when using narrow terminals
- runtests now returns error for the test if the name is longer
- replace the "..." with a singe space
Closes#22492
These functions were previously untested in the test suite. This is just
a set of basic invokes to make sure they work. The core of these
functions is identical and is tested already.
- curl_mvfprintf
- curl_mvprintf
- curl_mvsnprintf
- curl_mvsprintf
- curl_mvaprintf
Closes#22472
- add CURLUE_BACKSLASH that can be returned when a backslash was used
where a forward one probably was intended.
- make CURLUE_NO_HOST higher priority than port number errors for URLs
without hostname. Like in "http://::1"
- shortened some URL parser error strings
Extend test 1560 to verify.
Reported-by: kit-ty-kate on github
Fixes#22337Closes#22408
- Switch the Windows SSPI identity struct to SEC_WINNT_AUTH_IDENTITY_EX
to use !ntlm in PackageList to prevent NTLM from being offered.
- For GSS filter out NTLMSSP OID, and restrict via gss_set_neg_mechs()
to prevent NTLM from being offered.
- Extend the GSS-API debug stub layer to support the NTLM blocking logic
without a real Kerberos environment.
- Update test 2057 to check that negotiate auth is silently skipped with
no Authorization header when only NTLM stub credentials are available.
- Add SPNEGO NTLM blocking test 2093 which verifies that Kerberos
credentials still succeed when NTLM is blocked within SPNEGO.
- Suppress tests valgrind leak for MIT krb5 gss_display_status, since
the leak is in the library and not in curl.
To suppress the tests valgrind leak, the wildcard '...' bridges over an
anonymous frame inside libgssapi_krb5.so that valgrind reports as '???'.
Signed-off-by: Matthew John Cheetham <mjcheetham@outlook.com>
Aided-by: Johannes Schindelin
Closes https://github.com/curl/curl/pull/21315
Closes https://github.com/curl/curl/pull/22410
Fix test 1712 to pass curl C by setting `COLUMNS` to the highest
accepted value, and adjust expected results. To avoid envs with varying
lengths of `LOGDIR` affect the outcome.
Apply the same fix to test 459, though it wasn't affected in curl CI.
Also sync up test 433 `COLUMNS` value with these two tests for
consistency.
Ref: #22381
Follow-up to 8e3a2a64d1#20666Closes#22388
Do not let unrelated credentials from a redirected URL bypass the
cross-host auth boundary
Verified by test 5023 to 5025
Follow-up to a55731050eCloses#22395
Add support for signing outgoing HTTP requests per RFC 9421 using
Ed25519 or HMAC-SHA256 algorithms.
New libcurl options:
- CURLOPT_HTTPSIG: signing algorithm ("ed25519" or "hmac-sha256")
- CURLOPT_HTTPSIG_KEY: path to hex-encoded key file
- CURLOPT_HTTPSIG_KEYID: key identifier for Signature-Input
- CURLOPT_HTTPSIG_HEADERS: space-separated components to sign
New CLI flags: --httpsig, --httpsig-key, --httpsig-keyid,
--httpsig-headers
The crypto layer follows the sha256.c multi-backend pattern with
implementations for OpenSSL (EVP_DigestSign) and wolfSSL
(wc_ed25519_sign_msg). HMAC-SHA256 uses the existing Curl_hmacit()
infrastructure which works on all backends.
Verified by test 5000 to 5021
Assisted-by: Daniel Stenberg
Signed-off-by: Sameeh Jubran <sameeh@wolfssl.com>
Closes#22386Closes#21239
- connclose/streamclose/connkeep() remove description string that was
never used anywhere. Add trace statements where reasons for closing
were not already traced and maybe not obvious.
- multi_remove_handle: only lookup former connection in pool when
transfer is set to connect only
- test1554: adapt expectations now that pool is less often locked
Closes#22379
Fix handling of DoH response that only asks for HTTPS records.
Add test 2117 for checking that a HTTPS-RR resolve is processed,
even though the actual answer is invalid.
Closes#22372
Use separate dns cache entries for addresses (A+AAAA) and HTTPS-RR
results. That makes also "negative" results independent of each other.
Dns cache entries, once in use, can no longer be modified safely, as
concurrent use would require each access then to be done under lock. By
using separate entries, we can update a HTTPS entry without needing to
duplicate an existing address entry for the same host+port.
Connection filters can now ask for DNS resolves. This works at any time
during connection setup and while connect is going on.
`Curl_conn_dns_add_addr_resolve()` and
`Curl_conn_dns_add_https_resolve()` are used for that. They check if the
resolve is already ongoing, can be added to a resolve not started yet
or, as last resort, create a new filter instance and add it to the
connection (it's easier to add more filters than making the same filter
handle multiple resolves. Since DNS filters are removed once the
connection is established, there is no later penalty).
HTTPS-RR queries are added by the `HTTPS-CONNECT`, `SSL` and `QUIC`
filters. The latter will only do that when ECH is configured and
supported. That means we trigger HTTPS-RR queries only when the results
matter.
Add test_22_06 for ALPN influenced via HTTPS-RR when tunneling through a
proxy. This did not work before.
Adjust test2100 to use https: as the previous http: URL no longer
triggers HTTPS-RR resolves.
Closes#22216
For threaded resolving, added an additional socket/eventfd pair to the
multi handle for notifications from threads. The original "double use"
of the standard wakeup pair did lead to regressions for apps.
The API definition of curl_multi_poll/wait/wakeup is pretty tight
regarding what effects what and adding notifications on top of that
broke what apps perceived to be the contract.
Fixes#22272
Reported-by: Sergei Zimmerman
Closes#22274
- Adjust pollset during connection shutdown.
- Separate the FIRSTSOCKET and SECONDSOCKET check so that one being in
connect or shutdown no longer adds poll events for the other.
- Fix shutdown state evaluation (as detailed below).
- Add a unit test for Curl_conn_adjust_pollset.
- Add a client for event testing.
- Test that shutdown sockets stay with the socket callback until done.
The pollset predicate now reads the connection's own shutdown state
rather than going through data->conn, and the EXPIRE_SHUTDOWN arming in
cshutdn_perform() works again: next_expire_ms started at 0 and could
never be set, so a peer that never sends a close_notify would still park
its connection forever, timeout or not.
Reported-by: pszemus@users.noreply.github.com
Fixes https://github.com/curl/curl/issues/22282
Closes https://github.com/curl/curl/pull/22304
`test798` is the only test fetching `http://localhost:%HTTPPORT` without
`-4`, since it needs the hostname for its folded `domain=localhost`
cookie. curl tries `::1` first, but the test HTTP server is IPv4-only.
On the BSDs, IPv4 and IPv6 have separate ephemeral port namespaces, so
with every test server binding port 0, another parallel runner's
IPv6-bound server can hold the same numeric port. curl then connects to
the wrong runner's server, which cannot open its own `log/N/test798` and
closes without a response, giving exit 52 and an empty `server.input`.
This PR fixes this flake by adding `-4` matches what tests 389 and 392
already do. Linux is immune because wildcard IPv6 binds occupy the IPv4
port too. Seen in
https://github.com/curl/curl/actions/runs/29229170329/job/86749470571.
Closes#22318
A CR or LF in the CURLOPT_FTP_ACCOUNT or CURLOPT_FTP_ALTERNATIVE_TO_USER
string split the control-channel command line and smuggled a second FTP
command. Reject a byte below 0x20 in both values before the command is
built.
Closes#22301
Test 1701 did a HTTP/2 Upgrade in a HTTP/1.1 POST request. This will no
longer be supported in the upcoming nghttp2 release for security reasons
(Upgrade requests with body a re messy, error prone and can easily lead
to connection confusion).
As no other server we have supports this, remove the test.
Reported-by: firexinghe on github
Fixes#22280Closes#22291
The test provides an invalid username, but no ':' with password, so that
the curl tool tries to read the password from stdin. That makes the test
hang, also in CI.
The check for invalid characters is happening when the CURLOPT_USERPWD
is applied to the URL, so when the transfer is started. Maybe the check
should happen also in setopt already.
The test hangs because tool_paramhlp.c:548 `checkpasswd()` does not see
a ':' and tries to read the password from stdin.
Fix: add a ':' in the tests invalid user setting, so that the read does
not happen.
Follow-up to ecaa344a7aCloses#22264
Add classic tests 3223-3226 and extend tests/http/test_12_reuse.py so that
connection reuse decisions are explicit:
- partial/aborted HTTP/1.1 responses must not reuse the connection
(premature multi_done closes non-multiplexed conns)
- different target hostnames must not share a connection even when they
resolve to the same address (url_match_destination)
- HTTP Basic credentials are per-request (PROTOPT_CREDSPERREQUEST), so
different -u values still reuse; assert that with num_connects
- same host positive control reuses (num_connects 1 then 0)
Closes#22224
A sasl zero-length normal response is an empty string, unlike an initial
response. Since the encoding of a zero-length initial response is
protocol dependent, move its handling to the protocol-specific sendauth
procedure. Similarly, do not check for an '=' server message, as this
should normally never occur.
Update the erroneous tests accordingly.
Closes#22218
A gzip Content-Encoding response holding more than one gzip member made
curl decode the first member and then fail the transfer with a bare
CURLE_WRITE_ERROR and no hint about the cause. Detect the trailing
member and fail with a message that explains what happened.
Fixes#22156Closes#22172
Also repurpose existing build-time feature checks into unit test 1961,
to verify.
Prior to this patch these functions were auto-detected with both
autotools and cmake. In case of autotools there was an extra
verification phase ensuring the functions work as expected. This step
required running the function, thus was limited to non-cross-builds. For
cross-builds and CMake it always used the system implementation if
present. On Windows it always used the local implementation, because
availability/use is complicated there.
After this patch all platforms, always use the local implementation,
which is known to be accurate. This makes curl behave more consistently,
and simplifies the build process, a fixes cross-builds and CMake
auto-detection differences.
Also:
- test1960: enable unconditionally.
- checksrc: disallow globally, allowlist in `block_ip.c` example.
- dnsd: verify ntop result for NULL before passing to printf.
Ref: https://github.com/curl/curl/pull/22137#issuecomment-4797440983
Ref: #22137
Ref: 8537a5b0bc#16577Closes#22170
A custom header name padded with blanks (`Authorization :`) missed the
exact-length compare and slipped past the cross-host Authorization and
Cookie drop, forwarding the header to the redirect target. Trim the
parsed name in both the request and proxy CONNECT header builders.
Closes#22178
- GHA/windows: mention `IgnoreStandardErrorWarningFormat=true`
in comment.
- cmake: fix MIT/GNU GSS order in messages.
- drop some exclamation marks from messages.
- drop redundant ending newlines from messages.
- fold/unfold where possible.
- fix indent, whitespace, typos and other nits.
Closes#22185
