cfilter/conn: change send/recv function signatures. Unify the
calling/return conventions in our send/receive handling.
Curl_conn_recv(), adjust pnread type
Parameter `pnread` was a `ssize_t *`, but `size_t *` is better since the
function returns any error in its `CURLcode` return value.
Closes#17546
In shutdown, the result of a bufq_write() is intentionally ignored, but
it was not obvious why. Add a (void) cast to declare intent and a
comment explaining why.
Closes#17354
Move curlx_ functions into its own subdir.
The idea is to use the curlx_ prefix proper on these functions, and use
these same function names both in tool, lib and test suite source code.
Stop the previous special #define setup for curlx_ names.
The printf defines are now done for the library alone. Tests no longer
use the printf defines. The tool code sets its own defines. The printf
functions are not curlx, they are publicly available.
The strcase defines are not curlx_ functions and should not be used by
tool or server code.
dynbuf, warnless, base64, strparse, timeval, timediff are now proper
curlx functions.
When libcurl is built statically, the functions from the library can be
used as-is. The key is then that the functions must work as-is, without
having to be recompiled for use in tool/tests. This avoids symbol
collisions - when libcurl is built statically, we use those functions
directly when building the tool/tests. When libcurl is shared, we
build/link them separately for the tool/tests.
Assisted-by: Jay Satiro
Closes#17253
With a meta_hash at each connection (similar to easy handle, let
multi_ev.c store its pollsets as meta data, no longer needing its own
hashes.
This eliminates the last use of Curl_hash_offt. Remove it.
Closes#17095
When calling recvmsg(), always set up the msg structures for
each call as there are OS implemenations that change members
of msg.
Fixes#17120
Reported-by: Harry Sintonen
Closes#17131
Do not set a transport parameter idle timeout, meaning we have no such
thing from our side. The remote setting then applies.
In ngtcp2, set its "keep-alive" timer to prevent a possible remote idle
timeout to tear down the connection while we have active transfers on
that connection.
Closes#17057
Change multi's book keeping of transfers to no longer use lists, but a
special table and bitsets for unsigned int values.
`multi-xfers` is the `uint_tbl` where `multi_add_handle()` inserts a new
transfer which assigns it a unique identifier `mid`. Use bitsets to keep
track of transfers that are in state "process" or "pending" or
"msgsent".
Use sparse bitsets to replace `conn->easyq` and event handlings tracking
of transfers per socket. Instead of pointers, keep the mids involved.
Provide base data structures and document them in docs/internal:
* `uint_tbl`: a table of transfers with `mid` as lookup key,
handing out a mid for adds between 0 - capacity.
* `uint_bset`: a bitset keeping unsigned ints from 0 - capacity.
* `uint_spbset`: a sparse bitset for keeping a small number of
unsigned int values
* `uint_hash`: for associating `mid`s with a pointer.
This makes the `mid` the recommended way to refer to transfers inside
the same multi without risk of running into a UAF.
Modifying table and bitsets is safe while iterating over them. Overall
memory requirements are lower as with the double linked list apprach.
Closes#16761
With the new addition of OpenSSL QUIC API support and the support in
ngtcp2 main branch, make the necessary adjustments in curl to support
this combination.
- add support in configure.ac to detect the feature OPENSSL_QUIC_API2 in
openssl
- initialise ngtcp2 properly in this combination
- add a Curl_vquic_init() for global initialisation that ngtcp2 likes
for performance reasons
- add documentation on how to build in docs/HTTP3.md
- add CI testing in http3-linux.yml
Assisted-by: Viktor Szakats
Closes#17027
by including headers using "../[header]" when done from C files in
subdirectories, we do not need to specify the lib source dir as an
include path and we reduce the risk of header name collisions with
headers in the SDK using the same file names.
Idea-by: Kai Pastor
Ref: #16949Closes#16991
Before this patch, standard `E*` errno codes were redefined on Windows,
onto matching winsock2 `WSA*` error codes, which have different values.
This broke uses where using the `E*` value in non-socket context, or
other places expecting a POSIX `errno`, e.g. file I/O, threads, IDN or
interfacing with dependencies.
Fix it by introducing a curl-specific `SOCKE*` set of macros that map to
`WSA*` on Windows and standard POSIX codes on other platforms. Then
verify and update the code to use `SOCKE*` or `E*` macro depending on
context.
- Add `SOCKE*` macros that map to either winsock2 or POSIX error codes.
And use them with `SOCKERRNO` or in contexts requiring
platform-dependent socket error codes.
This fixes `E*` uses which were supposed be POSIX values, not `WSA*`
socket errors, on Windows:
- lib/curl_multibyte.c
- lib/curl_threads.c
- lib/idn.c
- lib/vtls/gtls.c
- lib/vtls/rustls.c
- src/tool_cb_wrt.c
- src/tool_dirhie.c
- Ban `E*` codes having a `SOCKE*` mapping, via checksrc.
Authored-by: Daniel Stenberg
- Add exceptions for `E*` codes used in file I/O, or other contexts
requiring POSIX error codes.
Also:
- ftp: fix missing `SOCKEACCES` mapping for Windows.
- add `SOCKENOMEM` for `Curl_getaddrinfo()` via `asyn-thread.c`.
- tests/server/sockfilt: fix to set `SOCKERRNO` in local `select()`
override on Windows.
- lib/inet_ntop: fix to return `WSAEINVAL` on Windows, where `ENOSPC` is
used on other platforms. To simulate Windows' built-in `inet_ntop()`,
as tested on a Win10 machine.
Note:
- WINE returns `STATUS_INVALID_PARAMETER` = `0xC000000D`.
- Microsoft documentation says it returns `WSA_INVALID_PARAMETER`
(= `ERROR_INVALID_PARAMETER`) 87:
https://learn.microsoft.com/windows/win32/api/ws2tcpip/nf-ws2tcpip-inet_ntop#return-value
- lib/inet_ntop: drop redundant `CURL_SETERRNO(ENOSPC)`.
`inet_ntop4()` already sets it before returning `NULL`.
- replace stray `WSAEWOULDBLOCK` with `USE_WINSOCK` macro to detect
winsock2.
- move existing `SOCKE*` mappings from `tests/server` to
`curl_setup_once.h`.
- add missing `EINTR`, `EINVAL` constants for WinCE.
Follow-up to abf80aae38#16612
Follow-up to d69425ed7d#16615
Bug: https://github.com/curl/curl/pull/16553#issuecomment-2704679377Closes#16621
The issues found fell into these categories, with the applied fixes:
- const was accidentally stripped.
Adjust code to not cast or cast with const.
- const/volatile missing from arguments, local variables.
Constify arguments or variables, adjust/delete casts. Small code
changes in a few places.
- const must be stripped because an API dependency requires it.
Strip `const` with `CURL_UNCONST()` macro to silence the warning out
of our control. These happen at API boundaries. Sometimes they depend
on dependency version, which this patch handles as necessary. Also
enable const support for the zlib API, using `ZLIB_CONST`. Supported
by zlib 1.2.5.2 and newer.
- const must be stripped because a curl API requires it.
Strip `const` with `CURL_UNCONST()` macro to silence the warning out
of our immediate control. For example we promise to send a non-const
argument to a callback, though the data is const internally.
- other cases where we may avoid const stripping by code changes.
Also silenced with `CURL_UNCONST()`.
- there are 3 places where `CURL_UNCONST()` is cast again to const.
To silence this type of warning:
```
lib/vquic/curl_osslq.c:1015:29: error: to be safe all intermediate
pointers in cast from 'unsigned char **' to 'const unsigned char **'
must be 'const' qualified [-Werror=cast-qual]
lib/cf-socket.c:734:32: error: to be safe all intermediate pointers in
cast from 'char **' to 'const char **' must be 'const' qualified
[-Werror=cast-qual]
```
There may be a better solution, but I couldn't find it.
These cases are handled in separate subcommits, but without further
markup.
If you see a `-Wcast-qual` warning in curl, we appreciate your report
about it.
Closes#16142
Quiche needs to find easy handles to events. Do this by iterating
over the filters stream hash and lookup the easy handle on a match.
This O(+streams-in-filter) vs O(all easy handles), at least once
we fix the multi lookup to use a hash.
Closes#16607
Iterate over the filters stream hash instead, lookup easy handles
at the multi when needed.
This also limits to pollset array sizes to the number of streams
on the connection and not the total number of transfers in the multi.
Closes#16611
Add a standalong hash table for curl_offt_t as key. This allows a
smaller memory footprint and faster lookups as we do not need to deal
with variable key lengths.
Use in all places we had the standard hash for this purpose.
Closes#16442
Enable TLS Early Data for wolfSSL:
- merge WOLFSSL_CTX and WOLFSSL setup from ngtcp2 with the general
implemenation in wolfssl.c
- enable for QUIC via ngtcp2
- give Curl_vquic_tls_init() a `struct alpn_spec` like used for the TCP
case. Adapt gnutls and other users.
- enable pytest test cases for early data with wolfSSL
and while this messes up wolfssl.c anyway, do
- rename all struct/functions with prefix 'wolfssl_' to 'wssl_' to not
pollute that name prefix
- rename `ctx/handle` to `ssl_ctx/ssl`, as used in openssl case
Closes#16167
Remove `blocking` argument from cfilter's connect method.
Implement blocking behaviour in Curl_conn_connect() instead for all
filter chains.
Update filters implementations. Several of which did never use the
paramter (QUIC for example). Simplifies connect handling in TLS filters
that no longer need to loop
Fixed a blocking connect call in FTP when waiting on a socket accept()
which only worked because the filter did not implement it.
Closes#16397
- Remove the workaround that disabled peer verification in DEBUGBUILDs
when CA certs were provided.
The workaround was part of a TODO that disabled verification in
DEBUGBUILDs with a CAfile/path because apparently there's no way to set
those options in msh3 and that caused some tests to fail. Instead the
tests should fail and this problem should not be covered up.
Ref: https://github.com/curl/curl/pull/16327#issuecomment-2661039423
Closes https://github.com/curl/curl/pull/16342
- add hex and octal parsers to the Curl_str_* family
- make curlx_strtoofft use these parsers
- remove all use of strtol() and strtoul() in library code
- generally use Curl_str_* more than strtoofft, for stricter parsing
- supports 64-bit universally, instead of 'long' which differs in size
between platforms
Extended the unit test 1664 to verify hex and octal parsing.
Closes#16336
The variable `conn->httpversion` was used for several purposes and it
was unclear at which time the value represents what.
- rename `conn->httpversion` to `conn->httpversion_seen`
This makes clear that the variable only records the last
HTTP version seen on the connection - if any. And that it
no longer is an indication of what version to use.
- Change Alt-Svc handling to no longer modify `conn->httpversion`
but set `data->state.httpwant` for influencing the HTTP version
to use on a transfer.
- Add `data->req.httpversion_sent` to have a record of what
HTTP version was sent in a request
- Add connection filter type CF_TYPE_HTTP
- Add filter query `CF_QUERY_HTTP_VERSION` to ask what HTTP
filter version is in place
- Lookup filters HTTP version instead of using `conn->httpversion`
Test test_12_05 now switches to HTTP/1.1 correctly and the
expectations have been fixed.
Removed the connection fitler "is_httpN()" checks and using
the version query instead.
Closes#16073
Make transfer attach/detach to/from connections chepaer.
- the "attach" event was no longer implemented by any filter
- the "detach" did the same as the "done" event for the filters
who still implemented it. It should be superfluous as the "done"
must always happen.
Closes#16067
This discussion:
https://github.com/openssl/openssl/discussions/23339#discussion-6094341
Specifically item number 2 (Send Blocking) was raised by the curl team,
noting that SSL_want_write returning false was not a good indicator of
when a stream is writeable. The suggestion in that discussion was to use
SSL_poll with an SSL_POLL_EVENT_W flag instead, as that is a proper
indication of when an SSL_object will allow writing without blocking.
While ssl_want_write updates its state based on the last error
encountered (implying a need to retry an operation to update the
last_error state again), SSL_poll checks stream buffer status during the
call, giving it more up to date information on request. This is the
method used by our guide demos (quic-hq-interop specifically), and it
works well.
This change has been run through the curl test suite, and shown to pass
all tests. However, given the initial problem description I'm not sure
if there is a test case that explicitly checks for blocking and
unblocking of streams. As such some additional testing may be warranted.
Closes#15909
Adds the experimental feature `ssls-export` to libcurl and curl for
importing and exporting SSL sessions from/to a file.
* add functions to libcurl API
* add command line option `--ssl-sessions <filename>` to curl
* add documenation
* add support in configure
* add support in cmake
+ add pytest case
Closes#15924
When a QUIC TLS session announced early data support and
'CURLSSLOPT_EARLYDATA' is set for the transfer, send initial request and
body (up to the 128k we buffer) as 0RTT when curl is built with
ngtcp2+gnutls.
QUIC 0RTT needs not only the TLS session but the QUIC transport
paramters as well. Store those and the earlydata max value together with
the session in the cache.
Add test case for h3 use of this. Enable quic early data in nghttpx for
testing.
Closes#15667
This is debug code, but since the caller might use the value even when
this function returns error, it needs to be cleared properly here.
Spotted by Clang-tidy
Closes#15807
