Add support for CURLOPT_READFUNCTION with WebSocket urls when *not* in
connect-only mode, e.g. when using curl_multi_perform.
Install the callback function and set CURLOPT_UPLOAD. Return
CURL_READFUNC_PAUSE when having nothing more to send and unpause the
transfer when more data is ready.
This will send the read bytes in a WebSocket BINARY frame.
Add support for this mode in the pytest "ws_data" client and have all
tests run in 'curl_ws_send/recv' and 'peform' mode as well.
Add `curl_ws_start_frame()`. Document, cover in libcurl-ws.md and
explain the READFUNCTION mode for websockets.
Add example `websocket-updown` for this.
Closes#17683
Deduce that the transfer response expects headers by the protocol
handler implementing `write_resp_hd` callback. This eleminates the
`getheader` parameter in the `Curl_xfer_setup_*()` methods.
Add an implementation to RTSP for `write_resp_hd`, joining the HTTP
protocol in the only handlers having it.
Reverse the default of request's `header` bit that signals that headers
are expected. Default is now FALSE, set to TRUE when setting up the
transfer by presence of `write_resp_hd` in the protocol handler.
Closes#18218
Make variants for transfers that send/receive or do both with just the
parameters they need. Split out the shutdown setting into a separate
function. Only FTP bothers with that.
Closes#18203
The `connectdata` members `sockfd` and `writesockfd` needed to by either
CURL_SOCKET_BAD or a copy of one of `conn->sock[2]`. When equal to one,
that index was used to send/recv/poll the proper socket or connection
filter chain.
Replace those with `send_idx` and `recv_idx` which are either -1, 0 or 1
to indicate which socket/filter to send/receive on.
Closes#18179
When a HTTP version has been negotiated via ALPN, set the member
`conn->httpversion_seen` accordingly. This allows pending transfers to
reuse multiplexed http connections before the response to the first
transfer has arrived.
Fixes#18177
Reported-by: IoannisGS on github
Closes#18181
- Consistently keep options within ranges
- Reduce the maximum maxredirs value to fit a signed short
- Removed comments as the place to document the options is not here
Closes#18174
`getsock()` calls operated on a global limit that could
not be configure beyond 16 sockets. This is no longer adequate
with the new happy eyeballing strategy.
Instead, do the following:
- make `struct easy_pollset` dynamic. Starting with
a minimal room for two sockets, the very common case,
allow it to grow on demand.
- replace all protocol handler getsock() calls with pollsets
and a CURLcode to return failures
- add CURLcode return for all connection filter `adjust_pollset()`
callbacks, since they too can now fail.
- use appropriately in multi.c and multi_ev.c
- fix unit2600 to trigger pollset growth
Closes#18164
Since the content varies during connection setup and while doing it
(eyeballing), remove these strcut from `connectdata` and replace use
with querying the connection filters. Those keep that information
already.
Change the info logging of established connections to also give the
local address and port.
Closes#17960
Add a connection filter query to obtained the negotiated ALPN
protocol to check in setup/protocols how the connection needs
to behave.
Remove the members `alpn` and `proxy_alpn` from `connectdata`.
Closes#17947
This became an issue after promoting curl compiler warnings to errors in
curl-for-win. The code is correct. It over-allocates a struct to store
variable sized data past its length. Similar code is present in
`lib/smb.c`, silenced earlier.
Seen in linux-musl-debian-testing-gcc curl-for-win builds, gcc 14.2.0,
RISC-V (but not amd64/aarch64), unity, debian:testing (trixie):
musl:
```
In file included from /curl/_r64-linux-musl-bld/lib/CMakeFiles/libcurl_object.dir/Unity/unity_0_c.c:184:
In function 'Curl_http_req_make',
inlined from 'Curl_http_proxy_create_CONNECT' at /curl/lib/http_proxy.c:252:12:
/curl/lib/http.c:4373:3: error: 'memcpy' offset [137, 142] from the object at 'req' is out of the bounds of
referenced subobject 'method' with type 'char[1]' at offset 136 [-Werror=array-bounds=]
4373 | memcpy(req->method, method, m_len);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /curl/lib/urldata.h:182,
from /curl/lib/altsvc.c:32,
from /curl/_r64-linux-musl-bld/lib/CMakeFiles/libcurl_object.dir/Unity/unity_0_c.c:4:
/curl/lib/http.h: In function 'Curl_http_proxy_create_CONNECT':
/curl/lib/http.h:230:8: note: subobject 'method' declared here
230 | char method[1];
| ^~~~~~
```
Ref: https://github.com/curl/curl/actions/runs/16527769182/job/46745369844?pr=18025#step:3:5798
Ref: https://github.com/curl/curl-for-win/actions/runs/16525969694/job/46739239206#step:3:5958
glibc (with unity batch):
```
In file included from /usr/riscv64-linux-gnu/include/string.h:548,
from /curl/lib/curl_setup_once.h:33,
from /curl/lib/curl_setup.h:823,
from /curl/lib/http.c:25,
from /curl/_r64-linux-gnu-bld/lib/CMakeFiles/libcurl_object.dir/Unity/unity_2_c.c:4:
In function 'memcpy',
inlined from 'Curl_http_req_make' at /curl/lib/http.c:4373:3,
inlined from 'Curl_http_proxy_create_CONNECT' at /curl/lib/http_proxy.c:252:12:
/usr/riscv64-linux-gnu/include/bits/string_fortified.h:29:10: error: '__builtin_memcpy' offset [137, 142]
from the object at 'req' is out of the bounds of
referenced subobject 'method' with type 'char[1]' at offset 136 [-Werror=array-bounds=]
29 | return __builtin___memcpy_chk (__dest, __src, __len,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
30 | __glibc_objsize0 (__dest));
| ~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /curl/lib/urldata.h:182,
from /curl/lib/http.c:50:
/curl/lib/http.h: In function 'Curl_http_proxy_create_CONNECT':
/curl/lib/http.h:230:8: note: subobject 'method' declared here
230 | char method[1];
| ^~~~~~
```
Ref: https://github.com/curl/curl-for-win/actions/runs/16538174468/job/46775731055#step:3:5936
Ref: f45df099f3
Follow-up to 14f26f5ee7#16187
Cherry-picked from #18025Closes#18030
- vtls: fix unused variable and symbols.
- ftp: fix unused variable.
- http: fix unused variables.
- smtp: fix unsued variable.
- wolfssl: fix unused variable with !proxy.
- libssh: fix unused argument.
- curl_trc: sync guards between declaration and definition.
- curl_trc: add missing guard for `Curl_trc_ssls` when !verbose.
- curl_trc: fix errors with !http + http3.
- curl_trc: fix missing function with !http + nghttp2.
- cf-h2-proxy: disable when !http + nghttp2, to avoid calling undeclared
functions.
- sha256: fix missing declaration in rare configs.
- md4: fix symbol conflict when building GnuTLS together with AWS-LC or
wolfSSL. By prioritizing the latter two. AWS-LC has no option
to disable the clashing symbol. wolfSSL does, but the most seamless is
to skip including GnuTLS's standalone `md4.h` to avoid the clash.
- build: fix errors with !http + nghttp2.
- build: catch !ssl + ssls-export combination in source. Convert
build-level errors to warnings.
- build: fix errors with !http + http3.
- build: fix building curl tool and unit1302 in rare combinations.
By always compiling base64 curlx functions.
- cmake: add `_CURL_SKIP_BUILD_CERTS` internal option.
To disable automatically building certs with the testdeps target.
To improve performance when testing builds.
(used locally to find the failing builds fixed in this PR.)
Closes#17962
The `transport` to use for a transfer, e.g. TCP/QUIC/UNIX/UDP, is
initially selected by options and protocol used. This is set at the
`struct connectdata` as `transport` member.
During connection establishment, this transport may change due to
Alt-Svc or Happy-Eyeballing. Most common is the switch from TCP to QUIC.
Rename the connection member to `transport_wanted` and add a way to
query the connection for the transport in use via a new connection
filter query.
The filter query can also be used in the happy eyeballing attempts when
code needs to know which transport is used by the "filter below". This
happens in wolfssl initialization, as one example.
Closes#17923
Drop `strcasecompare` and `strncasecompare` in favor of libcurl API
calls `curl_strequal` and `curl_strnequal` respectively.
Also drop unnecessary `strcase.h` includes. Include `curl/curl.h`
instead where it wasn't included before.
Closes#17772
All arguments and local variables in `http_header_s` were unused when
both `CURL_DISABLE_COOKIES` and `CURL_DISABLE_HSTS` were defined.
Closes https://github.com/curl/curl/pull/17753
Remove structs for negotiate, krb5, ntlm and gsasl from connectdata and
store them as connection meta data with auto cleanup.
De-complexify sasl mech selection by moving code into static functions.
Closes#17557
When inspecting a possible follow HTTP request, the result of a rewind
of the upload data was ignored as it was not clear at that point in time
if the request would become a GET.
This initiated the followup, rewound again, which failed again and
terminated the follow up.
This was confusing to users as it was not clear of the follow up was
done or not.
Fix: fail the early rewind when the request is not converted to GET.
Fixes#17472Closes#17474
Reported-by: Jeroen Ooms
Move curlx_ functions into its own subdir.
The idea is to use the curlx_ prefix proper on these functions, and use
these same function names both in tool, lib and test suite source code.
Stop the previous special #define setup for curlx_ names.
The printf defines are now done for the library alone. Tests no longer
use the printf defines. The tool code sets its own defines. The printf
functions are not curlx, they are publicly available.
The strcase defines are not curlx_ functions and should not be used by
tool or server code.
dynbuf, warnless, base64, strparse, timeval, timediff are now proper
curlx functions.
When libcurl is built statically, the functions from the library can be
used as-is. The key is then that the functions must work as-is, without
having to be recompiled for use in tool/tests. This avoids symbol
collisions - when libcurl is built statically, we use those functions
directly when building the tool/tests. When libcurl is shared, we
build/link them separately for the tool/tests.
Assisted-by: Jay Satiro
Closes#17253
A "TE" request header is allowed in HTTP/2 when it only carries the
"trailers" value. RFC 9113 ch. 8.2.2. Check client supplied TE values
for the "trailers" token and only pass that one in a HTTP/2 request.
Add test_01_17 to verify.
Fixes#17122
Reported-by: epicmkirzinger on github
Closes#17128
By allocating the method string as part of the struct, the previous
fixed size limit (23 bytes) can be avoided. It would previously make
"curl -X [long string]" work against http://localhost but fail against
https://curl.se with no clear error message.
Closes#16729
It was introduced in 294136b754, but not shipped in a release. It
has caused problems and after checking, the browsers don't insist on it
even if RFC 9112 says it is mandatory.
Adjust test 2 to do a response without the space.
Closes#16728
The issues found fell into these categories, with the applied fixes:
- const was accidentally stripped.
Adjust code to not cast or cast with const.
- const/volatile missing from arguments, local variables.
Constify arguments or variables, adjust/delete casts. Small code
changes in a few places.
- const must be stripped because an API dependency requires it.
Strip `const` with `CURL_UNCONST()` macro to silence the warning out
of our control. These happen at API boundaries. Sometimes they depend
on dependency version, which this patch handles as necessary. Also
enable const support for the zlib API, using `ZLIB_CONST`. Supported
by zlib 1.2.5.2 and newer.
- const must be stripped because a curl API requires it.
Strip `const` with `CURL_UNCONST()` macro to silence the warning out
of our immediate control. For example we promise to send a non-const
argument to a callback, though the data is const internally.
- other cases where we may avoid const stripping by code changes.
Also silenced with `CURL_UNCONST()`.
- there are 3 places where `CURL_UNCONST()` is cast again to const.
To silence this type of warning:
```
lib/vquic/curl_osslq.c:1015:29: error: to be safe all intermediate
pointers in cast from 'unsigned char **' to 'const unsigned char **'
must be 'const' qualified [-Werror=cast-qual]
lib/cf-socket.c:734:32: error: to be safe all intermediate pointers in
cast from 'char **' to 'const char **' must be 'const' qualified
[-Werror=cast-qual]
```
There may be a better solution, but I couldn't find it.
These cases are handled in separate subcommits, but without further
markup.
If you see a `-Wcast-qual` warning in curl, we appreciate your report
about it.
Closes#16142
- cf-h1-proxy: check return code and return error if the parser fails
- http: make the Retry-After parser check for a date string first then
number to avoid mis-parsing the begining of a date as a number
Closes#16548
With this change, the argument passed to the CURLOPT_FOLLOWLOCATION
option is now instead a "mode" instead of just a boolean. Documentation
is extended to describe the two new modes.
Test 1571 to 1581 verify.
Closes#16473
- replace several ISSPACE() with ISBLANK(), since the former also skips
CR and LF which in most cases should not occur where this macro is
used
- after this commit, there is no ISSPACE() user left in libcurl code, but
unfortunately tool and test code use the macro so it cannot be removed.
Closes#16520
- make it only accept version 1.0, as that is the version curl supports
- convert the parser to use strparse
- the status code max is now 999, but it does allow != 3 digits
Closes#16435
Avoids having to use the correct index into the line. Avoids repeated
use of is_valid_auth_separator.
Require that the following letter is not an alnum instead of checking
explicitly for ch == '\0' || ch == ',' || ISSPACE(ch). After all, the
point is to not erroneously match another auth string using the same
prefix.
Follow-up to b75620b9a0Closes#16406
Add a 'wanted' major HTTP version bitmask next to the 'allowed' bitmask
in HTTP version negotiation. This will try connections as specified in
'wanted', but enabled Alt-Svc and HTTPS-RR to redirect to other major
HTTP versions, if those are 'allowed'.
Changes libcurl internal default to `CURL_HTTP_VERSION_NONE` and removes
the code in curl that sets `CURL_HTTP_VERSION_2TLS` if the command line
does not say anything else.
Closes#16117
Translate the `data->set.httpwant` which is one of the consts from the
public API (CURL_HTTP_VERSION_*) into a major version mask plus
additional flags for internal handling.
`Curl_http_neg_init()` does the translation and flags setting in http.c,
using new internal consts CURL_HTTP_V1x, CURL_HTTP_V2x and CURL_HTTP_V3x
for the major versions. The flags are
- only_10: when the application explicity asked fro HTTP/1.0
- h2_upgrade: when the application asks for upgrading 1.1 to 2.
- h2_prior_knowledge: when directly talking h2 without ALPN
- accept_09: when a HTTP/0.9 response is acceptable.
The Alt-Svc and HTTPS RR redirections from one ALPN to another obey the
allowed major versions. If a transfer has only h3 enabled, Alt-Svc
redirection to h2 is ignored.
This is the current implementation. It can be debated if Alt-Svc should
be able to override the allowed major versions. Added test_12_06 to
verify the current restriction.
Closes#16100
Since the ISBLANK() and ISSPACE() macros check for specific matches,
there is no point in using while(*ptr && ISSPACE(*ptr)) etc, as the
'*ptr' check is then superfluous.
Closes#16363
- add hex and octal parsers to the Curl_str_* family
- make curlx_strtoofft use these parsers
- remove all use of strtol() and strtoul() in library code
- generally use Curl_str_* more than strtoofft, for stricter parsing
- supports 64-bit universally, instead of 'long' which differs in size
between platforms
Extended the unit test 1664 to verify hex and octal parsing.
Closes#16336
