By returning error separately on parse errors and avoiding magic
numbers, this function can now return 0 or -1 as proper dates when such
a date string is provided.
Closes#18445
- Avoid checking what's always true. The ftpcode pointer is always
passed in, so use it.
- Simplified an indent level somewhat
- Split out two functions from the state machine
Closes#18403
Deduce that the transfer response expects headers by the protocol
handler implementing `write_resp_hd` callback. This eleminates the
`getheader` parameter in the `Curl_xfer_setup_*()` methods.
Add an implementation to RTSP for `write_resp_hd`, joining the HTTP
protocol in the only handlers having it.
Reverse the default of request's `header` bit that signals that headers
are expected. Default is now FALSE, set to TRUE when setting up the
transfer by presence of `write_resp_hd` in the protocol handler.
Closes#18218
Make variants for transfers that send/receive or do both with just the
parameters they need. Split out the shutdown setting into a separate
function. Only FTP bothers with that.
Closes#18203
`getsock()` calls operated on a global limit that could
not be configure beyond 16 sockets. This is no longer adequate
with the new happy eyeballing strategy.
Instead, do the following:
- make `struct easy_pollset` dynamic. Starting with
a minimal room for two sockets, the very common case,
allow it to grow on demand.
- replace all protocol handler getsock() calls with pollsets
and a CURLcode to return failures
- add CURLcode return for all connection filter `adjust_pollset()`
callbacks, since they too can now fail.
- use appropriately in multi.c and multi_ev.c
- fix unit2600 to trigger pollset growth
Closes#18164
Since the content varies during connection setup and while doing it
(eyeballing), remove these strcut from `connectdata` and replace use
with querying the connection filters. Those keep that information
already.
Change the info logging of established connections to also give the
local address and port.
Closes#17960
This patch bumps the size of these constants from `int` to `long`, while
keeping their actual values the same. It may cause incompatibilities in
user code, requiring the bump of holder variables and/or adding casts:
- CURLFTP_CREATE_DIR
- CURLFTP_CREATE_DIR_NONE
- CURLFTP_CREATE_DIR_RETRY
- CURLFTPAUTH_DEFAULT
- CURLFTPAUTH_SSL
- CURLFTPAUTH_TLS
- CURLFTPMETHOD_DEFAULT
- CURLFTPMETHOD_MULTICWD
- CURLFTPMETHOD_NOCWD
- CURLFTPMETHOD_SINGLECWD
- CURLFTPSSL_CCC_ACTIVE
- CURLFTPSSL_CCC_NONE
- CURLFTPSSL_CCC_PASSIVE
Also:
- keep existing casts within the documentation to make sure it applies
to older curl versions as well.
Closes#17797
- vtls: fix unused variable and symbols.
- ftp: fix unused variable.
- http: fix unused variables.
- smtp: fix unsued variable.
- wolfssl: fix unused variable with !proxy.
- libssh: fix unused argument.
- curl_trc: sync guards between declaration and definition.
- curl_trc: add missing guard for `Curl_trc_ssls` when !verbose.
- curl_trc: fix errors with !http + http3.
- curl_trc: fix missing function with !http + nghttp2.
- cf-h2-proxy: disable when !http + nghttp2, to avoid calling undeclared
functions.
- sha256: fix missing declaration in rare configs.
- md4: fix symbol conflict when building GnuTLS together with AWS-LC or
wolfSSL. By prioritizing the latter two. AWS-LC has no option
to disable the clashing symbol. wolfSSL does, but the most seamless is
to skip including GnuTLS's standalone `md4.h` to avoid the clash.
- build: fix errors with !http + nghttp2.
- build: catch !ssl + ssls-export combination in source. Convert
build-level errors to warnings.
- build: fix errors with !http + http3.
- build: fix building curl tool and unit1302 in rare combinations.
By always compiling base64 curlx functions.
- cmake: add `_CURL_SKIP_BUILD_CERTS` internal option.
To disable automatically building certs with the testdeps target.
To improve performance when testing builds.
(used locally to find the failing builds fixed in this PR.)
Closes#17962
- codespell: break logic out into its own runnable script. Allowing
to run it on local machines.
- codespell: install via `pip`, bump to latest version.
- codespell: show version number in CI log.
- codespell: drop no longer needed word exception: `msdos`.
- codespell: include all curl source tree, except `packages` and
`winbuild`. Drop an obsolete file exclusion.
- add new spellchecker job using the `typos` tool. It includes
the codespell dictionary and a couple more. Use linuxbrew to install
it. This takes 10 seconds, while installing via `cargo` from source
would take over a minute.
- codespell: introduce an inline ignore filter compatible with `cspell`
Make `typos` recognize it, too. Move single exceptions inline.
Fix new typos found. Also rename variables and words to keep
spellchecking exceptions at minumum. This involves touching some tests.
Also switch base64 strings to `%b64[]` to avoid false positives.
Ref: https://github.com/crate-ci/typos/blob/master/docs/reference.md
Ref: https://github.com/codespell-project/codespell?tab=readme-ov-file#inline-ignore
Ref: https://github.com/codespell-project/codespell/issues/1212#issuecomment-1721152455
Ref: https://cspell.org/docs/Configuration/document-settingsCloses#17905
The `transport` to use for a transfer, e.g. TCP/QUIC/UNIX/UDP, is
initially selected by options and protocol used. This is set at the
`struct connectdata` as `transport` member.
During connection establishment, this transport may change due to
Alt-Svc or Happy-Eyeballing. Most common is the switch from TCP to QUIC.
Rename the connection member to `transport_wanted` and add a way to
query the connection for the transport in use via a new connection
filter query.
The filter query can also be used in the happy eyeballing attempts when
code needs to know which transport is used by the "filter below". This
happens in wolfssl initialization, as one example.
Closes#17923
Drop `strcasecompare` and `strncasecompare` in favor of libcurl API
calls `curl_strequal` and `curl_strnequal` respectively.
Also drop unnecessary `strcase.h` includes. Include `curl/curl.h`
instead where it wasn't included before.
Closes#17772
Allow prequotes to be sent after curl has changed the working directory,
just before the listing command if the URL is a directory.
FTP state machine is updated with the new FTP_LIST_PREQUOTE state and
FTP_RETR_LIST_TYPE type.
Test 754 verifies
Fixes#8602Closes#17616
Used to be a pointer set (and cleared) by the socket connection filters
to a struct in their contexts. Instead, add a filter query method to
obtain the pointer when needed.
Closes#17385
When `curl_easy_cleanup()` is invoked while still being part
of a multi handle, the code will auto-remove it. But since the
connection was detached first, the code in
`curl_multi_remove_handle()` that invalidates dirty connections
did not bite.
Attach the connection *after* the possible remove from a multi
handle, so that connection reuse can be prevented.
Add test753 to reproduce and verify the fix. This required adding
the new debug env var CURL_FTP_PWD_STOP, to have a transfer return
from multi_perform() early with a connection that does not show
and pending input.
Reported-by: Brian Harris
Fixes https://github.com/curl/curl/issues/17578
Closes https://github.com/curl/curl/pull/17607
The checks for a connection being "too long idle" or "too old" where
rounding down the elapsed time to seconds before comparing to the
configured max values. This caused connections to be reused for up to
999ms longer than intended.
Change the compares to scale the configured seconds up to ms, so
connection will properly be "too old" 1 ms after the coonfigured values.
Fixes sporadic failures of test1542 on platforms where "sleep(2)"
returnes before 2 full seconds on the internal clock where passed.
Reported-by: Christian Weisgerber
URL: https://curl.se/mail/lib-2025-06/0004.htmlCloses#17571
When a pingpong based protocol tries to perform a connection disconnect,
it sends a sort of "logout" command to the server, unless the connection
is deemed dead.
But the disconnect might happen before pingpong data has been completely
sent, in which case sending the "logout" will not work. Check the
pingpong state and do not "logout" when data is pending.
This was detected as a condition in fuzzing that triggered a debug
assert in the pingpong sending.
Closes#17555
When ftp_done() is called to terminate the transfer, it needs to tear
down any open SECONDARY filter chain. The condition on when to do that
was relying on there to be a valid socket. This is not sufficient as the
socket is only set *after* happy eyeballing has decided on one.
Instead of checking for a valid conn->sock, check if any connection
filter is installed.
Fixes#17482
Reported-by: Rasmus Melchior Jacobsen
Closes#17491
When TYPE was skipped for an immediate STORE command and the server
replied fast and the EPRT data connection was not ready, the transfer
was not initated, leading to no upload.
Fixes#17394Closes#17428
Reported-by: JoelAtWisetech on github
The `struct Curl_dns_entry *` used to established a connection
do not have the connection's lifetime, but the transfer's lifetime
(of the transfer that initiates the connect).
`Curl_dns_entry *` is reference counted with the "dns cache". That
cache might be owned by the multi or the transfer's share. In the
share, the reference count needs updating under lock.
Therefore, the dns entry can only be kept *and* released using the
same transfer it was initially looked up from. But a connection is
often discarded using another transfer.
So far, the problem of this has been avoided in clearing the connection's
dns entries in the "multi_don()" handling. So, connections had NULL
dns entries after the initial transfers and its connect had been handled.
Keeping the dns entries in data->state seems therefore a better choice.
Also: remove the `struct Curl_dns_entry *` from the connect filters
contexts. Use `data->state.dns` every time instead and fail correctly
when not present and needed.
Closes#17383
Move curlx_ functions into its own subdir.
The idea is to use the curlx_ prefix proper on these functions, and use
these same function names both in tool, lib and test suite source code.
Stop the previous special #define setup for curlx_ names.
The printf defines are now done for the library alone. Tests no longer
use the printf defines. The tool code sets its own defines. The printf
functions are not curlx, they are publicly available.
The strcase defines are not curlx_ functions and should not be used by
tool or server code.
dynbuf, warnless, base64, strparse, timeval, timediff are now proper
curlx functions.
When libcurl is built statically, the functions from the library can be
used as-is. The key is then that the functions must work as-is, without
having to be recompiled for use in tool/tests. This avoids symbol
collisions - when libcurl is built statically, we use those functions
directly when building the tool/tests. When libcurl is shared, we
build/link them separately for the tool/tests.
Assisted-by: Jay Satiro
Closes#17253
Enable it for `_MSC_VER <= 1943`.
Omit it for not yet CI-tested MSVC versions, to avoid hitting unfixed
warnings emitted by future toolchain releases. It means we need
to explicitly opt-in to newer MSVC versions while fixing any new issues.
The newly enabled warnings did not reveal new issues. It hints that we
catch those with clang/gcc. Yet, these warnings may be useful for local
development done with MSVC.
Also:
- disable and document warnings that don't seem useful, unactionable,
or unfixable.
- disable and document warnings found in Windows SDK headers.
- tidy up a few comments, also to avoid
`-Wdocumentation-unknown-command`, part of llvm/clang `-Wall`.
```
lib\dynhds.h(159,29): error : unknown command tag name [-Werror,-Wdocumentation-unknown-command]
lib\ftp.c(337,15): error : unknown command tag name [-Werror,-Wdocumentation-unknown-command]
```
(This patch did not end up enabling `-Wall` for clang-cl.)
Closes#17050
"asyn" is the internal name under which both c-ares and threaded
resolver operate. Make the naming more consistent. Implement the c-ares
resolver in `asyn-ares.*` and the threaded resolver in `asyn-thrdd.*`.
The common functions are in `asyn-base.c`.
When `CURLRES_ASYNCH` is defined, either of the two is used and
`data->state.async` exists. Members of that struct vary for the selected
implementation, but have the fields `hostname`, `port` and `ip_version`
always present. This are populated when the async resolving starts and
eliminate the need to pass them again when checking on the status and
processing the results of the resolving.
Add a `Curl_resolv_blocking()` to `hostip.h` that relieves FTP and SOCKS
from having to repeat the same code.
`Curl_resolv_check()` remains the function to check for status of
ongoing resolving. Now it also performs internally the check if the
needed DNS entry exists in the dnscache and if so, aborts the asnyc
operation. (libcurl right now does not check for duplicate resolve
attempts. an area for future improvements).
The number of functions in `asyn.h` has been reduced. There were subtle
difference in "cancel()" and "kill()" calls, both replaced by
`Curl_async_shutdown()` now. This changes behaviour for threaded
resolver insofar as the resolving thread is now always joined unless
`data->set.quick_exit` is set. Before this was only done on some code
paths. A future improvement would be a thread pool that keeps a limit
and also could handle joins more gracefully.
DoH, not previously tagged under "asny", has its struct `doh_probes` now
also in `data->state.async`, moved there from `data->req` because it
makes more sense. Further integration of DoH underneath the "asyn"
umbrella seems like a good idea.
Closes#16963
Add a DEBUGASSERT() in Curl_dyn_free() that checks that Curl_dyn_init()
has been performed before.
Fix code places that did it wrong.
Fixes#16725Closes#16775
Fix the broken implementation to have `data->state` carry pointers into
connectdata members. Always dup the memory and free when easy handle
closes.
Closes#16733
Before this patch, standard `E*` errno codes were redefined on Windows,
onto matching winsock2 `WSA*` error codes, which have different values.
This broke uses where using the `E*` value in non-socket context, or
other places expecting a POSIX `errno`, e.g. file I/O, threads, IDN or
interfacing with dependencies.
Fix it by introducing a curl-specific `SOCKE*` set of macros that map to
`WSA*` on Windows and standard POSIX codes on other platforms. Then
verify and update the code to use `SOCKE*` or `E*` macro depending on
context.
- Add `SOCKE*` macros that map to either winsock2 or POSIX error codes.
And use them with `SOCKERRNO` or in contexts requiring
platform-dependent socket error codes.
This fixes `E*` uses which were supposed be POSIX values, not `WSA*`
socket errors, on Windows:
- lib/curl_multibyte.c
- lib/curl_threads.c
- lib/idn.c
- lib/vtls/gtls.c
- lib/vtls/rustls.c
- src/tool_cb_wrt.c
- src/tool_dirhie.c
- Ban `E*` codes having a `SOCKE*` mapping, via checksrc.
Authored-by: Daniel Stenberg
- Add exceptions for `E*` codes used in file I/O, or other contexts
requiring POSIX error codes.
Also:
- ftp: fix missing `SOCKEACCES` mapping for Windows.
- add `SOCKENOMEM` for `Curl_getaddrinfo()` via `asyn-thread.c`.
- tests/server/sockfilt: fix to set `SOCKERRNO` in local `select()`
override on Windows.
- lib/inet_ntop: fix to return `WSAEINVAL` on Windows, where `ENOSPC` is
used on other platforms. To simulate Windows' built-in `inet_ntop()`,
as tested on a Win10 machine.
Note:
- WINE returns `STATUS_INVALID_PARAMETER` = `0xC000000D`.
- Microsoft documentation says it returns `WSA_INVALID_PARAMETER`
(= `ERROR_INVALID_PARAMETER`) 87:
https://learn.microsoft.com/windows/win32/api/ws2tcpip/nf-ws2tcpip-inet_ntop#return-value
- lib/inet_ntop: drop redundant `CURL_SETERRNO(ENOSPC)`.
`inet_ntop4()` already sets it before returning `NULL`.
- replace stray `WSAEWOULDBLOCK` with `USE_WINSOCK` macro to detect
winsock2.
- move existing `SOCKE*` mappings from `tests/server` to
`curl_setup_once.h`.
- add missing `EINTR`, `EINVAL` constants for WinCE.
Follow-up to abf80aae38#16612
Follow-up to d69425ed7d#16615
Bug: https://github.com/curl/curl/pull/16553#issuecomment-2704679377Closes#16621
The line conversion reader, added in crfl and prefer_ascii mode was
incrementing data->state.infilesize for every line end converted. This
results in the wrong size to start a retry of an upload.
Eliminate the increment and check upload size in FTP less precise when
conversions are done.
Bug: https://issues.oss-fuzz.com/issues/402476456Closes#16683
Further testing with timeouts in event based processing revealed that
our current shutdown handling in the connection pool was not clear
enough. Graceful shutdowns can only happen inside a multi handle and it
was confusing to track in the code which situation actually applies. It
seems better to split the shutdown handling off and have that code
always be part of a multi handle.
Add `cshutdn.[ch]` with its own struct to maintain connections being
shut down. A `cshutdn` always belongs to a multi handle and uses that
for socket/timeout monitoring.
The `cpool`, which can be part of a multi or share, either passes
connections to a `cshutdn` or terminates them with a one-time, best
effort.
Add an `admin` easy handle to each multi and share. This is used to
perform all maintenance operations where no "real" easy handle is
available. This solves the problem that the multi admin handle requires
some additional initialisation (e.g. timeout list).
The share needs its admin handle as it is often cleaned up when no other
transfer or multi handle exists any more. But we need a `data` in almost
every call.
Fix file:// handling of errors when adding a new connection to the pool.
Changes in `curl` itself:
- for parallel transfers, do not set a connection pool in the share,
rely on the multi's connection pool instead. While not a requirement
for the new `cshutdn` to work, this is
a) helpful in testing to trigger graceful shutdowns
b) a broader code coverage of libcurl via the curl tool
- on test_event with uv, cleanup the multi handle before returning from
parallel_event(). The uv struct is on the stack, cleanup of the multi
later will crash when it tries to register sockets. This is a "eat
your own dogfood" related fix.
Closes#16508
Make it possible to build curl for Windows CE using the CeGCC toolchain.
With both CMake and autotools, including tests and examples, also in CI.
The build configuration is the default one with Schannel enabled. No
3rd-party dependencies have been tested.
Also revive old code to make Schannel build with Windows CE, including
certificate verification.
Builds have been throughougly tested. But, I've made no functional tests
for this PR. Some parts (esp. file operations, like truncate and seek)
are stubbed out and likely broken as a result. Test servers build, but
they do not work on Windows CE. This patch substitutes `fstat()` calls
with `stat()`, which operate on filenames, not file handles. This may or
may not work and/or may not be secure.
About CeGCC: I used the latest available macOS binary build v0.59.1
r1397 from 2009, in native `mingw32ce` build mode. CeGCC is in effect
MinGW + GCC 4.4.0 + old/classic-mingw Windows headers. It targets
Windows CE v3.0 according to its `_WIN32_WCE` value. It means this PR
restores portions of old/classic-mingw support. It makes the Windows CE
codepath compatible with GCC 4.4.0. It also adds workaround for CMake,
which cannot identify and configure this toolchain out of the box.
Notes:
- CMake doesn't recognize CeGCC/mingw32ce, necessitating tricks as seen
with Amiga and MS-DOS.
- CMake doesn't set `MINGW` for mingw32ce. Set it and `MINGW32CE`
manually as a helper variable, in addition to `WINCE` which CMake sets
based on `CMAKE_SYSTEM_NAME`.
- CMake fails to create an implib for `libcurl.dll`, due to not
recognizing the platform as a Windowsy one. This patch adds the
necessary workaround to make it work.
- headers shipping with CeGCC miss some things curl needs for Schannel
support. Fixed by restoring and renovating code previously deleted
old-mingw code.
- it's sometime non-trivial to figure out if a fallout is WinCE,
mingw32ce, old-mingw, or GCC version-specific.
- WinCE is always Unicode. With exceptions: no `wmain`,
`GetProcAddress()`.
- `_fileno()` is said to convert from `FILE *` to `void *` which is
a Win32 file `HANDLE`. (This patch doesn't use this, but with further
effort it probably could be.)
https://stackoverflow.com/questions/3989545/how-do-i-get-the-file-handle-from-the-fopen-file-structure
- WinCE has no signals, current directory, stdio/CRT file handles, no
`_get_osfhandle()`, no `errno`, no `errno.h`. Some of this stuff is
standard C89, yet missing from this platform. Microsoft expects
Windows CE apps to use Win32 file API and `FILE *` exclusively.
- revived CeGCC here (not tested for this PR):
https://building.enlyze.com/posts/a-new-windows-ce-x86-compiler-in-2024/
On `UNDER_CE` vs. `_WIN32_WCE`: (This patch settled on `UNDER_CE`)
- A custom VS2008 WinCE toolchain does not set any of these.
The compiler binaries don't contain these strings, and has no compiler
option for targeting WinCE, hinting that a vanilla toolchain isn't
setting any of them either.
- `UNDER_CE` is automatically defined by the CeGCC compiler.
https://cegcc.sourceforge.net/docs/details.html
- `UNDER_CE` is similar to `_WIN32`, except it's not set automatically
by all compilers. It's not supposed to have any value, like a version.
(Though e.g. OpenSSL sets it to a version)
- `_WIN32_WCE` is the CE counterpart of the non-CE `_WIN32_WINNT` macro.
That does return the targeted Windows CE version.
- `_WIN32_WCE` is not defined by compilers, and relies on a header
setting it to a default, or the build to set it to the desired target
version. This is also how `_WIN32_WINNT` works.
- `_WIN32_WCE` default is set by `windef.h` in CeGCC.
- `_WIN32_WCE` isn't set to a default by MSVC Windows CE headers (the
ones I checked at least).
- CMake sets `_WIN32_WCE=<ver>`, `UNDER_CE`, `WINCE` for MSVC WinCE.
- `_WIN32_WCE` seems more popular in other projects, including CeGCC
itself. `zlib` is a notable exception amongst curl dependencies,
which uses `UNDER_CE`.
- Since `_WIN32_WCE` needs "certain" headers to have it defined, it's
undefined depending on headers included beforehand.
- `curl/curl.h` re-uses `_WIN32_WCE`'s as a self-guard, relying on
its not-(necessarily)-defined-by-default property:
25b445e479/include/curl/curl.h (L77)
Toolchain downloads:
- Windows:
https://downloads.sourceforge.net/cegcc/cegcc/0.59.1/cegcc_mingw32ce_cygwin1.7_r1399.tar.bz2
- macOS Intel:
https://downloads.sourceforge.net/cegcc/cegcc/0.59.1/cegcc_mingw32ce_snowleopard_r1397.tar.bz2Closes#15975
Remove `blocking` argument from cfilter's connect method.
Implement blocking behaviour in Curl_conn_connect() instead for all
filter chains.
Update filters implementations. Several of which did never use the
paramter (QUIC for example). Simplifies connect handling in TLS filters
that no longer need to loop
Fixed a blocking connect call in FTP when waiting on a socket accept()
which only worked because the filter did not implement it.
Closes#16397
- add hex and octal parsers to the Curl_str_* family
- make curlx_strtoofft use these parsers
- remove all use of strtol() and strtoul() in library code
- generally use Curl_str_* more than strtoofft, for stricter parsing
- supports 64-bit universally, instead of 'long' which differs in size
between platforms
Extended the unit test 1664 to verify hex and octal parsing.
Closes#16336
Instead of strtoul() and strtol() calls.
Easier API with better integer overflow detection and built-in max check
that now comes automatic everywhere this is used.
Closes#16319
