Hold them for half the normal lifetime. Helps when told to transfer N
URLs in quick succession that all use the same non-resolving hostname.
Done by storing a DNS entry with a NULL pointer for 'addr'.
Previously an attempt was made in #12406 by Björn Stenberg that was
ultimately never merged.
Closes#18157
In WebAssembly, using `TCP_NODELAY` fails with:
```
* Could not set TCP_NODELAY: Protocol not available
```
Add a new feature macro in `curl_setup.h` telling whether `TCP_NODELAY`
is known to be supported at runtime, when defined at compile-time.
Keep `TCP_NODELAY` guards at their current positions to ensure the
necessary headers (e.g. `netinet/tcp.h` and `netinet/in.h`) define it.
Reported-by: Jeroen Ooms
Fixes#17974Closes#18155
Adds `curl_off_t curl_multi_get_offt(CURLM *multi_handle, CURLMinfo_offt
info)` to the multi interface with enums:
* CURLMINFO_XFERS_CURRENT: current number of transfers
* CURLMINFO_XFERS_RUNNING: number of running transfers
* CURLMINFO_XFERS_PENDING: number of pending transfers
* CURLMINFO_XFERS_DONE: number of finished transfers to read
* CURLMINFO_XFERS_ADDED: total number of transfers added, ever
Add documentation for functions and info enums.
Add use in the curl command line tool to replace two static
variables counting the same "from the outside".
refs #17870Closes#17992
`getsock()` calls operated on a global limit that could
not be configure beyond 16 sockets. This is no longer adequate
with the new happy eyeballing strategy.
Instead, do the following:
- make `struct easy_pollset` dynamic. Starting with
a minimal room for two sockets, the very common case,
allow it to grow on demand.
- replace all protocol handler getsock() calls with pollsets
and a CURLcode to return failures
- add CURLcode return for all connection filter `adjust_pollset()`
callbacks, since they too can now fail.
- use appropriately in multi.c and multi_ev.c
- fix unit2600 to trigger pollset growth
Closes#18164
Instead of using integer seconds. Also: if the cache contains over
30,000 entries after first pruning, it makes anoter round and removes
all entries that are older than half the age of the oldest entry until
it goes below 30,000.
Closes#18160
- explains its purpose better
- make it global static const
- added an assert for a condition that should never happen (that we
also catch run-time)
Closes#18152
Setting the path component to contain the letters:
! $ & ' ( ) { } [ ] * + , ; = : @
now leaves them un-encoded when CURLU_URLENCODE is used.
Amended test 1560 to verify.
Reported-by: Jeroen Ooms
Fixes#17977Closes#18024
This patch bumps the size of these macros from `int` to `long`, while
keeping their actual values the same. It may cause incompatibilities in
user code, requiring the bump of holder variables and/or adding casts:
- CURLPROTO_HTTP
- CURLPROTO_HTTPS
- CURLPROTO_FTP
- CURLPROTO_FTPS
- CURLPROTO_SCP
- CURLPROTO_SFTP
- CURLPROTO_TELNET
- CURLPROTO_LDAP
- CURLPROTO_LDAPS
- CURLPROTO_DICT
- CURLPROTO_FILE
- CURLPROTO_TFTP
- CURLPROTO_IMAP
- CURLPROTO_IMAPS
- CURLPROTO_POP3
- CURLPROTO_POP3S
- CURLPROTO_SMTP
- CURLPROTO_SMTPS
- CURLPROTO_RTSP
- CURLPROTO_RTMP
- CURLPROTO_RTMPT
- CURLPROTO_RTMPE
- CURLPROTO_RTMPTE
- CURLPROTO_RTMPS
- CURLPROTO_RTMPTS
- CURLPROTO_GOPHER
- CURLPROTO_SMB
- CURLPROTO_SMBS
- CURLPROTO_MQTT
- CURLPROTO_GOPHERS
- CURLPROTO_ALL
Also:
- keep existing cast within the documentation to make sure it applies
to older curl versions as well.
Closes#18136
When an SSL_write() blocks we need to retry it with the
same length as before or stupid OpenSSL freaks out. Remember
it, limit any longer sends and fail shorter ones.
Fixes#18121
Reported-by: adamse on github
Closes#18132
test_02_36 tests h1/h2/h3 with urls longer than 1/16/32/64K.
Protocols behave the same until the size exceed 64k when h2 frame limits
bite and h3 exhibits a different http status.
Failed attempt to reproduce #18121Closes#18129
Add tests of CURLINFO_TLS_SSL_PTR and its returned value in test client
'hx-download'. Use obtained pointer to look up the negotiated TLS
version.
Update manpage of CURLINFO_TLS_SSL_PTR to also describe the behaviour of
wolfSSL similar to OpenSSL. Fix the wolfSSl implementation for TCP to
behave like that. Update the QUIC queries.
Fix rustls `get_internals()` to return the rustls_connection* and not
the address of the pointer.
Assisted-by: Viktor Szakats
Closes#18066
When `CURLOPT_HAPPY_EYEBALLS_TIMEOUT_MS` expires, start the next ip
connect attempt, but keep all ongoing attempts alive.
Separate happy-eyeballs connection filter into own source files.
Closes#18105
Schannel is not supported by UWP. SSPI is also required by Schannel in
curl, and SSPI also isn't supported by UWP.
mingw-w64 is able to create such build regardless (my guess: due to API
parts not accurately marked as UWP-only), but the binary is unlikely
to work. With MSVC the failure happens at build-time.
Ref: https://learn.microsoft.com/windows/win32/api/sspi/nf-sspi-initsecurityinterfacea#requirements
Ref: https://learn.microsoft.com/windows/win32/secauthn/initializesecuritycontext--schannel#requirements
Drop all UWP-related logic, including two related feature checks, that
can now be permanently enabled.
Also:
- build: show fatal error for Schannel in UWP mode.
- build: do not allow enabling SSPI in UWP mode.
- drop undocumented option `DISABLE_SCHANNEL_CLIENT_CERT`. Added without
mention in an unrelated commit. The PR text says to save size. On x64
this is 0.3%, or 4KB out of 1.3MB. The tiny gain doesn't justify
an extra build variant. Ref: 8beff43559
- move `MPROTO_SCHANNEL_CERT_SHARE_KEY` closer to its use.
- replace commented block with `#if 0`.
Reviewed-by: Jay Satiro
Follow-up to cd0ec4784c#17089Closes#18116
Always available in supported mingw-w64 and MSVC compilers, except
in UWP mode. For mingw32ce this macro is defined later in the code.
Also available in OpenWatcom 2.
ce6c37eb29/bld/w32api/include/winerror.mhCloses#18108
They are defined by all mingw-w64 versions and all supported MSVC
versions (VS2008 and up).
Also by OpenWatcom 2:
ce6c37eb29/bld/w32api/include/winerror.mh
mingw32ce misses `CRYPT_E_NOT_IN_REVOCATION_DATABASE`.
Closes#18092
They are defined by all mingw-w64 versions and all supported MSVC
versions (SDK 7.1A+).
Also by OpenWatcom 2:
ce6c37eb29/bld/w32api/include/wincrypt.mh
These aren't defined by mingw32ce. And likely defined by MS WinCE SDK,
but curl code doesn't use them in WinCE builds.
Closes#18093
In case they are used again in the future, the fallbacks are not
necessary for the supported mingw-w64 (v3+) and MSVC versions.
Follow-up to 6238888ca7#15621Closes#18091
Since the content varies during connection setup and while doing it
(eyeballing), remove these strcut from `connectdata` and replace use
with querying the connection filters. Those keep that information
already.
Change the info logging of established connections to also give the
local address and port.
Closes#17960
- add a message if a boolean is set to something not 1 or 0 (as it might be
made to mean something else in a future)
- use 's->' in all setopt_* functions
Closes#17887
New multi option CURLMOPT_NETWORK_CHANGED with a long bitmask value:
- CURLM_NWCOPT_CLEAR_CONNS: do not reuse existing connections, close all
idle connections.
- CURLM_NWCOPT_CLEAR_DNS: clear the multi's DNS cache.
All other bits reserved for future extensions.
Fixes#17225
Reported-by: ウさん
Closes#17613
Early mingw-w64 releases missed it, but by requiring v3.0, this is no
longer an issue. Supported Visual Studio SDKs also offer it.
Follow-up to a28f5f68b9#18010Closes#18057
Before curl 8.14.0, when pending was a list, `process_pending_handles()`
move a single transfer to processing. In 8.14.0 we changed that to move
all pending transfers to processing. This lead to unwanted performance
drops as reported in #18017.
Restore the old behaviour.
While the old behviour is better, the overall handling of "pending"
transfers is not optimal, since we do not keep track of the "condition"
a pending transfer is waiting on. This means, when moving a single,
pending transfer, we might move one that still cannot be processed while
another that could is kept pending.
Since we trigger `process_pending_handles()` from various changes, the
stalled pending will eventually make it to the processing queue, but
this is not optimal.
Fixes#18017
Reported-by: rm-rmonaghan on github
Closes#18056
This commit replaces the usage of the old deprecated sftp_async API with
the new sftp_aio API for remote file reading.
Signed-off-by: Eshan Kelkar <eshankelkar@galorithm.com>
Closes#17440
Fixing:
- HTTPS-RR builds with c-ares and Linux MUSL.
- curl-for-win minimal builds with Linux MUSL.
It should fix all other kinds of entaglement between curl's redefintions
of system symbols and system (or 3rd-party) headers sensitive to that.
It also syncs memory override behavior between unity & non-unity builds,
thus reducing build variations.
The idea is to define and declare everything once in `curl_setup.h`,
without overriding any system symbols with curl ones yet. Then, like
before this patch, override them, if necessary, in each source file via
`curl_memory.h` and `memdebug.h`, after including system headers.
To ensure a clean slate with no overrides at the beginning of each
source file, reset all of them unconditionally at the end of
`curl_setup.h`, by including `curl_mem_undef.h`. (This assumes
`curl_setup.h` is always included first, which is already the case
throughout the codebase.)
`curl_mem_undef.h` can also be included explicitly wherever overrides
are causing problems. E.g. in tests which use unity-style builds and
a previously included `curl_memory.h`/`memdebug.h` can be spilling into
other source files.
The simplified role of the two override headers:
- `curl_memory.h`: overrides system memory allocator functions to
libcurl ones, when memory tracing (aka `CURLDEBUG`) is disabled.
- `memdebug.h`: overrides system memory allocator and some other
functions to curl debug functions, when memory tracing is enabled.
Changed made in this patch, step-by-step:
- curl_memory.h: move allocator typedefs and protos to `curl_setup.h`.
- memdebug.h: move `ALLOC_*` macros to `curl_setup.h`.
- memdebug.h: move allocator protos to `curl_setup.h`.
- memdebug.h: move `Curl_safefree()` macro to `curl_setup.h`.
(it's a regular macro, with a one-time, global, definition.)
- curl_memory.h: move system symbol undefs to a new, separate header:
`curl_mem_undef.h`.
- curl_setup.h: include `curl_mem_undef.h` at the end, unconditionally,
to reset system symbol macros after each inclusion.
- handle `sclose()` and `fake_sclose()` in `curl_setup.h`. They are not
system symbols, a one-time definition does the job.
Also:
- GHA/linux: enable unity mode for the HTTP-RR c-ares MUSL job.
Follow-up to 17ab4d62e6#16413
That said, I'd still find it better to avoid redefining system macros.
To communicate clearly the fact that they are not the original system
calls and they do behave differently. And, it would allow dropping the
undef/redef dance in each source file, and maintaining the logic with
it. The "last #include files should be in this order" comments in each
source would also become unnecessary. Also the trick of using
`(func)` (or interim macros) to call the non-overridden function where
required. This method works for printf and most everything else already.
For `_tcsdup`, socket and fopen functions this could work without
disturbing the codebase much.
Ref: #16428 (clean reboot of)
Closes#17827
This patch bumps the size of these macros from `int` to `long`, while
keeping their actual values the same. It may cause incompatibilities in
user code, requiring the bump of holder variables and/or adding casts:
- CURLPROXY_HTTP
- CURLPROXY_HTTP_1_0
- CURLPROXY_HTTPS
- CURLPROXY_HTTPS2
- CURLPROXY_SOCKS4
- CURLPROXY_SOCKS4A
- CURLPROXY_SOCKS5
- CURLPROXY_SOCKS5_HOSTNAME
Also:
- keep existing cast within the documentation to make sure it applies
to older curl versions as well.
Closes#18054
Add a connection filter query to obtained the negotiated ALPN
protocol to check in setup/protocols how the connection needs
to behave.
Remove the members `alpn` and `proxy_alpn` from `connectdata`.
Closes#17947
mingw-w64 3.0 was released on 2013-09-20. Offered by Debian jessie.
1.0 and 2.0 were released in 2011. It seems unlikely that many people
use them. The oldest downloadable toolchain (that I know of) comes with
3.0. Due to this, older versions weren't CI tested, and probably seldom
tested elsewhere. The last bugfix update for both 1.0 and 2.0 was
released in 2015.
curl can now assume availability of these 3.0 features/fixes:
- 64-bit file offsets.
- `ADDRESS_FAMILY` type.
- `__MINGW_PRINTF_FORMAT` macro. (in public curl headers)
Public curl headers keep supporting older mingw-w64 versions.
Fixes#17984Closes#18010
- tests: merge cmake commands.
- tests: use `target_compile_definitions()`.
- tests/server: use generator expression for platform-specific macro.
- tests/unit: sync `Makefile.am` comment with cmake.
- tests/unit: merge two `AM_CPPFLAGS` lines to keep synced with cmake.
- tests: move macro definitions to `first.h` headers from build level.
`CURL_NO_OLDIES`, `CURL_DISABLE_DEPRECATION`, `WITHOUT_LIBCURL`,
`CURL_STATICLIB` (for servers).
To share more logic.
Pass `CURL_STATICLIB` in server on all platforms for simplicity.
(On non-Windows, it's a no-op. It's already done like this with curlu
and libcurltool.)
Also for lib:
- lib: merge commands.
- lib: sync macro order with tests (also in `Makefile.am`).
Closes#17768
Fix compile error when building with `--disable-verbose`.
Adjust pytest to skip when curl is not a debug build but needs
traces.
Follow-up to b453a447ceCloses#18053
Now that multi keeps the "dirty" bitset, the detection of possibly
stalling transfers needs to adapt. Before dirty, transfers needed
to expose a socket to poll or a timer to wait for.
Dirty transfer might no longer have a timer, but will run, so do
not need to report a socket. Adjust the assert condition.
Fixes#18046
Reported-by: Viktor Szakats
Closes#18051
This patch bumps the size of these constants from `int` to `long`, while
keeping their actual values the same. It may cause incompatibilities in
user code, requiring the bump of holder variables and/or adding casts:
- CURLFTP_CREATE_DIR
- CURLFTP_CREATE_DIR_NONE
- CURLFTP_CREATE_DIR_RETRY
- CURLFTPAUTH_DEFAULT
- CURLFTPAUTH_SSL
- CURLFTPAUTH_TLS
- CURLFTPMETHOD_DEFAULT
- CURLFTPMETHOD_MULTICWD
- CURLFTPMETHOD_NOCWD
- CURLFTPMETHOD_SINGLECWD
- CURLFTPSSL_CCC_ACTIVE
- CURLFTPSSL_CCC_NONE
- CURLFTPSSL_CCC_PASSIVE
Also:
- keep existing casts within the documentation to make sure it applies
to older curl versions as well.
Closes#17797
Syncing winbuild and VS Project File builds with the same fix applied
to cmake and autotools builds earlier.
Also fixes these warnings seen in the VisualStudioSolution (VS2013) job
on AppVeyor CI:
```
lib\hostip.c(148): warning C4090: 'function' : different 'const' qualifiers
lib\hostip.c(155): warning C4090: 'function' : different 'const' qualifiers
```
Ref: https://ci.appveyor.com/project/curlorg/curl/builds/52470650/job/gslnjrdxnd8b9mtv#L180
Went unnoticed because warnings are not promoted to error in these builds.
winbuild CI jobs did not hit this warning for some reason.
Follow-up to 8537a5b0bc#16577Closes#18045
Do not declare local inet_pton/inet_ntop implementations when they are
not used. In this case the same symbol is defined as a macro and mapped
to the system implementation.
Syncing this with their definitions.
Closes#18043
