Use non-deprecated CRT function variants on Windows.
- introduce `curlx_fdopen()`, `curlx_close()` and use them. Map them to
non-deprecated, underscored, CRT functions on Windows.
- replace `close()` uses with either `sclose()` (for sockets) or
`curlx_close()` (for files).
- map `fileno`, `unlink`, `isatty` to their non-deprecated, underscored,
versions on Windows.
- tool_dirhie: map `mkdir` to `_mkdir` on Windows.
- easy: use `_strdup()` on Windows, regardless of how `HAVE_STRDUP` is
set.
- cmake: assume `HAVE_STRDUP` on Windows. To allow dropping a detection
hack using `_CRT_NONSTDC_NO_DEPRECATE` with MSVC. Windows always has
`_strdup()` which the code uses, but also needs `HAVE_STRDUP` defined
to disable curl's own `strdup()` implementation.
- curl_setup.h: drop `_CRT_NONSTDC_NO_DEPRECATE` as no longer necessary.
Closes#20212
New define USE_SO_NOSIGPIPE in curl_setup.h, for now set whenever
SO_NOSIGPIPE is defined. Maybe overridden in the future on systems where
this does not work.
With USE_SO_NOSIGPIPE defined, set SO_NOSIGPIPE on all sockets created
by libcurl and fail the creation when setsockopt() fails.
Closes#20370
Reduce mentions of libcurl versions like "since 7.xx ..." in option
descriptions to reduce clutter and make the texts easier to read. Keep
them in, or move them to, the HISTORY or DEPRECATED sections
The last version 7 release (7.88.1) shipped on Februrary 20, 2023.
Closes#20369
Drop separate `TrackMemory` (aka `CURLDEBUG`) debug feature.
After recent changes (thread-safety,
193cb00ce9, and updates leading up to
it), `TrackMemory` is unlikely to cause build or runtime issues.
To simplify builds and debug options, enable `TrackMemory`
unconditionally for debug-enabled (aka `DEBUGBUILD`) builds. Before
this patch, this was already the default, with an option to disable
it, or enable it in non-debug-enabled builds.
Note, in practice these two debug options already went hand in hand. It
was not possible to toggle them separately for a long time due to bugs,
before 59dc9f7e69 (2024-05-28) fixed it.
This patch also removes/deprecates separate knobs and feature flags for
`TrackMemory`:
- autotools: `--enable-curldebug`/`--disable-curldebug`
- cmake: `-DENABLE_CURLDEBUG=ON`/`OFF`
- C macro: `CURLDEBUG`
- libcurl: `CURL_VERSION_CURLDEBUG` symbol deprecated in favor
of `CURL_VERSION_DEBUG`. They always return the same value after this
patch.
Also:
- drop `TrackMemory` from `curl -V` output.
- rename internal `CURLDEBUG` macro to `CURL_MEMDEBUG` internally.
To avoid confusion with `DEBUGBUILD`, but to keep guarding
`TrackMemory`-related internals for readability.
- runtests: bind `TrackMemory` to debug feature. Keep it a separate
test feature requirement, for clarity.
- CI: drop test builds for combinations of the two options.
- GHA/linux: no longer disable TrackMemory in the TSAN job.
Ref: https://github.com/curl/curl/pull/20328#issuecomment-3754528407Closes#20331
For reducing binary size. Also to remove (or greatly mitigate)
the side-effect of using "unity" builds. Similar to `-Wl,--gc-sections`
on non-Apple platforms.
For example with curl-for-win builds, macOS arm+intel:
curl (unity): 7.7MB -> 6.8MB
libcurl.dylib (unity): 7.2MB -> 6.4MB
trurl /w static libcurl (!unity): 535KB -> 251KB (same size with unity)
Ref: c4008d658aCloses#20346
- It is slower and uses more memory than the alternatives and is only
experimental in curl.
- We disable a few tests for OpenSSL-QUIC because of flakiness
- It gets little attention from OpenSSL and we have no expectation of the
major flaws getting corrected anytime soon.
- No one has spoken up for keeping it
- curl users building with vanilla OpenSSL can still use QUIC through the
means of ngtcp2
Closes#20226
With the same semantics as Apple SecTrust, in both libcurl and the curl
tool, when using non-Schannel TLS backends. In practice it means that
it makes TLS work without manually or implicitly configuring a CA bundle
`.crt` file, such as `curl-ca-bundle.crt`.
To enable:
- autotools: `--enable-ca-native`
- cmake: `-DCURL_CA_NATIVE=ON`
- CPPFLAGS: `-DCURL_CA_NATIVE`
When enabled:
- enables `CURLSSLOPT_NATIVE_CA` (libcurl) / `--ca-native`
and `--proxy-ca-native` (curl tool) options by default.
- unsafe search for an on-disk CA bundle gets disabled by default.
Equivalent to `--disable-ca-search` with autotools,
`-DCURL_DISABLE_CA_SEARCH=ON` with CMake.
- build-time detection of CA bundle and CA path gets disabled. As with
Apple SecTrust. This was already the default for Windows.
- native CA can be disabled at run-time with the `--no-ca-native`
and/or `--no-proxy-ca-native` command-line options.
Rationale: This build option:
- has a repeat and active interest from packagers and users.
- helps integrating curl with Windows for those who need this.
- it also applies to macOS: #17525
Shipped in curl 8.17.0.
- makes it trivial to use custom certs configured on the OS.
- frees applications/packagers/users from the task of securely
distributing, and keeping up-to-date, a CA bundle.
- frees potentially many curl tool from configuring a CA bundle manually
to access HTTPS (and other TLS) URLs. This is traditionally difficult
on Windows because there is no concept of a universal, protected,
non-world-writable, location on the file system to securely store
a CA bundle.
- allows using modern features regardless of Windows version. Some of
these features are not supported with Schannel (e.g. HTTP/3, ECH) on
any Windows version.
- is necessary for HTTP/3 builds, where bootstrapping a CA bundle is not
possible with Schannel, because MultiSSL is not an option, and HTTP/3
is not supported with Schannel.
Ref: #16181 (previous attempt)
Ref: https://github.com/curl/curl/discussions/9348
Ref: https://github.com/curl/curl/issues/9350
Ref: https://github.com/curl/curl/pull/13111
Ref: https://github.com/microsoft/vcpkg/pull/46459#issuecomment-3162068701
Ref: 22652a5a4c#14582
Ref: eefd03c572#18703Closes#18279
After this patch curl requires targeting Vista or newer, and a toolchain
with Vista support.
Supported MSVC compilers (VS2010+) all support Vista:
- VS2012+ target Win8 (or later) by default.
- VS2010 targets Win7 by default.
Supported mingw-w64 versions (v3+) all support Vista:
- mingw-w64 v9+ target Win10 by default.
- mingw-w64 v8 and older target Server 2003 (~XP) by default.
After this patch it may be necessary to override the default Windows
target version to Vista (or newer) via:
autotools: `CPPFLAGS=-D_WIN32_WINNT=0x0600`
cmake: `-DCURL_TARGET_WINDOWS_VERSION=0x0600`
- mingw-w64 v6+ allow changing the default at toolchain build-time.
Notes:
- For non-MSVC, non-mingw-w64 toolchains, `if_nametoindex` needs to be
allowlisted in `curl_setup.h`, if they do support it.
Fixes#17985 (discussion)
Closes#18009
- reorder functions to not need forward declarations.
- sync `ephiperfifo.c` and `evhiperfifo.c`.
- drop redundant casts for `calloc()` return value.
- ephiperfifo: silence unused variable warning.
- fix indent and apply clang-format more.
Closes#20296
To simplify the directory layout.
- OS400 and vms support move from `packages` to `projects`.
- Windows README and `generate.bat` files move from `projects`
to `projects/Windows`.
Closes#20271
To run checksrc and spacecheck on the source tree. Also for cmake
to sync up with autotools' `checksrc` target.
- cmake: `curl-lint`
With `-DCURL_LINT=ON`, checks run automatically for all targets.
- autotools: `lint`
Closes#20175
- asyn-thrdd.c: scope an include.
- apply more clang-format suggestions.
- tidy-up PP guard comments.
- delete empty line from the top of headers.
- add empty line after `curl_setup.h` include where missing.
- fix indent.
- CODE_STYLE.md: add `strcpy`.
Follow-up to 8636ad55df#20088
- lib1901.c: drop unnecessary line.
Follow-up to 436e67f65b#20076Closes#20070
Also:
- examples/hsts-preload: apply the same change as it's based on lib1915
in tests. Make a local clone of `curlx_strcopy()`. Then drop the
`_CRT_SECURE_NO_WARNINGS` hack, that's no longer necessary.
- curl_setup.h: delete `strcpy()` from the `_CRT_SECURE_NO_WARNINGS`
list.
Closes#20076
1. With `MBEDTLS_SSL_PROTO_TLS1_2` not enabled, the mbedTLS code was not
able to connect to any server due to broken logic in curl's
`mbed_set_ssl_version_min_max()`. Now it correctly sets the minimum
supported TLS version based on what is compiled in the library.
2. If debugging is enabled, move the debugging enabling earlier in the
`mbed_connect_step1()` so that verbose errors are actually displayed if
failures happen (see the previous point -- it would've made debugging
that issue easier).
3. Remove the constant `mbedtls_x509_crt_profile_fr` and instead use
mbedTLS-included profile `mbedtls_x509_crt_profile_next` with
`mbedtls_ssl_conf_cert_profile()`. This will follow the latest standards
as new mbedTLS versions are released (rather than being stuck-in-time
until someone comes along to fix what was hard-coded here). This has the
immediate benefit of no longer supporting SHA1 certs and insecure RSA
key-lengths (1024). This fix immediately prevents previously possible
MITM attacks (SHA1 hashes and RSA-1024 keys can be forged relatively
easily by nation-state actors and criminal organizations with
deep-pockets).
4. Added [predictive
resistance](https://mbed-tls.readthedocs.io/en/latest/kb/how-to/add-a-random-generator/#enabling-prediction-resistance)
to the random number generator (adding more entropy to the RNG).
5. Split the random number generator into initialization, the actual
random generation, and the "freeing" of the resources. This
significantly reduces the overhead of using the RNG.
6. Removed the separate RNG function in the TLS connect stage (instead
use the "main" one) and remove the ad-hoc threading support. Instead
properly document how to enable threading in mbedTLS. As it was, other
internals of mbedTLS could have race conditions (in the RSA module in
particular) if `MBEDTLS_THREADING_C` was *not* enabled. And if it is
enabled, then these race-conditions cannot happen. And also, if
MBEDTLS_THREADING_C is enabled then the RNG functions [are fully
thread-safe](https://mbed-tls.readthedocs.io/en/latest/kb/development/thread-safety-and-multi-threading/).
So, the previous ad-hoc threading support was both partial and broken.
7. Enable support for disabling `MBEDTLS_PEM_PARSE_C`.
8. Add support for `CURLOPT_SSLCERTTYPE` so user can specify `PEM` or
`DER` and get faster execution.
Closes#19983
Move existing `Curl_rename()` `rename()` wrapper from lib to
curlx/fopen, and make it a curlx macro/function. To allow using
the local worker function to fixup long filenames on Windows.
Then fix the Windows-specific rename implementation to support long
filenames. This operation may happen when using a cookie jar, HSTS cache
or alt-svc cache, via libcurl or the curl tool.
Before this patch, when passing a long filename to the above options,
a `<random>.tmp` file was left on the disk without renaming it to the
filename passed to curl. There was also 1 second delay for each
attempted rename operation.
Also:
- checksrc: ban raw `rename()` and `MoveFileEx*()` functions.
- Note: `Curl_rename()` returned 1 on failure before this patch, while
`curlx_rename()` returns -1 after, to match POSIX `rename()`.
Refs:
https://learn.microsoft.com/windows/win32/api/winbase/nf-winbase-movefileexahttps://learn.microsoft.com/windows/win32/fileio/maximum-file-path-limitation
Ref: #20040Closes#20042
It makes them work in Schannel's CA bundle loader, and curl tool's
set/get file timestamp operations (e.g. in `-R`/`--remote-time`). Also
to match file open operations, that already support long filenames.
E.g. when using `--remote-time`, fixing:
```
Warning: Failed to set filetime 1741363917 on outfile: CreateFile failed:
Warning: GetLastError 0x00000003
```
The special long filename logic is necessary to support Windows releases
prior to Windows 10 v1607. With the latter, it's possible to opt-in to
this behavior via a manifest setting. Note that Windows itself also needs
to opt-in to support this. Finally note that curl itself needs passing
`--globoff` to let long filenames through, pending #20044 and #20046.
Refs:
https://learn.microsoft.com/windows/win32/api/fileapi/nf-fileapi-createfileahttps://learn.microsoft.com/windows/win32/fileio/maximum-file-path-limitation
Ref: #8361
Inspired by: #19286
Inspired-by: Mathesh V
Closes#19286Closes#20040
Also:
- vquic-tls.h: do not include unused headers for non-H3 builds.
- autotools: stop looking for `openssl/x509.h` header.
- cmp-config.pl: delete exception for `openssl/x509.h`.
- examples: format/comment sync between the two touched files.
- openssl: drop unused `curlx/wait.h` include.
Closes#20049
Always use curlx_now() when calling Curl_pgrs_now(data). Tests with the
"manual" updates to now proved differ more then 100ms in parallel testing.
Add `curlx_nowp()` to set current time into a struct curltime.
Add `curlx_ptimediff_ms() and friends, passing pointers.
Update documentation.
Closes#19998
- apply more clang-format.
- lib/version: use `CURL_ARRAYSIZE()`.
- INSTALL-CMAKE.md: sync-up an option description with others.
- examples: delete unused main args.
- examples/ftpgetinfo: document `_CRT_SECURE_NO_WARNINGS` symbol.
- delete remaining stray duplicate lines.
- acinclude.m4: drop an unnecessary x-hack.
- vtls/mbedtls: join a URL split into two lines.
- src/tool_cb_see: add parentheses around macro expressions.
- src/tool_operate: move literals to the right side of comparisons.
- libtests: sync up fopen/fstat error messages between tests.
- curl_setup.h: replace `if ! defined __LP64` with `ifndef __LP64`.
I assume it makes no difference on Tandem systems, as the latter form
is already used in `include/curl/system.h`.
Closes#20018
