Rework the way `tool_hugehelp.c` is included in builds.
After this patch, with `./configure` and CMake `tool_hugehelp.c` is only
compiled when building with manuals enabled. With manuals disabled this
source file is not used anymore. The method is similar to how
8a3740bc8e implemented `tool_ca_embed.c`.
`./configure` always generates it as before, otherwise the build fails.
- winbuild: rework to not need `buildconf.bat`, but automatically use
`tool_hugehelp.c` if present (e.g. when building from an official
source tarball) and enable `USE_MANUAL` accordingly.
- `buildconf.bat`: after dropping `tool_hugehelp.c` generation, the only
logic left was `cp Makefile.dist Makefile`. This allowed to launch
winbuild builds via GNU Make in a Git repo. Drop this option together
with the batch file.
- build `libcurltool` without `USE_MANUAL` macro to exclude the manual
and the dependence on the generator commands. Drop relying on
`UNITTESTS` for this purpose.
Follow-up to 96843f4ef7#16068
- `src/mkhelp.pl`: include `tool_hugehelp.h` before using `USE_MANUAL`
to have it set in `config-*.h` builds with source tarballs created
with manual but without zlib.
Closes#16081
CMake builds using the Xcode generator broke with an error saying it
doesn't support multiple targets depending on the same custom commands.
These custom commands are generating `tool_hugehelp.c` and
`tool_c_embed.c` for the curl tool and libcurltool.
`unit1394` and `unit1604` tests use libcurltool to test tool-specific
functions. They don't need hugehelp and ca-embed. It's thus safe to
disable and exclude them when compiling the sources for libcurltool.
Use the `UNITTESTS` macro to detect a libcurltool build within C.
After this patch these sources are solely used for building the curl
tool. Making the build compatible with the CMake Xcode generator.
Apply the change to autotools too to keep build systems synchronized.
Follow-up to 12a6de2f66#16043Closes#16068
iOS:
- add jobs with autotools, CMake, CMake Xcode generator.
The Xcode generator is >10x slower than Unix Makefiles. Keep it
because it's the one recommended by CMake and for having its own
quirks we may want to know about.
- build, cache and use LibreSSL for these jobs.
With workaround for an iOS build issue fixed in master.
- make Xcode generator work by explicitly disabling code signing.
- make tests and examples build with the Xcode generator by setting
`-DMACOSX_BUNDLE_GUI_IDENTIFIER=se.curl`, to avoid
"Bundle identifier is missing" errors.
- cmake: disable `CURL_USE_PKGCONFIG` by default for Apple device.
- cmake: add `stdc++` library for BoringSSL and AWS-LC, with
`OPENSSL_USE_STATIC_LIBS=ON` set.
- cmake: add workaround for Xcode generator issue, where it cannot
handle two targets depending on one custom command. A better fix may
be dropping `tool_hugehelp.c` and `tool_ca_embed.c` from curltool
library. For a future PR.
Android:
- add vcpkg to Android jobs, enable dependencies.
Assisted-by: Tal Regev via #16045
- make vcpkg work with autotools.
- pass `--with-brotli` to autotools to detect the vcpkg-supplied brotli.
- enable BoringSSL for Android and add a job with it.
- silence 457 CMake configure warnings about the Android NDK CMake
scripts targeting freshly deprecated CMake versions.
These were much more involved than imagined. Basically nothing works out
of the box, and when combined, everything becomes a unique edge case.
autotools builds were a much easier to make work than CMake ones.
Also:
- GHA/non-native: re-sync names to be shorter and more aligned with
other workflows.
- GHA: add `persist-credentials: false` where missing.
Unresolved issues:
- `OPENSSL_ROOT_DIR` ignored/mis-used when pointing it to LibreSSL.
CMake seems to prepend the sysroot to the passed absolute directory.
Found no workaround.
- CMake when combined with Android, both the Google-recommended method
and the built-in CMake method fail to provide a way to avoid
`pkg-config` packages at system directories. Failed to find a knob
that can remove `/usr/include` from the search path. The workaround is
to disable zstd. (I enabled it by default in this release, maybe
premature?: f2adb3b6d7#15431)
Disabling `pkg-config` doesn't work because vcpkg dependencies do not
link without it.
- CMake's Xcode generator is slow because each `try_compile()` feature
check springs a new CMake + Xcode project taking a long time to run,
just to compile single-liner C files. A known issue, with no solution.
`-DCMAKE_MACOSX_BUNDLE=OFF` did not help, limiting build types to
a single one (e.g. `Debug`) also had no effect.
make | Xcode | GHA run
:---- | :---- | :--------------------------------------------------------------------
16s | 2m57s | https://github.com/curl/curl/actions/runs/12866334102/job/35868712426
23s | 4m13s | https://github.com/curl/curl/actions/runs/12868128013/job/35874212461
16s | 3m39s | https://github.com/curl/curl/actions/runs/12859073531/job/35849041880
14s | 2m23s | https://github.com/curl/curl/actions/runs/12858298423/job/35847201313
15s | 2m36s | https://github.com/curl/curl/actions/runs/12858058492/job/35846669761
19s | 3m19s | https://github.com/curl/curl/actions/runs/12868919430/job/35876601168Closes#16043
In effect it meant `_WIN32 && !USE_WIN32_LARGE_FILES`.
Replace it with these macros.
Also:
- configure: delete tautological check for small file support.
- configure: delete stray `_MSC_VER` reference. autotools does not
support MSVC.
- drop tautological checks for WinCE in `config-win32*.h` when setting
`USE_WIN32_LARGE_FILES`.
- merge related PP logic.
- prefer `#ifdef`, fix whitespace.
Suggested-by: Marcel Raad
Report: https://github.com/curl/curl/pull/15952#issuecomment-2580092328Closes#15968
- DJGPP 1.x (including `__GO32__`) (MS-DOS)
DJGPP 2.x support remains unchanged.
- Salford C (Windows)
- Turbo C (Windows 16-bit)
- Borland C++ < 5.2 (Windows 16-bit?)
- Pelles C < 2.8 (Windows)
These targets mapped `curl_off_t` to `long`. On Windows and MS-DOS
`long` is always 32-bit.
curl requires C compilers supporting 64-bit `curl_off_t` type since
835682661c#10597 (v8.0.0).
Also: drop remaining `__GO32__` and Salford C guards.
Closes#15957
Adds the experimental feature `ssls-export` to libcurl and curl for
importing and exporting SSL sessions from/to a file.
* add functions to libcurl API
* add command line option `--ssl-sessions <filename>` to curl
* add documenation
* add support in configure
* add support in cmake
+ add pytest case
Closes#15924
In OpenSSL < 3.0, the modularity was provided by mechanism called
"engines". This is supported in curl, but the engines got deprecated
with OpenSSL 3.0 in favor of more versatile providers.
This adds a support for OpenSSL Providers, to use PKCS#11 keys, namely
through the pkcs11 provider. This is done using similar approach as the
engines and this is automatically built in when the OpenSSL 3 and newer
is used.
Signed-off-by: Jakub Jelen <jjelen@redhat.com>
Closes#15587
Follow-up to 40c264db61 after discussions on IRC.
The new style is
name[0-99]=contents
and
name[0-99]@filename
A) This does not cause the same problems with old curl versions trying
the new syntax as this way will cause old curls just fail with syntax
error and not risk using the wrong file.
B) Adds the same byte range support for "normal" assigns, which the
previous syntax did not. Thus lets a user get a partial content of a
variable etc.
Added test 790 and 791 to verify non-file assigns with ranges.
Closes#15862
Use 'banfunc' and 'allowfunc' in .checksrc to specify which functions to
ban or allow to be used. This saves us from having to edit the script
going forward when we want to ban or allow specific functions.
This replaces a set of previous rules and all banned functions are now
checked with the BANNEDFUNC rule.
There is a set of default banned functions, shown by invoking
./checksrc.
Also, -a and -b options are added to specify allowed or banned functions
on the command line.
Closes#15835
build:
- autotools: fix to build generated sources for the `tidy` target.
- autotools: allow passing custom clang-tidy options via
`CURL_CLANG_TIDYFLAGS` env.
- cmake: add `CURL_CLANG_TIDY` option to configure for `clang-tidy`.
Also add:
- `CLANG_TIDY` variable to customize the `clang-tidy` tool.
- `CURL_CLANG_TIDYFLAGS` to pass custom options to `clang-tidy`.
- apply `--enable-werror` and `-DCURL_WERROR=ON` to `clang-tidy`.
CI/GHA:
- add clang-tidy job for Linux, using autotools and clang-tidy v18.
This one needs to disable `clang-analyzer-valist.Uninitialized`
to avoid false positives:
https://github.com/llvm/llvm-project/issues/40656
Duration: 5.5 minutes
- add clang-tidy job for macOS, using cmake and clang-tidy v19.
This one also covers tests and examples, and doesn't hit the false
positives seen with llvm v18 and earlier.
Duration: 4.5 minutes
- Linux/macOS: skip installing test dependencies when not building or
running tests.
fix fallouts reported by `clang-tidy`:
- lib:
- cf-h2-proxy: unused assignment in non-debug builds.
- cf-socket: silence warning.
FIXME: https://github.com/curl/curl/pull/15825#issuecomment-2561867769
- ftp: NULL passed to `strncmp()`.
- http2: NULL-ptr deref.
- mprintf: silence warning.
- src/tool_writeout: NULL passed to `fputs()`.
- examples:
- invalid file pointers.
- missing `fclose()`.
- tests:
- http/clients/hx-download: memory leaks on error.
- http/clients/hx-download: memory leak on repeat `-r` option.
- server: double `fclose()`.
https://www.man7.org/linux/man-pages/man3/fclose.3.html
- server: invalid file pointer/handle.
- server/getpart: unused assignments.
- server/mqttd: leak on failed `realloc()`.
- server/tftpd: NULL passed to `strcmp()`.
Closes#15825
Before this patch `CURL_LTO` only applied to builds explicitly
configured for `Release` or `RelWithDebInfo` via `CURL_BUILD_TYPE`.
After this patch it enables LTO regardess of build type.
Also:
- GHA/linux: enable LTO in an existing job.
Follow-up to 4ccf7622db#4799Closes#15829
To make them run clang-tidy correctly. clang-tidy occasionally finds
mistakes none of the other static code analyzers we use finds.
Also added the
-clang-analyzer-security.insecureAPI.DeprecatedOrUnsafeBufferHandling
flag, to make it not complain about memcpy()
"make tidy" in the build root works fine now. The previous clang-tidy CI
job was removed in e43c3b3e3e. It is probably time to bring it
back.
Closes#15813
Add `CURL_STRICMP()` macro that works on all platforms depending on
which lib C function is available.
Make sure to always use `_stricmp()` on Windows, which is the
non-deprecated, official API for this on this platform. Before this
patch it used a MinGW-specific call, or a deprecated compatibility
wrapper with MSVC.
Drop `stricmp` variant detections on Windows with autotools.
https://learn.microsoft.com/cpp/c-runtime-library/reference/stricmp-wcsicmp-mbsicmp-stricmp-l-wcsicmp-l-mbsicmp-l
Ref: #15652Closes#15788
And drop the prefix. This function was not use elsewhere and it should
certainly not be present in libcurl code when not used in the library.
Closes#15796
And use it from src and tests.
Syncing this functionality between platforms and build targets.
Also: Stop redefining `O_BINARY` in src, and use a local macro with
the same effect. `O_BINARY` is used in `CURL_SET_BINMODE()` to decide
if this functionality is supported, and redefining it makes this check
pass always in unity builds. The check is required for Apple OS, because
it offers a `setmode()` function, successfully detected by both CMake
and autotools, but that function has a different functionality and
signature than that expected by `CURL_SET_BINMODE()`.
Also:
- drop MetaWare High C (MS-DOS) support for set binmode.
- tests/libtest/Makefile.inc: dedupe comments.
- lib/curl_setup_once.h: tidy up feature guards for `io.h`, `fcntl.h`.
Ref: #15652Closes#15787
Let CURLINFO_QUEUE_TIME_T count only the time a transfer spends queued,
including possible redirect requests.
Add var 'time_queue' for reporting the time in write outs.
Add test for verifying correct reporting.
Closes#15512
Allowing --variable read a portion of provided files, makes curl work on
partial files for any options that accepts strings. Like --data and others.
The byte offset is provided within brackets, with a semicolon separator
like: --variable name@file;[100-200]"
Inspired by #14479
Assisted-by: Manuel Einfalt
Test 784 - 789. Documentation update provided.
Closes#15739
Adjusted test 186 to verify.
Regression in 9664d5a547, shipped in 8.11.1
Reported-by: IcedCoffeee on github
Assisted-by: Jay Satiro
Fixes#15761Closes#15762
`Makefile.mk` supported MS-DOS and Amiga, but `./configure` also
supported them in a better tested and more flexible way.
This patch also adds CMake support for MS-DOS/DJGPP and Amiga OS 3.
`Makefile.mk` was not maintained. Delete it in favour of first-tier
build methods.
Also include some non-MS-DOS/AmigaOS-specific tidy-up, see details at
the end of this message.
Details:
- fix/silence all MS-DOS/DJGPP build warnings and issues.
- add MS-DOS support to cmake.
- default to `ENABLE_THREADED_RESOLVER=OFF` for MS-DOS.
- add support for `WATT_ROOT`.
- use static libcurl with MS-DOS.
- fixup default CMake suffixes/prefixes for DJGPP.
- disable hidden symbols for MS-DOS. Not supported on MS-DOS.
- opt-in MS-DOS into `USE_UNIX_SOCKETS`.
- improve MS-DOS support in autotools.
- default to `--disable-threaded-resolver` for MS-DOS.
- make sure to use `close_s()` (from Watt-32) with autotools and cmake.
`Makefile.mk` used it before this patch.
- GHA: add DJGPP cmake (~30s) and autotools (~60s) build jobs.
Also build tests and examples with cmake.
- improve AmigaOS support in autotools:
- configure: detect `CloseSocket()` when it's a macro.
- configure: fix `IoctlSocket` detection on AmigaOS.
- curl-amissl.m4: pass AmiSSL libs to tests/servers.
- add AmigaOS3 support to cmake:
- cmake: fix `HAVE_IOCTLSOCKET_CAMEL` and
`HAVE_IOCTLSOCKET_CAMEL_FIONBIO` detections.
- set necessary system libs.
- add AmiSSL support.
- inet_ntop, inet_pton: fix using it for AmigaOS. cmake detects them,
and they did not compile with AmigaOS.
- cmake: better sync `gethostname` detection with autotools.
Fixes detection for AmigaOS, where `gethostname` is a macro.
- cmake: fix `sys/utime.h` detection on AmigaOS.
- cmake: force-disable `getaddrinfo` for AmigaOS.
- cmake: tweak threading and static/shared default for AmigaOS.
- cmake: rely on manual variable `AMIGA` to enable the platform.
- GHA: add AmigaOS cmake and autotools (~45s) jobs.
Also build tests and examples with cmake.
- INSTALL: update MS-DOS and AmigaOS build instructions.
- amigaos: fix `-Wpointer-sign` and
`zero or negative size array '_args'` in `Printf()`.
- amigaos: fix `-Wpointer-sign`
- amigaos: fix `-Wredundant-decls` `errno` and `h_errno`.
- amigaos: brute-force silence `lseek()` size warnings.
- amigaos: server/resolve: silence `-Wdiscarded-qualifiers`.
- amigaos: server/resolve: fix `-Wpointer-sign`.
- amigaos: fix `CURL_SA_FAMILY_T` type.
- nonblock: prefer `HAVE_IOCTLSOCKET_CAMEL_FIONBIO` for AmigaOS.
`ioctl` is also detected, but fails when used. Make the above override
it for a successful build.
Authored-by: Darren Banfi
Fixes#15537Closes#15603
- tftpd: prefer `HAVE_IOCTLSOCKET_CAMEL_FIONBIO` for AmigaOS.
- tftpd: tidy-up conditional code.
- curl: set stack size to 16384 for AmigaOS3/4
Overriding the default 4096.
Suggested-by: Darren Banfi
Ref: https://github.com/curl/curl/pull/15543#issuecomment-2498783123
Ref: https://wiki.amigaos.net/wiki/Controlling_Application_Stack
- functypes.h: fix `SEND_QUAL_ARG2` for AmigaOS.
- tftp: add missing cast in sendto() call for AmigaOS.
- getinfo: fix warning with AmigaOS.
- tool_operate: silence warning with AmigaOS
- amigaos: fix building libtests due to missing `RLIMIT_NOFILE`.
- curl_gethostname: silence warning for AmigaOS.
- ftp: silence `-Wtype-limits` for AmigaOS.
- libtest: fix timeval initialization for AmigaOS.
- examples: fix `timeval` initialization for AmigaOS.
- examples: silence warning for AmigaOS.
- configure: fix IPv6 detection for cross-builds.
- netrc: fix to build with AmigaOS cleanly.
- buildinfo: detect and add `DOS` tag for MS-DOS builds.
- buildinfo: add `AMIGA` to buildinfo.txt in auttools.
- build: move `USE_WATT32` macro definition to cmake/configure.
Non-MS-DOS/AmigeOS-specific tidy-ups:
- configure: sync `sa_family_t` detection with cmake.
- configure: sync `ADDRESS_FAMILY` detection signals with cmake.
- doh: use `CURL_SA_FAMILY_T`.
- lib: drop mingw-specific `CURL_SA_FAMILY_T` workaround.
- cmake: extend instead of override check-specific
configurations/requirements.
This allows to honor global requirements added earlier.
Necessary for AmigaOS for example.
- cmake: omit warning on disabled IPv6 for MS-DOS and AmigaOS.
No IPv6 support on these platforms. Also sync with autotools.
- lib1960: use libcurl `inet_pton()` wrapper.
- cmake: detect LibreSSL (to match autotools).
- cmake: say the specific OpenSSL flavour detected.
- hostip: add missing `HAVE_SOCKADDR_IN6_SIN6_SCOPE_ID` guard.
- lib: simplify classic mac feature guards.
Follow-up to a8861b6ccd#9764Closes#15543
Prefix (or suffix) curl-defined macro and function names with `curl`.
To avoid collisions with non-curl CMake and to make them recognizable
as curl-specific calls.
Closes#15498
In the function for handling 'type=' in the -F command line arguments,
we make the code more lax to accept more strings and thereby also avoid
the use of sscanf().
Closes#15683
This moves argument parsing logic for a number of options into sub
functions to reduce the overall complexity of the single getparameter()
function. pmccabe says it takes complexity down from 234 to 147.
The command line options that now has dedicated parser funtions are:
--continue-at, --ech, --header, --localport, --output, --quote, --range
--remote-name, --time-cond, --upload-file, --url, --verbose, --writeout
These parsers were selected for thise because they had more than 15
lines of logic in the main switch(). Detected like this:
git grep -hn 'case C_' tool_getparam.c |
cut -d: -f1 |
awk '{if(($1 - prev) > 15) { printf "%d\n", prev;} prev = $1;}'
Closes#15680
Timestamps in trace logs used a mix of realtime and monotonic time
sources, leading to fractional seconds carrying wrong values. Use
realtime only, so the correct nanoseconds are printed.
Fixes#15614
Reported-by: jethrogb on github
Closes#15641
Allowing both just creates a transfer with behaviors no user can
properly anticipate so better just deny the combo.
Fixes#15646
Reported-by: Harry Sintonen
Closes#15666
The MSVC UWP job in CI did not actually enable UWP. Fix this and
the fallouts discovered after enabling it.
- GHA/windows: make sure to enable UWP in MSVC vcpkg UWP job.
Use the CMake options and C flags already used for mingw-w64, but use
`WINAPI_FAMILY_PC_APP` instead of the deprecated `WINAPI_FAMILY_APP`.
(The former is not supported by mingw-w64, so leave it there as-is.)
Follow-up to cb22cfca69#14077
- GHA/windows: by default the MSVC UWP job became 2x-3x slower than
others after actually enabling UWP. Most of it is caused by
CMake/MSBuild automatically building full APPX containers for each
`.exe` target. This includes 21 CMake feature detections. Each
detection app is built into a 15MB APPX project, with code signing,
logos, etc. Example:
https://github.com/curl/curl/actions/runs/12056968170/job/33620610958
Disable this overhead for curl build targets via custom
`CMAKE_VS_GLOBALS` options. I've found no way to apply them to feature
detection targets, so those remain slow.
- cmake: automatically enable Unicode for UWP builds. It's required.
Also stop enabling it manually in the existing CI job.
- tests: fix `getpid()` use for Windows UWP:
```
tests\server\util.c(281,21): warning C4013: 'getpid' undefined; assuming extern returning int
```
Ref: https://github.com/curl/curl/actions/runs/12061215311/job/33632904249#step:11:38
- src/tool_doswin: disable `GetLoadedModulePaths()` for UWP.
mingw-w64 UWP was okay with this, but MS SDK headers are not.
This makes `--dump-module-paths` return empty for UWP builds.
```
src\tool_doswin.c(620,3): error C2065: 'MODULEENTRY32': undeclared identifier
src\tool_doswin.c(626,11): warning C4013: 'CreateToolhelp32Snapshot' undefined; assuming extern returning int
src\tool_doswin.c(626,36): error C2065: 'TH32CS_SNAPMODULE': undeclared identifier
src\tool_doswin.c(632,7): warning C4013: 'Module32First' undefined; assuming extern returning int
```
Ref: https://github.com/curl/curl/actions/runs/12055081933/job/33614629930#step:9:35
- examples: fix `websocket.c` to include `winsock2.h` before `windows.h`
to make it build with MSVC UWP:
```
include\curl\curl.h(143,16): error C2061: syntax error: identifier 'curl_socket_t'
include\curl\curl.h(143,16): error C2059: syntax error: ';'
include\curl\curl.h(417,52): error C2146: syntax error: missing ')' before identifier 'curlfd'
include\curl\curl.h(417,38): error C2081: 'curl_socket_t': name in formal parameter list illegal
```
Ref: https://github.com/curl/curl/actions/runs/12055317910/job/33615644427#step:14:126
- GHA/windows: silence linker warning with MSVC UWP builds:
```
LINK : warning LNK4075: ignoring '/INCREMENTAL' due to '/OPT:ICF' specification
```
Ref: https://github.com/curl/curl/actions/runs/12055696808/job/33616629610#step:11:38
- GHA/windows: set `/INCREMENTAL:NO` for all MSVC jobs to improve
performance a little.
- cmake: show `UWP` platform flag.
Ref: #15652Closes#15657
- GHA/windows: switch mingw-w64 UWP CI job to use UCRT.
`msvcr120_app` was missing `getch()` for example.
Follow-up to f988842d85#15637
This job tests compiling for UWP correctly, but the the resulting
`curl.exe` still doesn't look like a correct UWP app, now exiting
on startup with: `curl: error initializing curl library`.
- tool_getpass: restore `getch()` for UWP builds.
Follow-up to f988842d85#15637
- schannel: silence `-Werror=null-dereference` warning in mingw-w64 UWP:
```
lib/vtls/schannel_verify.c: In function 'Curl_verify_host':
lib/vtls/schannel_verify.c:558:33: error: null pointer dereference [-Werror=null-dereference]
558 | for(i = 0; i < alt_name_info->cAltEntry; ++i) {
| ~~~~~~~~~~~~~^~~~~~~~~~~
lib/vtls/schannel_verify.c:559:50: error: null pointer dereference [-Werror=null-dereference]
559 | PCERT_ALT_NAME_ENTRY entry = &alt_name_info->rgAltEntry[i];
| ~~~~~~~~~~~~~^~~~~~~~~~~~
```
Ref: https://github.com/curl/curl/actions/runs/12022656065/job/33515255397?pr=15638#step:19:27
Follow-up to 9640a8ef6f#15421
- GHA/windows: fix `find` command in MSVC job step.
Follow-up to 5f9411f953#15380
- GHA/windows: drop unnecessary `windowsappcompat` lib from mingw-w64
UWP job. Also drop related MSYS2 package.
- GHA/windows: cmake 3.31.0 still invokes `windres` with wrong options
with mingw-w64 UPW. Update curl version in comment accordingly.
- GHA/windows: tidy up mingw-w64 UWP spec logic, limit it to gcc.
- GHA/windows: update comments on `curl.exe` UWP startup errors.
Closes#15638
Fixing issue #15580 by renaming struct var to tool_var to avoid conflict
with the same structure name defined in AIX system headers.
Fixes#15580Closes#15581
Using this option (only available in debug builds) makes curl always
call curl_easy_duphandle() on the handle before using it.
To help us catch curl_easy_duphandle() mistakes better.
Add a CI job using this.
Bonus: the previous runtests option -e is now also supported as
--test-event
Closes#15504
