And drop the prefix. This function was not use elsewhere and it should
certainly not be present in libcurl code when not used in the library.
Closes#15796
And use it from src and tests.
Syncing this functionality between platforms and build targets.
Also: Stop redefining `O_BINARY` in src, and use a local macro with
the same effect. `O_BINARY` is used in `CURL_SET_BINMODE()` to decide
if this functionality is supported, and redefining it makes this check
pass always in unity builds. The check is required for Apple OS, because
it offers a `setmode()` function, successfully detected by both CMake
and autotools, but that function has a different functionality and
signature than that expected by `CURL_SET_BINMODE()`.
Also:
- drop MetaWare High C (MS-DOS) support for set binmode.
- tests/libtest/Makefile.inc: dedupe comments.
- lib/curl_setup_once.h: tidy up feature guards for `io.h`, `fcntl.h`.
Ref: #15652Closes#15787
Let CURLINFO_QUEUE_TIME_T count only the time a transfer spends queued,
including possible redirect requests.
Add var 'time_queue' for reporting the time in write outs.
Add test for verifying correct reporting.
Closes#15512
Allowing --variable read a portion of provided files, makes curl work on
partial files for any options that accepts strings. Like --data and others.
The byte offset is provided within brackets, with a semicolon separator
like: --variable name@file;[100-200]"
Inspired by #14479
Assisted-by: Manuel Einfalt
Test 784 - 789. Documentation update provided.
Closes#15739
Adjusted test 186 to verify.
Regression in 9664d5a547, shipped in 8.11.1
Reported-by: IcedCoffeee on github
Assisted-by: Jay Satiro
Fixes#15761Closes#15762
`Makefile.mk` supported MS-DOS and Amiga, but `./configure` also
supported them in a better tested and more flexible way.
This patch also adds CMake support for MS-DOS/DJGPP and Amiga OS 3.
`Makefile.mk` was not maintained. Delete it in favour of first-tier
build methods.
Also include some non-MS-DOS/AmigaOS-specific tidy-up, see details at
the end of this message.
Details:
- fix/silence all MS-DOS/DJGPP build warnings and issues.
- add MS-DOS support to cmake.
- default to `ENABLE_THREADED_RESOLVER=OFF` for MS-DOS.
- add support for `WATT_ROOT`.
- use static libcurl with MS-DOS.
- fixup default CMake suffixes/prefixes for DJGPP.
- disable hidden symbols for MS-DOS. Not supported on MS-DOS.
- opt-in MS-DOS into `USE_UNIX_SOCKETS`.
- improve MS-DOS support in autotools.
- default to `--disable-threaded-resolver` for MS-DOS.
- make sure to use `close_s()` (from Watt-32) with autotools and cmake.
`Makefile.mk` used it before this patch.
- GHA: add DJGPP cmake (~30s) and autotools (~60s) build jobs.
Also build tests and examples with cmake.
- improve AmigaOS support in autotools:
- configure: detect `CloseSocket()` when it's a macro.
- configure: fix `IoctlSocket` detection on AmigaOS.
- curl-amissl.m4: pass AmiSSL libs to tests/servers.
- add AmigaOS3 support to cmake:
- cmake: fix `HAVE_IOCTLSOCKET_CAMEL` and
`HAVE_IOCTLSOCKET_CAMEL_FIONBIO` detections.
- set necessary system libs.
- add AmiSSL support.
- inet_ntop, inet_pton: fix using it for AmigaOS. cmake detects them,
and they did not compile with AmigaOS.
- cmake: better sync `gethostname` detection with autotools.
Fixes detection for AmigaOS, where `gethostname` is a macro.
- cmake: fix `sys/utime.h` detection on AmigaOS.
- cmake: force-disable `getaddrinfo` for AmigaOS.
- cmake: tweak threading and static/shared default for AmigaOS.
- cmake: rely on manual variable `AMIGA` to enable the platform.
- GHA: add AmigaOS cmake and autotools (~45s) jobs.
Also build tests and examples with cmake.
- INSTALL: update MS-DOS and AmigaOS build instructions.
- amigaos: fix `-Wpointer-sign` and
`zero or negative size array '_args'` in `Printf()`.
- amigaos: fix `-Wpointer-sign`
- amigaos: fix `-Wredundant-decls` `errno` and `h_errno`.
- amigaos: brute-force silence `lseek()` size warnings.
- amigaos: server/resolve: silence `-Wdiscarded-qualifiers`.
- amigaos: server/resolve: fix `-Wpointer-sign`.
- amigaos: fix `CURL_SA_FAMILY_T` type.
- nonblock: prefer `HAVE_IOCTLSOCKET_CAMEL_FIONBIO` for AmigaOS.
`ioctl` is also detected, but fails when used. Make the above override
it for a successful build.
Authored-by: Darren Banfi
Fixes#15537Closes#15603
- tftpd: prefer `HAVE_IOCTLSOCKET_CAMEL_FIONBIO` for AmigaOS.
- tftpd: tidy-up conditional code.
- curl: set stack size to 16384 for AmigaOS3/4
Overriding the default 4096.
Suggested-by: Darren Banfi
Ref: https://github.com/curl/curl/pull/15543#issuecomment-2498783123
Ref: https://wiki.amigaos.net/wiki/Controlling_Application_Stack
- functypes.h: fix `SEND_QUAL_ARG2` for AmigaOS.
- tftp: add missing cast in sendto() call for AmigaOS.
- getinfo: fix warning with AmigaOS.
- tool_operate: silence warning with AmigaOS
- amigaos: fix building libtests due to missing `RLIMIT_NOFILE`.
- curl_gethostname: silence warning for AmigaOS.
- ftp: silence `-Wtype-limits` for AmigaOS.
- libtest: fix timeval initialization for AmigaOS.
- examples: fix `timeval` initialization for AmigaOS.
- examples: silence warning for AmigaOS.
- configure: fix IPv6 detection for cross-builds.
- netrc: fix to build with AmigaOS cleanly.
- buildinfo: detect and add `DOS` tag for MS-DOS builds.
- buildinfo: add `AMIGA` to buildinfo.txt in auttools.
- build: move `USE_WATT32` macro definition to cmake/configure.
Non-MS-DOS/AmigeOS-specific tidy-ups:
- configure: sync `sa_family_t` detection with cmake.
- configure: sync `ADDRESS_FAMILY` detection signals with cmake.
- doh: use `CURL_SA_FAMILY_T`.
- lib: drop mingw-specific `CURL_SA_FAMILY_T` workaround.
- cmake: extend instead of override check-specific
configurations/requirements.
This allows to honor global requirements added earlier.
Necessary for AmigaOS for example.
- cmake: omit warning on disabled IPv6 for MS-DOS and AmigaOS.
No IPv6 support on these platforms. Also sync with autotools.
- lib1960: use libcurl `inet_pton()` wrapper.
- cmake: detect LibreSSL (to match autotools).
- cmake: say the specific OpenSSL flavour detected.
- hostip: add missing `HAVE_SOCKADDR_IN6_SIN6_SCOPE_ID` guard.
- lib: simplify classic mac feature guards.
Follow-up to a8861b6ccd#9764Closes#15543
Prefix (or suffix) curl-defined macro and function names with `curl`.
To avoid collisions with non-curl CMake and to make them recognizable
as curl-specific calls.
Closes#15498
In the function for handling 'type=' in the -F command line arguments,
we make the code more lax to accept more strings and thereby also avoid
the use of sscanf().
Closes#15683
This moves argument parsing logic for a number of options into sub
functions to reduce the overall complexity of the single getparameter()
function. pmccabe says it takes complexity down from 234 to 147.
The command line options that now has dedicated parser funtions are:
--continue-at, --ech, --header, --localport, --output, --quote, --range
--remote-name, --time-cond, --upload-file, --url, --verbose, --writeout
These parsers were selected for thise because they had more than 15
lines of logic in the main switch(). Detected like this:
git grep -hn 'case C_' tool_getparam.c |
cut -d: -f1 |
awk '{if(($1 - prev) > 15) { printf "%d\n", prev;} prev = $1;}'
Closes#15680
Timestamps in trace logs used a mix of realtime and monotonic time
sources, leading to fractional seconds carrying wrong values. Use
realtime only, so the correct nanoseconds are printed.
Fixes#15614
Reported-by: jethrogb on github
Closes#15641
Allowing both just creates a transfer with behaviors no user can
properly anticipate so better just deny the combo.
Fixes#15646
Reported-by: Harry Sintonen
Closes#15666
The MSVC UWP job in CI did not actually enable UWP. Fix this and
the fallouts discovered after enabling it.
- GHA/windows: make sure to enable UWP in MSVC vcpkg UWP job.
Use the CMake options and C flags already used for mingw-w64, but use
`WINAPI_FAMILY_PC_APP` instead of the deprecated `WINAPI_FAMILY_APP`.
(The former is not supported by mingw-w64, so leave it there as-is.)
Follow-up to cb22cfca69#14077
- GHA/windows: by default the MSVC UWP job became 2x-3x slower than
others after actually enabling UWP. Most of it is caused by
CMake/MSBuild automatically building full APPX containers for each
`.exe` target. This includes 21 CMake feature detections. Each
detection app is built into a 15MB APPX project, with code signing,
logos, etc. Example:
https://github.com/curl/curl/actions/runs/12056968170/job/33620610958
Disable this overhead for curl build targets via custom
`CMAKE_VS_GLOBALS` options. I've found no way to apply them to feature
detection targets, so those remain slow.
- cmake: automatically enable Unicode for UWP builds. It's required.
Also stop enabling it manually in the existing CI job.
- tests: fix `getpid()` use for Windows UWP:
```
tests\server\util.c(281,21): warning C4013: 'getpid' undefined; assuming extern returning int
```
Ref: https://github.com/curl/curl/actions/runs/12061215311/job/33632904249#step:11:38
- src/tool_doswin: disable `GetLoadedModulePaths()` for UWP.
mingw-w64 UWP was okay with this, but MS SDK headers are not.
This makes `--dump-module-paths` return empty for UWP builds.
```
src\tool_doswin.c(620,3): error C2065: 'MODULEENTRY32': undeclared identifier
src\tool_doswin.c(626,11): warning C4013: 'CreateToolhelp32Snapshot' undefined; assuming extern returning int
src\tool_doswin.c(626,36): error C2065: 'TH32CS_SNAPMODULE': undeclared identifier
src\tool_doswin.c(632,7): warning C4013: 'Module32First' undefined; assuming extern returning int
```
Ref: https://github.com/curl/curl/actions/runs/12055081933/job/33614629930#step:9:35
- examples: fix `websocket.c` to include `winsock2.h` before `windows.h`
to make it build with MSVC UWP:
```
include\curl\curl.h(143,16): error C2061: syntax error: identifier 'curl_socket_t'
include\curl\curl.h(143,16): error C2059: syntax error: ';'
include\curl\curl.h(417,52): error C2146: syntax error: missing ')' before identifier 'curlfd'
include\curl\curl.h(417,38): error C2081: 'curl_socket_t': name in formal parameter list illegal
```
Ref: https://github.com/curl/curl/actions/runs/12055317910/job/33615644427#step:14:126
- GHA/windows: silence linker warning with MSVC UWP builds:
```
LINK : warning LNK4075: ignoring '/INCREMENTAL' due to '/OPT:ICF' specification
```
Ref: https://github.com/curl/curl/actions/runs/12055696808/job/33616629610#step:11:38
- GHA/windows: set `/INCREMENTAL:NO` for all MSVC jobs to improve
performance a little.
- cmake: show `UWP` platform flag.
Ref: #15652Closes#15657
- GHA/windows: switch mingw-w64 UWP CI job to use UCRT.
`msvcr120_app` was missing `getch()` for example.
Follow-up to f988842d85#15637
This job tests compiling for UWP correctly, but the the resulting
`curl.exe` still doesn't look like a correct UWP app, now exiting
on startup with: `curl: error initializing curl library`.
- tool_getpass: restore `getch()` for UWP builds.
Follow-up to f988842d85#15637
- schannel: silence `-Werror=null-dereference` warning in mingw-w64 UWP:
```
lib/vtls/schannel_verify.c: In function 'Curl_verify_host':
lib/vtls/schannel_verify.c:558:33: error: null pointer dereference [-Werror=null-dereference]
558 | for(i = 0; i < alt_name_info->cAltEntry; ++i) {
| ~~~~~~~~~~~~~^~~~~~~~~~~
lib/vtls/schannel_verify.c:559:50: error: null pointer dereference [-Werror=null-dereference]
559 | PCERT_ALT_NAME_ENTRY entry = &alt_name_info->rgAltEntry[i];
| ~~~~~~~~~~~~~^~~~~~~~~~~~
```
Ref: https://github.com/curl/curl/actions/runs/12022656065/job/33515255397?pr=15638#step:19:27
Follow-up to 9640a8ef6f#15421
- GHA/windows: fix `find` command in MSVC job step.
Follow-up to 5f9411f953#15380
- GHA/windows: drop unnecessary `windowsappcompat` lib from mingw-w64
UWP job. Also drop related MSYS2 package.
- GHA/windows: cmake 3.31.0 still invokes `windres` with wrong options
with mingw-w64 UPW. Update curl version in comment accordingly.
- GHA/windows: tidy up mingw-w64 UWP spec logic, limit it to gcc.
- GHA/windows: update comments on `curl.exe` UWP startup errors.
Closes#15638
Fixing issue #15580 by renaming struct var to tool_var to avoid conflict
with the same structure name defined in AIX system headers.
Fixes#15580Closes#15581
Using this option (only available in debug builds) makes curl always
call curl_easy_duphandle() on the handle before using it.
To help us catch curl_easy_duphandle() mistakes better.
Add a CI job using this.
Bonus: the previous runtests option -e is now also supported as
--test-event
Closes#15504
- appveyor: add build-only job for clang-cl.
- cmake: `-pedantic-errors` enables `-Werror,-Wlanguage-extension-token`
automatically, which makes `__int64` detection fail.
Explictly disable this compiler warning for clang-cl to make the
feature detection work and to accept `__int64` in the source code.
- cmake: disable `-Wlanguage-extension-token` warning for clang-cl
to fix these when encountering `__int64`:
```
lib/formdata.c(797,29): error : extension used [-Werror,-Wlanguage-extension-token]
lib/warnless.c(117,33): error : extension used [-Werror,-Wlanguage-extension-token]
lib/warnless.c(60,28): message : expanded from macro 'CURL_MASK_SCOFFT'
lib/warnless.c(59,38): message : expanded from macro 'CURL_MASK_UCOFFT'
include\curl/system.h(352,40): message : expanded from macro 'CURL_TYPEOF_CURL_OFF_T'
```
- make `__GNUC__` warning suppressions apply to `__clang__` too.
Necessary for clang-cl, which defines the latter, but not the former.
(Regular clang defines both.)
- examples: fix clang-cl compiler warning in `http2-upload.c`.
```
docs\examples\http2-upload.c(56,5): error : no previous prototype for function 'my_gettimeofday' [-Werror,-Wmissing-prototypes]
docs\examples\http2-upload.c(56,1): message : declare 'static' if the function is not intended to be used outside of this translation unit
```
- unit2604: add missing `#pragma GCC diagnostic pop`.
Follow-up to e53523fef0#14859
- unit1652: limit compiler warning suppression to GCC.
They do not affect clang builds.
Follow-up to 71cf0d1fca#14772Closes#15449
- renamed to url_proto_and_rewrite to better reveal what it does
- clarify the functionality in the top comment
- make it return CURLE_OUT_OF_MEMORY appropriately
- remove check for URL being set, use assert instead
Closes#15442
The second argument to curl_getdate() once took a time argument, but
that feature has been gone for decades, thus passing in a date there
makes no difference.
Closes#15420
- split up in a few smaller and easier to read functions
- simplify several sections
- avoid superfluous extra allocations
- remove unused debug code
Closes#15385
It reduces the number of synonym variables in the code.
Makes it easier to grok and grep.
- replace `CURL_SOURCE_DIR`
with `PROJECT_SOURCE_DIR`.
- replace `CURL_BINARY_DIR`
with `PROJECT_BINARY_DIR` or `CMAKE_CURRENT_BINARY_DIR`.
- replace a single use of `CMAKE_BINARY_DIR`
with `PROJECT_BINARY_DIR`.
- replace `CMAKE_CURRENT_*_DIR`
with `PROJECT_*_DIR` where it makes the code more uniform.
- quote an argument (formatting).
Closes#15331
The transfer_per_config is called once per new transfer. It now saves
the result of the first TLS backend check done so that subsequent
invokes are more efficient and reuses the existing knowledge.
This change also splits the logic into several smaller functions.
Closes#15323
Seen with curl-for-win linux-musl-from-mac build with gcc 9.2.0.
```
n file included from /Users/runner/work/curl-for-win/curl-for-win/curl/_x64-linux-musl-bld/src/CMakeFiles/curl.dir/Unity/unity_0_c.c:136:
/Users/runner/work/curl-for-win/curl-for-win/curl/_x64-linux-musl-bld/src/tool_ca_embed.c:4:28: warning: redundant redeclaration of 'curl_ca_embed' [-Wredundant-decls]
4 | extern const unsigned char curl_ca_embed[];
| ^~~~~~~~~~~~~
In file included from /Users/runner/work/curl-for-win/curl-for-win/curl/_x64-linux-musl-bld/src/CMakeFiles/curl.dir/Unity/unity_0_c.c:88:
/Users/runner/work/curl-for-win/curl-for-win/curl/src/tool_operate.c:107:28: note: previous declaration of 'curl_ca_embed' was here
107 | extern const unsigned char curl_ca_embed[];
| ^~~~~~~~~~~~~
```
https://github.com/curl/curl-for-win/actions/runs/11192203640/job/31116070669#step:3:4894
Follow-up to 8a3740bc8e#14059Closes#15307
It makes the callbacks get different signnatures when used from within
libcurl vs outside of it by libcurl-using applications (such as the
libtests) and this triggers UndefinedBehaviorSanitizer errors.
Closes#15289
This indicates that the file was created by curl which can help a user
determine the origin of a file. Like the other attributes, this is only
enabled with the --xattr option.
Closes#15237
Based on #14135, implement TLSv1.3 earlydata support for the curl
command line, libcurl and its implementation in GnuTLS.
If a known TLS session announces early data support, and the feature is
enabled *and* it is not a "connect-only" transfer, delay the TLS
handshake until the first request is being sent.
- Add --tls-earldata as new boolean command line option for curl.
- Add CURLSSLOPT_EARLYDATA to libcurl to enable use of the feature.
- Add CURLINFO_EARLYDATA_SENT_T to libcurl, reporting the amount of
bytes sent and accepted/rejected by the server.
Implementation details:
- store the ALPN protocol selected at the SSL session.
- When reusing the session and enabling earlydata, use exactly
that ALPN protocol for negoptiation with the server. When the
sessions ALPN does not match the connections ALPN, earlydata
will not be enabled.
- Check that the server selected the correct ALPN protocol for
an earlydata connect. If the server does not confirm or reports
something different, the connect fails.
- HTTP/2: delay sending the initial SETTINGS frames during connect,
if not connect-only.
Verification:
- add test_02_32 to verify earlydata GET with nghttpx.
- add test_07_70 to verify earlydata PUT with nghttpx.
- add support in 'hx-download', 'hx-upload' clients for the feature
Assisted-by: ad-chaos on github
Closes#15211
Before this patch `setmode()` was not detected with Cygwin/MSYS, because
it's a macro, not a function, and detection is looking for a function.
Switching to symbol detection doesn't work because it mis-detects it on
BSD systems which features a function with the same name but different
functionality and arguments.
Fix it by looking for a `_setmode()` function on Cygwin/MSYS, and use it
if available.
`_setmode()` is recommended over `setmode()` by Windows documentation so
use that on Windows too. It seems to be available on all supported
compilers, so omit detection.
https://learn.microsoft.com/cpp/c-runtime-library/reference/posix-setmodehttps://learn.microsoft.com/cpp/c-runtime-library/reference/setmode
Officially Windows requires argument `_O_BINARY` with an underscore.
`O_BINARY` is also supported but bound to conditions. Continue to use it
for simplicity. Cygwin supports `O_BINARY` (no underscore).
Closes#15169
By keeping the headers in memory until we know the target file name,
then output them all.
Previously this option combination would cause an error.
Add test 1310 and 1492 to verify. Adjusted test 1460 to work in the new
conditions.
Closes#15110
poll() on macOS 10.12 was deemed broken in 2016 when we discovered that
it misbehaves when provided with no sockets to wait for. The
HAVE_POLL_FINE is used to mark a poll() implementation that behaves
correctly: it *should* still wait the timeout time.
curl has therefore opted to use select() on Apple operating systems ever
since. To avoid the risk that this or other breakage cause problems.
However, using select() internally is also bad because it suffers from
problems when using file descriptors beyond 1024.
This change makes poll() used if it is present, but if there is no
sockets to wait for it avoids using poll() and instead falls back to
select() - but without any sockets to wait for there is no 1024 problem.
This removes all previous special-handling involving HAVE_POLL_FINE.
ref: https://daniel.haxx.se/blog/2016/10/11/poll-on-mac-10-12-is-broken/Closes#15096
