To make them more accurate.
Also:
- show Visual Studio version, where missing.
- ease the formatting.
- schannel_int.h: clang-tidy fallback code.
Used: `rg -l --sort=path CERT_FIND_HAS_PRIVATE_KEY`
Closes#21621
To include what's actually used.
Also:
- drop unused includes.
- scope includes where possible.
- drop `curlx/curlx.h` umbrella header.
- config2setopts: include `netinet/in.h` for Cygwin/MSYS2.
Previously included by chance via an unused curlx include.
Closes#20776
Also:
- support per-directory and per-upper-directory whitelist entries.
- convert badlist input grep tweak into the above format.
(except for 'And' which had just a few hits.)
- fix many code exceptions, but do not enforce.
(there also remain about 350 'will' uses in lib)
- fix badwords in example code, drop exceptions.
- badwords-all: convert to Perl.
To make it usable from CMake.
- FAQ: reword to not use 'will'. Drop exception.
Closes#20886
Instead of the first internal call to `curlx_verify_windows_version()`.
To avoid the chance of a race, potentially resulting in initializing
this address twice. AFAICT it could not cause an issue before this
patch.
Reported by Codex Security
Follow-up to b17ef873ae#18009Closes#20853
- vms/curlmsg_vms.h: delete unused/commented code.
- vtls/schannel_verify: sort includes.
- typecheck-gcc.h: fix indent and alignment.
- lib/config-win32.h: drop idle `#undef`.
- spacecheck: check for stray empty lines before after curly braces.
- make literals more readable: 1048576 -> 1024 * 1024
- scope variables.
- use ISO date in a comment.
- drop redundant parentheses.
- drop empty comments.
- unfold lines.
- duplicate/stray spaces in comments.
- fix indent, whitespace, minor typos.
Closes#20690
- fix internal macro `AN_APPLE_OS` reused between sources without
resetting it. It may potentially have left the system sha256
function unused.
- fix to define `WOLFSSL_OPTIONS_IGNORE_SYS` so that it always applies
to wolfSSL headers, also during feature detection.
- md4, md5, sha256: simplify fallback logic.
- delete 20+ unused macros.
- scope or move macros to avoid `-Wunused-macros` warnings.
- examples: delete unused code.
The warning detects macros defined but not used within the same C
source. It does not warn for macros defined in headers. It also works
with unity builds, but to a lesser extent.
Closes#20593
To avoid redefining the `fstat` system symbol, and to clarify
`struct_stat` is a curl symbol.
- introduce `curlx_fstat()` macro and use it.
- rename `struct_stat` to `curl_struct_stat`.
Also:
- tests: replace direct `curlx_win32_stat()` call with `curlx_stat()`.
- checksrc: disallow direct `_fstati64` and `fstat()` calls, except in
examples.
Closes#20496
Replacing `_WIN32`.
Also:
- tool_doswin: guard possibly non-portable socket code with
`USE_WINSOCK`. The socket is cast to `HANDLE` and passed to
win32 API `SetStdHandle(STD_INPUT_HANDLE, ...)`.
- lib/setup-win32.h: move `#undef`s before their `#define` pair.
Closes#20455
For general readability. Also to match the rest of the source code.
- bump `send()` result type from `int` to `ssize_t`.
- fix an `int` to be `curl_socklen_t`.
- `.S_un.S_addr` -> `.s_addr`.
- `SD_RECEIVE` -> `SHUT_RD`.
- `SD_SEND` -> `SHUT_WR`.
Follow-up to a81ab3e6db#20452
Follow-up to 9a2663322c#17572Closes#20457
For general readability. Also to match the rest of the source code.
- `SOCKADDR` -> `struct sockaddr`
- `SOCKADDR_IN` -> `struct sockaddr_in`
- `== SOCKET_ERROR` -> `== -1` or silent `!= 0`
Follow-up to 9a2663322c#17572Closes#20452
- `stdlib.h` and `string.h` is included via `curl_setup_once.h`,
allowing to drop from `tests/server` sources and `tool_doswin.c`.
- `stdlib.h` is also included via `setup-vms.h` (earlier than above),
allowing to drop it from `curl_addrinfo.h` on VMS.
Closes#20303
On MS-DOS (OOM and bad filename) and Windows (OOM only).
Given the rarity of both platform and error, we make a compromise and
return an unrelated libcurl error (43) in case of a bad output filename
on MS-DOS.
After:
```
$ CURL_FN_SANITIZE_OOM=1 wine curl.exe https://curl.se/ --output out.txt
curl: (27) Out of memory
$ CURL_FN_SANITIZE_BAD=1 wine curl.exe https://curl.se/ --output out.txt
Warning: bad output filename
curl: (43) A libcurl function was given a bad argument
$ CURL_FN_SANITIZE_OOM=1 wine curl.exe https://curl.se/index.html --globoff -O
curl: (27) Out of memory
$ CURL_FN_SANITIZE_BAD=1 wine curl.exe https://curl.se/index.html --globoff -O
curl: bad output filename
curl: (43) A libcurl function was given a bad argument
```
Before:
```
$ CURL_FN_SANITIZE_OOM=1 wine curl.exe https://curl.se/ --output out.txt
Warning: bad output glob
curl: (27) Out of memory
$ CURL_FN_SANITIZE_BAD=1 wine curl.exe https://curl.se/ --output out.txt
Warning: bad output glob
curl: (3) URL using bad/illegal format or missing URL
$ CURL_FN_SANITIZE_OOM=1 wine curl.exe https://curl.se/index.html --globoff -O
curl: Failed to extract a filename from the URL to use for storage
curl: (27) Out of memory
$ CURL_FN_SANITIZE_BAD=1 wine curl.exe https://curl.se/index.html --globoff -O
curl: Failed to extract a filename from the URL to use for storage
curl: (3) URL using bad/illegal format or missing URL
```
Ref: #20116 (simpler reboot of)
Ref: #20113#20121
Ref: 40c1748af5#20198
Ref: eb7f5b71e5#20143
Ref: 8c02407bef#20125Fixes#20044Closes#20199
A too long name is likely to cause a problem later anyway and get
reported there. We don't enforce file name lengths for any other
systems.
Assisted-by: Jay Satiro
Assisted-by: Viktor Szakats
Closes#20143
- `CURL_FN_SANITIZE_BAD=<any-value>` to simulate
`SANITIZE_ERR_INVALID_PATH`.
- `CURL_FN_SANITIZE_OOM=<any-value>` to simulate
`SANITIZE_ERR_OUT_OF_MEMORY`.
Both are Windows/MS-DOS-specific and require debug-enabled curl build.
Cherry-picked from #20116Closes#20125
- Use 32767-1 instead of PATH_MAX-1 (260-1) as the maximum allowable
length of a path in Windows.
Prior to this change the path sanitizer in Windows used 32767-1 as the
maximum length only for paths that had the "\\" prefix like
"\\?\longpath". Since then we added some workarounds to open longer
paths without "\\?\" prefix by normalizing the path and adding that
prefix, and the sanitizer is called before the prefix is added.
Bug: https://github.com/curl/curl/issues/20044
Reported-by: Viktor Szakats
Closes https://github.com/curl/curl/pull/20046
- they rarely catch any problems
- we have other ways to test different send/recv problems
- the number of such calls vary much more per invoke than others, making
memdebugging harder
- reducing the total number of fallible functions per test is good
- they were not used as intended anyway
Closes#20097
Clone a multibye conversion function into curlx/fopen, and use that
local copy from curlx/fopen functions. Adjust allocators in curlx/fopen
to use curl's in normal builds, and system allocators in TrackMemory
builds to avoid recursion.
This allows to switch curlx/multibyte functions to curl allocators in
all configurations, as they are no longer called by curlx/fopen, and
a recursive call can no longer happen.
After this patch the system allocator is only used in TrackMemory
Windows builds, within curlx `fopen`, `freopen`, `stat` and `open`
functions.
Also:
- test 1, 440, 767: raise allocation limitsto fit the extra allocations
in Windows Unicode builds.
- replace all uses of `curlx_unicodefree()` macro with `curlx_free()`
across the codebase.
- curlx/multibyte: delete `curlx_unicodefree()`.
- ldap: join Windows and non-Windows codepaths that became
identical after moving from `curlx_unicodefree()` to `curlx_free()`.
- vauth: drop a strdup from standard to curl allocator since
the original allocation is now already done by curl's.
- tool_doswin: drop now superfluous strdup from `FindWin32CACert()`.
- memanalyzer.pm: sync weirdo `calloc` log message with `malloc`'s.
Fixes#19748Closes#19845
Before this patch curl used the C preprocessor to override standard
memory allocation symbols: malloc, calloc, strdup, realloc, free.
The goal of these is to replace them with curl's debug wrappers in
`CURLDEBUG` builds, another was to replace them with the wrappers
calling user-defined allocators in libcurl. This solution needed a bunch
of workarounds to avoid breaking external headers: it relied on include
order to do the overriding last. For "unity" builds it needed to reset
overrides before external includes. Also in test apps, which are always
built as single source files. It also needed the `(symbol)` trick
to avoid overrides in some places. This would still not fix cases where
the standard symbols were macros. It was also fragile and difficult
to figure out which was the actual function behind an alloc or free call
in a specific piece of code. This in turn caused bugs where the wrong
allocator was accidentally called.
To avoid these problems, this patch replaces this solution with
`curlx_`-prefixed allocator macros, and mapping them _once_ to either
the libcurl wrappers, the debug wrappers or the standard ones, matching
the rest of the code in libtests.
This concludes the long journey to avoid redefining standard functions
in the curl codebase.
Note: I did not update `packages/OS400/*.c` sources. They did not
`#include` `curl_setup.h`, `curl_memory.h` or `memdebug.h`, meaning
the overrides were never applied to them. This may or may not have been
correct. For now I suppressed the direct use of standard allocators
via a local `.checksrc`. Probably they (except for `curlcl.c`) should be
updated to include `curl_setup.h` and use the `curlx_` macros.
This patch changes mappings in two places:
- `lib/curl_threads.c` in libtests: Before this patch it mapped to
libcurl allocators. After, it maps to standard allocators, like
the rest of libtests code.
- `units`: before this patch it mapped to standard allocators. After, it
maps to libcurl allocators.
Also:
- drop all position-dependent `curl_memory.h` and `memdebug.h` includes,
and delete the now unnecessary headers.
- rename `Curl_tcsdup` macro to `curlx_tcsdup` and define like the other
allocators.
- map `curlx_strdup()` to `_strdup()` on Windows (was: `strdup()`).
To fix warnings silenced via `_CRT_NONSTDC_NO_DEPRECATE`.
- multibyte: map `curlx_convert_*()` to `_strdup()` on Windows
(was: `strdup()`).
- src: do not reuse the `strdup` name for the local replacement.
- lib509: call `_strdup()` on Windows (was: `strdup()`).
- test1132: delete test obsoleted by this patch.
- CHECKSRC.md: update text for `SNPRINTF`.
- checksrc: ban standard allocator symbols.
Follow-up to b12da22db1#18866
Follow-up to db98daab05#18844
Follow-up to 4deea9396b#18814
Follow-up to 9678ff5b1b#18776
Follow-up to 10bac43b87#18774
Follow-up to 20142f5d06#18634
Follow-up to bf7375ecc5#18503
Follow-up to 9863599d69#18502
Follow-up to 3bb5e58c10#17827Closes#19626
Windows CE support was limited to successful builds with ming32ce
(a toolchain that hasn't seen an update since 2009, using an ancient gcc
version and "old mingw"-style SDK headers, that curl deprecated earlier).
Builds with MSVC were broken for a long time. mingw32ce builds were never
actually tested and runtime and unlikely to work due to missing stubs.
Windows CE toolchains also miss to comply with C89. Paired with lack of
demand and support for the platform, curl deprecated it earlier.
This patch removes support from the codebase to ease maintaining Windows
codepaths.
Follow-up to f98c0ba834#17924
Follow-up to 8491e6574c#17379
Follow-up to 2a292c3984#15975Closes#17927
- badwords.pl: add `-a` option to check all lines in source code files.
Before this patch indented lines were skipped (to avoid Markdown code
fences.)
- GHA/checksrc: use `-a` when verifying the source code.
- GHA/checksrc: disable `So` and `But` rules for source code.
- GHA/checksrc: add docs/examples to the verified sources.
- badwords.txt: delete 4 duplicates.
- badwords.txt: group and sort contractions.
- badwords.txt: allow ` url = `, `DIR`, `<file name`.
Closes#19536
As this is in the tool shutdown the impact of it was nothing.
Also, move two global variables to local.
Follow-up to 9a2663322c
Reported-by: Joshua Rogers
Closes#18996
For `GetLastError()` and `SECURITY_STATUS`:
0x-prefixed, 8-digit, lowercase, hex: 0x1234abcd
Also: say `GetLastError()` instead of `errno` in one message.
Closes#18877
Replace them by `curlx_open()` and `curlx_stat()`.
To make it obvious in the source code what is being executed.
Also:
- tests/server: stop overriding `open()` for test servers.
This is critical for the call made from the signal handler.
For other calls, it's an option to use `curlx_open()`, but
doesn't look important enough to do it, following the path
taken with `fopen()`.
Follow-up to 10bac43b87#18774
Follow-up to 20142f5d06#18634
Follow-up to bf7375ecc5#18503Closes#18776
Replace `WSASocketW()` with `CURL_SOCKET()`. Also replace a call
to `socketclose()` with `sclose()`. According to a comment,
`socketclose()` was chosen to silence test 1498 (and 2300) reporting
`MEMORY FAILURE`. These reports were accurate, and were caused by
calling `WSASocketW()` instead of `socket()` (now `CURL_SOCKET()`).
This also fixes the curl `sclose()` call on an error branch, which is
now correctly paired with a curl socket open. The mismatched open/close
calls caused an issue in TrackMemory-enabled (aka `CURLDEBUG`) builds.
Docs confirm that `socket()` is defaulting to overlapped I/O, matching
the replaced `WSASocketW()` call:
https://learn.microsoft.com/windows/win32/api/winsock2/nf-winsock2-socket#remarks
Also:
- checksrc: ban `WSASocket*()` functions.
- report `SOCKERRNO` instead of `GetLastError()` for socket calls,
to match the rest of the codebase.
Follow-up to 9a2663322c#17572Closes#18633
Before this patch `accept4()`, `socket()`, `socketpair()`, `send()` and
`recv()` system symbols were remapped via macros, using the same name,
to local curl debug wrappers. This patch replaces these overrides by
introducing curl-namespaced macros that map either to the system symbols
or to their curl debug wrappers in `CURLDEBUG` (TrackMemory) builds.
This follows a patch that implemented the same for `accept()`.
The old method required tricks to make these redefines work in unity
builds, and avoid them interfering with system headers. These tricks
did not work for system symbols implemented as macros.
The new method allows to setup these mappings once, without interfering
with system headers, upstream macros, or unity builds. It makes builds
more robust.
Also:
- checksrc: ban all mapped functions.
- docs/examples: tidy up checksrc rules.
Follow-up to 9863599d69#18502
Follow-up to 3bb5e58c10#17827Closes#18503
- update Microsoft documentation links.
(also drop language designator where present.)
- checksrc: allow longer than 78 character lines if they
contain a https URL. To make these links easier to use and parse.
- merge links that were split into two lines.
Closes#18626
To avoid hitting `-Wreserved-macro-identifier` where possible.
- amigaos: introduce local macro instead of reusing `__request()`.
- easy_lock: avoid redefining `__has_builtin()`.
Follow-up to 33fd57b8ff#9062
- rand: drop interim macro `_random()`.
- windows: rename local macro `_tcsdup()` to `Curl_tcsdup()`.
To avoid using the reserved macro namespace and to avoid
colliding with `_tcsdup()` as defined by Windows headers.
- checksrc: ban `_tcsdup()` in favor of `Curl_tcsdup()`.
- tool_doswin: avoid redefining `_use_lfn()` (MS-DOS).
- tool_findfile: limit `__NO_NET_API` hack to AmigaOS.
Syncing this pattern with `lib/netrc.c`.
Follow-up to 784a8ec2c1#16279
- examples/http2-upload: avoid reserved namespace for local macro.
More cases will be removed when dropping WinCE support via #17927.
Cases remain when defining external macros out of curl's control.
Ref: #18477Closes#18482
To avoid overriding the system symbol `accept`, which is a macro on some
systems (AIX), and thus can't be called via the `(function)` PP trick.
It's also problematic to reset such macro to its original value.
Follow-up to 3bb5e58c10#17827
Reported-by: Andrew Kirillov
Fixes#18500Closes#18501Closes#18502
The GlobalConfig only exists in a single instance and it has worked like
this since the dawn of time. It is about time we stop passing around
pointers to what was already essentially a global object and instead
just use a... global.
It simplifies things.
Closes#18213
To make all src and test code refer to curlx headers the same way.
Also:
- src: move `curlx.h` include to `tool_setup.h`.
- src/tool_setup.h: drop stray `curlx/timeval.h`.
- servers: de-duplicate `curlx.h` and `curl_setup.h` includes.
- libtests, units: drop stray curlx sub-headers in favor of
`<curlx/curlx.h>`.
- tests: include `curlx.h` with `<>` instead of `""`. To match
other parts of the codebase.
Closes#17680
Implements a seperate read thread for STDIN on Windows when curl is run
with -T/--upload-file .
This uses a similar technique to the nmap/ncat project, spawning a
seperate thread which creates a loop-back bound socket, sending STDIN
into this socket, and reading from the other end of said TCP socket in a
non-blocking way in the rest of curl.
Fixes#17451Closes#17572
Move curlx_ functions into its own subdir.
The idea is to use the curlx_ prefix proper on these functions, and use
these same function names both in tool, lib and test suite source code.
Stop the previous special #define setup for curlx_ names.
The printf defines are now done for the library alone. Tests no longer
use the printf defines. The tool code sets its own defines. The printf
functions are not curlx, they are publicly available.
The strcase defines are not curlx_ functions and should not be used by
tool or server code.
dynbuf, warnless, base64, strparse, timeval, timediff are now proper
curlx functions.
When libcurl is built statically, the functions from the library can be
used as-is. The key is then that the functions must work as-is, without
having to be recompiled for use in tool/tests. This avoids symbol
collisions - when libcurl is built statically, we use those functions
directly when building the tool/tests. When libcurl is shared, we
build/link them separately for the tool/tests.
Assisted-by: Jay Satiro
Closes#17253
