To allow building test binaries without test certs, replacing
`_CURL_SKIP_BUILD_CERTS` internal option with a build target that is
similar to `testdeps`, but without building the test certificates.
To make building test binaries a little bit faster, with less noisy
output, and without having to reconfigure the build.
Closes#20708
Tests are build in "unity"-style, by including sources into an umbrella
C files (similar to how CMake unity works). This does not play well with
clang-tidy, which seems to unconditionally ignore C sources included
like this. To fix it, curl's CMake implements a manual clang-tidy
support for tests, which compiles sources one-by-one, while also making
sure sources compile cleanly standalone (e.g. all sources need to
include `first.h`). The manual clang-tidy implementation is fragile, and
performance, in particular when targeting Windows, is abysmal.
This patch introduces an alternate solution, enabled by the
`_CURL_TESTS_CONCAT=ON` option. In this mode, umbrella sources include
the actual sources instead of `#including` them. Allowing to use CMake's
built-in clang-tidy support to compile them, with clang-tidy actually
checking the sources. Making the manual clang-tidy support unnecessary.
In the Windows CI job it results in a 4x performance improvement (4m ->
1m), making it practical to run clang-tidy on tests on Windows, in CI.
The main downside is that clang-tidy doesn't understand the `#line`
directive. Meaning issues found show the wrong filename and line number
next to them. It's not impossible to locate errors this way, but also
not convenient.
Minor/potential downside is that the concatenated source needs to be
reassembled each time an original source is updated. This may result in
more copying on the disk when used in local development. The largest
source is 1.4MB, so probably not a show-stopper on most machines.
Another is the complexity of maintaining two methods in parallel, which
may be necessary till clang-tidy understands `#line`:
https://github.com/llvm/llvm-project/issues/62405
This solution may in theory also enable adding clang-tidy support for
tests in autotools, though I haven't tried.
Targeted for curl CI for now, and used in a GHA/windows job. 100%
experimental, not recommended outside these.
Closes#20667
- tests: merge cmake commands.
- tests: use `target_compile_definitions()`.
- tests/server: use generator expression for platform-specific macro.
- tests/unit: sync `Makefile.am` comment with cmake.
- tests/unit: merge two `AM_CPPFLAGS` lines to keep synced with cmake.
- tests: move macro definitions to `first.h` headers from build level.
`CURL_NO_OLDIES`, `CURL_DISABLE_DEPRECATION`, `WITHOUT_LIBCURL`,
`CURL_STATICLIB` (for servers).
To share more logic.
Pass `CURL_STATICLIB` in server on all platforms for simplicity.
(On non-Windows, it's a no-op. It's already done like this with curlu
and libcurltool.)
Also for lib:
- lib: merge commands.
- lib: sync macro order with tests (also in `Makefile.am`).
Closes#17768
To simplify dependencies, and sync tunits and units builds further.
`curlcheck.h` already depended on logic implemented within libtests:
it referenced a global variable (`unitfail`) defined in `first.c` and
declared in `test.h`.
Also:
- rename to `unitcheck.h` to indicate it's meant for unit tests.
- make `unitcheck.h` include `first.h` instead of `test.h`.
This brings header use closer to libtests. It also includes
`curlx/curlx.h` for all unit tests by default now.
- move `unitfail` declaration from `test.h` to `first.h`.
To match its definition in `first.c`.
- drop now redundant per-test curlx header includes.
Closes#17868
To make `ninja units-clang-tidy` target work without manually building
core components first.
Also rename the clang-tidy test target generator macro to align its name
with the built-in `add_custom_target()` function.
Follow-up to c9bb9cd165#17750Closes#17810
In order to do unit tests for private functions, functions that are
marked UNITTEST but without a global scope in the library, functions
that do not have prototypes in their corresponding header file, unit
tests previously brought their own private prototype *copy* into the
unit test.
This was error-prone when the internal function changes but the change
might be missed in the unit test which then uses an outdated prototype
copy for testing.
This change removes the private prototypes from unit tests and instead
introduces a C file parser that parses the specific C files and extracts
the necessary unit test prototypes into a generated header file for unit
tests to use. This geneated lib/unitprotos.h header is then included by
unit tests that need private prototypes.
Assisted-by: Viktor Szakats
Closes#17750
It allows to pass `.h` files only where they are really needed, which is
`EXTRA_DIST` (or `*SOURCES`) for autotools.
Also:
- rename variables to be shorter and consistent.
- drop references to non-local headers. We don't need to pass them as
dist sources.
- drop empty variables and references, after the above.
- server: add placeholder `UTIL_H`.
Closes#17745
To make all src and test code refer to curlx headers the same way.
Also:
- src: move `curlx.h` include to `tool_setup.h`.
- src/tool_setup.h: drop stray `curlx/timeval.h`.
- servers: de-duplicate `curlx.h` and `curl_setup.h` includes.
- libtests, units: drop stray curlx sub-headers in favor of
`<curlx/curlx.h>`.
- tests: include `curlx.h` with `<>` instead of `""`. To match
other parts of the codebase.
Closes#17680
Replace existing `mk-unity.pl` `--embed` workaround with running
`clang-tidy` manually on individual test source instead. This aligns
with how clang-tidy works and removes `mk-unity.pl` from the solution.
Also:
- mqttd: fix potentially uninitialized buffer by zero filling it.
```
tests/server/mqttd.c:484:41: error: The left operand of '<<' is a garbage value
[clang-analyzer-core.UndefinedBinaryOperatorResult,-warnings-as-errors]
484 | payload_len = (size_t)(buffer[10] << 8) | buffer[11];
| ^
[...]
tests/server/mqttd.c:606:45: error: The left operand of '<<' is a garbage value
[clang-analyzer-core.UndefinedBinaryOperatorResult,-warnings-as-errors]
606 | topiclen = (size_t)(buffer[1 + bytes] << 8) | buffer[2 + bytes];
| ^
```
- sockfilt: fix potential out-of-bound pointer:
```
tests/server/sockfilt.c:1128:33: error: The 2nd argument to 'send' is a buffer
with size 17010 but should be a buffer with size equal to or greater than
the value of the 3rd argument (which is 18446744073709551615)
[clang-analyzer-unix.StdCLibraryFunctions,-warnings-as-errors]
1128 | ssize_t bytes_written = swrite(sockfd, buffer, buffer_len);
| ^
```
- clang-tidy: suppress bogus `bzero()` warnings that happens
inside the notorious `FD_ZERO()` macros, on macOS.
Ref: https://github.com/curl/curl/pull/17680#issuecomment-2991730158Closes#17705
Derive it from `$BUNDLE` instead. autotools seems to be already relying
on `$BUNDLE_SRC` being equal to `$BUNDLE.c`. (I haven't realized this
before aaebb45f58.)
Also drop redundant `nodist_<target>_SOURCE` lines in tunits and units.
Follow-up to aaebb45f58#17688
Follow-up to 2c27a67daa#17590Closes#17692
Use it from libtests' `first.c` and thus also from units, and tunits.
Also:
- cmake: drop stray `curltool` lib dependency for units.
- units: stop depending on `src` headers.
- tests/server: drop depending on `src` headers.
(the remaining one listed in the comments, `tool_xattr.h`, was not
actually used from servers.)
- tests/server: drop duplicate curlx headers.
(Except `warnless.h`, which is tricky on Windows.)
Closes#17672
To reduce the diff between tests CMakeFiles.txt, and syncing with
autotools, which already used the `BUNDLE` variable like this.
Also:
- fold lines that went over 132 chars after this change.
- autotools: sync order of macros with cmake.
Closes#17667
Instead of relying on CMake's built-in unity feature, use `mk-unity.pl`,
as already done with autotools. It simplified the build, shortens logs
and makes debugging easier because of the fewer build variations.
It also allows testing / fixing with cmake and those automatically apply
to autotools builds too. cmake builds can be much-much faster, esp.
when working the builds themselves.
It also enables "unity" in old cmake versions. Basically every test
target is a single generated .c source.
Also:
- drop a `lib` unity workaround for libtests with autotools after fixing
the issue in libtests itself. It drops a few exceptions and makes
libcurl build faster (in autotools unity).
- fix another `lib` autotools unity issue and drop the workaround for it
from `mk-unity.pl`. `srcdir` was missing from the header path.
- simplify `mk-unity.pl` command-lines, drop exclusions.
Follow-up to 2c27a67daa#17590Closes#17628
Make test bundles the default. Drop non-bundle build mode.
Also do all the optimizations and tidy-ups this allows, simpler builds,
less bundle exceptions, streamlined build mechanics.
Also rework the init/deinit macro magic for unit tests. The new method
allows using unique init/deinit function names, and calling them with
arguments. This is in turn makes it possible to reduce the use of global
variables.
Note this drop existing build options `-DCURL_TEST_BUNDLES=` from cmake
and `--enable-test-bundles` / `--disable-test-bundles` from autotools.
Also:
- rename test entry functions to have unique names: `test_<testname>`
This removes the last exception that was handled in the generator.
- fix `make dist` to not miss test sources with test bundles enabled.
- sync and merge `tests/mk-bundle.pl` into `scripts/mk-unity.pl`.
- mk-unity.pl: add `--embed` option and use it when `CURL_CLANG_TIDY=ON`
to ensure that `clang-tidy` does not miss external test C sources.
(because `clang-tidy` ignores code that's #included.)
- tests/unit: drop no-op setup/stop functions.
- tests: reduce symbol scopes, global macros, other fixes and tidy-ups.
- tool1621: fix to run, also fix it to pass.
- sockfilt: fix Windows compiler warning in certain unity include order,
by explicitly including `warnless.h`.
Follow-up to 6897aeb105#17468Closes#17590
Move curlx_ functions into its own subdir.
The idea is to use the curlx_ prefix proper on these functions, and use
these same function names both in tool, lib and test suite source code.
Stop the previous special #define setup for curlx_ names.
The printf defines are now done for the library alone. Tests no longer
use the printf defines. The tool code sets its own defines. The printf
functions are not curlx, they are publicly available.
The strcase defines are not curlx_ functions and should not be used by
tool or server code.
dynbuf, warnless, base64, strparse, timeval, timediff are now proper
curlx functions.
When libcurl is built statically, the functions from the library can be
used as-is. The key is then that the functions must work as-is, without
having to be recompiled for use in tool/tests. This avoids symbol
collisions - when libcurl is built statically, we use those functions
directly when building the tool/tests. When libcurl is shared, we
build/link them separately for the tool/tests.
Assisted-by: Jay Satiro
Closes#17253
To avoid adding this macro to the global `CURL_DEBUG_MACROS` variable,
which may be used for targets defined after unit tests, and where this
macro may not be necessary.
As of this commit unit tests are defined last, so extending the global
variable did not cause any issue.
Follow-up to 220eda34cd#17259Closes#17264
- unit tests need no tool code as they are libcurl unit tests
- unit test 1621 is now tunit test 1621 instead, as it tests tool code
- build unit tests with BUILDING_LIBCURL as they pretent to be libcurl
Closes#17259
It seems unnecessary and possibly unexpected to build test servers with
debug-enabled features and memory tracking whenever the tested curl is
built like that (which is a requirement for some tests, so curl is
mostly built like that when running tests.) It also makes building
servers a little bit faster with cmake for the most common cases.
You can apply debug options to `tests/server` with these new options:
- `./configure`: `--enable-server-debug`.
- cmake: `-DENABLE_SERVER_DEBUG`.
Also sync the way we pass these macros in autotools, with CMake builds.
Before this patch, autotools passed them via `curl_config.h`. After this
patch it passes them on the command-line, like cmake builds do.
This patch also make these option no longer passed to examples and
`http/client` in cmake builds, where they were no-ops anyway.
Ref: #15000Closes#16705
Prefix (or suffix) curl-defined macro and function names with `curl`.
To avoid collisions with non-curl CMake and to make them recognizable
as curl-specific calls.
Closes#15498
It reduces the number of synonym variables in the code.
Makes it easier to grok and grep.
- replace `CURL_SOURCE_DIR`
with `PROJECT_SOURCE_DIR`.
- replace `CURL_BINARY_DIR`
with `PROJECT_BINARY_DIR` or `CMAKE_CURRENT_BINARY_DIR`.
- replace a single use of `CMAKE_BINARY_DIR`
with `PROJECT_BINARY_DIR`.
- replace `CMAKE_CURRENT_*_DIR`
with `PROJECT_*_DIR` where it makes the code more uniform.
- quote an argument (formatting).
Closes#15331
- show `OpenSSL v3+` when detected (as in `./configure`).
(this string also makes its way to `curl-config`.)
- prefer `unset(VAR)` over `set(VAR)`.
Same effect, but `unset()` tells the intent unambiguously.
https://cmake.org/cmake/help/latest/command/set.html
- drop "implementation" from an `option()` description.
- FindGSS: replace legacy keyword alias with modern alternative.
https://cmake.org/cmake/help/latest/command/get_filename_component.html
- move `CURL_STATIC_CRT` logic next to its `option()`.
- improve order of `libcurl.pc`/`curl-config` variable init lines.
- tests: drop/shorten custom target names.
They inflated generated make files by 550KB.
Keep target name logic for sync between code snippets.
Follow-up to a2ef5d36b3#14660
- clear a variable after use.
- restore `STATUS` for `Features:`/`Protocols:` `message()`s:
Without it the output goes to stderr, and appears in red in CMake GUI.
It doesn't seem possible to show a line on stdout without leading
underscores to match `curl -V` and `./configure` output.
Partial revert of acbc6b703f#14197
- WindowsCache: move `HAVE_LINUX_TCP_H` into the header group.
- move strings to the same line as their `STRING` keyword.
- formatting in generated code.
- delete bogus comment.
- unfold lines for readability.
- fix a too long line. (for cmakelint)
- missing quotes, whitespace, comments.
Closes#14610
- reuse local variable names.
- sync `PROJECT_LABEL`, add where missing.
- namespace all target names.
- bind header directories to each target.
- tests/server: limit `CURL_STATICLIB` to Windows (as in autotools.)
- drop functions with a single caller.
Closes#14660
- quote string literals.
In the hope it improves syntax-highlighting and readability.
- use lowercase, underscore-prefixed local var names.
As a hint for scope, to help readability.
- prefer `pkg_search_module` (over `pkg_check_modules`).
They are the same, but `pkg_search_module` stops searching
at the first hit.
- more `IN LISTS` in `foreach()`.
- OtherTests.cmake: clear `CMAKE_EXTRA_INCLUDE_FILES` after use.
- add `PROJECT_LABEL` for http/client and unit test targets.
- sync `Find*` module comments and formatting.
- drop a few local variables.
- drop bogus `CARES_LIBRARIES` from comment.
- unquote numeric literal.
Follow-up to acbc6b703f#14197Closes#14388
- sync build-dir/source-dir header path order with autotools, by
including build-dir first, then source-dir.
This prevents out-of-tree builds breaking due to leftover generated
headers in the source tree.
- tests/unit: move `src` ahead of `libtest` in header path, syncing with
autotools.
- stop adding non-existing generated `include` dir to header path.
There are no generated `include` headers and this directory is either
missing in out-of-tree builds or the same as the one already added
globally via the root `CMakeLists.txt`.
- lib: stop adding a duplicate source include directory to the header
path.
It's already added globally via the root `CMakeLists.txt`.
- lib: stop adding the project root to the header path.
- docs/examples: drop internal header paths.
Examples do not and should not use internal headers.
- replace `curl_setup_once.h` in comments with `curl_setup.h`,
the header actually used, and also referred to in autotools comments.
- add comment why we need `src` in include path for `tests/server`.
- add quotes around header directories.
Closes#14416
- tidy-up comments.
- use lowercase, underscore prefixed names for internal variables.
- use `IN LISTS` and `IN ITEMS` in `foreach()` loops.
- rename variable name `OUTPUT` to a more distinctive one.
- tidy-up `STREQUAL` syntax.
- delete commented code.
- indent/whitespace.
Closes#14197
Before this patch, the `testdeps` build target required `-DCURLDEBUG`
be set either via `ENABLE_DEBUG=ON` or `ENABLE_CURLDEBUG=ON` to build
the curl unit tests.
After fixing build issues in #13694, we can drop this requirement and
build unit tests unconditionally.
Depends-on: #13694
Depends-on: #13697 (fix unit test issue revealed by Old Linux CI job)
Follow-up to 39e7c22bb4#11446Closes#13698
Add a `libcurlu`/`libcurltool` static library that is compiled only for
unit tests. We use `EXCLUDE_FROM_ALL` to make sure that they're not
built by default, they're only built if unit tests are built.
These libraries allow us to compile every unit test with CMake.
Closes#11446
- with `--proxy-http2` allow h2 ALPN negotiation to
forward proxies
- applies to http: requests against a https: proxy only,
as https: requests will auto-tunnel
- adding a HTTP/1 request parser in http1.c
- removed h2h3.c
- using new request parser in nghttp2 and all h3 backends
- adding test 2603 for request parser
- adding h2 proxy test cases to test_10_*
scorecard.py: request scoring accidentally always run curl
with '-v'. Removed that, expect double numbers.
labeller: added http1.* and h2-proxy sources to detection
Closes#10967
Adding `bufq`:
- at init() time configured to hold up to `n` chunks of `m` bytes each.
- various methods for reading from and writing to it.
- `peek` support to get access to buffered data without copy
- `pass` support to allow buffer flushing on write if it becomes full
- use case: IO buffers for dynamic reads and writes that do not blow up
- distinct from `dynbuf` in that:
- it maintains a read position
- writes on a full bufq return CURLE_AGAIN instead of nuking itself
- Init options:
- SOFT_LIMIT: allow writes into a full bufq
- NO_SPARES: free empty chunks right away
- a `bufc_pool` that can keep a number of spare chunks to
be shared between different `bufq` instances
Adding `dynhds`:
- a straightforward list of name+value pairs as used for HTTP headers
- headers can be appended dynamically
- headers can be removed again
- headers can be replaced
- headers can be looked up
- http/1.1 formatting into a `dynbuf`
- configured at init() with limits on header counts and total string
sizes
- use case: pass a HTTP request or response around without being version
specific
- express a HTTP request without a curl easy handle (used in h2 proxy
tunnels)
- future extension possibilities:
- conversions of `dynhds` to nghttp2/nghttp3 name+value arrays
Closes#10720
These are only built when a libcurl static library is available, since
we're not building a special libcurlu library yet and these tests rely
on private symbols that aren't available in the shared library. A few
unit tests do require libcurlu, so those are not built.
Closes#10722
Put only the test names into Makefile.inc so they can be used by both
cmake and automake. This will prevent the list of tests from becoming
out of date when they are also built under cmake.
- they are mostly pointless in all major jurisdictions
- many big corporations and projects already don't use them
- saves us from pointless churn
- git keeps history for us
- the year range is kept in COPYING
checksrc is updated to allow non-year using copyright statements
Closes#10205
Add licensing and copyright information for all files in this repository. This
either happens in the file itself as a comment header or in the file
`.reuse/dep5`.
This commit also adds a Github workflow to check pull requests and adapts
copyright.pl to the changes.
Closes#8869
This reverts commit 1cba36d216.
CMake provides properties that can be set on a target to rename the
output artifact without changing the name of a target.
Ref: #6899
* runtests.pl:
- Fix out-of-tree build under CMake when srcdir is not set. Default
srcdir to the location of runtests.pl.
- Add a hack to allow CMake to use the TFLAGS option as documented
in tests/README and used in scripts/travis/script.sh.
* Bump CMake version to 3.2 for USES_TERMINAL, dropping Debian Jessie
support (no one should care, it is already EOL.).
* Remove CTest since it defines its own 'test' target with no tests
since all unittests are already broken and not built by default.
* Add new test targets based on the options from Makefile.am. Since
new test targets are rarely added, I opted for duplicating the
runtests.pl options as opposed to creating a new Makefile.inc file.
Use top-level target names (test-x) instead of x-test since that is
used by CI and others.
Closes#5358
The default target should only build libcurl and curl. Add a dedicated
'testdeps' target which will be used later when running tests. Note that
unittests are currently broken in CMake and already excluded.
Closes#5368
Reported by the new script 'scripts/copyright.pl'. The script has a
regex whitelist for the files that don't need copyright headers.
Removed three (mostly usesless) README files from docs/
Closes#5141
Added unit test case 1655 to verify.
Close#4352
the code correctly finds the flaws in the old code,
if one temporarily restores doh.c to the old version.
