Tests are build in "unity"-style, by including sources into an umbrella C files (similar to how CMake unity works). This does not play well with clang-tidy, which seems to unconditionally ignore C sources included like this. To fix it, curl's CMake implements a manual clang-tidy support for tests, which compiles sources one-by-one, while also making sure sources compile cleanly standalone (e.g. all sources need to include `first.h`). The manual clang-tidy implementation is fragile, and performance, in particular when targeting Windows, is abysmal. This patch introduces an alternate solution, enabled by the `_CURL_TESTS_CONCAT=ON` option. In this mode, umbrella sources include the actual sources instead of `#including` them. Allowing to use CMake's built-in clang-tidy support to compile them, with clang-tidy actually checking the sources. Making the manual clang-tidy support unnecessary. In the Windows CI job it results in a 4x performance improvement (4m -> 1m), making it practical to run clang-tidy on tests on Windows, in CI. The main downside is that clang-tidy doesn't understand the `#line` directive. Meaning issues found show the wrong filename and line number next to them. It's not impossible to locate errors this way, but also not convenient. Minor/potential downside is that the concatenated source needs to be reassembled each time an original source is updated. This may result in more copying on the disk when used in local development. The largest source is 1.4MB, so probably not a show-stopper on most machines. Another is the complexity of maintaining two methods in parallel, which may be necessary till clang-tidy understands `#line`: https://github.com/llvm/llvm-project/issues/62405 This solution may in theory also enable adding clang-tidy support for tests in autotools, though I haven't tried. Targeted for curl CI for now, and used in a GHA/windows job. 100% experimental, not recommended outside these. Closes #20667 |
||
|---|---|---|
| .. | ||
| .gitignore | ||
| CMakeLists.txt | ||
| Makefile.am | ||
| Makefile.inc | ||
| README.md | ||
| unit1300.c | ||
| unit1302.c | ||
| unit1303.c | ||
| unit1304.c | ||
| unit1305.c | ||
| unit1307.c | ||
| unit1309.c | ||
| unit1323.c | ||
| unit1330.c | ||
| unit1395.c | ||
| unit1396.c | ||
| unit1397.c | ||
| unit1398.c | ||
| unit1399.c | ||
| unit1600.c | ||
| unit1601.c | ||
| unit1602.c | ||
| unit1603.c | ||
| unit1605.c | ||
| unit1606.c | ||
| unit1607.c | ||
| unit1608.c | ||
| unit1609.c | ||
| unit1610.c | ||
| unit1611.c | ||
| unit1612.c | ||
| unit1614.c | ||
| unit1615.c | ||
| unit1616.c | ||
| unit1620.c | ||
| unit1636.c | ||
| unit1650.c | ||
| unit1651.c | ||
| unit1652.c | ||
| unit1653.c | ||
| unit1654.c | ||
| unit1655.c | ||
| unit1656.c | ||
| unit1657.c | ||
| unit1658.c | ||
| unit1660.c | ||
| unit1661.c | ||
| unit1663.c | ||
| unit1664.c | ||
| unit1979.c | ||
| unit1980.c | ||
| unit2600.c | ||
| unit2601.c | ||
| unit2602.c | ||
| unit2603.c | ||
| unit2604.c | ||
| unit2605.c | ||
| unit3200.c | ||
| unit3205.c | ||
| unit3211.c | ||
| unit3212.c | ||
| unit3213.c | ||
| unit3214.c | ||
| unit3216.c | ||
Unit tests
The goal is to add tests for all functions in libcurl. If functions are too big and complicated, we should split them into smaller and testable ones.
Build Unit Tests
./configure --enable-debug is required for the unit tests to build. To
enable unit tests, there is a separate static libcurl built that is used
exclusively for linking unit test programs. Just build everything as normal,
and then you can run the unit test cases as well.
Run Unit Tests
Unit tests are run as part of the regular test suite. If you have built
everything to run unit tests, to can do 'make test' at the root level. Or you
can cd tests and make and then invoke individual unit tests with
./runtests.pl NNNN where NNNN is the specific test number.
Debug Unit Tests
If a specific test fails you get told. The test case then has output left in
the %LOGDIR subdirectory, but most importantly you can re-run the test again
using gdb by doing ./runtests.pl -g NNNN. That is, add a -g to make it
start up gdb and run the same case using that.
Write Unit Tests
We put tests that focus on an area or a specific function into a single C
source file. The source file should be named unitNNNN.c where NNNN is a
previously unused number.
Add your test to tests/unit/Makefile.inc (if it is a unit test). Add your
test data filename to tests/data/Makefile.am
You also need a separate file called tests/data/testNNNN (using the same
number) that describes your test case. See the test1300 file for inspiration
and the tests/FILEFORMAT.md documentation.
For the actual C file, here's a simple example:
#include "unitcheck.h"
#include "a libcurl header.h" /* from the lib directory */
static CURLcode test_unit9998(const char *arg)
{
UNITTEST_BEGIN_SIMPLE
/* here you start doing things and checking that the results are good */
fail_unless( size == 0 , "initial size should be zero" );
fail_if( head == NULL , "head should not be initiated to NULL" );
/* you end the test code like this: */
UNITTEST_END_SIMPLE
}
Here's an example using optional initialization and cleanup:
#include "unitcheck.h"
#include "a libcurl header.h" /* from the lib directory */
static CURLcode t9999_setup(void)
{
/* whatever you want done first */
return CURLE_OK;
}
static void t9999_stop(void)
{
/* done before shutting down and exiting */
}
static CURLcode test_unit9999(const char *arg)
{
UNITTEST_BEGIN(t9999_setup())
/* here you start doing things and checking that the results are good */
fail_unless( size == 0 , "initial size should be zero" );
fail_if( head == NULL , "head should not be initiated to NULL" );
/* you end the test code like this: */
UNITTEST_END(t9999_stop())
}