Commit graph

10 commits

Author SHA1 Message Date
Viktor Szakats
2c419fc14b
ci: tidy-ups
- http3-linux: add newlines for readability.
- http3-linux: use `make pytest` to run pytest.
- checksrc: use `|` multiline to sync with other uses.
- checksrc: prefer `$()`.
- prefer `>-` over `>` for folded option lists.
- cygwin: drop a `grep` no longer necessary.

Closes #15081
2024-09-29 17:39:55 +02:00
Stefan Eissing
7f3d598276
CI: improvements in test reliability and performance
- CI default: raise parallelism to 20
- CI valgind: set parallelism to 6
- CI non-native: adapt parallelism for OS builds
- CI Windows: no longer ignore FTP, TFTP, MQTT and SMTP
- CI Windows: restrict test timeout to 10 minutes
- CI Windows: do not run tests for msh3 build
- tests, various: restrict curl invocation to ipv4 to avoid talking to
  any ipv6 test server running in parallel
- tests: requiring http/2 server no longer needs to mention http server
- test 190: use a fixed timeout of 10 seconds instead of %FTPTIME2, as
  that value rises under parallel load beyond what the server is waiting
  and then produces different results
- test 1540: add debug logging
- testrunner: add verify checks for http/2 and http/3 server

Closes #15040
2024-09-27 16:47:37 +02:00
Viktor Szakats
71cf0d1fca
tests: speed up builds with single-binary test bundles
Add support for single-block binaries that contain all libtests and
unit tests respectively.

Enable with:
- autotools: `--enable-test-bundles`
- cmake: `-DCURL_TEST_BUNDLES=ON`

(They are compatible with `--enable-unity` and `-DCMAKE_UNITY_BUILD=ON`
options, for further speed-up.)

Makes libtests and unit tests build _fast_, needing little disk space
even in static mode. Similar to CMake unity mode, but with a custom
script, also supporting autotools builds.

The price is having to deal with symbols/macros colliding between
`lib*.c` and `unit*.c` sources. Maybe with naming conventions or other
solutions this can be improved gradually and reduce the need for manual
intervention by `mk-bundle.mk`. I've included a script that does the bulk
of detecting name collisions.

Also:
- CI: enable test bundles.
- CI: build tests in more jobs.
- lib2305: fix FILE handle leak.
- unit1661: fix memleak found by torture test by releasing the `bufref`
  structure in `unit_stop()` that was allocated in `unit_setup()`.
  ```
  test 1661...[bufref unit tests]
  Leak detected: memory still allocated: 13 bytes
   allocated by /home/runner/work/curl/curl/tests/unit/unit1661.c:70
   1661: torture FAILED: function number 1 in test.
  ```
  Ref: https://github.com/curl/curl/actions/runs/10967279334/job/30456745290?pr=14772#step:8:41

Similar test suite builds with autotools default and cmake+bundle+unity:
- GHA/Linux: 33s vs 7s
  https://github.com/curl/curl/actions/runs/10705668823/job/29681617374
- GHA/macOS 34s vs 2s
  https://github.com/curl/curl/actions/runs/10705668813/job/29681632885
- GHA/FreeBSD: 15m25 vs 6m21 (full workflow time, ~qemu)
  https://github.com/curl/curl/actions/runs/10705668811/job/29681607915
- GHA/Cygwin: 9m52 vs 32s
  https://github.com/curl/curl/actions/runs/10705668809/job/29681609965
- GHA/MSYS2: 3m52 vs 14s
  https://github.com/curl/curl/actions/runs/10705668808/job/29681624295
- GHA/mingw-w64: 5m45 vs 30s
  https://github.com/curl/curl/actions/runs/10705668808/job/29681628787

Autotools test suite builds compared between master -> `--enable-test-bundles`:
- GHA/Linux: 33s -> 9s (run tests: 22m23 -> 20m44)
  https://github.com/curl/curl/actions/runs/10710030193/job/29695932185
  https://github.com/curl/curl/actions/runs/10967831456/job/30458220344
- GHA/macOS: 25s -> 4s (run tests: 2m58 -> 2m24)
  https://github.com/curl/curl/actions/runs/10710030195/job/29695938444
  https://github.com/curl/curl/actions/runs/10967831452/job/30458225762
- GHA/non-native (FreeBSD): 4m8 -> 3m12 (full workflow time, ~qemu)
  https://github.com/curl/curl/actions/runs/10710030198/job/29695928401
  https://github.com/curl/curl/actions/runs/10967831458/job/30458212692
- GHA/Cygwin: 9m25 -> 1m9 (run tests: 9m19 -> 3m28)
  https://github.com/curl/curl/actions/runs/10710030212/job/29695928213
  https://github.com/curl/curl/actions/runs/10967831453/job/30458213268
- GHA/MSYS2: 3m54 -> 32s (run tests: 6m3 -> 3m59)
  https://github.com/curl/curl/actions/runs/10710030190/job/29704850591
  https://github.com/curl/curl/actions/runs/10967831449/job/30459280005
- GHA/mingw-w64: 5m42 -> 1m5 (run tests: 7m41 -> 5m36)
  https://github.com/curl/curl/actions/runs/10710030190/job/29704852058
  https://github.com/curl/curl/actions/runs/10967831449/job/30459280862
- Azure MSYS2 mingw64 openssl: 38m55 -> 11m58
  https://dev.azure.com/daniel0244/curl/_build/results?buildId=25546&view=logs&j=b58b8c59-0f61-52e9-0f9e-fad562a1e77f&t=0f9230a7-3b10-53ca-9938-700ece377c5e
  https://dev.azure.com/daniel0244/curl/_build/results?buildId=25547&view=logs&jobId=39473db1-3945-55d5-deb5-c218fad88dce&j=b58b8c59-0f61-52e9-0f9e-fad562a1e77f&t=0f9230a7-3b10-53ca-9938-700ece377c5e
- Azure Ubuntu default: 2m15 -> 55s (all build)
  https://dev.azure.com/daniel0244/curl/_build/results?buildId=25546&view=logs&j=9d58b9ac-e1e6-53b6-f83a-1f9f1d912522&t=a6b38d83-e7cf-5a9b-c762-a178412717b7
  https://dev.azure.com/daniel0244/curl/_build/results?buildId=25547&view=logs&jobId=39473db1-3945-55d5-deb5-c218fad88dce&j=9d58b9ac-e1e6-53b6-f83a-1f9f1d912522&t=a6b38d83-e7cf-5a9b-c762-a178412717b7

Cmake test suite builds compared between master -> `-DCURL_TEST_BUNDLES=ON` + unity:
- GHA/Linux: 29s -> 7s (run tests: 4m50 -> 4m57, 20m43 -> 20m45)
  https://github.com/curl/curl/actions/runs/10710030193/job/29695941814
  https://github.com/curl/curl/actions/runs/10705668823/job/29681622201
- GHA/Linux old: 44s -> 13s (bundle+no unity) (run tests: 5m5 -> 5m6)
  https://github.com/curl/curl/actions/runs/10718264094/job/29719794727
  https://github.com/curl/curl/actions/runs/10718653175/job/29721009613
- GHA/macOS: 32s -> 2s (run tests: 2m43 -> 2m40)
  https://github.com/curl/curl/actions/runs/10710030195/job/29695931956
  https://github.com/curl/curl/actions/runs/10705668813/job/29681638937
- GHA/non-native (*BSD): inconclusive (full workflow time, ~qemu)
  https://github.com/curl/curl/actions/runs/10710030198
  https://github.com/curl/curl/actions/runs/10705668811
- GHA/Cygwin: 3m9 -> 32s
  https://github.com/curl/curl/actions/runs/10710030212/job/29695929075
  https://github.com/curl/curl/actions/runs/10705668809/job/29681609965
- GHA/MSYS2: 2m24 -> 14s
  https://github.com/curl/curl/actions/runs/10710030190/job/29704850996
  https://github.com/curl/curl/actions/runs/10705668808/job/29681624295
- GHA/mingw-w64: 3m56 -> 30s (run tests: 4m2 -> 3m52)
  https://github.com/curl/curl/actions/runs/10710030190/job/29704852219
  https://github.com/curl/curl/actions/runs/10705668808/job/29681631393
- GHA/mingw-w64-old: 7m19 -> 1m44 (run tests: 3m30 -> 2m53)
  https://github.com/curl/curl/actions/runs/10710030190/job/29704849763
  https://github.com/curl/curl/actions/runs/10705668808/job/29681622329
- GHA/MSVC: 3m22 -> 13s (run tests: 9m43 -> 4m22)
  https://github.com/curl/curl/actions/runs/10710030190/job/29704850411
  https://github.com/curl/curl/actions/runs/10705668808/job/29681623313
- AppVeyor CI MSVC 2008: 4m3 -> 45s (full build)
- AppVeyor CI MSVC 2010: 2m56 -> 1m8 (full build)
- AppVeyor CI MSVC 2022: 10m19 -> 2m23 (full build)
  https://ci.appveyor.com/project/curlorg/curl/builds/50538455
  https://ci.appveyor.com/project/curlorg/curl/builds/50536558
- AppVeyor CI total build time: 10m30 (master) -> 6m48 (unity) -> 4m5 (bundle) -> 3m24 (bundle+unity) -> 5m7 (bundle+unity+all jobs building tests)

Closes #14772
2024-09-22 09:51:15 +02:00
Viktor Szakats
60c3d04465
autotools: add support for 'unity' builds, enable in CI
Implement the "unity" builds as known from CMake, but for autotools.
It's limited to `lib` and `src` (CMake also supports it in `tests`).

Enable with: `--enable-unity` (disabled by default)

Unity builds speed up builds significantly. Cygwin and Windows builds in
particular, but the effect is noticeable on most systems. It also allows
discovering unity issues with autotools, benefitting also CMake when
building the same combination. In CI it makes turnaround times quicker.

This closes build performance with CMake. autotools still lags behind
because it builds shared and static libcurl in two, separate passes.
CMake does it in one. Manpage compilation isn't batched, it is in CMake.
After unity and test bundle support the slowest parts of the build are
the configuration phase (which is effectively a tedious, non-parallel,
compilation and/or linking of 300+ tiny programs. The next bottleneck
is compiling individual examples and finally test servers (only slow
with autotools).

The autotools implementation is slightly less efficient than CMake,
because 3 sources are permanently excluded while in CMake this isn't
necessary and solved more efficiently while building libtests. There is
also no 'unity' support for tests, making them a less efficient also.

Enable it in CI for most `configure` jobs. Except in GHA/dist (though
it works fine there too), to use the default config there. Also skip for
the Linux AWC-LC job where it made builds time a few seconds longer
(reason undiscovered.)

Autotools test suite builds compared between master -> `--enable-unity`:
- GHA/Linux: 32s -> 12s
  https://github.com/curl/curl/actions/runs/10705668823/job/29681617374
  https://github.com/curl/curl/actions/runs/10742978889/job/29796766297
- GHA/macOS: 37s -> 10s
  https://github.com/curl/curl/actions/runs/10705668813/job/29681632885
  https://github.com/curl/curl/actions/runs/10742978699/job/29796768875
- GHA/FreeBSD: 15m25 -> 10m58 (full workflow time, ~qemu)
  https://github.com/curl/curl/actions/runs/10705668811/job/29681607915
  https://github.com/curl/curl/actions/runs/10742978937/job/29796766115
- GHA/Cygwin: 3m32 -> 1m21
  https://github.com/curl/curl/actions/runs/10705668809/job/29681609965
  https://github.com/curl/curl/actions/runs/10742978645/job/29796756933
- GHA/MSYS2: 2m42 -> 50s
  https://github.com/curl/curl/actions/runs/10705668808/job/29681621166
  https://github.com/curl/curl/actions/runs/10742978662/job/29799739289
- GHA/mingw-w64: 5m32 -> 1m23
  https://github.com/curl/curl/actions/runs/10705668808/job/29681628787
  https://github.com/curl/curl/actions/runs/10742978662/job/29799741568

Closes #14815
2024-09-20 23:53:33 +02:00
Viktor Szakats
45202cbba4
cmake: separate target for examples, optimize CI, fix fallouts
- Move `docs/examples` builds under a separate target.

- Make `BUILD_EXAMPLES` default to `ON`. It means to generate the rules
  for `docs/examples` by default, but not build them. To build them,
  an explicit `make curl-examples` (or ninja, etc) command is necessary.
  This syncs behaviour with autotools, and also how both cmake and
  autotools are building tests.

- GHA: update cmake jobs to use the new way of building examples.

- GHA: move examples build step at the end of the job, after building
  and running tests. This allows to have build and test run results
  faster, and leave the seldom-changing examples build to the end.
  Building examples is the slowest build step with no practical way to
  make them fast.

- appveyor: enable building examples in two old-MSVC jobs.

- examples: fix examples to build cleanly with old MSVC versions.

- GHA/non-native: move example build log under a GHA foldable section.

- GHA/windows: move building examples into separate step for Linux cross
  jobs.

Follow-up to dfdd978f7c #13491
Closes #14906
2024-09-20 23:53:33 +02:00
Viktor Szakats
e53523fef0
CI: move Azure jobs to GHA, fix fallouts, sshserver, runtests tweaks
CI:

- GHA/windows: enable OpenSSH server, SysInternals `handle`, `impacket`.
  Skip `impacket` on MSYS2 due to install failure.
  Skip OpenSSH server for old/standalone mingw-w64 (building curl
  without SSH there.)
- GHA/windows: make test tool installs a separate step.
- GHA/cygwin: enable OpenSSH server.
  Skip `impacket`: it's compiling for 7 minutes then breaks.
  Skip `stunnel` due to sluggish test run performance.
  (This update is unrelated to Azure jobs.)
- GHA/linux: migrate Linux jobs from Azure CI.
- GHA/linux: migrate scanbuild job from Azure CI.
- GHA/linux: enable libssh2 in a job. Also enable valgrind.
- CI/windows: enable SSPI in two jobs.
- CI/windows: disable zlib in one more job.
- CI/windows: improve `if` condition checking GnuTLS.
- CI/windows: ignore SFTP/SCP tests as necessary.
  - universally ignore SCP tests, they fail everywhere.
  - ignore test 612.
  - ignore test 613 616 618 with MSYS2 mingw-w64.
  - ignore test 614 with libssh.
  - ignore all SFTP with MSYS2 native.
  - ignore all SFTP with vcpkg with `libssh2[core,zlib]`.
  - ignore a couple of SFTP tests with MSYS2 mingw-w64.
  (This matches settings on Azure CI.)
- GHA/windows: ignore failing 1451 'Basic SMB request' test for
  old mingw-w64 7.3.0 (but not for 9.5.0!):
  ```
  2024-09-11 21:45:59,738 ERROR smbComNegotiate: b'NT LM 0.12\x00' is not in list
  [...]
  curl: (7) Could not connect to server
  [...]
  FAIL 1451: 'Basic SMB request' SMB
  ```
  Ref: https://github.com/curl/curl/actions/runs/10816280746/job/30007130770#step:13:3546
- CI/Azure: delete, now moved to GHA.

sshserver:

- sshserver: fix permissions for SSH host key on Windows, allowing sshd
  to launch.
- sshserver: fix initializing config paths with Cygwin/MSYS2 sshd.
- sshserver: fix initializing config paths in parallel builds.
- sshserver: delete redundant `DenyUsers`. This also opens the way
  to allow multiple usernames.
- sshserver: fix `AllowUsers` for GHA/windows, by allowing the
  domainless username again.
  Follow-up to 3ee7c676ec #5721
- sshserver: fix `Deprecated option` warnings in `sshd_config`:
  ```
  D:/a/curl/curl/bld/tests/log/2/server/curl_sshd_config line 7: Deprecated option AuthorizedKeysFile2
  D:/a/curl/curl/bld/tests/log/2/server/curl_sshd_config line 25: Deprecated option KeyRegenerationInterval
  D:/a/curl/curl/bld/tests/log/2/server/curl_sshd_config line 35: Deprecated option RhostsRSAAuthentication
  D:/a/curl/curl/bld/tests/log/2/server/curl_sshd_config line 36: Deprecated option RSAAuthentication
  D:/a/curl/curl/bld/tests/log/2/server/curl_sshd_config line 37: Deprecated option ServerKeyBits
  D:/a/curl/curl/bld/tests/log/2/server/curl_sshd_config line 41: Deprecated option UseLogin
  ```
- sshserver: fix `Deprecated option` warnings in `ssh_config` with
  Cygwin/MSYS2.
- sshserver: fix dumping config files due to the filenames missing their
  full paths.
- sshserver: add workaround to make `logmsg` messages visible.
  Before this patch they only went to a file and never shown.

runtests:

- runtests: log details when these Windows commands are called:
  `handle`, `taskkill`, `tasklist`.
- runtests: add documentation links to Windows tools:
  `handle`, `taskkill`, `tasklist`, `icacls`.
- runtests: add `-t` (kill whole tree) option to `taskkill` in
  `servers.pm`, syncing it with the other `taskkill` call.
  Follow-up to bc72a78a11 #14488
- runtests: show warning if Sysinternals `handle` tool is missing.
- runtests: drop Windows XP Home compatibility `tskill` call.
  The call was made on all Windows versions. It's possibly overkill to
  do this, because XP Home is probably rarely used for running curl
  tests these days. In case it's needed, it'd be better to put it under
  an explicit option.
- runtests: show Perl version and path.

Fix/silence fallouts:

- unit2603: fix building with disabled HTTP support.
- unit2604: silence `-Woverlength-strings` warnings in C89 mode.
- test437, test1614: fix to pass with no-IPv6 builds.

Closes #14859
2024-09-16 10:00:17 +02:00
Viktor Szakats
5f93cad086
GHA: replace make with ninja in Cygwin, MSYS2 and mingw-w64 cmake jobs
Speed-ups:
- cygwin build: 47s -> 29s
- cygwin build tests: 3m12 -> 2m32
- MSYS2 native build: 29s -> 16s
- MSYS2 native build tests: 2m24 -> 1m52
- MSYS2/mingw-w64: already had ninja installed and used implicitly.
- linux-mingw-w64-cross: 31s -> 25s

before:
cygwin: https://github.com/curl/curl/actions/runs/10866939355/job/30155134299
windows: https://github.com/curl/curl/actions/runs/10866939361

after:
cygwin: https://github.com/curl/curl/actions/runs/10867861215/job/30157121784
windows: https://github.com/curl/curl/actions/runs/10867861208?pr=14914

Closes #14914
2024-09-15 13:57:21 +02:00
Viktor Szakats
d620ec6791
CI: add test timeouts, more cmake build tests, fix VS2010 C warning
- GHA/macos: set timeout for test runs.
  Double the value for autotools to fit torture tests.
- GHA/cygwin: reduce test run timeout for autotools.
- GHA/cygwin: enable building tests with cmake.
- GHA/windows: enable building tests with MSYS cmake.
- GHA/windows: enable building tests with MSVC UWP.
- appveyor: enable building tests with VS2008 x86 and VS2010 x64.
- tests: add workaround compiler warnings when building with VS2010:
  ```
  tests\server\util.c(482): warning C4306: 'type cast' : conversion from 'int' to 'void (__cdecl *)(int)' of greater size
  tests\server\util.c(486): warning C4306: 'type cast' : conversion from 'int' to 'void (__cdecl *)(int)' of greater size
  tests\server\util.c(490): warning C4306: 'type cast' : conversion from 'int' to 'void (__cdecl *)(int)' of greater size
  tests\server\util.c(720): warning C4306: 'type cast' : conversion from 'int' to 'void (__cdecl *)(int)' of greater size
  tests\server\util.c(726): warning C4306: 'type cast' : conversion from 'int' to 'void (__cdecl *)(int)' of greater size
  tests\server\util.c(732): warning C4306: 'type cast' : conversion from 'int' to 'void (__cdecl *)(int)' of greater size
  tests\server\util.c(781): warning C4306: 'type cast' : conversion from 'int' to 'void (__cdecl *)(int)' of greater size
  tests\server\util.c(785): warning C4306: 'type cast' : conversion from 'int' to 'void (__cdecl *)(int)' of greater size
  tests\server\util.c(789): warning C4306: 'type cast' : conversion from 'int' to 'void (__cdecl *)(int)' of greater size
  ```
  Ref: https://ci.appveyor.com/project/curlorg/curl/builds/50485633/job/4ujlwxod3cexmn2q#L1535

With this, tests are built in all GHA cygwin/windows jobs.

Timeouts avoid long runs with runaway issues, example:
https://github.com/curl/curl/actions/runs/10575522173

Closes #14700
2024-08-27 23:40:48 +02:00
Viktor Szakats
f73f6bf9f8
GHA: add yamlcheck
Uses `yamllint`.

Also:
- fix warning for existing YAML files:
  ```
  [truthy] truthy value should be one of [false, true]
  ```

Closes #14650
2024-08-23 18:42:55 +02:00
Viktor Szakats
2d8464c4cb
GHA/windows: move Cygwin into its own workflow
Cygwin runs stable but slow, while native Windows in the same workflow
runs fast but unreliable (hangs). GHA requires all jobs to finish before
allowing to re-run failed ones. Before this patch this meant waiting for
the slow Cygwin jobs to re-run hung native jobs. After this patch Cygwin
jobs run in their own workflow, allowing to re-run broken Windows jobs
earlier.

Follow-up to 0d12528729 #14236
Closes #14366
2024-08-03 20:21:25 +02:00