Commit graph

6407 commits

Author SHA1 Message Date
Ethan Wilkes
fa3d1e7d43
ws: corrected curlws_cont to reflect its documented purpose
Verified in test 2311

Closes #16512
2025-03-06 16:38:05 +01:00
Stefan Eissing
556f722fe3
tests/certs: cleanup
Keep only the generated files needed for tests. Place generated
intermediaries in `tests/certs/gen` where they are ignored by git. No
longer generated `*.dhp` files.

Have a shorter naming scheme: `test-ca` instead of `EdelCurlRoot-ca` and
`test-localhost` instead of `Server-localhost-sv`, etc.

Remove the `stunnel` certificate as it was nearly a duplicate of
`test-localhost`.

No longer copy a generated certificates to `tests/stunnel.pem`. Let test
server default to `certs/test-localhost.pem` instead.

Closes #16593
2025-03-06 16:02:34 +01:00
Viktor Szakats
8fbd372e0d
tests: mark tests 1631, 1632 flaky
We already marked them flaky in GHA/macos CI. They are also flaky in
other CI jobs, in other OSes, with multiple TLS backends:
- MSVC/LibreSSL: https://github.com/curl/curl/actions/runs/13683996410/job/38262956317
- MSVC/wolfSSL: https://github.com/curl/curl/actions/runs/13680682695/job/38252047077
- FreeBSD/OpenSSL3: https://github.com/curl/curl/actions/runs/13690910863/job/38283867721#step:3:1

Ref: fa461b4eff #14486

Closes #16584
2025-03-06 13:17:15 +01:00
Viktor Szakats
c38f9089f3
lib1156: pass longs to curl_easy_setopt()
To fix this test on macOS with the gcc compiler.

Also unignore test 1156 in GHA/macos.

Ref: c349bd668c #14097 (issue 15.)
Ref: 7b0240c077 #16539
Ref: 2ec00372a1 #16482

Closes #16579
2025-03-06 01:35:30 +01:00
Viktor Szakats
fd54bed51a
curl.h: switch CURL_HTTP_VERSION* enums to long constants
It fixes tests 1539, and 2402, 2404 (for non-Secure Transport), on macOS
with the gcc compiler.

Also unignore these tests in GHA/macos for non-secure transport.

Ref: c349bd668c #14097 (issue 15.)
Ref: 7b0240c077 #16539
Ref: 2ec00372a1 #16482

Closes #16580
2025-03-06 01:01:24 +01:00
Daniel Stenberg
f9ec5d89cb
tests/certs: drop the empty .dhp files
They are not used for anything.

Closes #16576
2025-03-05 23:41:57 +01:00
Viktor Szakats
2485d3edaa
runtests: check and report if diff tool is missing
To make it apparent which CI jobs are missing this tool, so we can
install it to improve the runtests log.

Correction to the followed-up commit: `diff` is not installed via the
`gcc` package but via `automake`. Meaning it needs be installed manually
for MSYS cmake jobs.

Follow-up to e6c78e18da #16571
Closes #16578
2025-03-05 20:28:15 +01:00
Stefan Eissing
9bfa64f850
gnutls: set priority via --ciphers
No longer ignore the `--ciphers` argument in gnutls curl builds, but use
it to set the gnutls priority string.

When the set ciphers start with '+', '-' or '!', it is *appended* to the
curl generated priority string. Otherwise it replaces the curl one
completely.

Add test_17_18 to check various combinations.

Closes #16557
2025-03-05 13:51:56 +01:00
Daniel Stenberg
34cbf95267
tests: make sure 'commands.log' is generated in the correct logdir
Closes #16568
2025-03-05 11:37:38 +01:00
Stefan Eissing
e542fd9da1
http2: detect session being closed on ingress handling
nghttp2 will on its own send GOAWAY frames, closing the connection, when
internal processing of frames runs into errors. This may not become
visible in a direct error code from a call to nghttp2.

Check for session being closed on ingress processing (on sending, we
already did that) and report an error if so. In addition, monitor
outgoing GOAWAY not initiated by us so that the user will get a fail
message when that happens.

Add some more long response header tests.

Closes #16544
2025-03-05 10:59:27 +01:00
Stefan Eissing
aa042210d9
http2: add on_invalid_frame callback for error detection
When the server sends HEADER/CONTINUATION frames that exceed nghttp2's
size, this error is being reported via the on_invalid_frame_recv
callback. Without registering there, it will go unnoticed.

RST the stream when such a frame is encountered.

Closes #16544
2025-03-05 10:59:19 +01:00
Viktor Szakats
aa0d70fef7
build: tidy-ups around inet_pton
- cmake: make the `inet_pton`/`inet_ntop` detection codepath explicit
  for WinCE. To not rely on an empty `HAVE_WIN32_WINNT`.

- tests/server/mqttd: drop `inet_pton` header and lib source.

- tests/server: move `inet_pton` lib source to a variable.

Closes #16563
2025-03-05 03:10:45 +01:00
Viktor Szakats
6f1031e9d5
tests/server: replace strerror with sstrerror in socksd
To follow the `errno` -> `SOCKERRNO` update.

Missed from the previous commit.

Follow-up to adcfd4fb3e #16553
2025-03-04 18:32:51 +01:00
Viktor Szakats
adcfd4fb3e
tests/server: replace errno with SOCKERRNO in sockfilt, socksd, sws
To correctly read the winsock2 result code on Windows.

Follow-up to de2126b182 #5241
Ref: 5e855bbd18 (r38507132)
Ref: #14854
Closes #16553
2025-03-04 18:23:40 +01:00
Daniel Stenberg
57ac8aed3f
tests/data/Makefile.am: fix trailing slash issue
Follow-up from 6758aa722d

Fixes #16556
Closes #16558
2025-03-04 16:43:39 +01:00
tiymat
6758aa722d
IMAP: add CURLOPT_UPLOAD_FLAGS and --upload-flags
Set properties on the uploaded resource.

Test 3209 and 3210 verify.

Closes #15970
2025-03-04 15:21:16 +01:00
Viktor Szakats
9213e4e497
tests/server: round of tidy-ups
Dedupe, merge macros, globals, make symbols local where possible.
Drop unused macros and headers. Drop `DEFAULT_LOGFILE` macro in favour
of `--logfile` command-line option.

Ref: #15000
Closes #16525
2025-03-04 04:17:44 +01:00
Viktor Szakats
387311012c
tidy-up: alphasort lists, indentation/whitespace, pp
- cmake/win32-cache: alphasort items.
- configure.ac: alphasort `CURL_CHECK_FUNC_*` checks.
- configure.ac: alphasort `AC_CHECK_FUNCS` checks.
- prefer `#ifdef`/`#ifndef`.
- lib/asyn-thread: drop unused value of `USE_HTTPSRR_ARES`.
- lib/formdata: drop unused header `libgen.h`.
- indentation, whitespace.

Closes #16490
2025-03-04 01:46:53 +01:00
Stefan Eissing
35cfb081d2
pytest: check overlarge response headers
Add test_01 11, 12 and 13 to check various very long response
headers (accumulated and single) with http/1.1 and h2.

Closes #16541
2025-03-03 15:46:55 +01:00
Daniel Stenberg
fb13923dd6
lib: add CURLFOLLOW_OBEYCODE and CURLFOLLOW_FIRSTONLY
With this change, the argument passed to the CURLOPT_FOLLOWLOCATION
option is now instead a "mode" instead of just a boolean. Documentation
is extended to describe the two new modes.

Test 1571 to 1581 verify.

Closes #16473
2025-03-03 11:35:48 +01:00
Stefan Eissing
0d3b5937b3
OpenSSL/quictls: add support for TLSv1.3 early data
based on #16450

Adds support for TLSv1.3 early data for TCP and QUIC via ngtcp2.

Closes #16477
2025-03-03 09:27:04 +01:00
Stefan Eissing
df672695e5
shutdowns: split shutdown handling from connection pool
Further testing with timeouts in event based processing revealed that
our current shutdown handling in the connection pool was not clear
enough. Graceful shutdowns can only happen inside a multi handle and it
was confusing to track in the code which situation actually applies. It
seems better to split the shutdown handling off and have that code
always be part of a multi handle.

Add `cshutdn.[ch]` with its own struct to maintain connections being
shut down. A `cshutdn` always belongs to a multi handle and uses that
for socket/timeout monitoring.

The `cpool`, which can be part of a multi or share, either passes
connections to a `cshutdn` or terminates them with a one-time, best
effort.

Add an `admin` easy handle to each multi and share. This is used to
perform all maintenance operations where no "real" easy handle is
available. This solves the problem that the multi admin handle requires
some additional initialisation (e.g. timeout list).

The share needs its admin handle as it is often cleaned up when no other
transfer or multi handle exists any more. But we need a `data` in almost
every call.

Fix file:// handling of errors when adding a new connection to the pool.

Changes in `curl` itself:

- for parallel transfers, do not set a connection pool in the share,
  rely on the multi's connection pool instead. While not a requirement
  for the new `cshutdn` to work, this is

  a) helpful in testing to trigger graceful shutdowns
  b) a broader code coverage of libcurl via the curl tool

- on test_event with uv, cleanup the multi handle before returning from
  parallel_event(). The uv struct is on the stack, cleanup of the multi
  later will crash when it tries to register sockets. This is a "eat
  your own dogfood" related fix.

Closes #16508
2025-03-02 11:13:34 +01:00
Stefan Eissing
1aa69221be
hash_offt: standalone hash for curl_off_t
Add a standalong hash table for curl_offt_t as key. This allows a
smaller memory footprint and faster lookups as we do not need to deal
with variable key lengths.

Use in all places we had the standard hash for this purpose.

Closes #16442
2025-03-01 18:42:10 +01:00
Stefan Eissing
2809723ddf
pytest: test negotiate with http proxy
Fixes #14973
Reported-by: stevenpackardblp on github

When curl negotiated with a http: proxy for a https: request, it
wrongly believed there must be an SSL filter present, which during
CONNECT, there is not.

25b445e fixed this. This PR adds a pytest case for the setup.

Closes #16136
2025-03-01 18:40:40 +01:00
Zhaoming Luo
7211576442
tests: fix comment in lib533
Closes #16523
2025-02-28 16:12:20 +01:00
Viktor Szakats
08c7c937dc
tidy-up: prefer return over exit(), fix fallouts
To avoid breaking the control flow and align to majority of code
already using `return`.

`exit()` has the side-effect of suppressing leak detection in cases.
Fix fallouts detected after switching to `return`.

- configure:
  - fix `getaddrinfo` run test to call `freeaddrinfo()` to pacify ASAN,
    and call `WSACleanup()` to deinit winsock2.
  - fix `getifaddrs` run test to call `freeifaddrs()` to pacify ASAN.
- tests/server:
  - setup `atexit(win32_cleanup)` via `win32_init()`.
  - return 2 instead of 1 on winsock2 init failures.
  - sws: goto cleanup instead of `exit()` in `http_connect()`.
    Follow-up to 02dfe71937 #7235
- tests/client/http:
  - cleanup memory to pacify ASAN in `h2-upgrade-extreme`,
    `tls-session-reuse`.
- examples:
  - block_ip: fix memory leak reported by CI.
  - http2-upload: avoid handle leaks.

Untouched `exit()` calls, made from callbacks:
- docs/examples: ephiperfifo.c, ghiper.c, hiperfifo.c
- tests/libtest: lib582.c, lib655.c, lib670.c
- tests/server: tftpd.c

Closes #16507
2025-02-28 13:11:41 +01:00
Stefan Eissing
2e585f5640
test488: set --output-dir
Otherwise the downloaded files land in the `tests` directory and
show up in git status.

Closes #16519
2025-02-28 12:47:31 +01:00
Viktor Szakats
5a021aba41
cmake: improve httpd detection for pytest
Look for `httpd` in addition to `apache2`, like `./configure` does.
It fixes detection with macOS Homebrew for example.

Closes #16515
2025-02-28 03:40:40 +01:00
Viktor Szakats
cba83bfb10
runtests: accept CURL_DIRSUFFIX without ending slash
Follow-up to 3585796049 #16452
Closes #16506
2025-02-27 12:11:23 +01:00
Daniel Stenberg
bc24c60512
urlapi: fix redirect from file:// with query, and simplify
- fix redirect from file:// URL with query part
- find_host_sep() simplify
- urlencode_str() simplify
- redirect_url() simplify
- made more const char *
- add more redirect URL test cases to test 1560

Closes #16498
2025-02-27 11:51:19 +01:00
Viktor Szakats
c028a243f2
build: set -O3 and tune WinCE in CI, fix getpart, vtls_scache fallouts
- GHA/windows/WinCE:
  - set `-O3 -DNDEBUG` C flags manually for the CMake mingw32ce build.
    CMake doesn't recognize the platform and fails to add them. To match
    autotools (using `-O2`), and hit similar compiler warnings.
  - enable parallel builds for cmake.
  - tune parallelism for cmake using unity batches.
  - tune parallelism for autotools.

  Follow-up to 2a292c3984 #15975

- tests: fix potentially uninitialized value in `readline()` in
  `getpart.c`. Detected by gcc 4.4.0 `-O2` (Windows CE) jobs:
  ```
  tests/server/getpart.c: In function 'getpart':
  tests/server/getpart.c:298: error: 'datalen' may be used uninitialized in this function
  ```
  Ref: https://github.com/curl/curl/actions/runs/13522595237/job/37785147505?pr=16476#step:11:25
  Follow-up to 592880a3ca

- vtls_scache: rework returning pointer to avoid compiler warning seen
  with `-O3` gcc 4.4.0 builds (Windows CE/schannel):
  ```
  lib/vtls/schannel.c: In function 'schannel_connect_step1':
  lib/vtls/vtls_scache.c:975: error: dereferencing pointer 'old_cred.4474' does break strict-aliasing rules
  lib/vtls/vtls_scache.c:985: error: dereferencing pointer 'old_cred.4474' does break strict-aliasing rules
  lib/vtls/schannel.c:959: note: initialized from here
  ```
  Ref: https://github.com/curl/curl/actions/runs/13523868335/job/37789610845#step:9:25
  Follow-up to fa0ccd9f1f #15774

Closes #16476
2025-02-26 21:23:11 +01:00
Viktor Szakats
a1d00da81e
test1167: catch #defines with extra whitespace
Before this patch, it missed this in `curl/curl.h`:
```c
#  define __has_declspec_attribute(x) 0
```
After this patch:
```
test 1167...[Verify curl prefix of public symbols in header files]
/usr/bin/perl -I. -I.  returned 1, when expecting 0
 1167: exit FAILED
== Contents of files in the log/14/ dir after test 1167
=== Start of file server.cmd
 Testnum 1167
=== End of file server.cmd
=== Start of file stdout1167
 Bad symbols in public header files:
   __has_declspec_attribute(x)
=== End of file stdout1167
FAIL 1167: 'Verify curl prefix of public symbols in header files' source analysis
TESTFAIL: These test cases failed: 1167
```

Ref: #16491
Closes #16496
2025-02-26 14:01:40 +01:00
Daniel Stenberg
f5527e57f8
tests: make sure gdb gets the right path for -x
Regression from #16452

Closes #16495
2025-02-26 13:12:04 +01:00
Viktor Szakats
bc42010f66
GHA/http3-linux: build out-of-tree, make test2502 support it
To sync with the rest of core workflows.

Also fixup test2502 failing for out-of-tree builds due to:
```
== Info: error reading ca cert file ./certs/EdelCurlRoot-ca.cacert (Error while reading file.)
```
Ref: https://github.com/curl/curl/actions/runs/13525575035/job/37795171282?pr=16480#step:23:3608

Cherry-picked from #16480
Closes #16481
2025-02-26 12:35:03 +01:00
Daniel Stenberg
d485177151
altsvc: rewrite parser using strparse
Extend test 1654.

Closes #16454
2025-02-26 08:36:08 +01:00
Daniel Stenberg
7826927d9b
libtest/libprereq.c: set CURLOPT_FOLLOWLOCATION with a long
Previously this used '1', which as an int. The option needs a long.

Closes #16487
2025-02-26 00:06:24 +01:00
Viktor Szakats
84332d49fb
runtests: drop recognizing 'winssl' as Schannel
Follow-up to 180501cb02 #3504
Closes #16467
2025-02-25 12:59:10 +01:00
Viktor Szakats
31fd77fb3b
runtests: recognize AWS-LC as OpenSSL
Fixes (GHA/linux, AWS-LC jobs):
```
test 0307 SKIPPED: curl lacks OpenSSL support
test 0308 SKIPPED: curl lacks OpenSSL support
[...]
```
Ref: https://github.com/curl/curl/actions/runs/13511134270/job/37751473424#step:42:104

Follow-up to 34ef4fab22 #10320
Closes #16466
2025-02-25 12:59:09 +01:00
Daniel Stenberg
6306476fc3
tool_getparam: make --url support a file with URLs
It implies -O used for each URL.

Mention in the --url documentation.

Test 488 and 489 verify.

Closes #16099
2025-02-25 09:16:54 +01:00
Yedaya Katsman
a55b5b7c62
rustls: add support for CERTINFO
This allows you to use the `certs` and `num_certs` writeout variables in
the curl tool, and getting information about the server certificates
using CURLINFO_CERTINFO.

Closes #16459
2025-02-25 07:59:39 +01:00
Viktor Szakats
3175984ab2
build: drop unused getpart tool
Drop the `getpart` test tool from standard builds. This tool was not
used by tests.

Also:
- make it easier to build it standalone for testing `getpart.c` on
  the command-line.
- reduce local var scopes in source.

Closes #16460
2025-02-25 01:52:13 +01:00
Viktor Szakats
a7bcf25c83
runtests: drop ref to unused external function [ci skip]
Follow-up to 3585796049 #16452
2025-02-24 23:07:51 +01:00
Viktor Szakats
3585796049
runtests: support multi-target cmake, drop workarounds from CI
Support multi-target cmake builds via `CURL_DIRSUFFIX` env. For example:
`export CURL_DIRSUFFIX=Debug/`.

Multi-target generators place their output to `src/<subdir>/`,
`lib/<subdir>/`, `tests/server/<subdir>`, `tests/libtest/<subdir>` and
`tests/unit/<subdir>/` by default. Before this patch, `runtests.pl`
couldn't run on such builds because it expected the binaries under the
their `<subdir>`-less directories. This patch allows to set such subdir
and make `runtests.pl` find the binaries. In CI we use multi-target
builds with tests for MSVC. It also helps Xcode-generator builds, though
in CI we don't have such job running tests.

There may be better solutions to configure this, but passing a custom
value to `runtests.pl` including its subprocesses is somewhat tricky.
The reason the configuration value expects the slash at the end is
because MSYS is automagically expanding the env to a (wrong) absolute
path if the slash is in the front.

Also:
- drop the `-DCMAKE_RUNTIME_OUTPUT_DIRECTORY_*=` workaround from CI.
- replace `resolve` references in tests with a new `%RESOLVE` variable.
  It didn't use a filename extension before. After this patch it uses
  `exe_ext('TOOL')`. I'm not sure if this is the correct choice vs.
  `exe_ext('SRV')`.
- fix `-c` option format in manual.
- fix some whitespace.

Note, in CI we still tweak `CMAKE_RUNTIME_OUTPUT_DIRECTORY_*` in jobs
which share steps between `./configure` and cmake. It's easier that way.

Ref: #15000
Cherry-picked from #16394

Closes #16452
2025-02-24 21:00:30 +01:00
Stefan Eissing
edd573d980
wolfssl: tls early data support
Enable TLS Early Data for wolfSSL:

- merge WOLFSSL_CTX and WOLFSSL setup from ngtcp2 with the general
  implemenation in wolfssl.c
- enable for QUIC via ngtcp2
- give Curl_vquic_tls_init() a `struct alpn_spec` like used for the TCP
  case. Adapt gnutls and other users.
- enable pytest test cases for early data with wolfSSL

and while this messes up wolfssl.c anyway, do

- rename all struct/functions with prefix 'wolfssl_' to 'wssl_' to not
  pollute that name prefix
- rename `ctx/handle` to `ssl_ctx/ssl`, as used in openssl case

Closes #16167
2025-02-24 10:01:51 +01:00
Daniel Stenberg
e1b3d46944
tool_operate: fail SSH transfers without server auth
This now insists on using a server auth option unless --insecure is
provided. As an added bonus, it now also only checks for the knownhosts
file once (if found).

Ref: #16197
Closes #16205
2025-02-23 17:54:21 +01:00
Daniel Stenberg
4c5099868e
http: make the RTSP version check stricter
- make it only accept version 1.0, as that is the version curl supports
- convert the parser to use strparse
- the status code max is now 999, but it does allow != 3 digits

Closes #16435
2025-02-22 15:07:31 +01:00
Stefan Eissing
cfc657a48d
multi: event based rework
Rework the event based handling of transfers and connections to
be "localized" into a single source file with clearer dependencies.

- add multi_ev.c and multi_ev.h
- add docs/internal/MULTI-EV.md to explain the overall workings
- only do event handling book keeping when the socket callback
  is set
- add handling for "connection only" event tracking, when internal
  easy handles are used that are not really tied to a connection.
  Used in connection pool.
- remove transfer member "last_poll" and connections "shutdown_poll"
  and keep all that internal to multi_ev.c
- add CURL_TRC_M() for tracing of "multi" related things, including
  event handling and connection pool operations. Add new trace
  feature "multi" for trace config.
  multi traces will show exactly what is going on in regard to
  event handling.
- multi: trace transfers "mstate" in every CURL_TRC_M() call
- make internal trace buffer 2048 bytes and end the silliness
  with +n here -m there. Adjust test 1652 expectations of resulting
  length and input edge cases.
- add trace feature "lib-ids" to perfix libcurl traces with transfer
  and connection ids. Useful for debugging libcurl applications.

Closes #16308
2025-02-22 14:47:40 +01:00
Daniel Stenberg
886f5dea80
test76: add test with comma-separated list of WWW-Auth methods
The support for this is not 100% but at least this works.

Closes #16425
2025-02-22 14:09:37 +01:00
Daniel Stenberg
a6e8ead533
var: add a '64dec' function that can base64 decode a string
Verified in test 455 and 487.

If the provided string cannot be base64-decoded, it will instead use
"[64dec-fail]" (without the quotes).

Documented

Ref: #16288
Closes #16330
2025-02-22 11:38:52 +01:00
Stefan Eissing
54c07fd6c5
curl: add write-out variable 'tls_earlydata'
Add the write-out variable 'tls_earlydata' to inspect the amount of
TLSv1.3 early data that had been sent.

Closes #15956
2025-02-22 10:02:39 +01:00