Commit graph

704 commits

Author SHA1 Message Date
Daniel Stenberg
7c140f6b2d
runtests: set 'oldlibssh' for libssh versions before 0.9.6
... and make test 1459 check for the different return code then.

Closes #8444
2022-02-14 08:26:44 +01:00
Fabian Keil
2cb0eaba75 runtests.pl: tolerate test directories without Makefile.inc
Silences the following warnings when using a Makefile.inc-free
TESTDIR using the "-o" argument:

   readline() on closed filehandle D at ./runtests.pl line 592.
   Use of uninitialized value $disttests in pattern match (m//) at
   ./runtests.pl line 3602.

Closes https://github.com/curl/curl/pull/8379
2022-02-03 16:38:36 -05:00
Patrick Monnerat
52826d3b79
runtests.pl: disable debuginfod
Valgrind and gdb implement this feature: as this highly slows down tests,
disable it.

Closes #8291
2022-01-17 22:26:52 +01:00
Fabian Keil
736847611a
runtests.pl: support the nonewline attribute for the data part
Added to FILEFORMAT

Closes #8239
2022-01-10 15:54:54 +01:00
Daniel Stenberg
223f26c28a
mesalink: remove support
Mesalink has ceased development. We can no longer encourage use of it.
It seems to be continued under the name TabbySSL, but no attempts have
(yet) been to make curl support it.

Fixes #8188
Closes #8191
2022-01-10 11:27:59 +01:00
Daniel Stenberg
cfb4f8b626
runtests: make 'wolfssl' a testable feature
Closes #8252
2022-01-09 18:08:14 +01:00
Daniel Stenberg
1fefa1a856
runtests: make 'rustls' a testable feature 2022-01-09 16:06:19 +01:00
Daniel Stenberg
4432234aca
runtests: allow client/file to specify multiple directories
... and make sure to mkdir them all
2022-01-07 10:08:35 +01:00
Fabian Keil
43eef423d6
runtests.pl: properly print the test if it contains binary zeros 2022-01-07 10:01:44 +01:00
Daniel Stenberg
d2f712ddd0
runtests: add verify/socks check
If used, this data is compared with the data in log/socksd-request.log
which the socksd server logs.

Added to FILEFORMAT.md
2022-01-03 22:32:05 +01:00
Daniel Stenberg
ee2ca5826b
tool_operate: warn if too many output arguments were found
More output instructions than URLs is likely a user error.

Add test case 371 to verify

Closes #8210
2022-01-03 15:55:38 +01:00
Daniel Stenberg
f4cafa4f0d
runtests: make the SSH library a testable feature
libssh2, libssh and wolfssh
2021-11-14 16:13:44 +01:00
Marc Hoersken
9121032fe3
CI/runtests.pl: restore -u flag, but remove it from CI runs
This makes it possible to use -u again for local testing,
but removes the flag from CI config files and make targets.

Reviewed-by: Daniel Stenberg

Partially reverts #7841
Closes #7921
2021-10-31 11:28:25 +01:00
Daniel Stenberg
8a534f8838
runtests: split out ignored tests
Report ignore tests separately from the actual fails.

Don't exit non-zero if test servers couldn't get killed.

Assisted-by: Jay Satiro

Fixes #7818
Closes #7841
2021-10-12 17:00:50 +02:00
Mats Lindestam
d1e7d9197b
libssh2: add SHA256 fingerprint support
Added support for SHA256 fingerprint in command line curl and in
libcurl.

Closes #7646
2021-09-26 23:20:53 +02:00
Marc Hoersken
60efeb1e0d
runtests: add option -u to error on server unexpectedly alive
Let's try to actually handle the server unexpectedly alive
case by first making them visible on CI builds as failures.

This is needed to detect issues with killing of the test
servers completely including nested process chains with
multiple PIDs per test server (including bash and perl).

On Windows/cygwin platforms this is especially helpful with
debugging PID mixups due to cygwin using its own PID space.

Reviewed-by: Daniel Stenberg
Closes #7180
2021-09-09 21:28:26 +02:00
Marc Hoersken
5b1c2dd1db
tests/*server.pl: flush output before executing subprocess
Also avoid shell processes staying around by using exec.
This is necessary to avoid output data being buffering
inside the process chain of Perl, Bash/Shell and our
test server binaries. On non-Windows systems the exec
will also make the subprocess replace the intermediate
shell, but on Windows it will at least bind the processes
together since there is no real fork or exec available.

See: https://cygwin.com/cygwin-ug-net/highlights.html
and: https://docs.microsoft.com/cpp/c-runtime-library/exec-wexec-functions
Ref: https://github.com/curl/curl/pull/7530#issuecomment-900949010

Reviewed-by: Daniel Stenberg
Reviewed-by: Jay Satiro
Closes #7530
2021-08-18 20:59:36 +02:00
Marc Hoersken
893f39c596
tests/runtests.pl: cleanup copy&paste mistakes and unused code
Reviewed-by: Jay Satiro
Part of #7481
2021-07-25 19:59:14 +02:00
Gealber Morales
791937b881
mqtt: add support for username and password
Minor-edits-by: Daniel Stenberg
Added test 2200 to 2205

Closes #7243
2021-06-12 23:50:13 +02:00
Daniel Stenberg
f60f51721c
runtests: init $VERSION to avoid warnings when using -l 2021-06-11 14:26:32 +02:00
Daniel Stenberg
dd47b0cce9
runtests: also find the last test in Makefile.inc
Closes #7209
2021-06-10 08:42:52 +02:00
Daniel Stenberg
76035e612a
runtests: skip disabled tests unless -f is used
To make it easier to write ranges like '115 to 229' without that
explicitly enabling tests that are listed in DISABLED, this makes
runtests always skip disabled tests unless the -f command line option is
used.

Previously the code attempted to not run such tests, but didn't do it
correctly.

Closes #7212
2021-06-08 23:35:37 +02:00
Daniel Stenberg
265b14d6b3
metalink: remove
Warning: this will make existing curl command lines that use metalink to
stop working.

Reasons for removal:

1. We've found several security problems and issues involving the
   metalink support in curl. The issues are not detailed here. When
   working on those, it become apparent to the team that several of the
   problems are due to the system design, metalink library API and what
   the metalink RFC says. They are very hard to fix on the curl side
   only.

2. The metalink usage with curl was only very briefly documented and was
   not following the "normal" curl usage pattern in several ways, making
   it surprising and non-intuitive which could lead to further security
   issues.

3. The metalink library was last updated 6 years ago and wasn't so
   active the years before that either. An unmaintained library means
   there's a security problem waiting to happen. This is probably reason
   enough.

4. Metalink requires an XML parsing library, which is complex code (even
   the smaller alternatives) and to this day often gets security
   updates.

5. Metalink is not a widely used curl feature. In the 2020 curl user
   survey, only 1.4% of the responders said that they'd are using it. In
   2021 that number was 1.2%. Searching the web also show very few
   traces of it being used, even with other tools.

6. The torrent format and associated technology clearly won for
   downloading large files from multiple sources in parallel.

Cloes #7176
2021-06-07 08:14:25 +02:00
Daniel Stenberg
b193fd3bbd
runtests: enable 'hyper mode' only for HTTP tests
The 'hyper mode' makes line-ending checks work in the test suite for
when hyper is used. Now it also requires that HTTP or HTTPS are
mentioned as keywords to be enabled so that it doesn't wrongly adjusts
tests for other protocols.

This makes test 271 (TFTP) work again in hyper enabled builds.

Closes #7185
2021-06-03 23:08:47 +02:00
Daniel Stenberg
3c0f4622cd
runtests: parse data/Makefile.inc instead of using make
The warning about missing entries in that file then doesn't require that
the Makefile has been regenerated which was confusing.

The scan for the test num is a little more error prone than before
(since now it doesn't actually verify that it is legitimate Makefile
syntax), but I think it is good enough.

Closes #7177
2021-06-02 23:21:18 +02:00
Daniel Stenberg
6fc805d0c1
test972: verify the json output with jsonlint
Make sure one of the azure jobs has jsonlint installed so that the test
runs there.

Ref: #6905
2021-04-17 11:03:40 +02:00
Marc Hoersken
311c31ec8e
runtests.pl: kill processes locking test log files
Introduce a new runtests.pl command option: -rm

For now only required and implemented for Windows.
Ignore stunnel logs due to long running processes.

Requires Sysinternals handle[64].exe to be on PATH.

Reviewed-by: Jay Satiro

Ref: #6058
Closes #6179
2021-03-01 20:19:01 +01:00
Viktor Szakats
bd6b54ba1f
tests: disable .curlrc in more environments
by also setting CURL_HOME and XDG_CONFIG_HOME envvars to the local
directory.

Reviewed-by: Daniel Stenberg
Fixes #6595
Closes #6596
2021-02-11 22:06:02 +00:00
Stephan Szabo
1269c80af1 file: Support unicode urls on windows
Closes https://github.com/curl/curl/pull/6501
2021-02-09 02:45:51 -05:00
Fabian Keil
203dc00718
runtests.pl: add %TESTNUMBER variable to make copying tests more convenient 2021-02-09 08:16:40 +01:00
Fabian Keil
8ad31a8c6e
runtests.pl: add an -o option to change internal variables
runtests.pl has lots of internal variables one might want to
change in certain situations, but adding a dedicated option
for every single one of them isn't practical.

Usage:
./runtests.pl -o TESTDIR=$privoxy_curl_test_dir -o HOSTIP=10.0.0.1 ...
2021-02-09 00:28:29 +01:00
Fabian Keil
e6d55f647d
runtests.pl: cleanups
- show the summarized test result in the last line of the report
- do not use $_ after mapping it to a named variable
  Doing that makes the code harder to follow.
- log the restraints sorted by the number of their occurrences
- fix language when logging restraints that only occured once
- let runhttpserver() use $TESTDIR instead of $srcdir
  ... so it works if a non-default $TESTDIR is being used.
2021-02-09 00:28:29 +01:00
Fabian Keil
3f0bef2b53
runtests.pl: add an -E option to specify an exclude file
It can contain additional restraints for test numbers,
keywords and tools.

The idea is to let third parties like the Privoxy project
distribute an exclude file with their tarballs that specifies
which curl tests are not expected to work when using Privoxy
as a proxy, without having to fork the whole curl test suite.

The syntax could be changed to be extendable and maybe
more closely reflect the "curl test" syntax. Currently
it's a bunch of lines like these:

test:$TESTNUMBER:Reason why this test with number $TESTNUMBER should be skipped
keyword:$KEYWORD:Reason why tests whose keywords contain the $KEYWORD should be skipped
tool:$TOOL:Reason why tests with tools that contain $TOOL should be skipped

To specify multiple $TESTNUMBERs, $KEYWORDs and $TOOLs
on a single line, split them with commas.
2021-02-09 00:28:28 +01:00
Fabian Keil
b47ee58fb7
runtests.pl: add -L parameter to require additional perl libraries
This is useful to change the behaviour of the script without
having to modify the file itself, for example to use a custom
compareparts() function that ignores header differences that
are expected to occur when an external proxy is being used.

Such differences are proxy-specific and thus the modifications
should be maintained together with the proxy.
2021-02-09 00:28:28 +01:00
Fabian Keil
4421359e77
runtests.pl: add a -P option to specify an external proxy
... that should be used when executing the tests.

The assumption is that the proxy is an HTTP proxy.

This option should be used together with -L to provide
a customized compareparts() version that knows which
proxy-specific header differences should be ignored.

This option doesn't work for all test types yet.
2021-02-09 00:28:28 +01:00
Daniel Stenberg
f451206266
runtests: preprocess DISABLED to allow conditionals
... with this function provided, we can disable tests for specific
environments and setups directly within this file.

Closes #6477
2021-01-19 10:04:41 +01:00
Daniel Stenberg
bbfad7e8a1
runtests: turn preprocessing into a separate function
... and remove all other variable substitutions as they're now done once
and for all in the preprocessor.
2021-01-19 10:04:41 +01:00
Fabian Keil
4f61fd87b2
misc: fix typos
Bug: https://curl.se/mail/lib-2021-01/0063.html
Closes #6434
2021-01-11 15:12:25 +01:00
Daniel Stenberg
9d7281ff17
global_init: debug builds allocates a byte in init
... to make build tools/valgrind warn if no curl_global_cleanup is
called.

This is conditionally only done for debug builds with the env variable
CURL_GLOBAL_INIT set.

Closes #6410
2021-01-06 15:13:45 +01:00
Emil Engler
725ec470e2
language: s/behaviour/behavior/g
We currently use both spellings the british "behaviour" and the american
"behavior". However "behavior" is more used in the project so I think
it's worth dropping the british name.

Closes #6395
2021-01-02 23:35:59 +01:00
Daniel Stenberg
48b85c46f1
runtests: add support for gophers, gopher over TLS 2020-12-15 12:58:19 +01:00
Daniel Stenberg
34bb59a868
runtests: add 'wakeup' as a feature 2020-12-11 17:42:17 +01:00
Daniel Stenberg
3e92799171
runtests.pl: fix "uninitialized value" warning
follow-up to e12825c642
2020-12-11 13:53:23 +01:00
Daniel Stenberg
e12825c642
runtests: add support for %if [feature] conditions
... to make tests run differently or expect different results depending
on what features that are present or not in curl.

Bonus: initial minor 'Hyper' awareness but nothing is using that yet

Closes #6304
2020-12-11 13:16:02 +01:00
Daniel Stenberg
082422b189 runtests: make 'c-ares' a "feature" to depend on
... also added to the docs.
2020-11-26 17:24:24 +01:00
Daniel Stenberg
4d2f800677
curl.se: new home
Closes #6172
2020-11-04 23:59:47 +01:00
Daniel Stenberg
7385610d0c
hsts: add support for Strict-Transport-Security
- enable in the build (configure)
- header parsing
- host name lookup
- unit tests for the above
- CI build
- CURL_VERSION_HSTS bit
- curl_version_info support
- curl -V output
- curl-config --features
- CURLOPT_HSTS_CTRL
- man page for CURLOPT_HSTS_CTRL
- curl --hsts (sets CURLOPT_HSTS_CTRL and works with --libcurl)
- man page for --hsts
- save cache to disk
- load cache from disk
- CURLOPT_HSTS
- man page for CURLOPT_HSTS
- added docs/HSTS.md
- fixed --version docs
- adjusted curl_easy_duphandle

Closes #5896
2020-11-03 16:08:42 +01:00
Daniel Stenberg
61630a155e
runtests: show keywords when no tests ran
To help out future debugging, runtests now outputs the list of keywords
when it fails because no tests ran.

Ref: #6120
Closes #6126
2020-10-26 23:05:19 +01:00
Daniel Stenberg
4282d5f9d0
runtests: revert the mistaken edit of $CURL
Regression from c4693adc62
2020-10-19 12:01:00 +02:00
Daniel Stenberg
88bf689f94
runtests: return error if no tests ran
... and make TESTFAIL stand out a little better by adding newlines
before and after.

Reported-by: Marc Hörsken
Issue: #6052
Closes #6053
2020-10-15 15:50:11 +02:00