runtests.pl: add an -E option to specify an exclude file

It can contain additional restraints for test numbers,
keywords and tools.

The idea is to let third parties like the Privoxy project
distribute an exclude file with their tarballs that specifies
which curl tests are not expected to work when using Privoxy
as a proxy, without having to fork the whole curl test suite.

The syntax could be changed to be extendable and maybe
more closely reflect the "curl test" syntax. Currently
it's a bunch of lines like these:

test:$TESTNUMBER:Reason why this test with number $TESTNUMBER should be skipped
keyword:$KEYWORD:Reason why tests whose keywords contain the $KEYWORD should be skipped
tool:$TOOL:Reason why tests with tools that contain $TOOL should be skipped

To specify multiple $TESTNUMBERs, $KEYWORDs and $TOOLs
on a single line, split them with commas.
This commit is contained in:
Fabian Keil 2012-11-17 12:12:42 +01:00 committed by Daniel Stenberg
parent b47ee58fb7
commit 3f0bef2b53
No known key found for this signature in database
GPG key ID: 5CC908FDB71E12C2
4 changed files with 90 additions and 1 deletions

36
tests/data/test1182 Normal file
View file

@ -0,0 +1,36 @@
<testcase>
<info>
<keywords>
runtests.pl
</keywords>
</info>
#
# Client-side
<client>
<server>
none
</server>
<name>
Verify that runtests.pl accepts an exclude file with the -E option
</name>
<command type="perl">
%SRCDIR/runtests.pl -o TESTDIR=%SRCDIR/log/data -o LOGDIR=%SRCDIR/log/log -E %SRCDIR/log/test1182-exclude-file 1
</command>
<precheck>
mkdir %SRCDIR/log/data ; mkdir %SRCDIR/log/log; cp %SRCDIR/data/test1 %SRCDIR/log/data; echo 'test:1:Test should not run for unit test 1182' > %SRCDIR/log/test1182-exclude-file
</precheck>
<postcheck>
grep -q "Test should not run for unit test 1182" %SRCDIR/log/stdout1182
</postcheck>
</client>
<verify>
<errorcode>
1
</errorcode>
</verify>
</testcase>