romenskiy2012/curl
1
0
Fork 0
mirror of https://github.com/curl/curl.git synced 2026-08-25 06:03:32 +03:00
Code Issues Projects Releases Packages Wiki Activity

runtests: refactor the main test loop into two

Browse source 
The test loop now has an initial loop that first runs through all
possible tests to build a set of those to attempt on this run based on
features and keywords and only then goes through that new list to run
them.  This actually makes it three loops through all tests cases, as
there is an existing loop that gathers possible test numbers from the
test files on disk.

This has two minor effects on the output: all the tests that will be
skipped are displayed at the start (instead of being interspersed with
other tests) and the -l option no longer shows a count of tests at the
end or a (misleading) statement that tests have run successfully. The
skipped tests are also omitted from the test results sent to AppVeyor
and Azure in CI builds.

Another effect is a reduction in the amount of work considered part of
the "Test definition reading and preparation time" reported with -r
making those figures slightly lower than before.

Ref: #10818
This commit is contained in:
Dan Fandrich 2023-04-20 17:41:31 -07:00
parent 020cf1c117
commit a549e046b1
3 changed files with 126 additions and 71 deletions
Download patch file Download diff file Expand all files Collapse all files

35
tests/runner.pm
View file

@ -50,6 +50,7 @@ BEGIN {
# these are for debugging only
our @EXPORT_OK = qw(
readtestkeywords
singletest_preprocess
);
}
@ -191,6 +192,21 @@ sub prepro {
}
#######################################################################
# Load test keywords into %keywords hash
#
sub readtestkeywords {
my @info_keywords = getpart("info", "keywords");
# Clear the list of keywords from the last test
%keywords = ();
for my $k (@info_keywords) {
chomp $k;
$keywords{$k} = 1;
}
}
#######################################################################
# Memory allocation test and failure torture testing.
#
@ -349,6 +365,7 @@ sub torture {
}
#######################################################################
# restore environment variables that were modified in test
sub restore_test_env {
my $deleteoldenv = $_[0]; # 1 to delete the saved contents after restore
@ -880,9 +897,21 @@ sub singletest_postcheck {
# Get ready to run a single test case
sub runner_test_preprocess {
my ($testnum)=@_;
my %testtimings;
# timestamp test preparation start
# TODO: this metric now shows only a portion of the prep time; better would
# be to time singletest_preprocess below instead
$testtimings{"timeprepini"} = Time::HiRes::time();
###################################################################
# Load test metadata
# ignore any error here--if there were one, it would have been
# caught during the selection phase and this test would not be
# running now
loadtest("${TESTDIR}/test${testnum}");
readtestkeywords();
###################################################################
# Start the servers needed to run this test case
my $why = singletest_startservers($testnum, \%testtimings);
@ -891,14 +920,14 @@ sub runner_test_preprocess {
###############################################################
# Generate preprocessed test file
# This must be done after the servers are started so server
# variables are available for substitution.
singletest_preprocess($testnum);
###############################################################
# Set up the test environment to run this test case
singletest_setenv();
###############################################################
# Check that the test environment is fine to run this test case
if (!$listonly) {