tests: avoid hard-coded CRLFs in more sections

- `reply/data*`, `verify/stdout`, `verify/stderr`, `verify/file*`,
  `verify/proxy`:
  - make `crlf="yes"` force CRLF to all lines, instead of just applying
    to HTTP protocol headers.
  - add support for `crlf="headers"` that only converts HTTP protocol
    header lines to CRLF. (previously done via `crlf="yes"`.)
  - use `crlf="headers"` where possible.

- `reply/connect*`:
  - add support for `crlf="yes"` and `crlf="headers"`.
  - use them where possible.

- `client/file*`, `client/stdin`:
  - add support for `crlf="yes"`.
  - use it where possible.

- `reply/data*`, `verify/protocol`:
  - replace existing uses of `crlf="yes"` with `crlf="headers`" where it
    does not change the result.

Reducing the number of `tests/data/test*`:
- CRLF newlines from 10295 to 1985. (119985 lines total)
- files with mixed newlines from 656 to 113. (1890 files total)

After this patch there remain 141 sections with mixed newlines, where
the mixing is not split between headers/non-headers. There is no obvious
pattern here. Some of the CRLF uses might be accidental, or
non-significant. They will be tackled in a future patch.

Follow-up to 6cf3d7b1b1 #19318
Follow-up to 4d2a05d3fe #19284

Closes #19313
This commit is contained in:
Viktor Szakats 2025-10-31 15:50:01 +01:00
parent 6adefe8ad0
commit 63e9721b63
No known key found for this signature in database
GPG key ID: B5ABD165E2AEF201
1096 changed files with 10495 additions and 10422 deletions

View file

@ -299,7 +299,7 @@ sub prepro {
my (@entiretest) = @_;
my $show = 1;
my @out;
my $data_crlf;
my $data_crlf = "";
my @pshow;
my @altshow;
my $plvl;
@ -351,21 +351,26 @@ sub prepro {
next;
}
if($show) {
# The processor does CRLF replacements in the <data*> sections if
# necessary since those parts might be read by separate servers.
if($s =~ /^ *<data(.*)\>/) {
if($1 =~ /crlf="yes"/) {
$data_crlf = 1;
# The processor does CRLF replacements in the <data*> and <connect*>
# sections if necessary since those parts might be read by separate
# servers.
if($s =~ /^ *<(data|connect)(.*)\>/) {
if($2 =~ /crlf="yes"/) {
$data_crlf = "yes";
}
elsif($2 =~ /crlf="headers"/) {
$data_crlf = "headers";
}
}
elsif(($s =~ /^ *<\/data/) && $data_crlf) {
$data_crlf = 0;
elsif(($s =~ /^ *<\/(data|connect)/) && $data_crlf ne "") {
$data_crlf = "";
}
subvariables(\$s, $testnum, "%");
subbase64(\$s);
subsha256base64file(\$s);
substrippemfile(\$s);
subnewlines(0, \$s) if($data_crlf);
subnewlines(1, \$s) if($data_crlf eq "yes");
subnewlines(0, \$s) if($data_crlf eq "headers");
push @out, $s;
}
}
@ -758,7 +763,6 @@ sub singletest_prepare {
logmsg " $testnum: IGNORED: Section client=>file has no name attribute\n";
return -1;
}
my $fileContent = join('', @inputfile);
# make directories if needed
my $path = dirname($filename);
@ -775,11 +779,15 @@ sub singletest_prepare {
}
if(open(my $outfile, ">", "$filename")) {
binmode $outfile; # for crapage systems, use binary
if($fileattr{'nonewline'}) {
# cut off the final newline
chomp($fileContent);
chomp($inputfile[-1]);
}
print $outfile $fileContent;
if($fileattr{'crlf'}) {
subnewlines(1, \$_) for @inputfile;
}
print $outfile join('', @inputfile);
close($outfile);
} else {
logmsg "ERROR: cannot write $filename\n";
@ -947,6 +955,10 @@ sub singletest_run {
chomp($stdintest[-1]);
}
if($hash{'crlf'}) {
subnewlines(1, \$_) for @stdintest;
}
writearray($stdinfile, \@stdintest);
$cmdargs .= " <$stdinfile";