tests: avoid hard-coded CRLFs in more sections

- `reply/data*`, `verify/stdout`, `verify/stderr`, `verify/file*`,
  `verify/proxy`:
  - make `crlf="yes"` force CRLF to all lines, instead of just applying
    to HTTP protocol headers.
  - add support for `crlf="headers"` that only converts HTTP protocol
    header lines to CRLF. (previously done via `crlf="yes"`.)
  - use `crlf="headers"` where possible.

- `reply/connect*`:
  - add support for `crlf="yes"` and `crlf="headers"`.
  - use them where possible.

- `client/file*`, `client/stdin`:
  - add support for `crlf="yes"`.
  - use it where possible.

- `reply/data*`, `verify/protocol`:
  - replace existing uses of `crlf="yes"` with `crlf="headers`" where it
    does not change the result.

Reducing the number of `tests/data/test*`:
- CRLF newlines from 10295 to 1985. (119985 lines total)
- files with mixed newlines from 656 to 113. (1890 files total)

After this patch there remain 141 sections with mixed newlines, where
the mixing is not split between headers/non-headers. There is no obvious
pattern here. Some of the CRLF uses might be accidental, or
non-significant. They will be tackled in a future patch.

Follow-up to 6cf3d7b1b1 #19318
Follow-up to 4d2a05d3fe #19284

Closes #19313

docs/tests/FILEFORMAT.md

@@ -241,7 +241,7 @@ similar.
 
 ## `<reply>`
 
-### `<data [nocheck="yes"] [sendzero="yes"] [hex="yes"] [nonewline="yes"] [crlf="yes"]>`
+### `<data [nocheck="yes"] [sendzero="yes"] [hex="yes"] [nonewline="yes"] [crlf="yes|headers"]>`
 
 data to be sent to the client on its request and later verified that it
 arrived safely. Set `nocheck="yes"` to prevent the test script from verifying
@@ -269,16 +269,19 @@ used as "raw" data.
 `nonewline=yes` means that the last byte (the trailing newline character)
 should be cut off from the data before sending or comparing it.
 
-`crlf=yes` forces *header* newlines to become CRLF even if not written so in
-the source file. Note that this makes runtests.pl parse and "guess" what is a
-header and what is not in order to apply the CRLF line endings appropriately.
+`crlf=yes` forces the newlines to become CRLF even if not written so in the
+test.
+
+`crlf=headers` forces *header* newlines to become CRLF even if not written so
+in the source file. Note that this makes runtests.pl parse and "guess" what is
+a header and what is not in order to apply the CRLF line endings appropriately.
 
 For FTP file listings, the `<data>` section is be used *only* if you make sure
 that there has been a CWD done first to a directory named `test-[NUM]` where
 `NUM` is the test case number. Otherwise the ftp server cannot know from which
 test file to load the list content.
 
-### `<dataNUM [crlf="yes"]>`
+### `<dataNUM [crlf="yes|headers"]>`
 
 Send back this contents instead of the `<data>` one. The `NUM` is set by:
 
@@ -297,15 +300,22 @@ to complete a transfer. The response to each request is found in its own data
 section. Validating the entire negotiation sequence can be done by specifying
 a `datacheck` section.
 
-### `<connect>`
+### `<connect [crlf="yes|headers"]>`
 The connect section is used instead of the 'data' for all CONNECT
 requests. The remainder of the rules for the data section then apply but with
 a connect prefix.
 
+`crlf=yes` forces the newlines to become CRLF even if not written so in the
+test.
+
+`crlf=headers` forces *header* newlines to become CRLF even if not written so
+in the source file. Note that this makes runtests.pl parse and "guess" what is
+a header and what is not in order to apply the CRLF line endings appropriately.
+
 ### `<socks>`
 Address type and address details as logged by the SOCKS proxy.
 
-### `<datacheck [mode="text"] [nonewline="yes"] [crlf="yes"]>`
+### `<datacheck [mode="text"] [nonewline="yes"] [crlf="yes|headers"]>`
 if the data is sent but this is what should be checked afterwards. If
 `nonewline=yes` is set, runtests cuts off the trailing newline from the data
 before comparing with the one actually received by the client.
@@ -313,7 +323,7 @@ before comparing with the one actually received by the client.
 Use the `mode="text"` attribute if the output is in text mode on platforms
 that have a text/binary difference.
 
-### `<datacheckNUM [nonewline="yes"] [mode="text"] [crlf="yes"]>`
+### `<datacheckNUM [nonewline="yes"] [mode="text"] [crlf="yes|headers"]>`
 The contents of numbered `datacheck` sections are appended to the non-numbered
 one.
 
@@ -613,13 +623,16 @@ parameter is the not negative integer number of seconds for the delay. This
 'delay' attribute is intended for specific test cases, and normally not
 needed.
 
-### `<file name="%LOGDIR/filename" [nonewline="yes"]>`
+### `<file name="%LOGDIR/filename" [nonewline="yes"][crlf="yes"]>`
 This creates the named file with this content before the test case is run,
 which is useful if the test case needs a file to act on.
 
 If `nonewline="yes"` is used, the created file gets the final newline stripped
 off.
 
+`crlf=yes` forces the newlines to become CRLF even if not written so in the
+test.
+
 ### `<file1>`
 1 to 4 can be appended to 'file' to create more files.
 
@@ -629,12 +642,15 @@ off.
 
 ### `<file4>`
 
-### `<stdin [nonewline="yes"]>`
+### `<stdin [nonewline="yes"][crlf="yes"]>`
 Pass this given data on stdin to the tool.
 
 If `nonewline` is set, we cut off the trailing newline of this given data
 before comparing with the one actually received by the client
 
+`crlf=yes` forces the newlines to become CRLF even if not written so in the
+test.
+
 ## `<disable>`
 
 If `test-duphandle` is a listed item here, this is not run when
@@ -677,14 +693,14 @@ test.
 in the source file. Note that this makes runtests.pl parse and "guess" what is
 a header and what is not in order to apply the CRLF line endings appropriately.
 
-### `<proxy [nonewline="yes"][crlf="yes"]>`
+### `<proxy [nonewline="yes"][crlf="yes|headers"]>`
 
 The protocol dump curl should transmit to an HTTP proxy (when the http-proxy
 server is used), if `nonewline` is set, we cut off the trailing newline of
 this given data before comparing with the one actually sent by the client The
 `<strip>` and `<strippart>` rules are applied before comparisons are made.
 
-### `<stderr [mode="text"] [nonewline="yes"] [crlf="yes"]>`
+### `<stderr [mode="text"] [nonewline="yes"] [crlf="yes|headers"]>`
 This verifies that this data was passed to stderr.
 
 Use the mode="text" attribute if the output is in text mode on platforms that
@@ -693,10 +709,14 @@ have a text/binary difference.
 `crlf=yes` forces the newlines to become CRLF even if not written so in the
 test.
 
+`crlf=headers` forces *header* newlines to become CRLF even if not written so
+in the source file. Note that this makes runtests.pl parse and "guess" what is
+a header and what is not in order to apply the CRLF line endings appropriately.
+
 If `nonewline` is set, we cut off the trailing newline of this given data
 before comparing with the one actually received by the client
 
-### `<stdout [mode="text"] [nonewline="yes"] [crlf="yes"] [loadfile="filename"]>`
+### `<stdout [mode="text"] [nonewline="yes"] [crlf="yes|headers"] [loadfile="filename"]>`
 This verifies that this data was passed to stdout.
 
 Use the mode="text" attribute if the output is in text mode on platforms that
@@ -708,6 +728,10 @@ before comparing with the one actually received by the client
 `crlf=yes` forces the newlines to become CRLF even if not written so in the
 test.
 
+`crlf=headers` forces *header* newlines to become CRLF even if not written so
+in the source file. Note that this makes runtests.pl parse and "guess" what is
+a header and what is not in order to apply the CRLF line endings appropriately.
+
 `loadfile="filename"` makes loading the data from an external file.
 
 ### `<limit>`
@@ -718,11 +742,18 @@ that the set limits are not exceeded. Supported limits:
     Allocations: [number of allocation calls]
     Maximum allocated: [maximum concurrent memory allocated]
 
-### `<file name="%LOGDIR/filename" [mode="text"]>`
+### `<file name="%LOGDIR/filename" [mode="text"] [crlf="yes|headers"]>`
 The file's contents must be identical to this after the test is complete. Use
 the mode="text" attribute if the output is in text mode on platforms that have
 a text/binary difference.
 
+`crlf=yes` forces the newlines to become CRLF even if not written so in the
+test.
+
+`crlf=headers` forces *header* newlines to become CRLF even if not written so
+in the source file. Note that this makes runtests.pl parse and "guess" what is
+a header and what is not in order to apply the CRLF line endings appropriately.
+
 ### `<file1>`
 1 to 4 can be appended to 'file' to compare more files.