mime: do not perform more than one read in a row

Input buffer filling may delay the data sending if data reads are slow.
To overcome this problem, file and callback data reads do not accumulate
in buffer anymore. All other data (memory data and mime framing) are
considered as fast and still concatenated in buffer.
As this may highly impact performance in terms of data overhead, an early
end of part data check is added to spare a read call.
When encoding a part's data, an encoder may require more bytes than made
available by a single read. In this case, the above rule does not apply
and reads are performed until the encoder is able to deliver some data.

Tests 643, 644, 645, 650 and 654 have been adapted to the output data
changes, with test data size reduced to avoid the boredom of long lists of
1-byte chunks in verification data.
New test 667 checks mimepost using single-byte read callback with encoder.
New test 668 checks the end of part data early detection.

Fixes #4826
Reported-by: MrdUkk on github
This commit is contained in:
Patrick Monnerat 2020-03-07 11:51:00 +01:00 committed by Daniel Stenberg
parent 96972ec1c0
commit 1e4cb333ef
No known key found for this signature in database
GPG key ID: 5CC908FDB71E12C2
15 changed files with 679 additions and 77 deletions

99
tests/data/test668 Normal file
View file

@ -0,0 +1,99 @@
<testcase>
<info>
<keywords>
HTTP
HTTP POST
HTTP MIME POST
</keywords>
</info>
#
# Server-side
<reply>
<data>
HTTP/1.1 200 OK
Date: Thu, 09 Nov 2010 14:49:00 GMT
Server: test-server/fake swsclose
Connection: close
Content-Type: text/html
hello
</data>
<datacheck>
HTTP/1.1 200 OK
Date: Thu, 09 Nov 2010 14:49:00 GMT
Server: test-server/fake swsclose
Connection: close
Content-Type: text/html
hello
</datacheck>
</reply>
# Client-side
<client>
<server>
http
</server>
# tool is what to use instead of 'curl'
<tool>
lib668
</tool>
<name>
HTTP mimepost early end of data detection
</name>
<command>
http://%HOSTIP:%HTTPPORT/668
</command>
<file name="log/file668.txt">
This is data from a file
</file>
</client>
#
# Verify data after the test has been "shot"
<verify>
<strippart>
s/^--------------------------[a-z0-9]*/------------------------------/
s/boundary=------------------------[a-z0-9]*/boundary=----------------------------/
</strippart>
# Note that the stripping above removes 12 bytes from every occurrence of the
# boundary string and since 5 of them are in the body contents, we see
# (5*12) == 60 bytes less
<protocol>
POST /668 HTTP/1.1
Host: %HOSTIP:%HTTPPORT
Accept: */*
Transfer-Encoding: chunked
Content-Type: multipart/form-data; boundary=----------------------------
Expect: 100-continue
c1
------------------------------
Content-Disposition: form-data; name="field1"
dummy
------------------------------
Content-Disposition: form-data; name="field2"
5
dummy
91
------------------------------
Content-Disposition: form-data; name="field3"; filename="file668.txt"
Content-Type: text/plain
49
This is data from a file
--------------------------------
0
</protocol>
</verify>
</testcase>