HTTP: implement Brotli content encoding

This uses the brotli external library (https://github.com/google/brotli).
Brotli becomes a feature: additional curl_version_info() bit and
structure fields are provided for it and CURLVERSION_NOW bumped.

Tests 314 and 315 check Brotli content unencoding with correct and
erroneous data.

Some tests are updated to accomodate with the now configuration dependent
parameters of the Accept-Encoding header.

docs/INTERNALS.md

@@ -644,9 +644,9 @@ Content Encoding
  [HTTP/1.1][4] specifies that a client may request that a server encode its
  response. This is usually used to compress a response using one (or more)
  encodings from a set of commonly available compression techniques. These
- schemes include 'deflate' (the zlib algorithm), 'gzip' and 'compress'. A
- client requests that the server perform an encoding by including an
- Accept-Encoding header in the request document. The value of the header
+ schemes include 'deflate' (the zlib algorithm), 'gzip' 'br' (brotli) and
+ 'compress'. A client requests that the server perform an encoding by including
+ an Accept-Encoding header in the request document. The value of the header
  should be one of the recognized tokens 'deflate', ... (there's a way to
  register new schemes/tokens, see sec 3.5 of the spec). A server MAY honor
  the client's encoding request. When a response is encoded, the server
@@ -661,9 +661,10 @@ Content Encoding
 
 ## Supported content encodings
 
- The 'deflate' and 'gzip' content encodings are supported by libcurl. Both
- regular and chunked transfers work fine.  The zlib library is required for
- this feature.
+ The 'deflate', 'gzip' and 'br' content encodings are supported by libcurl.
+ Both regular and chunked transfers work fine.  The zlib library is required
+ for the 'deflate' and 'gzip' encodings, while the brotli decoding library is
+ for the 'br' encoding.
 
 ## The libcurl interface
 
@@ -674,10 +675,10 @@ Content Encoding
  where string is the intended value of the Accept-Encoding header.
 
  Currently, libcurl does support multiple encodings but only
- understands how to process responses that use the "deflate" or "gzip"
- Content-Encoding, so the only values for [`CURLOPT_ACCEPT_ENCODING`][5]
- that will work (besides "identity," which does nothing) are "deflate"
- and "gzip". If a response is encoded using the "compress" or methods,
+ understands how to process responses that use the "deflate", "gzip" and/or
+ "br" content encodings, so the only values for [`CURLOPT_ACCEPT_ENCODING`][5]
+ that will work (besides "identity," which does nothing) are "deflate",
+ "gzip" and "br". If a response is encoded using the "compress" or methods,
  libcurl will return an error indicating that the response could
  not be decoded.  If <string> is NULL no Accept-Encoding header is generated.
  If <string> is a zero-length string, then an Accept-Encoding header

docs/RESOURCES

@@ -81,3 +81,5 @@ This document lists documents and standards used by curl.
   RFC 4616 - PLAIN authentication
 
   RFC 4954 - SMTP Authentication
+
+  RFC 7932 - Brotli Compressed Data Format

docs/TODO

@@ -64,9 +64,8 @@
  5.4 HTTP Digest using SHA-256
  5.5 auth= in URLs
  5.6 Refuse "downgrade" redirects
- 5.7 Brotli compression
- 5.8 QUIC
- 5.9 Leave secure cookies alone
+ 5.7 QUIC
+ 5.8 Leave secure cookies alone
 
  6. TELNET
  6.1 ditch stdin
@@ -514,13 +513,7 @@ This is not detailed in any FTP specification.
  Consider a way to tell curl to refuse to "downgrade" protocol with a redirect
  and/or possibly a bit that refuses redirect to change protocol completely.
 
-5.7 Brotli compression
-
- Brotli compression performs better than gzip and is being implemented by
- browsers and servers widely. The algorithm: https://github.com/google/brotli
- The Firefox bug: https://bugzilla.mozilla.org/show_bug.cgi?id=366559
-
-5.8 QUIC
+5.7 QUIC
 
  The standardization process of QUIC has been taken to the IETF and can be
  followed on the [IETF QUIC Mailing
@@ -530,7 +523,7 @@ This is not detailed in any FTP specification.
  implemented. This, to allow other projects to benefit from the work and to
  thus broaden the interest and chance of others to participate.
 
-5.9 Leave secure cookies alone
+5.8 Leave secure cookies alone
 
  Non-secure origins (HTTP sites) should not be allowed to set or modify
  cookies with the 'secure' property:

docs/libcurl/curl_version_info.3

@@ -72,6 +72,12 @@ typedef struct {
 
   const char *libssh_version; /* human readable string */
 
+  /* when 'age' is 4 or higher (7.57.0 or later), the members below also
+     exist  */
+  unsigned int brotli_ver_num; /* Numeric Brotli version
+                                  (MAJOR << 24) | (MINOR << 12) | PATCH */
+  const char *brotli_version; /* human readable string. */
+
 } curl_version_info_data;
 .fi
 
@@ -160,6 +166,8 @@ libcurl was built with support for HTTPS-proxy.
 libcurl was built with multiple SSL backends. For details, see
 \fIcurl_global_sslset(3)\fP.
 (Added in 7.56.0)
+.IP CURL_VERSION_BROTLI
+supports HTTP Brotli content encoding using libbrotlidec (Added in 7.57.0)
 .RE
 \fIssl_version\fP is an ASCII string for the OpenSSL version used. If libcurl
 has no SSL support, this is NULL.

docs/libcurl/symbols-in-versions

@@ -699,6 +699,7 @@ CURLUSESSL_ALL                  7.17.0
 CURLUSESSL_CONTROL              7.17.0
 CURLUSESSL_NONE                 7.17.0
 CURLUSESSL_TRY                  7.17.0
+CURLVERSION_FIFTH               7.57.0
 CURLVERSION_FIRST               7.10
 CURLVERSION_FOURTH              7.16.1
 CURLVERSION_NOW                 7.10
@@ -829,6 +830,7 @@ CURL_TIMECOND_NONE              7.9.7
 CURL_TLSAUTH_NONE               7.21.4
 CURL_TLSAUTH_SRP                7.21.4
 CURL_VERSION_ASYNCHDNS          7.10.7
+CURL_VERSION_BROTLI             7.57.0
 CURL_VERSION_CONV               7.15.4
 CURL_VERSION_CURLDEBUG          7.19.6
 CURL_VERSION_DEBUG              7.10.6