connections: introduce http/3 happy eyeballs

New cfilter HTTP-CONNECT for h3/h2/http1.1 eyeballing.
- filter is installed when `--http3` in the tool is used (or
  the equivalent CURLOPT_ done in the library)
- starts a QUIC/HTTP/3 connect right away. Should that not
  succeed after 100ms (subject to change), a parallel attempt
  is started for HTTP/2 and HTTP/1.1 via TCP
- both attempts are subject to IPv6/IPv4 eyeballing, same
  as happens for other connections
- tie timeout to the ip-version HAPPY_EYEBALLS_TIMEOUT
- use a `soft` timeout at half the value. When the soft timeout
  expires, the HTTPS-CONNECT filter checks if the QUIC filter
  has received any data from the server. If not, it will start
  the HTTP/2 attempt.

HTTP/3(ngtcp2) improvements.
- setting call_data in all cfilter calls similar to http/2 and vtls filters
  for use in callback where no stream data is available.
- returning CURLE_PARTIAL_FILE for prematurely terminated transfers
- enabling pytest test_05 for h3
- shifting functionality to "connect" UDP sockets from ngtcp2
  implementation into the udp socket cfilter. Because unconnected
  UDP sockets are weird. For example they error when adding to a
  pollset.

HTTP/3(quiche) improvements.
- fixed upload bug in quiche implementation, now passes 251 and pytest
- error codes on stream RESET
- improved debug logs
- handling of DRAIN during connect
- limiting pending event queue

HTTP/2 cfilter improvements.
- use LOG_CF macros for dynamic logging in debug build
- fix CURLcode on RST streams to be CURLE_PARTIAL_FILE
- enable pytest test_05 for h2
- fix upload pytests and improve parallel transfer performance.

GOAWAY handling for ngtcp2/quiche
- during connect, when the remote server refuses to accept new connections
  and closes immediately (so the local conn goes into DRAIN phase), the
  connection is torn down and a another attempt is made after a short grace
  period.
  This is the behaviour observed with nghttpx when we tell it to  shut
  down gracefully. Tested in pytest test_03_02.

TLS improvements
- ALPN selection for SSL/SSL-PROXY filters in one vtls set of functions, replaces
  copy of logic in all tls backends.
- standardized the infof logging of offered ALPNs
- ALPN negotiated: have common function for all backends that sets alpn proprty
  and connection related things based on the negotiated protocol (or lack thereof).

- new tests/tests-httpd/scorecard.py for testing h3/h2 protocol implementation.
  Invoke:
    python3 tests/tests-httpd/scorecard.py --help
  for usage.

Improvements on gathering connect statistics and socket access.
- new CF_CTRL_CONN_REPORT_STATS cfilter control for having cfilters
  report connection statistics. This is triggered when the connection
  has completely connected.
- new void Curl_pgrsTimeWas(..) method to report a timer update with
  a timestamp of when it happend. This allows for updating timers
  "later", e.g. a connect statistic after full connectivity has been
  reached.
- in case of HTTP eyeballing, the previous changes will update
  statistics only from the filter chain that "won" the eyeballing.
- new cfilter query CF_QUERY_SOCKET for retrieving the socket used
  by a filter chain.
  Added methods Curl_conn_cf_get_socket() and Curl_conn_get_socket()
  for convenient use of this query.
- Change VTLS backend to query their sub-filters for the socket when
  checks during the handshake are made.

HTTP/3 documentation on how https eyeballing works.

TLS improvements
- ALPN selection for SSL/SSL-PROXY filters in one vtls set of functions, replaces
  copy of logic in all tls backends.
- standardized the infof logging of offered ALPNs
- ALPN negotiated: have common function for all backends that sets alpn proprty
  and connection related things based on the negotiated protocol (or lack thereof).

Scorecard with Caddy.
- configure can be run with `--with-test-caddy=path` to specify which caddy to use for testing
- tests/tests-httpd/scorecard.py now measures download speeds with caddy

pytest improvements
- adding Makfile to clean gen dir
- adding nghttpx rundir creation on start
- checking httpd version 2.4.55 for test_05 cases where it is needed. Skipping with message if too old.
- catch exception when checking for caddy existance on system.

Closes #10349
This commit is contained in:
Stefan Eissing 2023-02-01 17:13:12 +01:00 committed by Daniel Stenberg
parent b7aaf074e5
commit 671158242d
No known key found for this signature in database
GPG key ID: 5CC908FDB71E12C2
61 changed files with 3599 additions and 1249 deletions

View file

@ -267,10 +267,11 @@ static CURLcode CONNECT_host(struct Curl_easy *data,
}
#ifndef USE_HYPER
static CURLcode start_CONNECT(struct Curl_easy *data,
struct connectdata *conn,
static CURLcode start_CONNECT(struct Curl_cfilter *cf,
struct Curl_easy *data,
struct tunnel_state *ts)
{
struct connectdata *conn = cf->conn;
char *hostheader = NULL;
char *host = NULL;
const char *httpv;
@ -476,7 +477,7 @@ static CURLcode recv_CONNECT_resp(struct Curl_cfilter *cf,
{
CURLcode result = CURLE_OK;
struct SingleRequest *k = &data->req;
curl_socket_t tunnelsocket = cf->conn->sock[ts->sockindex];
curl_socket_t tunnelsocket = Curl_conn_cf_get_socket(cf, data);
char *linep;
size_t perline;
int error;
@ -665,12 +666,13 @@ static CURLcode recv_CONNECT_resp(struct Curl_cfilter *cf,
#else /* USE_HYPER */
/* The Hyper version of CONNECT */
static CURLcode start_CONNECT(struct Curl_easy *data,
struct connectdata *conn,
static CURLcode start_CONNECT(struct Curl_cfilter *cf,
struct Curl_easy *data,
struct tunnel_state *ts)
{
struct connectdata *conn = cf->conn;
struct hyptransfer *h = &data->hyp;
curl_socket_t tunnelsocket = conn->sock[ts->sockindex];
curl_socket_t tunnelsocket = Curl_conn_cf_get_socket(cf, data);
hyper_io *io = NULL;
hyper_request *req = NULL;
hyper_headers *headers = NULL;
@ -971,7 +973,7 @@ static CURLcode CONNECT(struct Curl_cfilter *cf,
case TUNNEL_INIT:
/* Prepare the CONNECT request and make a first attempt to send. */
DEBUGF(LOG_CF(data, cf, "CONNECT start"));
result = start_CONNECT(data, cf->conn, ts);
result = start_CONNECT(cf, data, ts);
if(result)
goto out;
tunnel_go_state(cf, ts, TUNNEL_CONNECT, data);
@ -1125,15 +1127,13 @@ static int http_proxy_cf_get_select_socks(struct Curl_cfilter *cf,
curl_socket_t *socks)
{
struct tunnel_state *ts = cf->ctx;
struct connectdata *conn = cf->conn;
int fds;
DEBUGASSERT(conn);
fds = cf->next->cft->get_select_socks(cf->next, data, socks);
if(!fds && cf->next->connected && !cf->connected) {
/* If we are not connected, but the filter "below" is
* and not waiting on something, we are tunneling. */
socks[0] = conn->sock[cf->sockindex];
socks[0] = Curl_conn_cf_get_socket(cf, data);
if(ts) {
/* when we've sent a CONNECT to a proxy, we should rather either
wait for the socket to become readable to be able to get the
@ -1347,15 +1347,13 @@ static int cf_haproxy_get_select_socks(struct Curl_cfilter *cf,
struct Curl_easy *data,
curl_socket_t *socks)
{
struct connectdata *conn = cf->conn;
int fds;
DEBUGASSERT(conn);
fds = cf->next->cft->get_select_socks(cf->next, data, socks);
if(!fds && cf->next->connected && !cf->connected) {
/* If we are not connected, but the filter "below" is
* and not waiting on something, we are sending. */
socks[0] = conn->sock[cf->sockindex];
socks[0] = Curl_conn_cf_get_socket(cf, data);
return GETSOCK_WRITESOCK(0);
}
return fds;