curl/lib/vtls/vtls.h
Stefan Eissing 671158242d
connections: introduce http/3 happy eyeballs
New cfilter HTTP-CONNECT for h3/h2/http1.1 eyeballing.
- filter is installed when `--http3` in the tool is used (or
  the equivalent CURLOPT_ done in the library)
- starts a QUIC/HTTP/3 connect right away. Should that not
  succeed after 100ms (subject to change), a parallel attempt
  is started for HTTP/2 and HTTP/1.1 via TCP
- both attempts are subject to IPv6/IPv4 eyeballing, same
  as happens for other connections
- tie timeout to the ip-version HAPPY_EYEBALLS_TIMEOUT
- use a `soft` timeout at half the value. When the soft timeout
  expires, the HTTPS-CONNECT filter checks if the QUIC filter
  has received any data from the server. If not, it will start
  the HTTP/2 attempt.

HTTP/3(ngtcp2) improvements.
- setting call_data in all cfilter calls similar to http/2 and vtls filters
  for use in callback where no stream data is available.
- returning CURLE_PARTIAL_FILE for prematurely terminated transfers
- enabling pytest test_05 for h3
- shifting functionality to "connect" UDP sockets from ngtcp2
  implementation into the udp socket cfilter. Because unconnected
  UDP sockets are weird. For example they error when adding to a
  pollset.

HTTP/3(quiche) improvements.
- fixed upload bug in quiche implementation, now passes 251 and pytest
- error codes on stream RESET
- improved debug logs
- handling of DRAIN during connect
- limiting pending event queue

HTTP/2 cfilter improvements.
- use LOG_CF macros for dynamic logging in debug build
- fix CURLcode on RST streams to be CURLE_PARTIAL_FILE
- enable pytest test_05 for h2
- fix upload pytests and improve parallel transfer performance.

GOAWAY handling for ngtcp2/quiche
- during connect, when the remote server refuses to accept new connections
  and closes immediately (so the local conn goes into DRAIN phase), the
  connection is torn down and a another attempt is made after a short grace
  period.
  This is the behaviour observed with nghttpx when we tell it to  shut
  down gracefully. Tested in pytest test_03_02.

TLS improvements
- ALPN selection for SSL/SSL-PROXY filters in one vtls set of functions, replaces
  copy of logic in all tls backends.
- standardized the infof logging of offered ALPNs
- ALPN negotiated: have common function for all backends that sets alpn proprty
  and connection related things based on the negotiated protocol (or lack thereof).

- new tests/tests-httpd/scorecard.py for testing h3/h2 protocol implementation.
  Invoke:
    python3 tests/tests-httpd/scorecard.py --help
  for usage.

Improvements on gathering connect statistics and socket access.
- new CF_CTRL_CONN_REPORT_STATS cfilter control for having cfilters
  report connection statistics. This is triggered when the connection
  has completely connected.
- new void Curl_pgrsTimeWas(..) method to report a timer update with
  a timestamp of when it happend. This allows for updating timers
  "later", e.g. a connect statistic after full connectivity has been
  reached.
- in case of HTTP eyeballing, the previous changes will update
  statistics only from the filter chain that "won" the eyeballing.
- new cfilter query CF_QUERY_SOCKET for retrieving the socket used
  by a filter chain.
  Added methods Curl_conn_cf_get_socket() and Curl_conn_get_socket()
  for convenient use of this query.
- Change VTLS backend to query their sub-filters for the socket when
  checks during the handshake are made.

HTTP/3 documentation on how https eyeballing works.

TLS improvements
- ALPN selection for SSL/SSL-PROXY filters in one vtls set of functions, replaces
  copy of logic in all tls backends.
- standardized the infof logging of offered ALPNs
- ALPN negotiated: have common function for all backends that sets alpn proprty
  and connection related things based on the negotiated protocol (or lack thereof).

Scorecard with Caddy.
- configure can be run with `--with-test-caddy=path` to specify which caddy to use for testing
- tests/tests-httpd/scorecard.py now measures download speeds with caddy

pytest improvements
- adding Makfile to clean gen dir
- adding nghttpx rundir creation on start
- checking httpd version 2.4.55 for test_05 cases where it is needed. Skipping with message if too old.
- catch exception when checking for caddy existance on system.

Closes #10349
2023-02-02 09:57:34 +01:00

286 lines
11 KiB
C

#ifndef HEADER_CURL_VTLS_H
#define HEADER_CURL_VTLS_H
/***************************************************************************
* _ _ ____ _
* Project ___| | | | _ \| |
* / __| | | | |_) | |
* | (__| |_| | _ <| |___
* \___|\___/|_| \_\_____|
*
* Copyright (C) Daniel Stenberg, <daniel@haxx.se>, et al.
*
* This software is licensed as described in the file COPYING, which
* you should have received as part of this distribution. The terms
* are also available at https://curl.se/docs/copyright.html.
*
* You may opt to use, copy, modify, merge, publish, distribute and/or sell
* copies of the Software, and permit persons to whom the Software is
* furnished to do so, under the terms of the COPYING file.
*
* This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY
* KIND, either express or implied.
*
* SPDX-License-Identifier: curl
*
***************************************************************************/
#include "curl_setup.h"
struct connectdata;
struct ssl_config_data;
struct ssl_primary_config;
struct Curl_ssl_session;
#define SSLSUPP_CA_PATH (1<<0) /* supports CAPATH */
#define SSLSUPP_CERTINFO (1<<1) /* supports CURLOPT_CERTINFO */
#define SSLSUPP_PINNEDPUBKEY (1<<2) /* supports CURLOPT_PINNEDPUBLICKEY */
#define SSLSUPP_SSL_CTX (1<<3) /* supports CURLOPT_SSL_CTX */
#define SSLSUPP_HTTPS_PROXY (1<<4) /* supports access via HTTPS proxies */
#define SSLSUPP_TLS13_CIPHERSUITES (1<<5) /* supports TLS 1.3 ciphersuites */
#define SSLSUPP_CAINFO_BLOB (1<<6)
#define ALPN_ACCEPTED "ALPN: server accepted "
#define VTLS_INFOF_NO_ALPN \
"ALPN: server did not agree on a protocol. Uses default."
#define VTLS_INFOF_ALPN_OFFER_1STR \
"ALPN: offers %s"
#define VTLS_INFOF_ALPN_ACCEPTED_1STR \
ALPN_ACCEPTED "%s"
#define VTLS_INFOF_ALPN_ACCEPTED_LEN_1STR \
ALPN_ACCEPTED "%.*s"
/* Curl_multi SSL backend-specific data; declared differently by each SSL
backend */
struct multi_ssl_backend_data;
struct Curl_cfilter;
CURLsslset Curl_init_sslset_nolock(curl_sslbackend id, const char *name,
const curl_ssl_backend ***avail);
#ifndef MAX_PINNED_PUBKEY_SIZE
#define MAX_PINNED_PUBKEY_SIZE 1048576 /* 1MB */
#endif
#ifndef CURL_SHA256_DIGEST_LENGTH
#define CURL_SHA256_DIGEST_LENGTH 32 /* fixed size */
#endif
/* see https://www.iana.org/assignments/tls-extensiontype-values/ */
#define ALPN_HTTP_1_1_LENGTH 8
#define ALPN_HTTP_1_1 "http/1.1"
#define ALPN_HTTP_1_0_LENGTH 8
#define ALPN_HTTP_1_0 "http/1.0"
#define ALPN_H2_LENGTH 2
#define ALPN_H2 "h2"
#define ALPN_H3_LENGTH 2
#define ALPN_H3 "h3"
/* conservative sizes on the ALPN entries and count we are handling,
* we can increase these if we ever feel the need or have to accomodate
* ALPN strings from the "outside". */
#define ALPN_NAME_MAX 10
#define ALPN_ENTRIES_MAX 3
#define ALPN_PROTO_BUF_MAX (ALPN_ENTRIES_MAX * (ALPN_NAME_MAX + 1))
struct alpn_spec {
const char entries[ALPN_ENTRIES_MAX][ALPN_NAME_MAX];
size_t count; /* number of entries */
};
struct alpn_proto_buf {
unsigned char data[ALPN_PROTO_BUF_MAX];
int len;
};
CURLcode Curl_alpn_to_proto_buf(struct alpn_proto_buf *buf,
const struct alpn_spec *spec);
CURLcode Curl_alpn_to_proto_str(struct alpn_proto_buf *buf,
const struct alpn_spec *spec);
CURLcode Curl_alpn_set_negotiated(struct Curl_cfilter *cf,
struct Curl_easy *data,
const unsigned char *proto,
size_t proto_len);
/**
* Get the ALPN specification to use for talking to remote host.
* May return NULL if ALPN is disabled on the connection.
*/
const struct alpn_spec *
Curl_alpn_get_spec(struct Curl_easy *data, struct connectdata *conn);
/**
* Get the ALPN specification to use for talking to the proxy.
* May return NULL if ALPN is disabled on the connection.
*/
const struct alpn_spec *
Curl_alpn_get_proxy_spec(struct Curl_easy *data, struct connectdata *conn);
char *Curl_ssl_snihost(struct Curl_easy *data, const char *host, size_t *olen);
bool Curl_ssl_config_matches(struct ssl_primary_config *data,
struct ssl_primary_config *needle);
bool Curl_clone_primary_ssl_config(struct ssl_primary_config *source,
struct ssl_primary_config *dest);
void Curl_free_primary_ssl_config(struct ssl_primary_config *sslc);
curl_sslbackend Curl_ssl_backend(void);
#ifdef USE_SSL
int Curl_ssl_init(void);
void Curl_ssl_cleanup(void);
/* tell the SSL stuff to close down all open information regarding
connections (and thus session ID caching etc) */
void Curl_ssl_close_all(struct Curl_easy *data);
CURLcode Curl_ssl_set_engine(struct Curl_easy *data, const char *engine);
/* Sets engine as default for all SSL operations */
CURLcode Curl_ssl_set_engine_default(struct Curl_easy *data);
struct curl_slist *Curl_ssl_engines_list(struct Curl_easy *data);
/* init the SSL session ID cache */
CURLcode Curl_ssl_initsessions(struct Curl_easy *, size_t);
void Curl_ssl_version(char *buffer, size_t size);
/* Certificate information list handling. */
void Curl_ssl_free_certinfo(struct Curl_easy *data);
CURLcode Curl_ssl_init_certinfo(struct Curl_easy *data, int num);
CURLcode Curl_ssl_push_certinfo_len(struct Curl_easy *data, int certnum,
const char *label, const char *value,
size_t valuelen);
CURLcode Curl_ssl_push_certinfo(struct Curl_easy *data, int certnum,
const char *label, const char *value);
/* Functions to be used by SSL library adaptation functions */
/* Lock session cache mutex.
* Call this before calling other Curl_ssl_*session* functions
* Caller should unlock this mutex as soon as possible, as it may block
* other SSL connection from making progress.
* The purpose of explicitly locking SSL session cache data is to allow
* individual SSL engines to manage session lifetime in their specific way.
*/
void Curl_ssl_sessionid_lock(struct Curl_easy *data);
/* Unlock session cache mutex */
void Curl_ssl_sessionid_unlock(struct Curl_easy *data);
/* Kill a single session ID entry in the cache
* Sessionid mutex must be locked (see Curl_ssl_sessionid_lock).
* This will call engine-specific curlssl_session_free function, which must
* take sessionid object ownership from sessionid cache
* (e.g. decrement refcount).
*/
void Curl_ssl_kill_session(struct Curl_ssl_session *session);
/* delete a session from the cache
* Sessionid mutex must be locked (see Curl_ssl_sessionid_lock).
* This will call engine-specific curlssl_session_free function, which must
* take sessionid object ownership from sessionid cache
* (e.g. decrement refcount).
*/
void Curl_ssl_delsessionid(struct Curl_easy *data, void *ssl_sessionid);
/* get N random bytes into the buffer */
CURLcode Curl_ssl_random(struct Curl_easy *data, unsigned char *buffer,
size_t length);
/* Check pinned public key. */
CURLcode Curl_pin_peer_pubkey(struct Curl_easy *data,
const char *pinnedpubkey,
const unsigned char *pubkey, size_t pubkeylen);
bool Curl_ssl_cert_status_request(void);
bool Curl_ssl_false_start(struct Curl_easy *data);
void Curl_free_multi_ssl_backend_data(struct multi_ssl_backend_data *mbackend);
#define SSL_SHUTDOWN_TIMEOUT 10000 /* ms */
CURLcode Curl_ssl_cfilter_add(struct Curl_easy *data,
struct connectdata *conn,
int sockindex);
CURLcode Curl_cf_ssl_insert_after(struct Curl_cfilter *cf_at,
struct Curl_easy *data);
CURLcode Curl_ssl_cfilter_remove(struct Curl_easy *data,
int sockindex);
#ifndef CURL_DISABLE_PROXY
CURLcode Curl_ssl_cfilter_proxy_add(struct Curl_easy *data,
struct connectdata *conn,
int sockindex);
CURLcode Curl_cf_ssl_proxy_insert_after(struct Curl_cfilter *cf_at,
struct Curl_easy *data);
#endif /* !CURL_DISABLE_PROXY */
/**
* Get the SSL configuration that is used on the connection.
* This returns NULL if no SSL is configured.
* Otherwise it returns the config of the first (highest) one that is
* either connected, in handshake or about to start
* (e.g. all filters below it are connected). If SSL filters are present,
* but neither can start operating, return the config of the lowest one
* that will first come into effect when connecting.
*/
struct ssl_config_data *Curl_ssl_get_config(struct Curl_easy *data,
int sockindex);
/**
* Get the primary SSL configuration from the connection.
* This returns NULL if no SSL is configured.
* Otherwise it returns the config of the first (highest) one that is
* either connected, in handshake or about to start
* (e.g. all filters below it are connected). If SSL filters are present,
* but neither can start operating, return the config of the lowest one
* that will first come into effect when connecting.
*/
struct ssl_primary_config *
Curl_ssl_get_primary_config(struct Curl_easy *data,
struct connectdata *conn,
int sockindex);
/**
* True iff the underlying SSL implementation supports the option.
* Option is one of the defined SSLSUPP_* values.
* `data` maybe NULL for the features of the default implementation.
*/
bool Curl_ssl_supports(struct Curl_easy *data, int ssl_option);
/**
* Get the internal ssl instance (like OpenSSL's SSL*) from the filter
* chain at `sockindex` of type specified by `info`.
* For `n` == 0, the first active (top down) instance is returned.
* 1 gives the second active, etc.
* NULL is returned when no active SSL filter is present.
*/
void *Curl_ssl_get_internals(struct Curl_easy *data, int sockindex,
CURLINFO info, int n);
extern struct Curl_cftype Curl_cft_ssl;
extern struct Curl_cftype Curl_cft_ssl_proxy;
#else /* if not USE_SSL */
/* When SSL support is not present, just define away these function calls */
#define Curl_ssl_init() 1
#define Curl_ssl_cleanup() Curl_nop_stmt
#define Curl_ssl_close_all(x) Curl_nop_stmt
#define Curl_ssl_set_engine(x,y) CURLE_NOT_BUILT_IN
#define Curl_ssl_set_engine_default(x) CURLE_NOT_BUILT_IN
#define Curl_ssl_engines_list(x) NULL
#define Curl_ssl_initsessions(x,y) CURLE_OK
#define Curl_ssl_free_certinfo(x) Curl_nop_stmt
#define Curl_ssl_kill_session(x) Curl_nop_stmt
#define Curl_ssl_random(x,y,z) ((void)x, CURLE_NOT_BUILT_IN)
#define Curl_ssl_cert_status_request() FALSE
#define Curl_ssl_false_start(a) FALSE
#define Curl_ssl_get_internals(a,b,c,d) NULL
#define Curl_ssl_supports(a,b) FALSE
#define Curl_ssl_cfilter_add(a,b,c) CURLE_NOT_BUILT_IN
#define Curl_ssl_cfilter_proxy_add(a,b,c) CURLE_NOT_BUILT_IN
#define Curl_ssl_get_config(a,b) NULL
#define Curl_ssl_cfilter_remove(a,b) CURLE_OK
#endif
#endif /* HEADER_CURL_VTLS_H */