mirror of
https://github.com/curl/curl.git
synced 2026-08-24 13:03:35 +03:00
connections: introduce http/3 happy eyeballs
New cfilter HTTP-CONNECT for h3/h2/http1.1 eyeballing.
- filter is installed when `--http3` in the tool is used (or
the equivalent CURLOPT_ done in the library)
- starts a QUIC/HTTP/3 connect right away. Should that not
succeed after 100ms (subject to change), a parallel attempt
is started for HTTP/2 and HTTP/1.1 via TCP
- both attempts are subject to IPv6/IPv4 eyeballing, same
as happens for other connections
- tie timeout to the ip-version HAPPY_EYEBALLS_TIMEOUT
- use a `soft` timeout at half the value. When the soft timeout
expires, the HTTPS-CONNECT filter checks if the QUIC filter
has received any data from the server. If not, it will start
the HTTP/2 attempt.
HTTP/3(ngtcp2) improvements.
- setting call_data in all cfilter calls similar to http/2 and vtls filters
for use in callback where no stream data is available.
- returning CURLE_PARTIAL_FILE for prematurely terminated transfers
- enabling pytest test_05 for h3
- shifting functionality to "connect" UDP sockets from ngtcp2
implementation into the udp socket cfilter. Because unconnected
UDP sockets are weird. For example they error when adding to a
pollset.
HTTP/3(quiche) improvements.
- fixed upload bug in quiche implementation, now passes 251 and pytest
- error codes on stream RESET
- improved debug logs
- handling of DRAIN during connect
- limiting pending event queue
HTTP/2 cfilter improvements.
- use LOG_CF macros for dynamic logging in debug build
- fix CURLcode on RST streams to be CURLE_PARTIAL_FILE
- enable pytest test_05 for h2
- fix upload pytests and improve parallel transfer performance.
GOAWAY handling for ngtcp2/quiche
- during connect, when the remote server refuses to accept new connections
and closes immediately (so the local conn goes into DRAIN phase), the
connection is torn down and a another attempt is made after a short grace
period.
This is the behaviour observed with nghttpx when we tell it to shut
down gracefully. Tested in pytest test_03_02.
TLS improvements
- ALPN selection for SSL/SSL-PROXY filters in one vtls set of functions, replaces
copy of logic in all tls backends.
- standardized the infof logging of offered ALPNs
- ALPN negotiated: have common function for all backends that sets alpn proprty
and connection related things based on the negotiated protocol (or lack thereof).
- new tests/tests-httpd/scorecard.py for testing h3/h2 protocol implementation.
Invoke:
python3 tests/tests-httpd/scorecard.py --help
for usage.
Improvements on gathering connect statistics and socket access.
- new CF_CTRL_CONN_REPORT_STATS cfilter control for having cfilters
report connection statistics. This is triggered when the connection
has completely connected.
- new void Curl_pgrsTimeWas(..) method to report a timer update with
a timestamp of when it happend. This allows for updating timers
"later", e.g. a connect statistic after full connectivity has been
reached.
- in case of HTTP eyeballing, the previous changes will update
statistics only from the filter chain that "won" the eyeballing.
- new cfilter query CF_QUERY_SOCKET for retrieving the socket used
by a filter chain.
Added methods Curl_conn_cf_get_socket() and Curl_conn_get_socket()
for convenient use of this query.
- Change VTLS backend to query their sub-filters for the socket when
checks during the handshake are made.
HTTP/3 documentation on how https eyeballing works.
TLS improvements
- ALPN selection for SSL/SSL-PROXY filters in one vtls set of functions, replaces
copy of logic in all tls backends.
- standardized the infof logging of offered ALPNs
- ALPN negotiated: have common function for all backends that sets alpn proprty
and connection related things based on the negotiated protocol (or lack thereof).
Scorecard with Caddy.
- configure can be run with `--with-test-caddy=path` to specify which caddy to use for testing
- tests/tests-httpd/scorecard.py now measures download speeds with caddy
pytest improvements
- adding Makfile to clean gen dir
- adding nghttpx rundir creation on start
- checking httpd version 2.4.55 for test_05 cases where it is needed. Skipping with message if too old.
- catch exception when checking for caddy existance on system.
Closes #10349
This commit is contained in:
parent
b7aaf074e5
commit
671158242d
61 changed files with 3599 additions and 1249 deletions
|
|
@ -59,19 +59,41 @@ class EnvConfig:
|
|||
self.config = DEF_CONFIG
|
||||
# check cur and its features
|
||||
self.curl = CURL
|
||||
self.curl_features = []
|
||||
self.curl_props = {
|
||||
'version': None,
|
||||
'os': None,
|
||||
'features': [],
|
||||
'protocols': [],
|
||||
'libs': [],
|
||||
'lib_versions': [],
|
||||
}
|
||||
self.curl_protos = []
|
||||
p = subprocess.run(args=[self.curl, '-V'],
|
||||
capture_output=True, text=True)
|
||||
if p.returncode != 0:
|
||||
assert False, f'{self.curl} -V failed with exit code: {p.returncode}'
|
||||
for l in p.stdout.splitlines(keepends=False):
|
||||
if l.startswith('curl '):
|
||||
m = re.match(r'^curl (?P<version>\S+) (?P<os>\S+) (?P<libs>.*)$', l)
|
||||
if m:
|
||||
self.curl_props['version'] = m.group('version')
|
||||
self.curl_props['os'] = m.group('os')
|
||||
self.curl_props['lib_versions'] = [
|
||||
lib.lower() for lib in m.group('libs').split(' ')
|
||||
]
|
||||
self.curl_props['libs'] = [
|
||||
re.sub(r'/.*', '',lib) for lib in self.curl_props['lib_versions']
|
||||
]
|
||||
if l.startswith('Features: '):
|
||||
self.curl_features = [feat.lower() for feat in l[10:].split(' ')]
|
||||
self.curl_props['features'] = [
|
||||
feat.lower() for feat in l[10:].split(' ')
|
||||
]
|
||||
if l.startswith('Protocols: '):
|
||||
self.curl_protos = [prot.lower() for prot in l[11:].split(' ')]
|
||||
self.curl_props['protocols'] = [
|
||||
prot.lower() for prot in l[11:].split(' ')
|
||||
]
|
||||
self.nghttpx_with_h3 = re.match(r'.* nghttp3/.*', p.stdout.strip())
|
||||
log.error(f'nghttpx -v: {p.stdout}')
|
||||
log.debug(f'nghttpx -v: {p.stdout}')
|
||||
|
||||
self.http_port = self.config['test']['http_port']
|
||||
self.https_port = self.config['test']['https_port']
|
||||
|
|
@ -81,6 +103,7 @@ class EnvConfig:
|
|||
self.apxs = self.config['httpd']['apxs']
|
||||
if len(self.apxs) == 0:
|
||||
self.apxs = None
|
||||
self._httpd_version = None
|
||||
|
||||
self.examples_pem = {
|
||||
'key': 'xxx',
|
||||
|
|
@ -110,7 +133,39 @@ class EnvConfig:
|
|||
self.nghttpx = None
|
||||
else:
|
||||
self.nghttpx_with_h3 = re.match(r'.* nghttp3/.*', p.stdout.strip()) is not None
|
||||
log.error(f'nghttpx -v: {p.stdout}')
|
||||
log.debug(f'nghttpx -v: {p.stdout}')
|
||||
|
||||
self.caddy = self.config['caddy']['caddy']
|
||||
if len(self.caddy) == 0:
|
||||
self.caddy = 'caddy'
|
||||
if self.caddy is not None:
|
||||
try:
|
||||
p = subprocess.run(args=[self.caddy, 'version'],
|
||||
capture_output=True, text=True)
|
||||
if p.returncode != 0:
|
||||
# not a working caddy
|
||||
self.caddy = None
|
||||
except:
|
||||
self.caddy = None
|
||||
self.caddy_port = self.config['caddy']['port']
|
||||
|
||||
@property
|
||||
def httpd_version(self):
|
||||
if self._httpd_version is None and self.apxs is not None:
|
||||
p = subprocess.run(args=[self.apxs, '-q', 'HTTPD_VERSION'],
|
||||
capture_output=True, text=True)
|
||||
if p.returncode != 0:
|
||||
raise Exception(f'{self.apxs} failed to query HTTPD_VERSION: {p}')
|
||||
self._httpd_version = p.stdout.strip()
|
||||
return self._httpd_version
|
||||
|
||||
def _versiontuple(self, v):
|
||||
v = re.sub(r'(\d+\.\d+(\.\d+)?)(-\S+)?', r'\1', v)
|
||||
return tuple(map(int, v.split('.')))
|
||||
|
||||
def httpd_is_at_least(self, minv):
|
||||
hv = self._versiontuple(self.httpd_version)
|
||||
return hv >= self._versiontuple(minv)
|
||||
|
||||
def is_complete(self) -> bool:
|
||||
return os.path.isfile(self.httpd) and \
|
||||
|
|
@ -146,14 +201,46 @@ class Env:
|
|||
def have_h3_server() -> bool:
|
||||
return Env.CONFIG.nghttpx_with_h3
|
||||
|
||||
@staticmethod
|
||||
def have_h2_curl() -> bool:
|
||||
return 'http2' in Env.CONFIG.curl_props['features']
|
||||
|
||||
@staticmethod
|
||||
def have_h3_curl() -> bool:
|
||||
return 'http3' in Env.CONFIG.curl_features
|
||||
return 'http3' in Env.CONFIG.curl_props['features']
|
||||
|
||||
@staticmethod
|
||||
def curl_uses_lib(libname: str) -> bool:
|
||||
return libname.lower() in Env.CONFIG.curl_props['libs']
|
||||
|
||||
@staticmethod
|
||||
def curl_lib_version(libname: str) -> str:
|
||||
prefix = f'{libname.lower()}/'
|
||||
for lversion in Env.CONFIG.curl_props['lib_versions']:
|
||||
if lversion.startswith(prefix):
|
||||
return lversion[len(prefix):]
|
||||
return 'unknown'
|
||||
|
||||
@staticmethod
|
||||
def curl_os() -> bool:
|
||||
return Env.CONFIG.curl_props['os']
|
||||
|
||||
@staticmethod
|
||||
def curl_version() -> bool:
|
||||
return Env.CONFIG.curl_props['version']
|
||||
|
||||
@staticmethod
|
||||
def have_h3() -> bool:
|
||||
return Env.have_h3_curl() and Env.have_h3_server()
|
||||
|
||||
@staticmethod
|
||||
def httpd_version() -> str:
|
||||
return Env.CONFIG.httpd_version
|
||||
|
||||
@staticmethod
|
||||
def httpd_is_at_least(minv) -> bool:
|
||||
return Env.CONFIG.httpd_is_at_least(minv)
|
||||
|
||||
def __init__(self, pytestconfig=None):
|
||||
self._verbose = pytestconfig.option.verbose \
|
||||
if pytestconfig is not None else 0
|
||||
|
|
@ -214,6 +301,14 @@ class Env:
|
|||
def h3_port(self) -> str:
|
||||
return self.CONFIG.h3_port
|
||||
|
||||
@property
|
||||
def caddy(self) -> str:
|
||||
return self.CONFIG.caddy
|
||||
|
||||
@property
|
||||
def caddy_port(self) -> str:
|
||||
return self.CONFIG.caddy_port
|
||||
|
||||
@property
|
||||
def curl(self) -> str:
|
||||
return self.CONFIG.curl
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue