Commit graph

996 commits

Author SHA1 Message Date
Stefan Eissing
db91dbbf2c
curl_log: for failf/infof and debug logging implementations
- new functions and macros for cfilter debugging
 - set CURL_DEBUG with names of cfilters where debug logging should be
   enabled
 - use GNUC __attribute__ to enable printf format checks during compile

Closes #10271
2023-01-12 10:14:17 +01:00
Jay Satiro
260fea215a sendf: fix build for Linux TCP fastopen
- Fix the remote addr struct dereference.

- Include cf-socket.h in urldata.h.

Follow-up to 6a8d7ef9 which changed conn->ipaddr (Curl_addrinfo* )
member to conn->remote_addr (Curl_sockaddr_ex *) several days ago.

Reported-by: Stephan Guilloux

Fixes https://github.com/curl/curl/issues/10249
Closes https://github.com/curl/curl/pull/10250
2023-01-08 03:32:06 -05:00
Daniel Stenberg
1485e89213
misc: reduce struct and struct field sizes
- by using BIT() instead of bool
- imap: shrink struct
- ftp: make state 'unsigned char'
- ftp: sort ftp_conn struct entries on size
- urldata: use smaller fields for SSL version info storage
- pop3: reduce the pop3_conn struct size
- smtp: reduce the size of the smtp structs

Closes #10186
2023-01-04 15:36:36 +01:00
Stefan Eissing
6a8d7ef981
cf-socket: keep sockaddr local in the socket filters
- copy `struct Curl_addrinfo` on filter setup into context
- remove `struct Curl_addrinfoi *` with `struct Curl_sockaddr_ex *` in
  connectdata that is set and NULLed by the socket filter
- this means we have no reference to the resolver info in connectdata or
  its filters
- trigger the CF_CTRL_CONN_INFO_UPDATE event when the complete filter
  chain reaches connected status
- update easy handle connection information on CF_CTRL_DATA_SETUP event.

Closes #10213
2023-01-03 17:33:29 +01:00
Daniel Stenberg
2bc1d775f5
copyright: update all copyright lines and remove year ranges
- they are mostly pointless in all major jurisdictions
- many big corporations and projects already don't use them
- saves us from pointless churn
- git keeps history for us
- the year range is kept in COPYING

checksrc is updated to allow non-year using copyright statements

Closes #10205
2023-01-03 09:19:21 +01:00
Daniel Stenberg
becfe2ec78
urldata: cease storing TLS auth type
The only TLS auth type libcurl ever supported is SRP and that is the
default type. Since nobody ever sets any other type, there is no point
in wasting space to store the set type and code to check the type.

If TLS auth is used, SRP is now implied.

Closes #10181
2022-12-31 16:56:21 +01:00
Stefan Eissing
71b7e01610
lib: connect/h2/h3 refactor
Refactoring of connection setup and happy eyeballing. Move
nghttp2. ngtcp2, quiche and msh3 into connection filters.

 - eyeballing cfilter that uses sub-filters for performing parallel connects
 - socket cfilter for all transport types, including QUIC
 - QUIC implementations in cfilter, can now participate in eyeballing
 - connection setup is more dynamic in order to adapt to what filter did
   really connect.  Relevant to see if a SSL filter needs to be added or
   if SSL has already been provided
 - HTTP/3 test cases similar to HTTP/2
 - multiuse of parallel transfers for HTTP/3, tested for ngtcp2 and quiche

 - Fix for data attach/detach in VTLS filters that could lead to crashes
   during parallel transfers.
 - Eliminating setup() methods in cfilters, no longer needed.
 - Improving Curl_conn_is_alive() to replace Curl_connalive() and
   integrated ssl alive checks into cfilter.
 - Adding CF_CNTRL_CONN_INFO_UPDATE to tell filters to update
   connection into and persist it at the easy handle.

 - Several more cfilter related cleanups and moves:
   - stream_weigth and dependency info is now wrapped in struct
     Curl_data_priority
   - Curl_data_priority members depend is available in HTTP2|HTTP3
   - Curl_data_priority members depend on NGHTTP2 support
   - handling init/reset/cleanup of priority part of url.c
   - data->state.priority same struct, but shallow copy for compares only

 - PROTOPT_STREAM has been removed
   - Curl_conn_is_mulitplex() now available to check on capability

 - Adding query method to connection filters.
   - ngtcp2+quiche: implementing query for max concurrent transfers.

 - Adding is_alive and keep_alive cfilter methods. Adding DATA_SETUP event.
   - setting keepalive timestamp on connect
   - DATA_SETUP is called after the connection has been completely
     setup (but may not connected yet) to allow filters to initialize
     data members they use.

 - there is no socket to be had with msh3, it is unclear how select
   shall work

 - manual test via "curl --http3 https://curl.se" fail with "empty
   reply from server".

 - Various socket/conn related cleanups:
   - Curl_socket is now Curl_socket_open and in cf-socket.c
   - Curl_closesocket is now Curl_socket_close and in cf-socket.c
   - Curl_ssl_use has been replaced with Cur_conn_is_ssl
   - Curl_conn_tcp_accepted_set has been split into
     Curl_conn_tcp_listen_set and Curl_conn_tcp_accepted_set
     with a clearer purpose

Closes #10141
2022-12-30 16:43:19 +01:00
Daniel Stenberg
076a2f6291
share: add sharing of HSTS cache among handles
Closes #10138
2022-12-27 15:22:01 +01:00
Daniel Stenberg
db07301fbb
urldata: remove unused struct fields, made more conditional
- source_quote, source_prequote and source_postquote have not been used since
  5e0d9aea3; September 2006

- make several fields conditional on proxy support

- make three quote struct fields conditional on FTP || SSH

- make 'mime_options' depend on MIME

- make trailer_* fields depend on HTTP

- change 'gssapi_delegation' from long to unsigned char

- make 'localportrange' unsigned short instead of int

- conn->trailer now depends on HTTP

Closes #10147
2022-12-23 08:45:05 +01:00
Daniel Stenberg
f4883ba966
urldata: make set.http200aliases conditional on HTTP being present
And make a few SSH-only fields depend on SSH

Closes #10140
2022-12-22 23:07:46 +01:00
Daniel Stenberg
31b9a32260
urldata: make 'ftp_create_missing_dirs' depend on FTP || SFTP
Closes #10139
2022-12-22 23:04:47 +01:00
Daniel Stenberg
af5999a674
urldata: move the cookefilelist to the 'set' struct
The cookiefile entries are set into the handle and should remain set for
the lifetime of the handle so that duplicating it also duplicates the
list. Therefore, the struct field is moved from 'state' to 'set'.

Fixes #10133
Closes #10134
2022-12-22 16:06:37 +01:00
Daniel Stenberg
57d2d9b6be
lib: use size_t or int etc instead of longs
Since long is not using a consistent data size in curl builds, making it
often "waste" 32 bits.

Closes #10088
2022-12-14 12:31:01 +01:00
Daniel Stenberg
1b3973180b
lib: rewind BEFORE request instead of AFTER previous
This makes a big difference for cases when the rewind is not actually
necessary to perofm (for example HTTP response code 301 converts to GET)
and therefore the rewind can be avoided. In particular for situations
when that rewind fails, for example when reading from a pipe or similar.

Reported-by: Ali Utku Selen

Fixes #9735
Closes #9958
2022-11-25 08:30:26 +01:00
Stefan Eissing
af22c2a546
vtls: localization of state data in filters
- almost all backend calls pass the Curl_cfilter intance instead of
   connectdata+sockindex
 - ssl_connect_data is remove from struct connectdata and made internal
   to vtls
 - ssl_connect_data is allocated in the added filter, kept at cf->ctx

 - added function to let a ssl filter access its ssl_primary_config and
   ssl_config_data this selects the propert subfields in conn and data,
   for filters added as plain or proxy
 - adjusted all backends to use the changed api
 - adjusted all backends to access config data via the exposed
   functions, no longer using conn or data directly

cfilter renames for clear purpose:

 - methods `Curl_conn_*(data, conn, sockindex)` work on the complete
   filter chain at `sockindex` and connection `conn`.
 - methods `Curl_cf_*(cf, ...)` work on a specific Curl_cfilter
   instance.
 - methods `Curl_conn_cf()` work on/with filter instances at a
   connection.
 - rebased and resolved some naming conflicts
 - hostname validation (und session lookup) on SECONDARY use the same
   name as on FIRST (again).

new debug macros and removing connectdata from function signatures where not
needed.

adapting schannel for new Curl_read_plain paramter.

Closes #9919
2022-11-22 14:25:50 +01:00
Daniel Stenberg
c7cd781a63
urldata: change port num storage to int and unsigned short
Instead of long.

Closes #9946
2022-11-18 23:54:38 +01:00
Alexandre Ferrieux
49798cac83
CURLOPT_QUICK_EXIT: don't wait for DNS thread on exit
Fixes #2975
Closes #9147
2022-11-18 00:17:27 +01:00
Stefan Eissing
a8e6351e12
proxy: refactor haproxy protocol handling as connection filter
Closes #9893
2022-11-15 15:56:25 +01:00
Daniel Stenberg
bf12c2bed6
lib: remove bad set.opt_no_body assignments
This struct field MUST remain what the application set it to, so that
handle reuse and handle duplication work.

Instead, the request state bit 'no_body' is introduced for code flows
that need to change this in run-time.

Closes #9888
2022-11-11 15:56:05 +01:00
Stefan Eissing
dafdb20a26
lib: connection filters (cfilter) addition to curl:
- general construct/destroy in connectdata
 - default implementations of callback functions
 - connect: cfilters for connect and accept
 - socks: cfilter for socks proxying
 - http_proxy: cfilter for http proxy tunneling
 - vtls: cfilters for primary and proxy ssl
 - change in general handling of data/conn
 - Curl_cfilter_setup() sets up filter chain based on data settings,
   if none are installed by the protocol handler setup
 - Curl_cfilter_connect() boot straps filters into `connected` status,
   used by handlers and multi to reach further stages
 - Curl_cfilter_is_connected() to check if a conn is connected,
   e.g. all filters have done their work
 - Curl_cfilter_get_select_socks() gets the sockets and READ/WRITE
   indicators for multi select to work
 - Curl_cfilter_data_pending() asks filters if the have incoming
   data pending for recv
 - Curl_cfilter_recv()/Curl_cfilter_send are the general callbacks
   installed in conn->recv/conn->send for io handling
 - Curl_cfilter_attach_data()/Curl_cfilter_detach_data() inform filters
   and addition/removal of a `data` from their connection
 - adding vtl functions to prevent use of Curl_ssl globals directly
   in other parts of the code.

Reviewed-by: Daniel Stenberg
Closes #9855
2022-11-11 15:17:51 +01:00
Fata Nugraha
11ad25fff4
http: do not send PROXY more than once
Unlike `CONNECT`, currently we don't keep track whether `PROXY` is
already sent or not. This causes `PROXY` header to be sent twice during
`MSTATE_TUNNELING` and `MSTATE_PROTOCONNECT`.

Closes #9878
Fixes #9442
2022-11-10 12:24:48 +01:00
Michael Drake
1fdca35ddd
curl.h: add CURLOPT_CA_CACHE_TIMEOUT option
Adds a new option to control the maximum time that a cached
certificate store may be retained for.

Currently only the OpenSSL backend implements support for
caching certificate stores.

Closes #9620
2022-11-08 10:06:12 +01:00
Michael Drake
3c16697ebd
openssl: reduce CA certificate bundle reparsing by caching
Closes #9620
2022-11-08 10:06:12 +01:00
Daniel Stenberg
52cc4a85fd
style: use space after comment start and before comment end
/* like this */

/*not this*/

checksrc is updated accordingly

Closes #9828
2022-10-30 22:31:29 +01:00
Ayesh Karunaratne
4484270afc
misc: typo and grammar fixes
- Replace `Github` with `GitHub`.
- Replace `windows` with `Windows`
- Replace `advice` with `advise` where a verb is used.
- A few fixes on removing repeated words.
- Replace `a HTTP` with `an HTTP`

Closes #9802
2022-10-27 10:01:30 +02:00
Viktor Szakats
0c327464ca
tidy-up: delete parallel/unused feature flags
Detecting headers and lib separately makes sense when headers come in
variations or with extra ones, but this wasn't the case here. These were
duplicate/parallel macros that we had to keep in sync with each other
for a working build. This patch leaves a single macro for each of these
dependencies:

- Rely on `HAVE_LIBZ`, delete parallel `HAVE_ZLIB_H`.

  Also delete CMake logic making sure these two were in sync, along with
  a toggle to turn off that logic, called `CURL_SPECIAL_LIBZ`.

  Also delete stray `HAVE_ZLIB` defines.

  There is also a `USE_ZLIB` variant in `lib/config-dos.h`. This patch
  retains it for compatibility and deprecates it.

- Rely on `USE_LIBSSH2`, delete parallel `HAVE_LIBSSH2_H`.

  Also delete `LIBSSH2_WIN32`, `LIBSSH2_LIBRARY` from
  `winbuild/MakefileBuild.vc`, these have a role when building libssh2
  itself. And `CURL_USE_LIBSSH`, which had no use at all.

  Also delete stray `HAVE_LIBSSH2` defines.

- Rely on `USE_LIBSSH`, delete parallel `HAVE_LIBSSH_LIBSSH_H`.

  Also delete `LIBSSH_WIN32`, `LIBSSH_LIBRARY` and `HAVE_LIBSSH` from
  `winbuild/MakefileBuild.vc`, these were the result of copy-pasting the
  libssh2 line, and were not having any use.

- Delete unused `HAVE_LIBPSL_H` and `HAVE_LIBPSL`.

Reviewed-by: Daniel Stenberg

Closes #9652
2022-10-06 15:30:13 +00:00
Viktor Szakats
7313ffebfe
docs: spelling nits
- MingW -> MinGW (Minimalist GNU for Windows)
- f.e. -> e.g.
- some whitespace and punctuation.

Reviewed-by: Daniel Stenberg

Closes #9622
2022-09-29 21:29:04 +00:00
Patrick Monnerat
91e06e687a
lib: prepare the incoming of additional protocols
Move the curl_prot_t to its own conditional block. Introduce symbol
PROTO_TYPE_SMALL to control it.

Fix a cast in a curl_prot_t assignment.
Remove an outdated comment.

Follow-up to cd5ca80.

Closes #9534
2022-09-23 08:39:44 +02:00
Daniel Stenberg
cd5ca80f00
urldata: use a curl_prot_t type for storing protocol bits
This internal-use-only storage type can be bumped to a curl_off_t once
we need to use bit 32 as the previous 'unsigned int' can no longer hold
them all then.

The websocket protocols take bit 30 and 31 so they are the last ones
that fit within 32 bits - but cannot properly be exported through APIs
since those use *signed* 32 bit types (long) in places.

Closes #9481
2022-09-13 11:26:49 +02:00
Daniel Stenberg
664249d095
ws: initial websockets support
Closes #8995
2022-09-09 15:11:14 +02:00
Daniel Stenberg
472f1cbe7e
NPN: remove support for and use of
Next Protocol Negotiation is a TLS extension that was created and used
for agreeing to use the SPDY protocol (the precursor to HTTP/2) for
HTTPS. In the early days of HTTP/2, before the spec was finalized and
shipped, the protocol could be enabled using this extension with some
servers.

curl supports the NPN extension with some TLS backends since then, with
a command line option `--npn` and in libcurl with
`CURLOPT_SSL_ENABLE_NPN`.

HTTP/2 proper is made to use the ALPN (Application-Layer Protocol
Negotiation) extension and the NPN extension has no purposes
anymore. The HTTP/2 spec was published in May 2015.

Today, use of NPN in the wild should be extremely rare and most likely
totally extinct. Chrome removed NPN support in Chrome 51, shipped in
June 2016. Removed in Firefox 53, April 2017.

Closes #9307
2022-09-05 07:39:02 +02:00
Daniel Stenberg
15f7feac22
urldata: move smaller fields down in connectdata struct
By (almost) sorting the struct fields in connectdata in a decending size
order, having the single char ones last, we reduce the number of holes
in the struct and thus the amount of storage needed.

Closes #9280
2022-08-08 19:57:49 +02:00
Daniel Stenberg
a29d34b0b7
urldata: make 'negnpn' use less storage
The connectdata struct field 'negnpn' never holds a value larger than
30, so an unsigned char saves 3 bytes struct space.

Closes #9279
2022-08-08 14:31:06 +02:00
Daniel Stenberg
a550831023
urldata: make three *_proto struct fields smaller
Use 'unsigned char' for storage instead of the enum, for three GSSAPI
related fields in the connectdata struct.

Closes #9278
2022-08-08 14:30:25 +02:00
Daniel Stenberg
0f23341953
connect: set socktype/protocol correctly
So that an address used from the DNS cache that was previously used for
QUIC can be reused for TCP and vice versa.

To make this possible, set conn->transport to "unix" for unix domain
connections ... and store the transport struct field in an unsigned char
to use less space.

Reported-by: ウさん
Fixes #9274
Closes #9276
2022-08-08 13:26:11 +02:00
Daniel Stenberg
8cd57d6e39
urldata: change second proxytype field to unsigned char to match
To avoid "enumerated type mixed with another type"

Closes #9179
2022-07-23 13:39:03 +02:00
Daniel Stenberg
dcbc9226bc
urldata: make state.httpreq an unsigned char
To match set.method used for the same purpose.

Closes #9179
2022-07-23 13:38:57 +02:00
Daniel Stenberg
fe8f1512e5
ftp: use a correct expire ID for timer expiry
This was an accurate error pointed out by the icc warning: enumerated
type mixed with another type

Ref: #9179
Closes #9184
2022-07-20 00:50:19 +02:00
Daniel Stenberg
fe93bb4d1f
doh: move doh related struct definitions to doh.h
and make 'dnstype' in 'struct dnsprobe' use the DNStype to fix the icc compiler warning:

  doh.c(924): error #188: enumerated type mixed with another type

Reported-by: Matthew Thompson
Ref #9156
Closes #9174
2022-07-18 11:55:26 +02:00
Daniel Stenberg
1b3d59acf9
urldata: reduce size of four ftp related members
ftp_filemethod, ftpsslauth and ftp_ccc are now uchars

accepttimeout is now unsigned int - almost 50 days ought to be enough
for this value.

Closes #9106
2022-07-05 10:18:48 +02:00
Daniel Stenberg
dcb7d4f904
urldata: reduce three type-members from int to uchar
- timecondition
 - proxytype
 - method

... previously used their enum type in the struct, which made them
unnecesarily large.

Closes #9105
2022-07-05 10:17:31 +02:00
Daniel Stenberg
671cc8e110
urldata: make 'ftp_create_missing_dirs' a uchar
It only ever holds the values 0-2.

Closes #9103
2022-07-05 10:10:49 +02:00
Daniel Stenberg
3622bf4e79
urldata: change 4 timeouts to unsigned int from long
They're not used for that long times anyway, 32 bit milliseconds is long
enough.

Closes #9101
2022-07-05 00:15:54 +02:00
Daniel Stenberg
fe14ff6150
urldata: make 'use_netrc' a uchar
Closes #9102
2022-07-04 23:04:24 +02:00
Daniel Stenberg
3fa343a35c
urldata: make 'buffer_size' an unsigned int
It is already capped at READBUFFER_MAX which fits easily in 32 bits.

Closes #9098
2022-07-04 23:03:44 +02:00
Daniel Stenberg
cb17b12b01
urldata: remove the unused 'rtspversion' struct member
Closes #9100
2022-07-04 19:44:10 +02:00
Daniel Stenberg
1d85d2e4f9
urldata: make 'use_port' an usigned short
... instead of a long. It is already enforced to not attempt to set any
value outside of 16 bits unsigned.

Closes #9099
2022-07-04 19:42:40 +02:00
Daniel Stenberg
ccc8092b05
urldata: store dns cache timeout in an int
68 years ought to be enough for most.

Closes #9097
2022-07-04 19:28:50 +02:00
Daniel Stenberg
e6f8445ede
setopt: add CURLOPT_PROTOCOLS_STR and CURLOPT_REDIR_PROTOCOLS_STR
... as replacements for deprecated CURLOPT_PROTOCOLS and
CURLOPT_REDIR_PROTOCOLS as these new ones do not risk running into the
32 bit limit the old ones are facing.

CURLINFO_PROTCOOL is now deprecated.

The curl tool is updated to use the new options.

Added test 1597 to verify the libcurl protocol parser.

Closes #8992
2022-07-04 08:35:09 +02:00
Daniel Stenberg
193215db3c
digest: simplify a switch() to a simple if 2022-07-04 08:27:21 +02:00