Commit graph

14249 commits

Author SHA1 Message Date
Daniel Stenberg
804d5293f8
urlapi: skip path checks if path is just "/"
As a miniscule optimization, treat a path of the length 1 as the same as
non-existing, as it can only be a single leading slash, and that's what
we do for no paths as well.

Closes #10385
2023-02-01 23:04:45 +01:00
Viktor Szakats
48eb71ade4
tls: fixes for wolfssl + openssl combo builds
1. Add `USE_WOLFSSL` to the TLS backend priority list in
   `lib/curl_ntlm_core.c`.

2. Fix `lib/curl_ntlm_core.h` to respect TLS backend priority, bringing
   it in sync with the above list and `lib/curl_ntlm_core.c` itself.

   Reported-by: Mark Roszko
   Ref: https://github.com/curl/curl/issues/10321

3. Allow enabling both wolfSSL and OpenSSL at the same time in
   `lib/Makefile.mk` bringing this in line with cmake/autotools builds.
   Update logic to select the crypto-specific lib for `ngtcp2`, which
   supports a single TLS backend at the same time.

Closes #10322
2023-02-01 09:47:16 +00:00
Kvarec Lezki
9caa7bc930
cookies: fp is always not NULL
Closes #10383
2023-02-01 07:57:06 +01:00
Daniel Stenberg
fc8ad0b23c
escape: use table lookup when adding %-codes to output
On my dev host, this code runs 7.8 times faster.

Closes #10377
2023-01-31 22:42:21 +01:00
Daniel Stenberg
f1f8acb3b9
escape: hex decode with a lookup-table
Makes the decoding 2.8 times faster in my tests.

Closes #10376
2023-01-31 15:03:19 +01:00
Daniel Stenberg
1ca483a40c
cf-socket: fix build error wo TCP_FASTOPEN_CONNECT
Follow-up to 5651a36d1a

Closes #10378

Reviewed-by: Stefan Eissing
2023-01-31 14:35:39 +01:00
Stefan Eissing
91eb197a9e
CI: add pytest github workflow to CI test/tests-httpd on a HTTP/3 setup
Closes #10317
2023-01-31 12:04:23 +01:00
Stefan Eissing
d55de24dce
connect: fix strategy testing for attempts, timeouts and happy-eyeball
- add test2600 as a unit test that triggers various connect conditions
  and monitors behaviour, available in a debug build only.

- this exposed edge cases in connect.c that have been fixed

Closes #10312
2023-01-31 11:28:14 +01:00
Stefan Eissing
5651a36d1a
cf-socket: improvements in socket I/O handling
- Curl_write_plain/Curl_read_plain have been eliminated. Last code use
  now uses Curl_conn_send/recv so that requests use conn->send/revc
  callbacks which defaults to cfilters use.
- Curl_recv_plain/Curl_send_plain have been internalized in cf-socket.c.
- USE_RECV_BEFORE_SEND_WORKAROUND (active on Windows) has been moved
  into cf-socket.c. The pre_recv buffer is held at the socket filter
  context.  `postponed_data` structures have been removed from
  `connectdata`.
- the hanger in HTTP/2 request handling was a result of read buffering
  on all sends and the multi handling is not prepared for this. The
  following happens:

   - multi preforms on a HTTP/2 easy handle
   - h2 reads and processes data
   - this leads to a send of h2 data
   - which receives and buffers before the send
   - h2 returns
   - multi selects on the socket, but no data arrives (its in the buffer already)
   the workaround now receives data in a loop as long as there is something in
   the buffer. The real fix would be for multi to change, so that `data_pending`
   is evaluated before deciding to wait on the socket.

io_buffer, optional, in cf-socket.c, http/2 sets state.drain if lower
filter have pending data.

This io_buffer is only available/used when the
-DUSE_RECV_BEFORE_SEND_WORKAROUND is active, e.g. on Windows
configurations. It also maintains the original checks on protocol
handler being HTTP and conn->send/recv not being replaced.

The HTTP/2 (nghttp2) cfilter now sets data->state.drain when it finds
out that the "lower" filter chain has still pending data at the end of
its IO operation. This prevents the processing from becoming stalled.

Closes #10280
2023-01-31 11:23:59 +01:00
Daniel Stenberg
a3bcfab4b5
openssl: only use CA_BLOB if verifying peer
Reported-by: Paul Groke
Bug: https://curl.se/mail/lib-2023-01/0070.html
Fixes #10351
Closes #10359
2023-01-31 11:10:42 +01:00
Daniel Stenberg
4c48fb4933
ws: unstick connect-only shutdown
As this mode uses blocking sockets, it must set them back to
non-blocking in disconnect to avoid the risk of getting stuck.

Closes #10366
2023-01-30 15:39:04 +01:00
Daniel Stenberg
097544959a
ws: remove bad assert
Reported-by: Stanley Wucw
Fixes #10347
Closes #10366
2023-01-30 15:38:55 +01:00
Daniel Stenberg
fc10d86011
openssl: adapt to boringssl's error code type
BoringSSL uses uint32_t, OpenSSL uses 'unsigned 'long'

Closes #10360
2023-01-29 22:35:40 +01:00
Daniel Stenberg
0c3d54269e
dict: URL decode the entire path always
Reported-by: dekerser on github
Fixes #10298
Closes #10354
2023-01-29 16:03:40 +01:00
Stefan Eissing
22d3e89e6f
vtls: do not null-check when we already assume cf-ctx exists
Fixes #10361
Closes #10362
2023-01-29 15:39:09 +01:00
Ryan Schmidt
6a45abfbf6 connect: Fix build when not ENABLE_IPV6
Check for ENABLE_IPV6 before accessing AF_INET6. Fixes build failure
introduced in 1c5d8ac.

Closes https://github.com/curl/curl/pull/10344
2023-01-26 03:32:24 -05:00
Ryan Schmidt
a17101e347 cf-socket: Fix build when not HAVE_GETPEERNAME
Remove remaining references to conn and sockfd, which were removed from
the function signature when conninfo_remote was renamed to
conn_set_primary_ip in 6a8d7ef.

Closes https://github.com/curl/curl/pull/10343
2023-01-26 03:30:26 -05:00
Stefan Eissing
9e93bd47c2 vtls: Manage current easy handle in nested cfilter calls
The previous implementation cleared `data` so the outer invocation lost
its data, which could lead to a crash.

Bug: https://github.com/curl/curl/issues/10336
Reported-by: Fujii Hironori

Closes https://github.com/curl/curl/pull/10340
2023-01-26 03:05:01 -05:00
Cherish98
bde24fac7c openssl: don't log raw record headers
- Skip content type SSL3_RT_HEADER in verbose TLS output.

This commit prevents bogus and misleading verbose TLS header messages as
discussed in #10299.

Assisted-by: Peter Wu

Closes https://github.com/curl/curl/pull/10299
2023-01-24 03:31:07 -05:00
Stefan Eissing
f8da4f2f2d vtls: fix hostname handling in filters
- Copy the hostname and dispname to ssl_connect_data.

Use a copy instead of referencing the `connectdata` instance since this
may get free'ed on connection reuse.

Reported-by: Stefan Talpalaru
Reported-by: sergio-nsk@users.noreply.github.com

Fixes https://github.com/curl/curl/issues/10273
Fixes https://github.com/curl/curl/issues/10309

Closes https://github.com/curl/curl/pull/10310
2023-01-20 00:40:18 -05:00
Sergey Bronnikov
11708d6f00 lib: fix typos
Closes https://github.com/curl/curl/pull/10307
2023-01-17 03:39:25 -05:00
Jay Satiro
c4cd0e2be9 openssl: Don't ignore CA paths when using Windows CA store (redux)
.. and remove 'experimental' designation from CURLSSLOPT_NATIVE_CA.

This commit restores the behavior of CURLSSLOPT_NATIVE_CA so that it
does not override CURLOPT_CAINFO / CURLOPT_CAPATH, or the hardcoded
default locations. Instead the native Windows CA store can be used at
the same time.

---

This behavior was originally added over two years ago in abbc5d60
(#5585) but then 83393b1a (#7892) broke it over a year ago, I assume
inadvertently.

The CURLSSLOPT_NATIVE_CA feature was marked experimental and likely
rarely used.

Ref: https://github.com/curl/curl/pull/5585
Ref: https://github.com/curl/curl/pull/7892
Ref: https://curl.se/mail/lib-2023-01/0019.html

Closes https://github.com/curl/curl/pull/10244
2023-01-17 03:32:58 -05:00
Daniel Stenberg
abae4e31a2
ws: fix autoping handling
Reported-by: Alexey Savchuk
Fixes #10289
Closes #10294
2023-01-13 15:35:50 +01:00
Daniel Stenberg
521da2dbd3
curl_log: avoid printf() format checking with mingw
Since it does not seem to like %zu and more

Follow-up to db91dbbf2

Fixes #10291
Closes #10292
2023-01-13 13:37:21 +01:00
Stefan Eissing
12157aa924
openssl: make the BIO_METHOD a local variable in the connection filter
This avoids UAF issues when `curl_global_cleanup()` is called before all
transfers have been completely handled. Unfortunately this seems to be a
more common pattern than we like.

Closes #10285
2023-01-12 23:43:56 +01:00
Daniel Stenberg
7d3b167f48
curl: output warning at --verbose output for debug-enabled version
+ a libcurl warning in the debug output

Assisted-by: Jay Satiro

Ref: https://curl.se/mail/lib-2023-01/0039.html
Closes #10278
2023-01-12 17:59:26 +01:00
Daniel Stenberg
8956cd4f9a
connect: fix access of pointer before NULL check
Detected by Coverity CID 1518992

Closes #10284
2023-01-12 15:31:21 +01:00
Daniel Gustafsson
4579626c27
easyoptions: Fix header printing in generation script
The optiontable.pl script prints the header comment when generating
easyoptions.c, but it wasn't escaping all characters which jumbled the
curl ascii logo.  Fix by escaping.

Cloes #10275
2023-01-12 14:59:38 +01:00
Stefan Eissing
db91dbbf2c
curl_log: for failf/infof and debug logging implementations
- new functions and macros for cfilter debugging
 - set CURL_DEBUG with names of cfilters where debug logging should be
   enabled
 - use GNUC __attribute__ to enable printf format checks during compile

Closes #10271
2023-01-12 10:14:17 +01:00
Nick Banks
9f0c7795eb
msh3: update to v0.6
Closes #10192
2023-01-10 13:50:08 +01:00
Stefan Eissing
088c08a7e4
ngtcp2: add CURLOPT_SSL_CTX_FUNCTION support for openssl+wolfssl
Using common method for SSL_CTX initialization of verfiy peer and CA
settings. This also provides X509_STORE sharing to become available for
ngtcp2+openssl HTTP/3.

Reported-by: violetlige on github

Fixes #10222
Closes #10239
2023-01-10 13:41:21 +01:00
Daniel Stenberg
c34fcbbad8
cf-socket: make infof() call use %zu for size_t output
Detected by Coverity CID 1518986 and CID 1518984

Closes #10268
2023-01-10 11:51:34 +01:00
Stefan Eissing
33ac97e1cb
tests-httpd: basic infra to run curl against an apache httpd plus nghttpx for h3
- adding '--with-test-httpd=<path>' to configure non-standard apache2
  install
- python env and base classes for running httpd
- basic tests for connectivity with h1/h2/h3
- adding test cases for truncated responses in http versions.
- adding goaway test for HTTP/3.
- adding "stuttering" tests with parallel downloads in chunks with
  varying delays between chunks.

- adding a curltest module to the httpd server, adding GOAWAY test.
    - mod_curltest now installs 2 handlers
      - 'echo': writing as response body what came as request body
      - 'tweak': with query parameters to tweak response behaviour
- marked known fails as skip for now

Closes #10175
2023-01-09 17:40:04 +01:00
Stefan Eissing
1c5d8acf79
quic: improve connect error message, debugging info, fix false connect report
- ECONNECTREFUSED has not its own fail message in quic filters
- Debug logging in connect eyballing improved
- Fix bug in ngtcp2/quiche that could lead to false success reporting.

Reported-by: Divy Le Ray

Fixes #10245
Closes #10248
2023-01-09 13:23:20 +01:00
Stefan Eissing
0ef2926881
quiche: fix build without any HTTP/2 implementation
Fixes #10260
Closes #10263
2023-01-09 11:09:34 +01:00
Stefan Eissing
4303093cd5
msh3: run again in its cfilter
- test 2500, single GET works
- test 2501, single POST stalls
- test 2502, multiple, sequential GETs each use a new connection since
  MsH3ConnectionGetState(qconn) no longer reports CONNECTED after one
  GET.

Closes #10204
2023-01-08 11:06:30 +01:00
Jay Satiro
260fea215a sendf: fix build for Linux TCP fastopen
- Fix the remote addr struct dereference.

- Include cf-socket.h in urldata.h.

Follow-up to 6a8d7ef9 which changed conn->ipaddr (Curl_addrinfo* )
member to conn->remote_addr (Curl_sockaddr_ex *) several days ago.

Reported-by: Stephan Guilloux

Fixes https://github.com/curl/curl/issues/10249
Closes https://github.com/curl/curl/pull/10250
2023-01-08 03:32:06 -05:00
Daniel Stenberg
b45b6b618d
setopt: move the SHA256 opt within #ifdef libssh2
Because only the libssh2 backend not supports it and thus this should
return error if this option is used other backends.

Reported-by: Harry Sintonen

Closes #10255
2023-01-07 23:01:52 +01:00
Patrick Monnerat
ee0f73919a
nss: implement data_pending method
NSS currently uses the default Curl_none_data_pending() method which
always returns false, causing TLS buffered input data to be missed.

The current commit implements the nss_data_pending() method that properly
monitors the presence of available TLS data.

Ref:#10077

Closes #10225
2023-01-07 14:19:31 +01:00
jonrumsey
1f693e0aca
x509asn1: fix compile errors and warnings
Various small issues when built for GSKit

Closes #10238
2023-01-05 23:51:27 +01:00
Daniel Stenberg
3f3ddee066
http_proxy: do not assign data->req.p.http use local copy
Avoid the tricky reusing of the data->req.p.http pointer for http proxy
tunneling.

Fixes #10194
Closes #10234
2023-01-05 23:43:02 +01:00
Stefan Eissing
446267c555
quic: rename vquic implementations, fix for quiche build.
- quiche in debug mode did not build, fixed.
- moved all vquic implementation files to prefix curl_* to avoid
  the potential mixups between provided .h files and our own.
- quich passes test 2500 and 2502. 2501, the POST, fail with
  the body being rejected. Quich bug?

Closes #10242
2023-01-05 23:39:49 +01:00
Stefan Eissing
16bb32e104
sectransp: fix for incomplete read/writes
SecureTransport expects result code errSSLWouldBlock when the requested
length could not be sent/recieved in full. The previous code returned
noErr, which let SecureTransport to believe that the IO had terminated
prematurely.

Fixes #10227
Closes #10235
2023-01-05 16:04:12 +01:00
Jay Satiro
602964ec5e scripts: set file mode +x on all perl and shell scripts
- Set all scripts +x, ie 644 => 755.

Prior to this change some scripts were not executable and therefore
could not be called directly.

~~~
git ls-files -s \*.{sh,pl,py} | grep -v 100755
~~~

Closes https://github.com/curl/curl/pull/10219
2023-01-05 02:34:24 -05:00
Daniel Stenberg
1485e89213
misc: reduce struct and struct field sizes
- by using BIT() instead of bool
- imap: shrink struct
- ftp: make state 'unsigned char'
- ftp: sort ftp_conn struct entries on size
- urldata: use smaller fields for SSL version info storage
- pop3: reduce the pop3_conn struct size
- smtp: reduce the size of the smtp structs

Closes #10186
2023-01-04 15:36:36 +01:00
Daniel Stenberg
7ad8a7ba9e
noproxy: support for space-separated names is deprecated
To be removed in July 2024.

Assisted-by: Michael Osipov
Fixes #10209
Closes #10215
2023-01-04 09:15:37 +01:00
Andrei Rybak
bb393e521f
lib: fix typos in comments which repeat a word
Remove erroneously duplicated words in code comments of files
`lib.connect.c` and `lib/url.c`.

Closes #10220
2023-01-04 09:12:58 +01:00
Radek Brich
3ef31a16fc
cmake: set SOVERSION also for macOS
Closes #10214
2023-01-03 23:51:40 +01:00
Jay Satiro
fcd1b547d4 http2: fix compiler warning due to uninitialized variable
Prior to this change http2_cfilter_add could return an uninitialized
cfilter pointer in an OOM condition. In this case though, the pointer
is discarded and not dereferenced so there was no risk of a crash.
2023-01-03 17:15:26 -05:00
Stefan Eissing
6a8d7ef981
cf-socket: keep sockaddr local in the socket filters
- copy `struct Curl_addrinfo` on filter setup into context
- remove `struct Curl_addrinfoi *` with `struct Curl_sockaddr_ex *` in
  connectdata that is set and NULLed by the socket filter
- this means we have no reference to the resolver info in connectdata or
  its filters
- trigger the CF_CTRL_CONN_INFO_UPDATE event when the complete filter
  chain reaches connected status
- update easy handle connection information on CF_CTRL_DATA_SETUP event.

Closes #10213
2023-01-03 17:33:29 +01:00