Commit graph

13556 commits

Author SHA1 Message Date
Daniel Stenberg
907eea0804
Revert "urldata: move async resolver state from easy handle to connectdata"
This reverts commit 56a4db2e4e (#12198)

We want the c-ares channel to be held in the easy handle, not per
connection - for performance.

Closes #12524
2023-12-15 12:57:35 +01:00
Viktor Szakats
9f2d2290d1
openssl: re-match LibreSSL deinit with init
Earlier we switched to use modern initialization with LibreSSL v2.7.0
and up, but did not touch deinitialization [1]. Fix it in this patch.

Regression from bec0c5bbf3 #11611

[1] https://github.com/curl/curl/pull/11611#issuecomment-1668654014

Reported-by: Mike Hommey
Reviewed-by: Daniel Stenberg
Fixes #12525
Closes #12526
2023-12-15 09:26:04 +00:00
Daniel Stenberg
78a1814b33
libssh: supress warnings without version check
Define unconditionally.

Follow-up from d21bd2190c

Closes #12523
2023-12-14 23:47:48 +01:00
Daniel Stenberg
907dce2dc0
hostip: return error immediately when Curl_ip2addr() fails
Closes #12522
2023-12-14 22:57:28 +01:00
Theo
d21bd2190c
libssh: improve the deprecation warning dismissal
Previous code was compiler dependant, and dismissed all deprecation warnings
indiscriminately.

libssh provides a way to disable the deprecation warnings for libssh only, and
naturally this is the preferred way.

This commit uses that, to prevent the erroneous hiding of potential, unrelated
deprecation warnings.

Fixes #12519
Closes #12520
2023-12-14 18:05:53 +01:00
Daniel Stenberg
1da640abb6
readwrite_data: loop less
This function is made to loop in order to drain incoming data
faster. Completely removing the loop has a measerably negative impact on
transfer speeds.

Downsides with the looping include

- it might call the progress callback much more seldom. Especially if
  the write callback is slow.

- rate limiting becomes less exact

- a single transfer might "starve out" other parallel transfers

- QUIC timers for other connections can't be maintained correctly

The long term fix should be to remove the loop and optimize coming back
to avoid the transfer speed penalty.

This fix lower the max loop count to reduce the starvation problem, and
avoids the loop completely for when rate-limiting is in progress.

Ref: #12488
Ref: https://curl.se/mail/lib-2023-12/0012.html
Closes #12504
2023-12-14 16:13:28 +01:00
Stefan Eissing
8706b68010
lib: eliminate conn->cselect_bits
- use `data->state.dselect_bits` everywhere instead
- remove `bool *comeback` parameter as non-zero
  `data->state.dselect_bits` will indicate that IO is
  incomplete.

Closes #12512
2023-12-14 15:20:05 +01:00
Stefan Eissing
0f052808b3
connect: refactor Curl_timeleft()
- less local vars, "better" readability
- added documentation

Closes #12518
2023-12-14 15:18:32 +01:00
Dmitry Karpov
1e4cd51935
cookie: avoid fopen with empty file name
Closes #12514
2023-12-14 00:15:57 +01:00
Daniel Stenberg
ff74cef5d4
lib: reduce use of strncpy
- bearssl: select cipher without buffer copies
- http_aws_sigv4: avoid strncpy, require exact timestamp length
- http_aws_sigv4: use memcpy isntead of strncpy
- openssl: avoid strncpy calls
- schannel: check for 1.3 algos without buffer copies
- strerror: avoid strncpy calls
- telnet: avoid strncpy, return error on too long inputs
- vtls: avoid strncpy in multissl_version()

Closes #12499
2023-12-11 23:29:02 +01:00
Daniel Gustafsson
d65b8868c6
doh: remove unused local variable
The nurl variable is no longer used during probing following
a refactoring, so remove.

Closes #12491
2023-12-09 00:14:18 +01:00
Daniel Stenberg
7c992dd9f8
lib: rename Curl_strndup to Curl_memdup0 to avoid misunderstanding
Since the copy does not stop at a null byte, let's not call it anything
that makes you think it works like the common strndup() function.

Based on feedback from Jay Satiro, Stefan Eissing and Patrick Monnerat

Closes #12490
2023-12-08 17:22:33 +01:00
MAntoniak
13a1d1ace8
sendf: fix compiler warning with CURL_DISABLE_HEADERS_API
fix MSVC warning C4189: 'htype': local variable is initialized but not
referenced - when CURL_DISABLE_HEADERS_API is defined.

Closes #12485
2023-12-08 09:28:53 +01:00
MAntoniak
4dbc7acc19
vtls: remove the Curl_cft_ssl_proxy object if CURL_DISABLE_PROXY
Closes #12459
2023-12-07 09:31:05 +01:00
Daniel Stenberg
7309b9cbbf
lib: strndup/memdup instead of malloc, memcpy and null-terminate
- bufref: use strndup
 - cookie: use strndup
 - formdata: use strndup
 - ftp: use strndup
 - gtls: use aprintf instead of malloc + strcpy * 2
 - http: use strndup
 - mbedtls: use strndup
 - md4: use memdup
 - ntlm: use memdup
 - ntlm_sspi: use strndup
 - pingpong: use memdup
 - rtsp: use strndup instead of malloc, memcpy and null-terminate
 - sectransp: use strndup
 - socks_gssapi.c: use memdup
 - vtls: use dynbuf instead of malloc, snprintf and memcpy
 - vtls: use strdup instead of malloc + memcpy
 - wolfssh: use strndup

Closes #12453
2023-12-07 08:47:44 +01:00
Daniel Stenberg
63cdaefbc3
strdup: remove the memchr check from Curl_strndup
It makes it possible to clone a binary chunk of data.

Closes #12453
2023-12-07 08:47:28 +01:00
Daniel Stenberg
c0dd06ecb3
ftp: handle the PORT parsing without allocation
Also reduces amount of *cpy() calls.

Closes #12456
2023-12-07 08:46:22 +01:00
Daniel Stenberg
45cf4755e7
url: for disabled protocols, mention if found in redirect
To help users better understand where the URL (and denied scheme) comes
from. Also removed "in libcurl" from the message, since the disabling
can be done by the application.

The error message now says "not supported" or "disabled" depending on
why it was denied:

 Protocol "hej" not supported
 Protocol "http" disabled

And in redirects:

 Protocol "hej" not supported (in redirect)
 Protocol "http" disabled (in redirect)

Reported-by: Mauricio Scheffer
Fixes #12465
Closes #12469
2023-12-06 23:05:21 +01:00
Stefan Eissing
8b99e6f23f
sectransp_ make TLSCipherNameForNumber() available in non-verbose config
Reported-by: Cajus Pollmeier
Closes #12476
Fixes #12474
2023-12-06 23:03:35 +01:00
YX Hao
a17f041bea
lib: fix variable undeclared error caused by infof changes
`--disable-verbose` yields `CURL_DISABLE_VERBOSE_STRINGS` defined.
`infof` isn't `Curl_nop_stmt` anymore: dac293c.

Follow-up to dac293c

Closes #12470
2023-12-06 23:01:54 +01:00
Jacob Hoffman-Andrews
ad040e1262
rustls: implement connect_blocking
Closes #11647
2023-12-04 23:03:45 +01:00
Stefan Eissing
0fbbc80b24
ngtcp2: ignore errors on unknown streams
- expecially in is_alive checks on connections, we might
  see incoming packets on streams already forgotten and closed,
  leading to errors reported by nghttp3. Ignore those.

Closes #12449
2023-12-04 12:57:24 +01:00
z2_
c44671ed43 build: fix builds that disable protocols but not digest auth
- Build base64 functions if digest auth is not disabled.

Prior to this change if some protocols were disabled but not digest auth
then a build error would occur due to missing base64 functions.

Fixes https://github.com/curl/curl/issues/12440
Closes https://github.com/curl/curl/pull/12442
2023-12-01 23:26:53 -05:00
MAntoniak
0eda1f6c9f
connect: reduce number of transportation providers
Use only the ones necessary - the ones that are built-in. Saves a few
bytes in the resulting code.

Closes #12438
2023-12-01 23:14:07 +01:00
David Benjamin
56791f3e23
vtls: consistently use typedef names for OpenSSL structs
The foo_st names don't appear in OpenSSL public API documentation. The
FOO typedefs are more common. This header was already referencing
SSL_CTX via <openssl/ssl.h>. There is a comment about avoiding
<openssl/x509v3.h>, but OpenSSL actually declares all the typedefs in
<openssl/ossl_typ.h>, which is already included by <openssl/ssl.h> (and
every other OpenSSL header), so just use that. Though I've included it
just to be explicit.

(I'm also fairly sure including <openssl/ssl.h> already triggers the
Schannel conflicts anyway. The comment was probably just out of date.)

Closes #12439
2023-12-01 23:00:43 +01:00
Stefan Eissing
781cd3e864
ngtcp2: fix races in stream handling
- fix cases where ngtcp2 invokes callbacks on streams that
  nghttp3 has already forgotten. Ignore the NGHTTP3_ERR_STREAM_NOT_FOUND
  in these cases as it is normal behaviour.

Closes #12435
2023-12-01 17:52:36 +01:00
Stefan Eissing
83e4d61981
cf-socket: TCP trace output local address used in connect
Closes #12427
2023-12-01 10:18:19 +01:00
Daniel Stenberg
d1c2bb3d8d
multi: during ratelimit multi_getsock should return no sockets
... as there is nothing to wait for then, it just waits. Otherwise, this
causes much more CPU work and updates than necessary during ratelimit
periods.

Ref: https://curl.se/mail/lib-2023-11/0056.html
Closes #12430
2023-11-30 15:53:22 +01:00
Dmitry Karpov
6a0dc7cf23
transfer: abort pause send when connection is marked for closing
This handles cases of some bi-directional "upgrade" scenarios
(i.e. WebSockets) where sending is paused until some "upgrade" handshake
is completed, but server rejects the handshake and closes the
connection.

Closes #12428
2023-11-30 09:30:51 +01:00
Daniel Stenberg
395365ad2d
openssl: when a session-ID is reused, skip OCSP stapling
Fixes #12399
Reported-by: Alexey Larikov
Closes #12418
2023-11-28 22:59:10 +01:00
Daniel Stenberg
34e319954a
Curl_http_body: cleanup properly when Curl_getformdata errors
Reported-by: yushicheng7788 on github
Based-on-work-by: yushicheng7788 on github
Fixes #12410
Closes #12421
2023-11-28 22:57:22 +01:00
Daniel Stenberg
dbf4c4030a
conncache: use the closure handle when disconnecting surplus connections
Use the closure handle for disconnecting connection cache entries so
that anything that happens during the disconnect is not stored and
associated with the 'data' handle which already just finished a transfer
and it is important that details from the unrelated disconnect does not
taint meta-data in the data handle.

Like storing the response code.

This also adjust test 1506. Unfortunately it also removes a key part of
the test that verifies that a connection is closed since when this
output vanishes (because the closure handle is used), we don't know
exactly that the connection actually gets closed in this test...

Reported-by: ohyeaah on github
Fixes #12367
Closes #12405
2023-11-26 17:49:15 +01:00
Stefan Eissing
247defa753
quic: make eyeballers connect retries stop at weird replies
- when a connect immediately goes into DRAINING state, do
  not attempt retries in the QUIC connection filter. Instead,
  return CURLE_WEIRD_SERVER_REPLY
- When eyeballing, interpret CURLE_WEIRD_SERVER_REPLY as an
  inconclusive answer. When all addresses have been attempted,
  rewind the address list once on an inconclusive answer.
- refs #11832 where connects were retried indefinitely until
  the overall timeout fired

Closes #12400
2023-11-24 20:58:54 +01:00
Daniel Stenberg
3c30c165a8
sendf: fix comment typo 2023-11-24 14:36:25 +01:00
Daniel Stenberg
f27b8dba73
fopen: allocate the dir after fopen
Move the allocation of the directory name down to after the fopen() call
to allow that shortcut code path to avoid a superfluous malloc+free
cycle.

Follow-up to 73b65e94f3

Closes #12398
2023-11-24 13:24:08 +01:00
Stefan Eissing
5b65e7d1ae
transfer: cleanup done+excess handling
- add `SingleRequest->download_done` as indicator that
  all download bytes have been received
- remove `stop_reading` bool from readwrite functions
- move excess body handling into client download writer

Closes #12371
2023-11-24 13:22:07 +01:00
Daniel Stenberg
03cb1ff4d6
fopen: create new file using old file's mode
Because the function renames the temp file to the target name as a last
step, if the file was previously owned by a different user, not ORing
the old mode could otherwise end up creating a file that was no longer
readable by the original owner after save.

Reported-by: Loïc Yhuel
Fixes #12299
Closes #12395
2023-11-23 22:30:42 +01:00
Daniel Stenberg
73b65e94f3
fopen: create short(er) temporary file name
Only using random letters in the name plus a ".tmp" extension. Not by
appending characters to the final file name.

Reported-by: Maksymilian Arciemowicz

Closes #12388
2023-11-23 15:46:34 +01:00
Viktor Szakats
294194d689
openssl: enable infof_certstack for 1.1 and LibreSSL 3.6
Lower the barrier to enable `infof_certstack()` from OpenSSL 3 to
OpenSSL 1.1.x, and LibreSSL 3.6 or upper.

With the caveat, that "group name" and "type name" are missing from
the log output with these TLS backends.

Follow-up to b6e6d4ff8f #12030

Reviewed-by: Daniel Stenberg
Closes #12385
2023-11-23 12:41:48 +00:00
Daniel Stenberg
fafefdf9dc
urldata: fix typo in comment 2023-11-23 13:07:53 +01:00
Daniel Stenberg
0510e8b58c
lib: fix comment typos
Five separate ones, found by codespell

Closes #12390
2023-11-23 12:35:59 +01:00
Daniel Stenberg
2b0994c29a
cookie: lowercase the domain names before PSL checks
Reported-by: Harry Sintonen

Closes #12387
2023-11-23 10:08:56 +01:00
Viktor Szakats
006977859d
openssl: fix building with v3 no-deprecated + add CI test
- build quictls with `no-deprecated` in CI to have test coverage for
  this OpenSSL 3 configuration.

- don't call `OpenSSL_add_all_algorithms()`, `OpenSSL_add_all_digests()`.
  The caller code is meant for OpenSSL 3, while these two functions were
  only necessary before OpenSSL 1.1.0. They are missing from OpenSSL 3
  if built with option `no-deprecated`, causing build errors:
  ```
  vtls/openssl.c:4097:3: error: call to undeclared function 'OpenSSL_add_all_algorithms'; ISO C99 and later do not   support implicit function declarations [-Wimplicit-function-declaration]
  vtls/openssl.c:4098:3: error: call to undeclared function 'OpenSSL_add_all_digests'; ISO C99 and later do not   support implicit function declarations [-Wimplicit-function-declaration]
  ```
  Ref: https://ci.appveyor.com/project/curlorg/curl-for-win/builds/48587418?fullLog=true#L7667

  Regression from b6e6d4ff8f #12030
  Bug: https://github.com/curl/curl/issues/12380#issuecomment-1822944669
  Reviewed-by: Alex Bozarth

- vquic/curl_ngtcp2: fix using `SSL_get_peer_certificate` with
  `no-deprecated` quictls 3 builds.
  Do it by moving an existing solution for this from `vtls/openssl.c`
  to `vtls/openssl.h` and adjusting caller code.
  ```
  vquic/curl_ngtcp2.c:1950:19: error: implicit declaration of function 'SSL_get_peer_certificate'; did you mean   'SSL_get1_peer_certificate'? [-Wimplicit-function-declaration]
  ```
  Ref: https://github.com/curl/curl/actions/runs/6960723097/job/18940818625#step:24:1178

- curl_ntlm_core: fix `-Wunused-parameter`, `-Wunused-variable` and
  `-Wunused-function` when trying to build curl with NTLM enabled but
  without the necessary TLS backend (with DES) support.

Closes #12384
2023-11-23 02:03:02 +00:00
Viktor Szakats
e9a7d4a1c8
windows: use built-in _WIN32 macro to detect Windows
Windows compilers define `_WIN32` automatically. Windows SDK headers
or build env defines `WIN32`, or we have to take care of it. The
agreement seems to be that `_WIN32` is the preferred practice here.
Make the source code rely on that to detect we're building for Windows.

Public `curl.h` was using `WIN32`, `__WIN32__` and `CURL_WIN32` for
Windows detection, next to the official `_WIN32`. After this patch it
only uses `_WIN32` for this. Also, make it stop defining `CURL_WIN32`.

There is a slight chance these break compatibility with Windows
compilers that fail to define `_WIN32`. I'm not aware of any obsolete
or modern compiler affected, but in case there is one, one possible
solution is to define this macro manually.

grepping for `WIN32` remains useful to discover Windows-specific code.

Also:

- extend `checksrc` to ensure we're not using `WIN32` anymore.

- apply minor formatting here and there.

- delete unnecessary checks for `!MSDOS` when `_WIN32` is present.

Co-authored-by: Jay Satiro
Reviewed-by: Daniel Stenberg

Closes #12376
2023-11-22 15:42:25 +00:00
Stefan Eissing
1c37d472a0
url: ConnectionExists revisited
- have common pattern of `if not match, continue`
- revert pages long if()s to return early
- move dead connection check to later since it may
  be relatively expensive
- check multiuse also when NOT building with NGHTTP2
- for MULTIUSE bundles, verify that the inspected
  connection indeed supports multiplexing when in use
  (bundles may contain a mix of connection, afaict)

Closes #12373
2023-11-22 15:10:58 +01:00
Daniel Stenberg
9a8bb20d48
CURLMOPT_MAX_CONCURRENT_STREAMS: make sure the set value is within range
... or use the default value.

Also clarify the documentation language somewhat.

Closes #12382
2023-11-22 15:01:04 +01:00
Daniel Stenberg
2d06eebf28
urldata: make maxconnects a 32 bit value
"2^32 idle connections ought to be enough for anybody"

Closes #12375
2023-11-22 13:27:38 +01:00
Daniel Stenberg
cfdcdc8d2f
wolfssh: remove redundant static prototypes
vssh/wolfssh.c:346:18: error: redundant redeclaration of ‘wscp_recv’ [-Werror=redundant-decls]

Closes #12381
2023-11-22 09:39:08 +01:00
Daniel Stenberg
92efb39c00
setopt: remove superfluous use of ternary expressions
Closes #12374
2023-11-22 07:44:22 +01:00
Daniel Stenberg
7f766637aa
mime: store "form escape" as a single bit
Closes #12374
2023-11-22 07:44:18 +01:00