Use `data->progress.now` as the timestamp of proecssing a transfer.
Update it on significant events and refrain from calling `curlx_now()`
in many places.
The problem this addresses is
a) calling curlx_now() has costs, depending on platform. Calling it
every time results in 25% increase `./runtest` duration on macOS.
b) we used to pass a `struct curltime *` around to save on calls, but
when some method directly use `curx_now()` and some use the passed
pointer, the transfer experienes non-linear time. This results in
timeline checks to report events in the wrong order.
By keeping a timestamp in the easy handle and updating it there, no
longer invoking `curlx_now()` in the "lower" methods, the transfer
can observer a steady clock progression.
Add documentation in docs/internals/TIME-KEEPING.md
Reported-by: Viktor Szakats
Fixes#19935Closes#19961
OpenSSL 4 has plans to make ASN1_STRING opaque, which will break the
build, so convert the code to use accessors. ASN1_STRING_length() and
ASN1_STRING_type() go way back to SSLeay and ASN1_STRING_get0_data() is
OpenSSL 1.1 API present in BoringSSL since foreer and also available
since LibreSSL 2.7, so this should not cause compat issues with any
libcrypto in a supported version of the fork family.
https://github.com/openssl/openssl/issues/29117Closes#19831
Before this patch curl used the C preprocessor to override standard
memory allocation symbols: malloc, calloc, strdup, realloc, free.
The goal of these is to replace them with curl's debug wrappers in
`CURLDEBUG` builds, another was to replace them with the wrappers
calling user-defined allocators in libcurl. This solution needed a bunch
of workarounds to avoid breaking external headers: it relied on include
order to do the overriding last. For "unity" builds it needed to reset
overrides before external includes. Also in test apps, which are always
built as single source files. It also needed the `(symbol)` trick
to avoid overrides in some places. This would still not fix cases where
the standard symbols were macros. It was also fragile and difficult
to figure out which was the actual function behind an alloc or free call
in a specific piece of code. This in turn caused bugs where the wrong
allocator was accidentally called.
To avoid these problems, this patch replaces this solution with
`curlx_`-prefixed allocator macros, and mapping them _once_ to either
the libcurl wrappers, the debug wrappers or the standard ones, matching
the rest of the code in libtests.
This concludes the long journey to avoid redefining standard functions
in the curl codebase.
Note: I did not update `packages/OS400/*.c` sources. They did not
`#include` `curl_setup.h`, `curl_memory.h` or `memdebug.h`, meaning
the overrides were never applied to them. This may or may not have been
correct. For now I suppressed the direct use of standard allocators
via a local `.checksrc`. Probably they (except for `curlcl.c`) should be
updated to include `curl_setup.h` and use the `curlx_` macros.
This patch changes mappings in two places:
- `lib/curl_threads.c` in libtests: Before this patch it mapped to
libcurl allocators. After, it maps to standard allocators, like
the rest of libtests code.
- `units`: before this patch it mapped to standard allocators. After, it
maps to libcurl allocators.
Also:
- drop all position-dependent `curl_memory.h` and `memdebug.h` includes,
and delete the now unnecessary headers.
- rename `Curl_tcsdup` macro to `curlx_tcsdup` and define like the other
allocators.
- map `curlx_strdup()` to `_strdup()` on Windows (was: `strdup()`).
To fix warnings silenced via `_CRT_NONSTDC_NO_DEPRECATE`.
- multibyte: map `curlx_convert_*()` to `_strdup()` on Windows
(was: `strdup()`).
- src: do not reuse the `strdup` name for the local replacement.
- lib509: call `_strdup()` on Windows (was: `strdup()`).
- test1132: delete test obsoleted by this patch.
- CHECKSRC.md: update text for `SNPRINTF`.
- checksrc: ban standard allocator symbols.
Follow-up to b12da22db1#18866
Follow-up to db98daab05#18844
Follow-up to 4deea9396b#18814
Follow-up to 9678ff5b1b#18776
Follow-up to 10bac43b87#18774
Follow-up to 20142f5d06#18634
Follow-up to bf7375ecc5#18503
Follow-up to 9863599d69#18502
Follow-up to 3bb5e58c10#17827Closes#19626
Change the send parameter from `const void *` to `const uint8_t *` and
adapt calling code. Several had already unsigned chars and were casting.
Closes#19729
Change `inputbuff` parameter from `const char *` to `const uint8_t *` to
reflect the binary nature of the input bytes. Half the code was casting
unsigned char to signed already in calling.
Closes#19722
Add protocol handler flag `PROTOPT_CONN_REUSE` to indicate that the
protocol allows reusing connections for other tranfers. Add that
to all handlers that support it.
Create connections with `conn->bits.close = FALSE` and remove all
the `connkeep()` calls in protocol handlers setup/connect implementations.
`PROTOPT_CONN_REUSE` assures that the default behaviour applies
at the end of a transfer without need to juggle the close bit.
`conn->bits.close` now serves as an additional indication that a
connection cannot be reused. Only protocol handles that allow
reuse need to set it to override the default behaviour.
Remove all `connclose()` and `connkeep()` calls from connection
filters. Filters should not modify connection flags. They are
supposed to run in eyeballing situations where a filter is just
one of many determining the outcome.
Fix http response header handling to only honour `Connection: close`
for HTTP/1.x versions.
Closes#19333
When OpenSSL fails to verify the peer certificate, we checked for
one specific reason code and did not ask Apple SecTrust for any
other failure.
Always ask Apple SecTrust after OpenSSL fails when the `native_ca_store`
is enabled. If the user configures a CAfile or CApath, the native store
is disabled, so this does not affect use cases where users asks curl
to use a specific set of trust anchors.
Do the same for GnuTLS
Fixes#19636
Reported-by: ffath-vo on github
Closes#19638
If verifypeer and verifyhost are disabled, to not generate
a failf() message for failed verifications.
Fixes#19615
Reported-by: ncaklovic on github
Closes#19625
It also means that all supported OpenSSL versions and forks support
TLSv1.3 after this patch.
It reduces `openssl.c` size by more than 10%, or 400 LOC.
Ref: #18822Closes#18330
- badwords.pl: add `-a` option to check all lines in source code files.
Before this patch indented lines were skipped (to avoid Markdown code
fences.)
- GHA/checksrc: use `-a` when verifying the source code.
- GHA/checksrc: disable `So` and `But` rules for source code.
- GHA/checksrc: add docs/examples to the verified sources.
- badwords.txt: delete 4 duplicates.
- badwords.txt: group and sort contractions.
- badwords.txt: allow ` url = `, `DIR`, `<file name`.
Closes#19536
Rename `Curl_timeleft()` to `Curl_timeleft_ms()` to make the units in
the returned `timediff_t` clear. (We used to always have ms there, but
with QUIC started to sometimes calc ns as well).
Rename some assigned vars without `_ms` suffix for clarity as well.
Closes#19486
Also:
- replace `manpage` with `man page`, add to `badwords.txt`.
- badwords.pl: import `-w` feature from curl-www, syncing the two
scripts fully.
- badwords.txt: import missing items from curl-www, syncing the two
files fully.
- pyspelling.words: drop `cURL` allowed word.
Closes#19468
find_ssl_filter used while(cf && cf->next) and skipped the last node.
If the SSL filter was last, channel binding lookup failed and we returned
CURLE_BAD_FUNCTION_ARGUMENT. Switch to while(cf) so the tail is examined.
This bug was found with ZeroPath.
Closes#19229
The definition of these constants does not give a numeric ordering
and MAX_DEFAULT needs to be checked in addition of ciphers and QUIC
checks to apply correctly.
Fixes#19340
Reported-by: Peter Piekarski
Closes#19341
When openssl does not verify the certificate, but apple sectrust
does, we also pass it the ocsp stapled response when configured and
available.
When openssl does not verify the cert chain, it will also not be able
to verify the ocsp stapling. Do not call it if sectrust is the
verifier of the cert chain.
Fixes#19307
Reported-by: Harry Sintonen
Closes#19308
Replace the check if a ssl session cache is configured with
a function checking if it is configured *and* if an ssl session
cache is available.
During normal operations, a session cache is always there, however
for "connect-only" transfers this might not be the case. When such
transfers receive new sessions/tickets, they need to silently
discard those and not fail.
Reported-by: Marc Aldorasi
Fixes https://github.com/curl/curl/issues/18983
Closes https://github.com/curl/curl/pull/19251
Curl is one of the last callers of PKCS12_PBE_add(). It has been a noop
since OpenSSL 0.9.8k (2006) stubbed it out when moving the built-in PBE
algorithms to a static table:
b8f702a0afCloses#19201
In providercheck(), when failing to open the "store", the exit path
would not previously free the created UI_METHOD and instead leak this
resource.
Pointed out by ZeroPath
Closes#19114
After this patch, the codebase no longer overrides system printf
functions. Instead it explicitly calls either the curl printf functions
`curl_m*printf()` or the system ones using their original names.
Also:
- drop unused `curl_printf.h` includes.
- checksrc: ban system printf functions, allow where necessary.
Follow-up to db98daab05#18844
Follow-up to 4deea9396b#18814Closes#18866
Also:
- tests/server: replace local `sstrerror()` with `curlx_strerror()`.
- tests/server: show the error code next to the string, where missing.
- curlx: use `curl_msnprintf()` when building for src and tests.
(units was already using it.)
- lib: drop unused includes found along the way.
- curlx_strerror(): avoid compiler warning (and another similar one):
```
In file included from servers.c:14:
../../lib/../../lib/curlx/strerr.c: In function ‘curlx_strerror’:
../../lib/../../lib/curlx/strerr.c:328:32: error: ‘snprintf’ output may be truncated before the last format character [-Werror=format-truncation=]
328 | SNPRINTF(buf, buflen, "%s", msg);
| ^
../../lib/../../lib/curlx/strerr.c:47:18: note: ‘snprintf’ output 1 or more bytes (assuming 2) into a destination of size 1
47 | #define SNPRINTF snprintf
| ^
../../lib/../../lib/curlx/strerr.c:328:7: note: in expansion of macro ‘SNPRINTF’
328 | SNPRINTF(buf, buflen, "%s", msg);
| ^~~~~~~~
```
Follow-up to 45438c8d6f#18823Closes#18840
```
lib/vtls/openssl.c: In function 'asn1_object_dump':
lib/vtls/openssl.c:299:42: error: passing argument 3 of 'i2t_ASN1_OBJECT' discards 'const' qualifier from pointer target type [-Werror=discarded-qualifiers]
299 | int i = i2t_ASN1_OBJECT(buf, (int)len, a);
| ^
In file included from /home/runner/djgpp/include/openssl/objects.h:965,
from /home/runner/djgpp/include/openssl/evp.h:94,
from /home/runner/djgpp/include/openssl/x509.h:73,
from /home/runner/djgpp/include/openssl/ssl.h:156,
from lib/curl_ntlm_core.c:71,
from bld/lib/CMakeFiles/libcurl_static.dir/Unity/unity_0_c.c:88:
/home/runner/djgpp/include/openssl/asn1.h:921:58: note: expected 'ASN1_OBJECT *' {aka 'struct asn1_object_st *'} but argument is of type 'const ASN1_OBJECT *' {aka 'const struct asn1_object_st *'}
921 | int i2t_ASN1_OBJECT(char *buf, int buf_len, ASN1_OBJECT *a);
| ~~~~~~~~~~~~~^
```
Ref: https://github.com/curl/curl/actions/runs/18236773678/job/51931937131?pr=18039
Follow-up to bb46d42407#18647Closes#18841
- configure/cmake support for enabling the option
- supported in OpenSSL and GnuTLS backends
- when configured, Apple SecTrust is the default trust store
for peer verification. When one of the CURLOPT_* for adding
certificates is used, that default does not apply.
- add documentation of build options and SSL use
Closes#18703
- it's just too random who got mentioned
- we can't mention all, so better consistently mention none
- make sure they all are mentioned in THANKS
- also remove some unnecessary comment ramblings
Closes#18803
- GHA/checkdocs: rename `spellcheck` job to `pyspelling` to say
the exact tool used.
- GHA/checkdocs: restore a comment.
- GHA/linux: add `-B .` to a cmake configure to avoid warning, and
future breakage.
- autotools: use correct casing for `Schannel`.
- doh: update RFC URL.
- drop redundant parenthesis.
- fix indentation, whitespace.
Closes#18756
Since it would indicate errors to the degree that continuing would just
risk hiding the earlier errors or make things weird.
Inspired by a report in Joshua's sarif data
Closes#18646
When OpenSSL reports SSL_ERROR_WANT_READ, set the io_need explicitly.
It should have already been set by the BIO, but be safe.
Reported in Joshua's sarif data
Closes#18733
