Change multi's book keeping of transfers to no longer use lists, but a
special table and bitsets for unsigned int values.
`multi-xfers` is the `uint_tbl` where `multi_add_handle()` inserts a new
transfer which assigns it a unique identifier `mid`. Use bitsets to keep
track of transfers that are in state "process" or "pending" or
"msgsent".
Use sparse bitsets to replace `conn->easyq` and event handlings tracking
of transfers per socket. Instead of pointers, keep the mids involved.
Provide base data structures and document them in docs/internal:
* `uint_tbl`: a table of transfers with `mid` as lookup key,
handing out a mid for adds between 0 - capacity.
* `uint_bset`: a bitset keeping unsigned ints from 0 - capacity.
* `uint_spbset`: a sparse bitset for keeping a small number of
unsigned int values
* `uint_hash`: for associating `mid`s with a pointer.
This makes the `mid` the recommended way to refer to transfers inside
the same multi without risk of running into a UAF.
Modifying table and bitsets is safe while iterating over them. Overall
memory requirements are lower as with the double linked list apprach.
Closes#16761
When handling connections (not transfers) related events, always use the
passed `conn` and not `data->conn` as the transfer does not need to have
the same connection.
Fix handling of conn pollset diffs to disregard the transfer.
Closes#16782
The issues found fell into these categories, with the applied fixes:
- const was accidentally stripped.
Adjust code to not cast or cast with const.
- const/volatile missing from arguments, local variables.
Constify arguments or variables, adjust/delete casts. Small code
changes in a few places.
- const must be stripped because an API dependency requires it.
Strip `const` with `CURL_UNCONST()` macro to silence the warning out
of our control. These happen at API boundaries. Sometimes they depend
on dependency version, which this patch handles as necessary. Also
enable const support for the zlib API, using `ZLIB_CONST`. Supported
by zlib 1.2.5.2 and newer.
- const must be stripped because a curl API requires it.
Strip `const` with `CURL_UNCONST()` macro to silence the warning out
of our immediate control. For example we promise to send a non-const
argument to a callback, though the data is const internally.
- other cases where we may avoid const stripping by code changes.
Also silenced with `CURL_UNCONST()`.
- there are 3 places where `CURL_UNCONST()` is cast again to const.
To silence this type of warning:
```
lib/vquic/curl_osslq.c:1015:29: error: to be safe all intermediate
pointers in cast from 'unsigned char **' to 'const unsigned char **'
must be 'const' qualified [-Werror=cast-qual]
lib/cf-socket.c:734:32: error: to be safe all intermediate pointers in
cast from 'char **' to 'const char **' must be 'const' qualified
[-Werror=cast-qual]
```
There may be a better solution, but I couldn't find it.
These cases are handled in separate subcommits, but without further
markup.
If you see a `-Wcast-qual` warning in curl, we appreciate your report
about it.
Closes#16142
`data->id` is unique in the same connection pool, but a multi may
involved more than one pool. `data->mid` is unique inside the multi and
since multi_ev lives inside one multi, the `mid` is the right thing to
use.
Closes#16545
Add a standalong hash table for curl_offt_t as key. This allows a
smaller memory footprint and faster lookups as we do not need to deal
with variable key lengths.
Use in all places we had the standard hash for this purpose.
Closes#16442
Rework the event based handling of transfers and connections to
be "localized" into a single source file with clearer dependencies.
- add multi_ev.c and multi_ev.h
- add docs/internal/MULTI-EV.md to explain the overall workings
- only do event handling book keeping when the socket callback
is set
- add handling for "connection only" event tracking, when internal
easy handles are used that are not really tied to a connection.
Used in connection pool.
- remove transfer member "last_poll" and connections "shutdown_poll"
and keep all that internal to multi_ev.c
- add CURL_TRC_M() for tracing of "multi" related things, including
event handling and connection pool operations. Add new trace
feature "multi" for trace config.
multi traces will show exactly what is going on in regard to
event handling.
- multi: trace transfers "mstate" in every CURL_TRC_M() call
- make internal trace buffer 2048 bytes and end the silliness
with +n here -m there. Adjust test 1652 expectations of resulting
length and input edge cases.
- add trace feature "lib-ids" to perfix libcurl traces with transfer
and connection ids. Useful for debugging libcurl applications.
Closes#16308
