- Checks for missing explicit `return` statements at the end of functions
that can return non-`None` values.
- Checks for classes that inherit from `object`.
- Checks for useless expressions.
- Within an `except*` clause, raise exceptions with `raise ... from err`
or `raise ... from None` to distinguish them from errors in exception
handling
- Checks for variable assignments that immediately precede a `return` of the
assigned variable.
- Checks for `else` statements with a `return` statement in the preceding
`if` block.
- Checks for unnecessary parentheses on raised exceptions.
Closes: #21258
Write out 9-length frames to client's WRITEFUNCTION
Read 0-length frames from READFUNCTION *if* the function
started a new frame via `curl_ws_start_frame()`.
Fixes#18286Closes#18332
Reported-by: Andriy Druk
Add support for CURLOPT_READFUNCTION with WebSocket urls when *not* in
connect-only mode, e.g. when using curl_multi_perform.
Install the callback function and set CURLOPT_UPLOAD. Return
CURL_READFUNC_PAUSE when having nothing more to send and unpause the
transfer when more data is ready.
This will send the read bytes in a WebSocket BINARY frame.
Add support for this mode in the pytest "ws_data" client and have all
tests run in 'curl_ws_send/recv' and 'peform' mode as well.
Add `curl_ws_start_frame()`. Document, cover in libcurl-ws.md and
explain the READFUNCTION mode for websockets.
Add example `websocket-updown` for this.
Closes#17683
libtests and clients were built the same way after recent overhauls.
libtests are used by runtests, clients by pytests.
Merge clients into libtests, aligning their entry function signature,
dropping common utility functions, and simplifying the build.
Note: After this patch `CURLDEBUG` applies to cli tests, when enabled.
Also:
- lib552: drop local copy-paste debug callback in favor of testtrace.
- lib552: drop local copy-paste dump function in favor of testtrace.
- clients: use `long` for HTTP version, drop casts.
- clients: replace local dump function in favor of testrace clone.
- sync cli test entry function prototype with libtests'.
- h2_serverpush: replace local trace callback with testtrace.
- de-duplicate 3 websocket close, ping, ping, functions. Kept the pong
iteration from `ws_pingpong`. Note: the pong clone in `lib2304` was
returning an error when `curl_ws_recv()` returned non-zero and
the payload matched the expected one anyway. After this patch, this
case returns success, as it does in `ws_pingpong`.
`lib2304` keeps passing, but I'm not sure if the previous behavior
was intentional.
- display full value in websocket close, ping, pong, drop casts.
Closes#18079
To make building the http client tests faster, with no duplication, by
using the build method that other test binaries already use.
The difference compared to other tests is that these don't use internal
libcurl headers or code. With the exception of `curl_config.h`, for
a feature macro.
Before this patch, these tests were built like examples.
Also:
- de-duplicate code and give unique names to colliding symbols.
- add local getopt implementation and enable all code for MSVC.
Adapted for curl via Public Domain source:
4e618ef782/getopt.h
Credits-to: Christopher Wellons
Thanks!
Closes#17627
This started out as regression tests for the `curl_ws_recv()` and
`curl_ws_send()` implementation and ended up with a bugfix, additional
protocol validation and minor logging improvements.
- Fix reset of fragmented message decoder state when a PING/PONG is
received in between message fragments.
- Fix undefined behavior (applying zero offset to null pointer) in
curl_ws_send() when the given buffer is NULL.
- Detect invalid overlong PING/PONG/CLOSE frames.
- Detect invalid fragmented PING/PONG/CLOSE frames.
- Detect invalid sequences of fragmented frames.
- a) A continuation frame (0x80...) is received without any ongoing
fragmented message.
- b) A new fragmented message is started (0x81/0x01/0x82/0x02...)
before the ongoing fragmented message has terminated.
- Made logs for invalid opcodes easier to understand.
- Moved noisy logs to the `CURL_TRC_WS` log level.
- Unified the prefixes for WebSocket log messages: `[WS] ...`
- Add env var `CURL_WS_FORCE_ZERO_MASK` in debug builds.
- If set, it forces the bit mask applied to outgoing payloads to
0x00000000, which effectively means the payload is not masked at
all. This drastically simplifies defining the expected `<protocol>`
data in test cases.
- 2700: Frame types
- 2701: Invalid opcode 0x3
- 2702: Invalid opcode 0xB
- 2703: Invalid reserved bit RSV1 _(replaces 2310)_
- 2704: Invalid reserved bit RSV2
- 2705: Invalid reserved bit RSV3
- 2706: Invalid masked server message
- 2707: Peculiar frame sizes _(part. replaces 2311)_
- 2708: Automatic PONG
- 2709: No automatic PONG _(replaces 2312)_
- 2710: Unsolicited PONG
- 2711: Empty PING/PONG/CLOSE
- 2712: Max sized PING/PONG/CLOSE
- 2713: Invalid oversized PING _(replaces 2307)_
- 2714: Invalid oversized PONG
- 2715: Invalid oversized CLOSE
- 2716: Invalid fragmented PING
- 2717: Invalid fragmented PONG
- 2718: Invalid fragmented CLOSE
- 2719: Fragmented messages _(part. replaces 2311)_
- 2720: Fragmented messages with empty fragments
- 2721: Fragmented messages with interleaved pong
- 2722: Invalid fragmented message without initial frame
- 2723: Invalid fragmented message without final frame
- 2305: curl_ws_recv() loop reading three larger frames
- This test involuntarily sent an invalid sequence of opcodes (0x01...,0x01...,0x81...) , but neither libcurl nor the test caught this! The correct sequence was tested in 2311 (0x01...,0x00...,0x80...). See below for 2311.
- Validation of the opcode sequence was added to libcurl and is now tested in 2723.
- Superseded by 2719 (fragmented message) and 2707 (large frames).
- 2307: overlong PING payload
- The tested PING payload length check was actually missing, but the test didn't catch this since it involuntarily sent an invalid opcode (0x19... instead of 0x89...) so that the expected error occurred, but for the wrong reason.
- Superseded by 2713.
- 2310: unknown reserved bit set in frame header
- Superseded by 2703 and extended by 2704 and 2705.
- 2311: curl_ws_recv() read fragmented message
- Superseded by 2719 (fragmented message) and 2707 (large frames).
- 2312: WebSockets no auto ping
- Superseded by 2709.
- No tests for `CURLOPT_WRITEFUNCTION`.
- No tests for sending of invalid frames/fragments.
Closes#17136
They take about 4.5 minutes of CI time in GHA/macos.
Also:
- autotools: improve `caddy`, `vsftpd` detection.
Bringing it closer to cmake.
- autotools: fix `--with-test-caddy=no`, `--with-test-vsftps=no`,
`--with-test-nghttpx=no` options.
- cmake: sync `nghttpx` default with autotools.
- pytest: disable failing mbedTLS tests on macOS CI.
- pytest: disable failing earlydata tests on macOS CI.
- GHA/macos: keep vsftpd pytests disabled due to lengthy run times.
- pytest: fix test_05_04 for LibreSSL. Ref: #17367
Authored-by: Stefan Eissing
Remaining issues:
- some unidentified tests taking a long time with mbedTLS:
`================= 462 passed, 278 skipped in 347.93s (0:05:47) =================`
Ref: https://github.com/curl/curl/actions/runs/15073354301/job/42374999041#step:17:1536
Workaround: not enabling pytest for mbedTLS jobs
- 17 FTP tests taking a long time (affecting all TLS backends):
without vsftpd:
`====================== 496 passed, 244 skipped in 56.15s =======================`
Ref: https://github.com/curl/curl/actions/runs/15073354301/job/42374998230#step:17:1536
with vsftpd:
`================= 513 passed, 227 skipped in 409.37s (0:06:49) =================`
Ref: https://github.com/curl/curl/actions/runs/15073678568/job/42376039672?pr=17362#step:17:1537
Workaround: force-disable vsftpd.
- 100 tests failing with SecureTransport. Let's ignore that due to imminent deprecation.
Ref: https://github.com/curl/curl/actions/runs/15055652333/job/42320873732#step:17:15362
Follow-up to 30ef79ed93#17295
Follow-up to 9147903366#16518Closes#17362
Require now pytest-xdist from tests/http/requirements.txt and
run pytest in 'auto' parallel mode (counts cpu cores).
For CI runs, set the worker count to 4, overriding the
core count of 2 exposed in the images.
- use Filelock to generate allocated ports at start for all
workers and have subsequent workers just read the file and
take the ports for their slot
- make httpd config clearing a function fixture so every test
starts with a clean httpd config
- have fixture `configures_httpd` as parameter of test cases
that configure httpd anyway, saving one reload
- add pytest-xdist and filelock to required pyhton modules
- add installs to ruff CI
- give live checks waiting for a server to start up longer time
- add fixtures to tests that rely on a server
- do not stop servers unnecessarily. failures may not start them
properly again, leading to unexpected fails in whatever follows
- add a https: port to httpd that is *not* back by QUIC to allow
failover tests without stopping the QUIC server
Closes#17295
- Fix a bug in EAGAIN handling when sending frames that led to a
corrupted last byte of the frame sent.
- Restore sanity to curl_ws_send() behaviour:
- Partial writes are reported as OK with the actual number of
payload bytes sent.
- CURLE_AGAIN is only returned when none of the payload bytes
(or for 0-length frames, not all of the frame header bytes)
could be sent.
- curl_ws_send() now behaves like a common send() call.
- Change 'ws-data' test client to allow concurrent send/recv
operations and vary frame sizes and repeat count.
- Add DEBUG env var CURL_WS_CHUNK_EAGAIN to simulate blocking
after a chunk of an encoded websocket frame has been sent.
- Add tests.
Prior to this change data corruption may occur when sending websocket
messages due to two bugs:
1) 3e64569a (precedes 8.10.0) caused a data corruption bug in the last
byte of frame of large messages.
2) curl_ws_send had non-traditional send behavior and could return
CURLE_AGAIN with bytes sent and expect the caller to adjust buffer
and buflen in a subsequent call. That behavior was not documented.
Reported-by: na-trium-144@users.noreply.github.com
Fixes https://github.com/curl/curl/issues/15865
Fixes https://github.com/curl/curl/issues/15865#issuecomment-2569870144
Closes https://github.com/curl/curl/pull/15901
- when data arrived in several chunks, the collection into
the passed buffer always started at offset 0, overwriting
the data already there.
adding test_20_07 to verify fix
- debug environment var CURL_WS_CHUNK_SIZE can be used to
influence the buffer chunk size used for en-/decoding.
Closes#12945
- Makefile support for building test specific clients in tests/http/clients
- auto-make of clients when invoking pytest
- added test_09_02 for server PUSH_PROMISEs using clients/h2-serverpush
- added test_02_21 for lib based downloads and pausing/unpausing transfers
curl url parser:
- added internal method `curl_url_set_authority()` for setting the
authority part of a url (used for PUSH_PROMISE)
http2:
- made logging of PUSH_PROMISE handling nicer
Placing python test requirements in requirements.txt files
- separate files to base test suite and http tests since use
and module lists differ
- using the files in the gh workflows
websocket test cases, fixes for we and bufq
- bufq: account for spare chunks in space calculation
- bufq: reset chunks that are skipped empty
- ws: correctly encode frames with 126 bytes payload
- ws: update frame meta information on first call of collect
callback that fills user buffer
- test client ws-data: some test/reporting improvements
Closes#11006
