curl: add --rate to set max request rate per time unit

--rate "12/m" - for 12 per minute or
--rate "5/h" - for 5 per hour

Removed from TODO

Closes #8671
This commit is contained in:
Daniel Stenberg 2022-05-23 17:59:56 +02:00
parent 64641d2170
commit 8f48b5d783
No known key found for this signature in database
GPG key ID: 5CC908FDB71E12C2
8 changed files with 103 additions and 15 deletions

View file

@ -149,7 +149,6 @@
18.4 --proxycommand
18.5 UTF-8 filenames in Content-Disposition
18.6 Option to make -Z merge lined based outputs on stdout
18.7 at least N milliseconds between requests
18.8 Consider convenience options for JSON and XML?
18.9 Choose the name of file in braces for complex URLs
18.10 improve how curl works in a windows console window
@ -999,17 +998,6 @@
https://github.com/curl/curl/issues/5175
18.7 at least N milliseconds between requests
Allow curl command lines issue a lot of request against services that limit
users to no more than N requests/second or similar. Could be implemented with
an option asking that at least a certain time has elapsed since the previous
request before the next one will be performed. Example:
$ curl "https://example.com/api?input=[1-1000]" -d yadayada --after 500
See https://github.com/curl/curl/issues/3920
18.8 Consider convenience options for JSON and XML?
Could we add `--xml` or `--json` to add headers needed to call rest API:

View file

@ -198,6 +198,7 @@ DPAGES = \
quote.d \
random-file.d \
range.d \
rate.d \
raw.d \
referer.d \
remote-header-name.d \

33
docs/cmdline-opts/rate.d Normal file
View file

@ -0,0 +1,33 @@
Long: rate
Arg: <max request rate>
Help: Request rate for serial transfers
Category: connection
Example: --rate 2/s $URL
Example: --rate 3/h $URL
Example: --rate 14/m $URL
Added: 7.84.0
See-also: limit-rate retry-delay
---
Specify the maximum transfer frequency you allow curl to use - in number of
transfer starts per time unit (sometimes called request rate). Without this
option, curl will start the next transfer as fast as possible.
If given several URLs and a transfer completes faster than the allowed rate,
curl will wait until the next transfer is started to maintain the requested
rate. This option has no effect when --parallel is used.
The request rate is provided as "N/U" where N is an integer number and U is a
time unit. Supported units are 's' (second), 'm' (minute), 'h' (hour) and 'd'
/(day, as in a 24 hour unit). The default time unit, if no "/U" is provided,
is number of transfers per hour.
If curl is told to allow 10 requests per minute, it will not start the next
request until 6 seconds have elapsed since the previous transfer was started.
This function uses millisecond resolution. If the allowed frequency is set
more than 1000 per second, it will instead run unrestricted.
When retrying transfers, enabled with --retry, the separate retry delay logic
is used and not this setting.
If this option is used several times, the last one will be used.

View file

@ -186,6 +186,7 @@
--quote (-Q) 5.3
--random-file 7.7
--range (-r) 4.0
--rate 7.84.0
--raw 7.16.2
--referer (-e) 4.0
--remote-header-name (-J) 7.20.0