From 2238f0921cb00b33958470e30dff6326ea6d5c65 Mon Sep 17 00:00:00 2001 From: Daniel Stenberg Date: Wed, 22 Apr 2026 00:52:16 +0200 Subject: [PATCH] curl: named globs in output file name for upload glob references Use parts of text from the upload filename field when that uses globbing by giving it a name the same way we do it for URL globs. For example, if you upload three files to a HTTP URL and want to save the corresponding responses in separate files: curl -T 'file{1,2,3}' https://upload.example/ -o 'response-#' Verified by test 2014 Closes #21407 --- docs/cmdline-opts/output.md | 22 +++++--- docs/cmdline-opts/upload-file.md | 26 ++++++--- src/tool_operate.c | 8 ++- src/tool_urlglob.c | 9 ++- src/tool_urlglob.h | 3 +- tests/data/Makefile.am | 2 +- tests/data/test2014 | 97 ++++++++++++++++++++++++++++++++ 7 files changed, 145 insertions(+), 22 deletions(-) create mode 100644 tests/data/test2014 diff --git a/docs/cmdline-opts/output.md b/docs/cmdline-opts/output.md index c1d823e324..b2d196d038 100644 --- a/docs/cmdline-opts/output.md +++ b/docs/cmdline-opts/output.md @@ -23,10 +23,10 @@ Example: # `--output` -Write output to the given file instead of stdout. If you are using globbing to -fetch multiple documents, you should quote the URL and you can use `#` -followed by a number in the filename. That variable is then replaced with the -current string for the URL being fetched. Like in: +Write output to the given file instead of stdout. If you are using globbing in +the URL to fetch multiple documents, you should quote the URL and you can use +`#` followed by a number in the filename. That variable gets replaced with the +current glob text. Like in: curl "http://{one,two}.example.com" -o "file_#1.txt" @@ -70,9 +70,9 @@ override curl's internal binary output in terminal prevention: Note that the binary output may be caused by the response being compressed, in which case you may want to use the --compressed option. -Starting in curl 8.21.0, the separate globbing parts can be named and -referenced by their names. The case sensitive alphanumeric name is set -enclosed within angle brackets after the opening character. Examples: +Since curl 8.21.0, the separate globbing parts can be named and referenced by +their names. The case sensitive alphanumeric name is set enclosed within angle +brackets after the opening character. Examples: curl "https://fun.example/{one,two}.jpg" -o "save-#" @@ -80,3 +80,11 @@ enclosed within angle brackets after the opening character. Examples: -o "save-#.txt" Referencing a named glob that is not set, causes an error. + +Since curl 8.21.0, you can use parts of the upload filename when it uses +globbing by setting a glob name and referencing it the same way you reference +named URL globs. For example, if you upload three files to a single fixed HTTP +URL and want to save the corresponding responses in separate files: + + curl -T 'file{1,2,3}' \ + https://upload.example/ -o 'response-#' diff --git a/docs/cmdline-opts/upload-file.md b/docs/cmdline-opts/upload-file.md index 5a2842e58a..1988d8afb7 100644 --- a/docs/cmdline-opts/upload-file.md +++ b/docs/cmdline-opts/upload-file.md @@ -26,13 +26,13 @@ Upload the specified local file to the remote URL. If there is no file part in the specified URL, curl appends the local file name to the end of the URL before the operation starts. You must use a -trailing slash (/) on the last directory to prove to curl that there is no +trailing slash (`/`) on the last directory to prove to curl that there is no filename or curl thinks that your last directory name is the remote filename to use. When putting the local filename at the end of the URL, curl ignores what is on -the left side of any slash (/) or backslash (\\) used in the filename and only -appends what is on the right side of the rightmost such character. +the left side of any slash (`/`) or backslash (`\\`) used in the filename and +only appends what is on the right side of the rightmost such character. Use the filename `-` (a single dash) to use stdin instead of a given file. Alternately, the filename `.` (a single period) may be specified instead of @@ -45,9 +45,19 @@ You can specify one --upload-file for each URL on the command line. Each --upload-file + URL pair specifies what to upload and to where. curl also supports globbing of the --upload-file argument, meaning that you can upload multiple files to a single URL by using the same URL globbing style supported -in the URL. +in the URL. Example: -When uploading to an SMTP server: the uploaded data is assumed to be RFC 5322 -formatted. It has to feature the necessary set of headers and mail body -formatted correctly by the user as curl does not transcode nor encode it -further in any way. + curl --upload-file 'file{1,2,3}' ftp://ftp.example/ + +Since curl 8.21.0, you can use parts of the upload filename when it uses +globbing by setting a glob name and referencing that in the same way you +reference named URL globs. For example, if you upload three files to a single +fixed HTTP URL and want to save the corresponding responses in separate files: + + curl -T 'file{1,2,3}' \ + https://upload.example/ -o 'response-#' + +When uploading to an SMTP server (aka "sending email"): the uploaded data is +assumed to be RFC 5322 formatted. It has to feature the necessary set of +headers and mail body formatted correctly by the user as curl does not +transcode nor encode it further in any way. diff --git a/src/tool_operate.c b/src/tool_operate.c index 62d40afd55..c5ac095cb9 100644 --- a/src/tool_operate.c +++ b/src/tool_operate.c @@ -1052,11 +1052,13 @@ static CURLcode setup_outfile(struct OperationConfig *config, return result; } } - else if(glob_inuse(&state->urlglob)) { - /* fill '#1' ... '#9' terms from URL pattern */ + else if(glob_inuse(&state->urlglob) || glob_inuse(&state->inglob)) { + /* expand '#1' ... '#9' references from URL pattern and named references + from the upload file glob */ SANITIZEcode sc; CURLcode result = - glob_match_url(&per->outfile, u->outfile, &state->urlglob, &sc); + glob_match_url(&per->outfile, u->outfile, &state->urlglob, + glob_inuse(&state->inglob) ? &state->inglob : NULL, &sc); if(sc) { if(sc == SANITIZE_ERR_OUT_OF_MEMORY) diff --git a/src/tool_urlglob.c b/src/tool_urlglob.c index a0dbb0bb6f..dd7a6a9d8e 100644 --- a/src/tool_urlglob.c +++ b/src/tool_urlglob.c @@ -703,7 +703,8 @@ CURLcode glob_next_url(char **globbed, struct URLGlob *glob) #define MAX_OUTPUT_GLOB_LENGTH (1024 * 1024) CURLcode glob_match_url(char **output, const char *filename, - struct URLGlob *glob, SANITIZEcode *sc) + struct URLGlob *glob, struct URLGlob *glob2, + SANITIZEcode *sc) { struct dynbuf dyn; const char *ifilename = filename; @@ -741,7 +742,11 @@ CURLcode glob_match_url(char **output, const char *filename, if(!curlx_str_until(&filename, &name, MAX_GLOBNAME_LEN, '>') && !curlx_str_single(&filename, '>')) { /* find the correct glob entry */ - pat = glob_find_name(glob, &name); + if(glob_inuse(glob)) + pat = glob_find_name(glob, &name); + if(!pat && glob2 && glob_inuse(glob2)) + /* scan the second glob list if there is one */ + pat = glob_find_name(glob2, &name); if(!pat) { /* when the name is given correctly, it needs to be an existing glob name, which makes this an error */ diff --git a/src/tool_urlglob.h b/src/tool_urlglob.h index abc279de73..e891258aa4 100644 --- a/src/tool_urlglob.h +++ b/src/tool_urlglob.h @@ -79,7 +79,8 @@ CURLcode glob_url(struct URLGlob *glob, const char *url, curl_off_t *urlnum, FILE *error); CURLcode glob_next_url(char **globbed, struct URLGlob *glob); CURLcode glob_match_url(char **output, const char *filename, - struct URLGlob *glob, SANITIZEcode *sc); + struct URLGlob *glob, struct URLGlob *glob2, + SANITIZEcode *sc); void glob_cleanup(struct URLGlob *glob); bool glob_inuse(struct URLGlob *glob); diff --git a/tests/data/Makefile.am b/tests/data/Makefile.am index 8dcf2d360c..b63c9c06a0 100644 --- a/tests/data/Makefile.am +++ b/tests/data/Makefile.am @@ -245,7 +245,7 @@ test1970 test1971 test1972 test1973 test1974 test1975 test1976 test1977 \ test1978 test1979 test1980 test1981 test1982 test1983 test1984 \ \ test2000 test2001 test2002 test2003 test2004 test2005 test2006 test2007 \ -test2008 test2009 test2010 test2011 test2012 test2013 \ +test2008 test2009 test2010 test2011 test2012 test2013 test2014 \ \ test2023 \ test2024 test2025 test2026 test2027 test2028 test2029 test2030 test2031 \ diff --git a/tests/data/test2014 b/tests/data/test2014 new file mode 100644 index 0000000000..a7fb078db6 --- /dev/null +++ b/tests/data/test2014 @@ -0,0 +1,97 @@ + + + + +HTTP +HTTP PUT + + + +# Server-side + + +HTTP/1.1 200 OK swsbounce +Date: Tue, 09 Nov 2010 14:49:00 GMT +Server: test-server/fake +Accept-Ranges: bytes +Content-Length: 6 +Content-Type: text/html + +-foo- + + +HTTP/1.1 200 OK +Date: Tue, 09 Nov 2010 14:49:00 GMT +Content-Length: 20 +Content-Type: text/html + +the second response + + + +# Client-side + + +http + + +upload with glob, output name based on upload glob + + +-T '%LOGDIR/upload{%LThej%GT1,2}' http://%HOSTIP:%HTTPPORT/%TESTNUMBER --silent '--output=%LOGDIR/out-#%LThej%GT' + + + +first! + + + +second + + + + +# Verify data after the test has been "shot" + + +PUT /%TESTNUMBER HTTP/1.1 +Host: %HOSTIP:%HTTPPORT +User-Agent: curl/%VERSION +Accept: */* +Content-Length: 7 + +first! +PUT /%TESTNUMBER HTTP/1.1 +Host: %HOSTIP:%HTTPPORT +User-Agent: curl/%VERSION +Accept: */* +Content-Length: 7 + +second + + +%EMPTY + + + +HTTP/1.1 200 OK swsbounce +Date: Tue, 09 Nov 2010 14:49:00 GMT +Server: test-server/fake +Accept-Ranges: bytes +Content-Length: 6 +Content-Type: text/html + +-foo- + + + +HTTP/1.1 200 OK +Date: Tue, 09 Nov 2010 14:49:00 GMT +Content-Length: 20 +Content-Type: text/html + +the second response + + + +