metalink: remove

Warning: this will make existing curl command lines that use metalink to
stop working.

Reasons for removal:

1. We've found several security problems and issues involving the
   metalink support in curl. The issues are not detailed here. When
   working on those, it become apparent to the team that several of the
   problems are due to the system design, metalink library API and what
   the metalink RFC says. They are very hard to fix on the curl side
   only.

2. The metalink usage with curl was only very briefly documented and was
   not following the "normal" curl usage pattern in several ways, making
   it surprising and non-intuitive which could lead to further security
   issues.

3. The metalink library was last updated 6 years ago and wasn't so
   active the years before that either. An unmaintained library means
   there's a security problem waiting to happen. This is probably reason
   enough.

4. Metalink requires an XML parsing library, which is complex code (even
   the smaller alternatives) and to this day often gets security
   updates.

5. Metalink is not a widely used curl feature. In the 2020 curl user
   survey, only 1.4% of the responders said that they'd are using it. In
   2021 that number was 1.2%. Searching the web also show very few
   traces of it being used, even with other tools.

6. The torrent format and associated technology clearly won for
   downloading large files from multiple sources in parallel.

Cloes #7176
This commit is contained in:
Daniel Stenberg 2021-06-01 16:24:00 +02:00
parent 9cf516adc6
commit 265b14d6b3
No known key found for this signature in database
GPG key ID: 5CC908FDB71E12C2
48 changed files with 65 additions and 3154 deletions

View file

@ -8,7 +8,6 @@
- multiple file upload on a single command line
- custom maximum transfer rate
- redirectable stderr
- metalink support (13)
- parallel transfers
## libcurl
@ -79,7 +78,7 @@
- download
- authentication
- Kerberos 5 (14)
- Kerberos 5 (13)
- active/passive using PORT, EPRT, PASV or EPSV
- single file size information (compare to HTTP HEAD)
- 'type=' URL support
@ -218,6 +217,4 @@
10. -
11. requires libidn2 or Windows
12. requires libz, brotli and/or zstd
13. requires libmetalink, and either an Apple or Microsoft operating
system, or OpenSSL, or GnuTLS, or NSS
14. requires a GSS-API implementation (such as Heimdal or MIT Kerberos)
13. requires a GSS-API implementation (such as Heimdal or MIT Kerberos)

View file

@ -952,31 +952,6 @@ previous example in an SFTP URL might look like:
IPv6 addresses provided other than in URLs (e.g. to the `--proxy`,
`--interface` or `--ftp-port` options) should not be URL encoded.
## Metalink
Curl supports Metalink (both version 3 and 4 (RFC 5854) are supported), a way
to list multiple URIs and hashes for a file. Curl will make use of the mirrors
listed within for failover if there are errors (such as the file or server not
being available). It will also verify the hash of the file after the download
completes. The Metalink file itself is downloaded and processed in memory and
not stored in the local file system.
Example to use a remote Metalink file:
curl --metalink http://www.example.com/example.metalink
To use a Metalink file in the local file system, use FILE protocol
(`file://`):
curl --metalink file:///example.metalink
Please note that if FILE protocol is disabled, there is no way to use a local
Metalink file at the time of this writing. Also note that if `--metalink` and
`--include` are used together, `--include` will be ignored. This is because
including headers in the response will break Metalink parser and if the
headers are included in the file described in Metalink file, hash check will
fail.
## Mailing Lists
For your convenience, we have several open mailing lists to discuss curl, its

View file

@ -1,27 +1,8 @@
Long: metalink
Help: Process given URLs as metalink XML file
Added: 7.27.0
Requires: metalink
Category: misc
---
This option can tell curl to parse and process a given URI as Metalink file
(both version 3 and 4 (RFC 5854) are supported) and make use of the mirrors
listed within for failover if there are errors (such as the file or server not
being available). It will also verify the hash of the file after the download
completes. The Metalink file itself is downloaded and processed in memory and
not stored in the local file system.
Example to use a remote Metalink file:
curl --metalink http://www.example.com/example.metalink
To use a Metalink file in the local file system, use FILE protocol (file://):
curl --metalink file:///example.metalink
Please note that if FILE protocol is disabled, there is no way to use a local
Metalink file at the time of this writing. Also note that if --metalink and
--include are used together, --include will be ignored. This is because
including headers in the response will break Metalink parser and if the
headers are included in the file described in Metalink file, hash check will
fail.
This option was previously used to specify a metalink resource. Metalink
support has unfortunately been disabled in curl since 7.78.0 due to security
reasons.

View file

@ -37,8 +37,8 @@ interaction.
curl offers a busload of useful tricks like proxy support, user
authentication, FTP upload, HTTP post, SSL connections, cookies, file transfer
resume, Metalink, and more. As you will see below, the number of features will
make your head spin!
resume and more. As you will see below, the number of features will make your
head spin!
curl is powered by libcurl for all transfer-related features. See
*libcurl(3)* for details.

View file

@ -50,8 +50,6 @@ This curl supports transfers of large files, files larger than 2GB.
.IP "libz"
Automatic decompression (via gzip, deflate) of compressed files over HTTP is
supported.
.IP "Metalink"
This curl supports Metalink.
.IP "MultiSSL"
This curl supports multiple TLS backends.
.IP "NTLM"

View file

@ -59,10 +59,6 @@ endif
ifndef LIBRTMP_PATH
LIBRTMP_PATH = ../../../librtmp-2.4
endif
# Edit the path below to point to the base of your libmetalink package.
ifndef LIBMETALINK_PATH
LIBMETALINK_PATH = ../../../libmetalink-0.1.3
endif
# Edit the path below to point to the base of your libexpat package.
ifndef LIBEXPAT_PATH
LIBEXPAT_PATH = ../../../expat-2.1.0
@ -220,9 +216,6 @@ endif
ifeq ($(findstring -ipv6,$(CFG)),-ipv6)
IPV6 = 1
endif
ifeq ($(findstring -metalink,$(CFG)),-metalink)
METALINK = 1
endif
ifeq ($(findstring -winssl,$(CFG)),-winssl)
WINSSL = 1
SSPI = 1
@ -354,18 +347,6 @@ ifdef WINIDN
curl_LDADD += -L"$(WINIDN_PATH)" -lnormaliz
endif
endif
ifdef METALINK
INCLUDES += -I"$(LIBMETALINK_PATH)/include"
CFLAGS += -DUSE_METALINK
curl_LDADD += -L"$(LIBMETALINK_PATH)/lib" -lmetalink
ifndef DYN
ifeq ($(findstring libexpat_metalink_parser.o,$(shell $(AR) t "$(LIBMETALINK_PATH)/lib/libmetalink.a")),libexpat_metalink_parser.o)
curl_LDADD += -L"$(LIBEXPAT_PATH)/lib" -lexpat
else
curl_LDADD += -L"$(LIBXML2_PATH)/lib" -lxml2
endif
endif
endif
ifdef SSPI
CFLAGS += -DUSE_WINDOWS_SSPI
endif