quote:

Getting popular software off the Internet can sometimes be a struggle, even with all the mirrors and BitTorrent Samaritans out there. When the Fedora project released Fedora Core 6 last month, for instance, even several dozen mirrors weren't enough to serve everyone, and torrent speeds weren't good enough because of a scarcity of seeders. But thanks to Metalink I was able to sleep while my FC6 ISOs were downloading.

Metalink is an open standard that claims to make downloading easier, faster, and more reliable by helping users extract the last drop of juice out of their connection. But Metalink isn't your run-of-the-mill download accelerator. It is in fact a framework for use by other download clients, and bundles traditional HTTP and FTP methods of downloading files along with BitTorrent.

The Metalink standard replaces static URLs with a .metalink file, which is a simple XML file. The file in turn contains locations of all the mirrors of the application you want to download. In addition to HTTP and FTP mirrors and rsync, Metalink supports several P2P methods as well, including BitTorrent, ed2k, and magnet links. For example, the OpenOffice.org metalink contains links to more than 50 HTTP and FTP servers and a torrent.

If I were using a regular hyperlink to download OpenOffice.org and the server went down midway through the transfer, I would be left with an incomplete transfer that could either be resumed or not depending on the download client. But with Metalink, if one server goes down, the client software simply jumps to another mirror and resumes downloading from that point. In effect, this increases reliability, as all listed servers would need to be down for the file to be unavailable.

Files downloaded with metalinks are automatically verified. Metalink supports both MD5SUM and SHA1SUM checksums as well as PGP signatures, which are embedded in the .metalink file itself. Checksums are unique for every file, like a fingerprint. If there's an error in transfer, or if someone has maliciously replaced a good file with a bad one, the checksums won't match. If that happened with another file transfer method, you'd have to download the file again from another mirror. With Metalink, if the application you are downloading has a torrent, Metalink can use a torrent's partial file or chunk checksums to verify mirror downloads as well. If only a small chunk of a download has errors, Metalink just re-downloads that part instead of the whole file.

<metalink version="3.0" xmlns="http://www.metalinker.org/">
  <files>
    <file name="example.ext">
    <verification>
      <hash type="md5">example-md5-hash</hash>
      <hash type="sha1">example-sha1-hash</hash>
      <signature type="pgp"/>
    </verification>
    <resources>
      <url type="ftp" location="us" preference="90">ftp://ftp.example1.com/example.ext</url>
      <url type="ftp" location="uk" preference="90">ftp://ftp.example2.com/example.ext</url>
      <url type="http" location="us" preference="90">http://www.example1.com/example.ext</url> 
      <url type="http" location="uk" preference="90">http://www.example2.com/example.ext</url>
      <url type="http" location="de" preference="90">http://www.example3.com/example.ext</url> 
      <url type="bittorrent" preference="100">http://www.ex.com/example.ext.torrent</url>
      <url type="magnet"/>
      <url type="ed2k"/>
    </resources>
    </file>
  </files>
</metalink>