Debian Package of the Day (static archived copy)

Article submitted by Cameron Dale. Guess what? We still need you to submit good articles about software you like!

Do you want to help out the Debian (or Ubuntu) project with some mirror bandwidth but don’t know how? Do you want to contribute somehow to Debian’s infrastructure, but you’re not a coder? Tired of getting slow download speeds when the mirrors are overloaded after a new release? Then Apt-P2P is for you.

After installing the apt-p2p package and making some minor changes to apt’s sources, all the files you download with apt (or aptitude, synaptic, gnome-apt, etc…) will be shared with other users, and any files you download will use other users (peers) to download from. However, if no other users have the file you want there’s no need to worry, Apt-P2P will happily fall back to downloading directly from the mirror so your download will not fail.

How it works

Apt-P2P is a daemon that runs in the background, responding to any requests from apt for files to download, and sharing any downloaded files with other users. The sharing is all done using HTTP, so it operates as both a server for the requests from apt and other peers, and as a client to download from other peers and mirrors. Also, if you go to http://localhost:9977/ in your browser on the machine Apt-P2P is running on, you will get a nice statistical display of what it is doing.

The main operation of Apt-P2P is the maintenance of a Distributed Hash Table (DHT) used to find and store peers to download from for each file. Whenever you download a file, apt-2p will first lookup the SHA1 hash of the file in the DHT. If it is found and has peers listed, then the downloading will occur from the peers (if there are only 1 or 2 peers, the mirror is used as well to speed up the download). If it is not found then the file is requested directly from the mirror. Once the download is complete, a new value is added to the DHT using the SHA1 hash of the file as the key, and including your contact info, so that other peers can then find you to download the file from.

That’s just a brief overview, but there are many hidden details that make things go smoother. For example, for larger files the SHA1 hashes of pieces of the file are stored in the DHT as well, which allows downloaders to break up large files among several peers to get better download speeds (similar to BitTorrent). For more information, you can go to the Apt-P2P home page: http://www.camrdale.org/apt-p2p/.

Comparison with other P2P programs

Other than DebTorrent, there aren’t any other peer-to-peer downloaders available for apt. There was apt-torrent, but it was never packaged in Debian, and now seems to be dead (no updates in 18 months). Comparing Apt-P2P with DebTorrent, Apt-P2P:

• Is faster at downloading from mirrors.
• Uses much less memory and a little less CPU.
• Can download almost all files (source files, Packages.bz2, etc…) from peers, not just .deb packages.
• Works with only the information apt has, no need for other files.
• Doesn’t require a new apt transport for communicating with apt.
• Is more modular and easier to understand as it is based on existing technologies (e.g. twisted).

Setup

Apt-P2P is available in testing (lenny) and unstable (sid), and will be available in Intrepid for Ubuntu. It can be installed by aptitude install apt-p2p.

First, it is VERY important to set up port forwarding if your machine is behind a firewall or router/NAT. The default port you need to forward is 9977, both UDP and TCP. More information on how to determine if you are reachable can be found here.

Setting up apt to use Apt-P2P as a proxy is easy, especially if you have used other proxy software (e.g. apt-proxy, apt-cacher, approx) in the past. The configuration change is the same, simply adding a localhost:9977/ to the front of the entries in your /etc/apt/sources.list file. For example, if you previously had this:

# Official
deb http://ftp.us.debian.org/debian etch main contrib non-free
deb-src http://ftp.us.debian.org/debian etch main contrib non-free

# Security Updates
deb http://security.debian.org/ etch/updates main contrib non-free
deb-src http://security.debian.org/ etch/updates main contrib non-free

# Unofficial
deb http://ftp.debian-unofficial.org/debian etch main contrib non-free restricted
deb-src http://ftp.debian-unofficial.org/debian etch main contrib non-free restricted

# Backports
deb http://www.backports.org/debian etch-backports main contrib non-free
deb-src http://www.backports.org/debian etch-backports main contrib non-free

Then, if you only want to share the official and backported packages, you would change it to this:

# Official
deb http://localhost:9977/ftp.us.debian.org/debian etch main contrib non-free
deb-src http://localhost:9977/ftp.us.debian.org/debian etch main contrib non-free

# Security Updates
deb http://security.debian.org/ etch/updates main contrib non-free
deb-src http://security.debian.org/ etch/updates main contrib non-free

# Unofficial
deb http://ftp.debian-unofficial.org/debian etch main contrib non-free restricted
deb-src http://ftp.debian-unofficial.org/debian etch main contrib non-free restricted

# Backports
deb http://localhost:9977/www.backports.org/debian etch-backports main contrib non-free
deb-src http://localhost:9977/www.backports.org/debian etch-backports main contrib non-free

Then, run an apt-get update and start installing packages.

The apt-p2p package is fairly new, so it’s available only in the testing and unstable distributions of Debian, and in the just released Intrepid Ibex, from Ubuntu.