Search

rdiff-backup: Easy incremental backups from the command line

October 26th, 2008 edited by Vicho

Storage is becoming cheaper and cheaper: you can find hard drives that cost less than a dollar per GiB. Buying an external hard drive to make backups (or even having a backup server) is a must if you value your work and what you have stored in your computer. However, doing backups should be easy enough to be done on a regular basis. The more automated, the better.

So, I find no excuse not to do regular backups and looked for a tool easy-to-use but powerful. rdiff-backup is a python script that helps doing local and remote incremental backups. To backup your $HOME to an external hard drive mounted in /media/backup simply do:

$ rdiff-backup $HOME /media/backup/home_backup

If after some days you want to backup your new files, run the same command to update the backup.

Now, in /media/backup/home_backup you have an exact copy of your home as it was when you did the last backup. If you want to restore a directory, you can just copy it:

$ cp -a /media/backup/home_backup/src/myprogram ~/src/

Which is equivalent to:

$ rdiff-backup --restore-as-of now /media/backup/home_backup/src/myprogram ~/src/

Of course, you can restore previous versions of the file. For example, to restore the source of myprogram as it was a mounth ago:

$ rdiff-backup --restore-as-of 1M /media/backup/home_backup/src/myprogram ~/src/

You can see all the incremental backups you have done executing:

$ rdiff-backup --list-increments /media/backup/home_backup

If you run out of space in your backup device and you’re sure you don’t need the backups you made three years ago, you can remove them with:

$ rdiff-backup --remove-older-than 3Y /media/backup/home_backup

rdiff-backup works exactly the same with remote directories. You need to have ssh access and rdiff-backup must be installed in the remote(s) machine(s). Note that in any example above, you can change the local directories to remote ones, so you can backup a remote machine locally, or do a backup of this machine to a remote backup-server. For example, say backup.mysite.org is your backup server. You can backup regularly using:

$ rdiff-backup local-dir/ user@backup.mysite.org::/remote-dir

If you use RSA or DSA authentication, you can even put that in a cron job.

See rdiff-backup documentation and other examples to discover all the functionality of this package.

Similar packages

Frontends for rdiff-backup:

  • keep is a GUI (KDE) frontend for rdiff-backup.
  • archfs is a fuse (filesystem in userspace) virtual filesystem that lets you browse each version of a rdiff-backup repository as if they were any other directory. Adam Sloboda has stated his intention to package archfs for Debian.
  • rdiff-backup-web (not in Debian, no WNPP yet) is a web frontend for rdiff-backup.

There are a ton of other programs to make backups. I will list here some of them (but this list is no where near complete) that are similar to rdiff-backup:

  • backup2l also makes local backups, but seems to miss the remote-backup feature.
  • backuppc is a perl script that also makes incremental backups, and has an http user interface to help manage and restore backups.
  • duplicity also makes remote incremental backups, but encrypts the data using gnupg. I haven’t test it myself, but it can be useful if you don’t trust the remote file server.
  • storebackup also makes local incremental backups. It makes a new tree in every snapshot, but disk space is preserved by hard-linking unchanged files.

Conclusions

Pros:

  • Easy to use. Now there’s no excuse not to do backups!
  • Works from the command line, so you can easily put it in a script or cron job.
  • Simple recovery from last snapshot, you can use standard tools like cp or find.

Cons:

  • Not having a GUI may scare some users.
  • It stores the last snapshot uncompressed, so depending on what you are backing up, it can be very space consuming. Older snapshots are compressed, which makes this con a not-so-con ;-).

rdiff-backup has been available in Debian since Sarge (perhaps even longer), and in Ubuntu since Dapper.

Posted in Debian, Ubuntu |

14 Responses

  1. Marc Says:

    Argh, I just started learning Rsync. Is this a better option than Rsync? I’m backing up my work laptop onto an external HD. Thanks for the writeup.

  2. Paul Says:

    “you can find hard drives that cost less than a dollar per GiB”

    I think you mean $1 per 10 GB, unless you’re paying $1K for a 1TB drive.

  3. Morten Siebuhr Says:

    I have used rdiff-backup from within backupninja for a few years now - I highly recommend it for simple backup-systems.

    Backupninja is more or less a backup scheduler; i.e. “save all system configuration at XX”, “hotcopy MySQL to this location”, “copy all files at /home, /etc, /var/backup to server_name:/srv/backup/client_name/”.

  4. Simon Oosthoek Says:

    rdiff-backup has another drawback: version dependence. Newer versions tend to be incompatible with older ones. On a single system, or on different machines running the same distro and release version, this is not a problem (backups can get automatically upgraded), but backups across different systems can run into this. So far I haven’t found a way to run different versions of rdiff-backup on a single system to accommodate backups from different systems in one location…

  5. amd-linux Says:

    Good, old rsync.

    I backup my server and virtual machines with rsync, once in a while.

    http://vale.homelinux.net/wordpress/2008/08/19/easy-and-simple-backup-of-a-live-linux-server/

    Easy, simple, reliable, fast.

  6. gwern Says:

    Paul: 10 gigs per dollar is a bit optimistic, I think. The best I see new on Newegg (according to http://forre.st/storage) is 8 gigs per dollar. Even a daredevil who is willing to trust a used drive is only going to get 9 gigs a dollar.

    Marc: rdiff-backup is a lot like rsync, except the crucial part of course is that rsync is merely copying directory trees. It doesn’t store old copies anywhere in any form. The use case here being you do a rsync backup, and then realize you want a file or change back which you just ‘updated’ on the remote machine as well.

  7. adam Says:

    archfs is looking for a sponsor

  8. Giovanni Says:

    Still using “wine syncback.exe”

  9. Claude Says:

    [rsync] doesn’t store old copies anywhere in any form.

    Sure it does. See the man page’s entry for the -b switch.

  10. flow Says:

    You’re right, it’s really very simple, very useful. However, another pro: It’s highly platform independent, so you can even store you’re data on a FAT32, NTFS, or whatsoever drive since it checks the FS and encodes file names depending on it’s capabilities.

    I for example back up my notebooks (ubuntu intrepid) data to a server at home running Windows XP (and cygwin..) via SSH.

  11. Adam Says:

    I’d also suggest looking a rsnapshot with is similar and also in Debian standard.

    I’d also checkout the excellent “Backup and Restore” book by W. Curtis Preston and the corresponding http://www.backupcentral.com/ website.

  12. DrCR Says:

    FYI rsnapshot’s hard link snapshot makes it perfect for stuff that’s constant like, say, a media RAID. rdiff comes into its own when you want to backup something that is more than rarely changed.

    Thanks for the blog! I’m looking at using rdiff for my Debian server OS install, while continuing to use rsnapshot for the mounted raid, assuming rdiff has an ignore option.

  13. toki Says:

    See the man page’s entry for the -b switch.

  14. tex Says:

    Don’t forget about rsnapshot and Dirivsh. Those are also great backup programs.