Savannah W-ITS

I started this post a while back and settled on Back in Time (Rsync based solution) for my needs.  The post includes references (and in some cases, brief descriptions) of other tools.  The best tool for any solution will vary based on requirements, budget & time – as usual, your mileage will vary – just remember the THREE Bs... 

The Scenario:

1. this discussion is will NOT cover solutions for production environments (but it might be starting point for deciding on a direction),
2. in this case we have several non-critical PCs & Linux systems,
3. all systems are connected via internal LAN,
4. some external storage is available (~1.5 TB external SATA disk),
5. we want to backup ‘important data’ and/or
6. we want to backup entire systems for easy/quick restore and
7. we want at a simple solution with low maintenance – just configure and turn ‘on’ and
8. we prefer to use a central system as the ‘backup server’.

A manual solution

• create a large SAMBA (Windows) share on any system on the network
• adjust SAMBA share permissions for global write access from the network
• or create sub-shares with permissions for each host
• manually copy files or disk images from client systems to the SAMBA share

An automated solution

• create large storage folder (i.e. using a set of pooled disks) on a Linux system designated to handle automated backups
• install automated backup client-tools on all client systems
• schedule automated backups
• after successful copy from client to backup server archive any data needed for long-term retention (save from backup disk pool to tape, CD, DVD, or other external media)
• solutions can include pull (files are requested by backup server) or push (client pushes files to backup server) approaches

Some simple, semi-automated backup solutions might be:

• a same-host-based backup solution is to create a mirror disk/partition and simply duplicate the entire disk to the mirror (if you are already using some level of RAID then you already have this.)  You still need some sort of application to automate the backup process (unless you want to include this as an on-demand, a  boot-up or a shut down procedure.)
• using one of the many GUI Rsync Tools, i.e. Grsync or perhaps Gnome Conduit (data sync locally or with remote storage providers, i.e. Picassa, Flickr, Gmail, etc.)
• a tool like Pdumpfs (2001 Ruby backup solution) run from cron
• non-gui Rsync wrapper-tools listed below like:  Dirvish, Bontmia, Duplicity, Rsnapshot, Back in Time – all usually run from cron with setup fairly simple (create a storage location, make any needed SSH tweaks to allow automated system access, test and created cron jobs, review on a regular basis)
• rdiffbackup
• rsnapshot – Perl script wrapped around rsync -

After looking at a number of Open Source backup solutions one tool seems to be quite popular as a cross-OS, system level solution:  RSYNC.  Many of the solutions mentioned above and  below either support rsync as an option or use rsync as the basis for the backup solution.  At this point you might consider reviewing the man page for rsync – and for a deeper understanding of how this awesome tool works I would suggest reviewing the PHd thesis written by Andrew Tridgell (3 chapters discuss rsync.)

For a more complete (and more complex) backup solution you might consider:

• BackupPC – “Perl based, ‘enterprise grade’ backup solution supporting Linux, Windows backup to disk…”
• Bacula – “The Network Backup Solution”
• Amanda – “Advanced Maryland Automatic Network Disk Archiver” – backup to disk and/or tape

---

Dirvish (rsync) - sounded good – disk-based snapshots ; Perl based; poor (non-helpful) error messages; config files must be manually created and backup paths need to match ‘vaults’, i.e. /backup_location/path/sub_folder/VAULT_NAME where VAULT_NAME = a valid SYSTEM_SERVER_NAME in /etc/dirvish/master.conf;  your mileage will hopefully be less confusing…

I modified the script to use a ‘backup’ user account on Linux systems, i.e. changed:

rsh             => ’ssh’,

to

rsh             => ’ssh -l backup’,

so I also had to take steps needed for ssh keys – using the ‘backup’ user account. Try the –dry-run option with –summary long and you will see what your configuration looks like.

dirvish --dry-run --vault SYSTEM_NAMES --summary long --init
client: drlinux
tree: /
rsh: ssh -l backup
Server: drquad
Bank: /BACKUPS/Linux/L-03-127GB
vault: SYSTEM_NAME(s)
branch: default
Image: 20091125
Reference: default
Image-now: 2009-11-25 22:34:15
Expire: +15 days == 2009-12-10 22:34:15
exclude:
 usr/src/
 lost+found/
 proc/
 media/
 spool/
 mnt/
 tmp/
 var/[a-v,y-z]*
 var/cache/apt/archives
 var/cache/man
 tmp
 var/tmp
SET permissions devices init numeric-ids stats xdev
UNSET checksum sparse whole-file zxfer

Command-Args: --dry-run --vault drlinux --summary long --init
Configfiles:
 /etc/dirvish/master.conf
 /BACKUPS/Linux/L-03-127GB/SYSTEM_NAME/dirvish/default.conf
bank:
 /BACKUPS/Linux/L-03-127GB
expire-default: +15 days
expire-rule:
 *   *     *   *         1    +3 months
 *   *     1-7 *         1    +1 year
 *   *     1-7 1,4,7,10  1
 *   10-20 *   *         *    +4 days
image-default: %Y%m%d
include:
 var/home_share
index: gzip
log: gzip
no-run: 1
summary: long

ACTION: rsync -vrltH --delete -pgo --stats -D --numeric-ids -x --exclude-from=/BACKUPS/Linux/L-03-127GB/SYSTEM_NAME/20091125/exclude SYSTEM_NAME:/ /BACKUPS/Linux/L-03-127GB/SYSTEM_NAME/20091125/tree

---

Bontmia (rsync – wrapped in shell script)

There is no ‘man page’ – simply call the script without arguments and you will be presented with the documentation…”

NAME
 Bontmia - Backup Over Network To Multiple Incremental Archives
 Version 0.14

SYNOPSIS
 bontmia --dest <dest. dir> [options] [source dir [source dir ...]]

DESCRIPTION
 Bontmia is a network-based backup tool that saves configurable
 numbers of last month, week, day, hour, and minute backups.
 Each backup is a complete snapshot of the original
 directories.  Only new and changed files are copied over the
 network when generating a snapshot.  Remote access is
 implemented securely using ssh.  Unchanged files are stored as
 hard links in the archive and therefore takes virtually no
 space.

This seems to be a simpler solution than Dirvish – no configuration required – just create your storage location(s), create your ssh logins on external machines and wrap things up by setting up cron jobs for each backup.  Unfortunately the Bontmia script does NOT seem to include a local host option (you could modify it to not use SSH for ‘local’ backups OR you have to configure SSH to work with the local machine…)

---

Duplicity (rsync – encrypted backup – GnuPG required)

---

Related posts:

1. Windows 7 won’t boot after Linux Live CD install… Whooops! The Fedora 12 LIVE CD provides an ‘install’ option...
2. Linux – lost your boot? Ok, you made YAW (yet another whoops!) and need a...
3. More about Unix/Linux file permissions How many times have you read or heard that the...
4. Linux firewall – iptables 101c GEO Blocking network access – blocking specific IP blocks (GEO-blocking...
5. Sharing application data via VirtualBox (Vista/Linux) After flipping between Vista (64 bit) and Fedora 8/9 I...