I’ve been managing many Windows and Unix servers in the last 10 years, and this I know for sure – backing up in Windows can be a painful experience, and most of the time it require significant investment in special backup software (which tend to cost thousands of dollars, usually more).
Even then, the software will become buggy once you start setting up complex backup scenario.
In Unix/Linux however, you have powerful scripting tools at your disposal, usually already included in the package. These tools are very flexible, enabling you to develop almost any kind of backup scheme.
It does require some sort of programming skill. Point-and-click admins will have a hard time at first, but let me tell you, do give it a try. You’ll find that it’s very much worth the trouble.
Both require investment in time & effort to develop a good backup strategy.
Before we progress, first here’s a few rules in regard to backup :
- You can never have too many backups.
I backed up my personal data to several locations – my other PC, and also my brothers’ PC. So in total, I have 3 copies of it.
One day, the hard drive in my main PC broke down. So I came to my brother, and asked him to copy my data which is in his PC. To my surprise, he said that his hard drive just died too. I ended up with only a single copy of my precious data.
I quickly replaced the dead hard drive, restored the only copy of my data there, and made a new backup script for it. Nowadays, my data is usually available in 5 or more locations. - Automate all of its processes.
If it require even the tiniest amount of manual intervention, believe me, it will end up not being executed. For once of twice, you may still willing to intervene. But when you need to do that everyday, it just won’t happen. - Check your backup.
Check the result / logs everyday. Try restoring the backup about every week. Do NOT skip this, or you will find out that the backup is actually not restorable when that very important server died on you.
These are the most important ones, and I confess to have suffered from one or more of it in the past.
You don’t have to, but it’s your choice.
Anyway, here’s a sample script to get you started backing up in Linux.
Backing up the whole hard drive, over the network.
#!/bin/bash
mkdir /mnt/backup
mount -t ext3 /dev/hdb1 /mnt/backup
chmod 777 /mnt/backup
cd /mnt/backup
/usr/bin/rsync -avuz --progress --rsh="ssh -l root -i /root/.ssh/id_dsa" 192.168.0.1:/ /mnt/backup
The 1st line is important – it tells the computer that we’d like this script to be processed by bash. Different shell has different syntax. So we need to be precise about this.
The 2nd line create the directory for mounting the backup drive. 3rd line mounts /dev/hdb1 (first partition of second IDE device) to /mnt/backup. 4th line gives full access to the drive. the 6th line does the actual backup process, copying only changed files on 192.168.10.101 to /mnt/backup.
It may seem simple at first, but make no mistake, rsync is one powerful tool. For example, quoted from rsync manual:
The rsync remote-update protocol allows rsync to transfer just the differences between two sets of files across the network connection, using an efficient checksum-search algorithm described in the technical report that accompanies this package.
This capability has enabled me to backup a 200 GB hard drive, over 100 Mbps network, in under 2 hours.
Without disturbing the 15+ users which are on that network as well. Simply amazing.
The next one is probably the kind of backup script you’d more often encounter – backup, compress, store to a safe location.
#!/bin/bash
tar cvf /backup/accounting-$(date +%Y%m%d).tar /home/accounting
bzip2 -9 /backup/accounting-$(date +%Y%m%d).tar
/usr/bin/scp -2 -i ~/.ssh/id_dsa /backup/accounting-$(date +%Y%m%d).tar.bz2 smith@192.168.0.10:/data/backup/
The 2nd line bundles up the whole content of /home/accounting into a single file named /backup/accounting-(today’s date).tar; example; /backup/accounting-20061230.tar would be the resulting file if this script is run on the 30th December 2006.
This trick needed to avoid the backup replacing the same file everytime it runs. This way, we’ll have multiple backups over time, instead of just one.
The 3rd line compresses the file above, as strong as possible (with the -9 switch)
The last line copies the file (now with .bz2 extension after compressed by bzip2) into directory /data/backup/ in a server with IP address of 192.168.0.10, as user smith.
The last example is a more complicated backup script.
I developed this to backup groups.or.id‘s (kinda like Yahoogroups) member database automatically, everyday to servers on different countries. Therefore, in case of disaster, the administrator can quickly restore the service on another server with little problem.
Backup member database, over the Internet.
#!/bin/bash
daftar_milis=( $(ls ~/) )
for element in $(seq 0 $((${#daftar_milis[@]} - 1)))
do
echo "---- MILIS: ${daftar_milis[$element]} ----" >> /backup/daftar-member-$(date +%Y%m%d).txt
/usr/bin/ezmlm-list ~/${daftar_milis[$element]} >> /backup/daftar-member-$(date +%Y%m%d).txt
done
/usr/bin/bzip2 -9 /backup/daftar-member-$(date +%Y%m%d).txt
/usr/bin/scp -2 -i ~/.ssh/id_dsa /backup/daftar-member-$(date +%Y%m%d).txt.bz2 harry@mydomain.com:/home/harry/backup/groups.or.id/
exit 0
A bit of background, the server uses ezmlm as the mailing list software, which is usually controlled by user “alias”.
2nd line is already interesting. Basically, we execute ls (which shows the content of directory) ~/. What is directory ~/ ? Well, the tilde character (~) is a shortcut for our home directory. So, when running this script as alias user, the “ls ~/” actually means “ls /var/qmail/alias/”
The result (list of files and directories) will then be stored in an (array) variable named “daftar_milis”
4th and 5th line sets us up for a looping. It will loop as many times as there are data in “daftar_milis”.
7th line will output a line, which is “—- MILIS: (current data in “daftar_milis”) —-“, and append ( >>) it into a file named /backup/daftar-member-(today’s date).txt
8th line runs ezmlm-list, which will list the members of the mailing list, and store in into the same file as above.
When all the data in “daftar_milis” has been processed, then the 12th line will be executed. It will compress the backup file with bzip2 compression.
Note that this compression algorithm is much more complex than standard Zip compression, therefore on a slow processor it may take a very long time to finish.
The 13th line will copy the backup file to a server somewhere on the Internet, on a secure tunnel encrypted with SSH2 protocol.
So there you are, a few examples to get you started backing up in Linux. Hope you find it useful.