Yes it’s still turned on by default at least in Solaris 10 update 2. That’ll teach me not to portscan new *nix installs where I’m not completely familiar with the OS defaults.
Seriously though, is there a good reason for telnet to be on by default in this day and age?

I’ve now got backup scripts happily creating copies of all my subversion repositories and MySQL databases every 24 hours. This is great but it means you end up with an awful lot of backups. I realy don’t need a backup from every day going back forever. But it is nice to have snapshots going back into history in case something subtle has gone wrong.

What I’d really like is to copy the oldest backups into a different directory every seven days, and delete all the backups in the main directory that are older than seven days. Of course I’ll then end up with piles of backups building up in the weekly directory. So I’d like to go through the weekly directory every month, copy the oldest backups into another directory and delete all the weekly backups that are more than one month old.

To do this I give you the snappily named rotate-backups

rotate-backup            rotates backups in a given directory weekly,or monthly

-b                       directory to rotate backups in
-f                       file containing list of backup dirs
-t                       time period (weekly,monthly)

The config file is just a new-line separated list of directories. To make it work I put a script in /etc/cron.weekly like:

rotate-backups -t weekly -f /etc/rotate-backups

and one in cron.monthly:

rotate-backups -t monthly -f /etc/rotate-backups

The backup script makes an assumption that backups created on the same day are from the same backup run. It copies the ‘oldest’ backups by copying files from the same day as the oldest file in the directory. This way it doesn’t have to know anything about what you are backing up or what your naming conventions are.

Also it culls old backups relative to the date of the latest file in the directory. This means that if you stop taking backups the script won’t keep deleting files until you have none left.

We run a subversion service for a number of different research groups. Generally we create a separate subversion repository for each group. Obviously looking after this data is important. It’s not a good idea to lose months of someone’s work.

Fortunately backing up a subversion repository is pretty simple. Subversion ships with a utility called svnadmin. One of the functions of which is to dump a repository to a file.

As the repository owner do:

svnadmin dump /path/to/repository > subversion_dump_file

I have a directory full of subversion repositories so what I really want is a script that will find all the subversion repositories in the directory and dump them with sensible filenames. With my usual lack of imagination I’ve called this script svn_backup. It runs svnadmin verify against each file in the directory it’s given. If any of them turn out to be subversion repositories it dumps them using svnadmin dump.

$ ./svn_backup
svn_backup  -s subversion_dir [-b backup_dir] [ -o logfile]
        script to backup subversion repositories

        -s      directory containging subversion repositories
        -b      directory to dump backups to. defaults to /tmp
        -o      file to output names of backups to

So I now have an entry in cron.daily like:

svn_backup -s /var/www/subversion -o /tmp/svn.log -b /var/backups/svn

The reason I write the backups to a log file is that it allows me to run a script once a week that copies the latest backups to Amazon’s S3 storage system.

The scripts:
svn_backup
s3svnbackup.py

Like most people I’ve got a number of MySQL servers with different databases running things like Wikis and other web-apps. MySQL ships with a handy little tool called mysqldump which can be used to dump ( as the name suggests ) a mysql DB to a text file containing SQL commands necessary to re-create it.

The first thing I like to do is to create a backup user that only has enough privileges to do the backup.

GRANT LOCK TABLES,SELECT,RELOAD,SHOW DATABASES ON *.* TO 'backup'@'localhost' IDENTIFIED BY 'mypasswd';
FLUSH PRIVILEGES;

with this done you should be able to do something like

mysqldump -hlocalhost -ubackup -pmypasswd --databases mysql > test_backup.mysql

With that in place it was an easy task to write a script that can read a config file for the backup login info and options to pass to mysqldump. This script has a number of benefits over sticking a mysqldump command straight into cron:

1. It stores it’s settings in an external config file so you can use the same script in several settings
2. It backs up each database on the server into a separate dump file.
3. The backup options are in the config file so you can back up different servers with different tweaks. e.g locking the DB for consistancy.

the line in the config file looks like

localhost:backup:mypasswd:--opt --add-drop-table

I add a quick script to /etc/cron.daily like

mysql_backup -b /var/backups/mysql -f /etc/mysql_backup

I can now sleep a bit easier. This isn’t the only way to back up mysql, proper DBAs will know all about replication and various other tricks.

Next time: subversion repositories.

Over the last couple of weeks I’ve been chipping away at the problem of our department having no backups whatsoever. Being a small department with few machines and a fairly small amount of data I’ve decided that systems like Bacula and Amanda are over-kill for our situation.

I’ve written a set of small scripts to handle our most pressing backup needs. Over the next few posts I’ll describe how I’ve backed things up and the scripts and tools I’ve used to do it. None of this is rocket-science but if it saves even one person, one hour of work it’ll have been worth writing down.

All the scripts in the next few posts can be found in the WeSC subversion repository.

Part 1: Mysql

Part 2: Subversion

Part 3: Rotating and Culling Backups