How I do backups: rsync is my friend :)

There are many many fancy backup programs out there, that have all sorts of cool features. I remember, a long time ago, using ntbackup (which saved my bacon and that of the business I worked for at the time), and later Backup Exec. Of course, now, there's cloud cloud cloud all over the place.. What if, however, you just want to backup from one place to another without a subscription fee?

Well, it took awhile, and the script's evolved over time, but I presently use rsync for backups. In conjunction with a filesystem that supports hard links, I can store date stamped copies of a given source, where the backup disk only stores changed files between copies, saving disk space - but all the files are ready for direct access within their respective directory heirarchies.

Your mileage may vary, you may indeed find fancier scripts, but this one works for me :)

#!/bin/bash

# Script written by Anthony Hogan

PROG_NAME=$(basename $0)

error_exit() {
exit_status=$1
log_message=$2
logger -st "$PROG_NAME[$$]" -- $log_message
exit $exit_status
}

logit() {
message="$1"
[ "${message}" == "" ] && logger -t "${PROG_NAME}[$$]" --
[ "${message}" != "" ] && logger -t "${PROG_NAME}[$$]" -- "${message}"
}


##############################################################################
### Items required
# Source path (on local filesystem or an rsync path)
# Destination folder (Where the backup will be placed)
# Destination timecode (If we're not autogenerating the date, it'll need to be specified

# To specify an RSYNC password, set the environment variable RSYNC_PASSWORD prior to script launch

BACKUP_SOURCE="user@remote:/path1 :path2 :path3"
BACKUP_FOLDER="/path/to/backup/media"

BACKUP_TIMECODE="$(date +%Y%m%d_%H%M%S)"
RSYNC_OPTIONS="-aHv --stats --no-inc-recursive --progress --delete-during --modify-window=1 --exclude Thumbs.db"
export RSYNC_RSH="ssh -o ServerAliveInterval=30 -o ServerAliveCountMax=6"

if [ "${BACKUP_SOURCE}" == "" ]; then
error_exit 1 "No backup source specified! Where am I supposed to be copying files from?"
fi

if [ "${BACKUP_FOLDER}" == "" ]; then
error_exit 1 "No backup folder specified! Where am I supposed to be sending the files to?"
fi

if [ "${BACKUP_TIMECODE}" == "" ]; then
error_exit 1 "No backup timecode specified! Why are you overriding the default with nothing?"
fi

BACKUP_DESTINATION="$(find "${BACKUP_FOLDER}" -maxdepth 1 -mindepth 1 -type d -name 'inprogress-[0-9]???????_??????' | sort | tail -n 1)"
[ "${BACKUP_DESTINATION}" == "" ] && BACKUP_DESTINATION="${BACKUP_FOLDER}/inprogress-${BACKUP_TIMECODE}"
BACKUP_DESTINATION_FINAL="${BACKUP_FOLDER}/${BACKUP_TIMECODE}"

logit "Commencing backup from: ${BACKUP_SOURCE} to: ${BACKUP_DESTINATION}"

# Check destination folder for a previous, completed rsync backup from which to hardlink unchanged files
BACKUP_LINKDEST="$(find "${BACKUP_FOLDER}" -mindepth 1 -maxdepth 1 -type d -name "[0-9]???????_??????" | sort | tail -n 1)"

# If no previous backup has been found, then don't include this
if [ "${BACKUP_LINKDEST}" == "" ]; then
logit "No previous backup found from which to hard link"
BACKUP_LINKDEST=""
else
logit "Previous backup found: ${BACKUP_LINKDEST}"
BACKUP_LINKDEST="--link-dest=${BACKUP_LINKDEST}"
fi

logit "Commencing rsync"
echo rsync $RSYNC_OPTIONS "${BACKUP_LINKDEST}" "${BACKUP_SOURCE}" "${BACKUP_DESTINATION}" || error_exit $? "rsync failed!"

if [ "${BACKUP_LINKDEST}" == "" ]; then
rsync $RSYNC_OPTIONS ${BACKUP_SOURCE} "${BACKUP_DESTINATION}"
rsync_success="$?"
else
rsync "${BACKUP_LINKDEST}" $RSYNC_OPTIONS ${BACKUP_SOURCE} "${BACKUP_DESTINATION}"
rsync_success="$?"
fi

echo "Rsync return value: ${rsync_success}"

case "${rsync_success}" in
0|24)
echo "rsync success!"
;;
*)
error_exit $? "rsync failed!"
;;
esac

logit "rsync complete"
mv -v "${BACKUP_DESTINATION}" "${BACKUP_DESTINATION_FINAL}" || error_exit $? "Failed moving: ${BACKUP_DESTINATION} to: ${BACKUP_DESTINATION_FINAL}"

logit "Moved: ${BACKUP_DESTINATION} to: ${BACKUP_DESTINATION_FINAL}"

It can be cron'd if the media's available, or triggered by udev rule (what it was initially written for). It doesn't accept arguments, but this'd be trivial to add back in. There are probably better ways to do this, but if you've got bash shell, SSH, rsync and a filesystem that rsync can hardlink on, this'll do it.

Did you find this helpful? Feel free to drop me a line!