Simple site backups to S3

Within the past month or so I’ve been as busy as can be working on building a production backend and frontend for a new cloud product, standing up monitoring for infrastructure and our clients, standing up elasticsearch, securing internal network and a host of other things. One of the things on my list was to ensure that our internal systems had proper backups and since AWS provides an affordable option for storage my company went with S3.

Below is a simple bash script template to use for backing up data to S3

#!/bin/bash

# Backup Settings
DB_NAME='db_name'
DB_USER='db_user'
DB_PASS='db_pass'
EPOCH=`date +%d%m%y%H%M`
BACKUP_DIR='/backup/'
DB_BACKUP_FILE="/backup/db/$DB_NAME-$EPOCH.sql.gz"
SITE_BACKUP_FILE="/backup/site/files-$EPOCH.tar.gz"
CONFIG_BACKUP_FILE="/backup/config/conf-$EPOCH.tar.gz"
SITE_PATH='/home/user/public'
MAIL_TO='noc@domain.com'

notify_noc() {
    MESSAGE=$1;
    /bin/mail -s "Backup failure on $HOSTNAME" $MAIL_TO
}

echo "Backing up database......."
/bin/mysqldump -u $DB_USER -p$DB_PASS $DB_NAME | /bin/gzip > $DB_BACKUP_FILE
if [ $? -eq 0 ]; then
    echo "Successfully backed up to $DB_BACKUP_FILE"
else
    echo "DB backup failed on $HOSTNAME"
    notify_noc "DB backup for $DB_NAME failed on $HOSTNAME"
fi

echo "Backing up site files......."
/bin/tar czf $SITE_BACKUP_FILE -C / $SITE_PATH
if [ $? -eq 0 ]; then
    echo "Successfully backed up site files to $SITE_BACKUP_FILE"
else
    echo "Tar failed on $HOSTNAME"
    notify_noc "Tar backup failed on $HOSTNAME"
fi

echo "Backing up config files......."
/bin/tar czf $CONFIG_BACKUP_FILE -C / /etc
if [ $? -eq 0 ]; then
    echo "Successfully backed up config files to $CONFIG_BACKUP_FILE"
else 
    echo "Tar failed on $HOSTNAME"
    notify_noc "Tar of config files failed on $HOSTNAME"
fi

echo "Cleaning up old backup files......."
find $BACKUP_DIR -mtime +5 -type f -delete;
if [ $? -eq 0 ]; then
    echo "Successfully cleaned up old backup files in $BACKUP_DIR"
else
    echo "Unable to clean up old backup files on $HOSTNAME"
    notify_noc "Unable to clean up old backup files on $HOSTNAME"
fi

/usr/local/bin/aws s3 sync /backup/ s3://backups.domain.com/backup/web/domain.com
if [ $? -eq 0 ]; then
    echo "Successfully backed up to AWS S3"
else
    echo "Failed backing up to AWS S3"
    notify_noc "Failed backing up to AWS S3 on $HOSTNAME"
fi

Now this script does have requirements of installing the AWS cli tool which you can do on any Linux box using the following commands.

curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" -o "awscli-bundle.zip"
unzip awscli-bundle.zip
sudo ./awscli-bundle/install -i /usr/local/aws -b /usr/local/bin/aws