Full backups using Duplicity, Rsync and GPG

28 || 0  || 2018-04-14 06:02:41 || root || || Bash

The script below allows you to create system backups on Linux systems using Duplicity, Rsync and GPG. Check comments in the code and don't forget to replace some placeholders with real values.

There are several snippets related to this topic:

  1. Incremental backups (duplicity + rsync + gpg)
  2. Full backups (duplicity + rsync + gpg)
  3. Backup to Amazon S3
  4. Configuring GPG keys

Please check the code to find links for other code snippets.

#!/bin/sh

# Incremental backups: https://snippets.yoctocloud.cc/snippets/2b0b3e2bc4714366b4e7afbc1ba555c4/
# Full backups: https://snippets.yoctocloud.cc/snippets/df3feac1d1e84945a5dd906219adfd55/
# Backups to S3: https://snippets.yoctocloud.cc/snippets/6c351cfa605a41c2bccd4588f4ee9b5a/
# Configuring GPG: soon

# You need to create your own GPG key before using this script
keyID="GPG_KEY_ID"
# This directory will be used for making backups
srcPath="/home/user/"
# Backups will be copied to server (192.168.1.100) using rsync. Install rsync on both server and client and configure in
dstPath="rsync://192.168.1.100//mnt/2tb/backups/laptop"
# This file contains list of excluded files/directories (one path per line)
excList="/home/user/backup_exclude_list.txt"
# Choose backup name if you are going to backup several devices
backupName="laptop_hp"

# Modify several lines below. They are related to email notifications. Use real email addresses, host and port
mailFrom="Backups <backups@example.com>"
mailTo="Firstname Lastname <firstname_lastname@gmail.com>"
mailSubject="Failed backup on HP laptop"
mailBody="Latest incremental backup process on HP laptop has been finished with errors. Please check script manually and try again later. You can find error details below:"
mailServer="email-smtp.eu-west-1.amazonaws.com"
mailPort=587

# This file contains GPG and email credentials, one per line (PASSPHRASE="", mailLogin="", mailPass="")
test -x $(which duplicity) || exit 0
. /root/.duplicity_file
export PASSPHRASE

# Command for making incremental backups
duplicity_output=$(/usr/bin/duplicity full --encrypt-key "$keyID" --sign-key "$keyID" --exclude-filelist "$excList" --name "$backupName" --num-retries 5 --timeout 30 --volsize 50 --asynchronous-upload "$srcPath" "$dstPath" 2>&1)
exit_code=$?

# If backup is finished with errors for some reason, script will send email about that
# Otherwise script will verify the latest full backup
if [ $exit_code != 0 ]
then
    mailBody="$mailBody\n\n\n$duplicity_output"
    /usr/bin/sendEmail -f "$mailFrom" -t "$mailTo" -u "$mailSubject" -m "$mailBody" -s "${mailServer}:${mailPort}" -o message-charset=utf-8 -o tls=yes -xu "$mailLogin" -xp "$mailPass"
    exit 1
else
    verify_output=$(/usr/bin/duplicity verify --encrypt-key "$keyID" --sign-key "$keyID" --name "$backupName" --num-retries 5 --timeout 30 "$dstPath" "$srcPath" 2>&1)
    verify_status=$?

# If backup is finished with errors for some reason, script will send email about that
# Otherwise script will remove backups older than 3rd full backup    
    if [ $verify_status != 0 ]
    then
        mailBody="$mailBody\n\n\n$verify_output"
        /usr/bin/sendEmail -f "$mailFrom" -t "$mailTo" -u "$mailSubject" -m "$mailBody" -s "${mailServer}:${mailPort}" -o message-charset=utf-8 -o tls=yes -xu "$mailLogin" -xp "$mailPass"
        exit 2
    else
        cleanup_output=$(/usr/bin/duplicity remove-all-but-n-full 3 --force --encrypt-key "$keyID" --sign-key "$keyID" --name "$backupName" --num-retries 5 --timeout 30 "$dstPath" 2>&1)
        cleanup_status=$?

# If backup is finished with errors for some reason, script will send email about that
# Otherwise script will remove all partial backups and other useless stuff 
        if [ $cleanup_status != 0 ]
        then
            mailBody="$mailBody\n\n\n$cleanup_output"
            /usr/bin/sendEmail -f "$mailFrom" -t "$mailTo" -u "$mailSubject" -m "$mailBody" -s "${mailServer}:${mailPort}" -o message-charset=utf-8 -o tls=yes -xu "$mailLogin" -xp "$mailPass"
            exit 3
        else
            extra_cleanup_output=$(/usr/bin/duplicity cleanup --force "$dstPath" 2>&1)
            extra_cleanup_status=$?

# If backup is finished with errors for some reason, script will send email about that
           if [ $extra_cleanup_status != 0 ]
           then
               mailBody="$mailBody\n\n\n$extra_cleanup_output"
               /usr/bin/sendEmail -f "$mailFrom" -t "$mailTo" -u "$mailSubject" -m "$mailBody" -s "${mailServer}:${mailPort}" -o message-charset=utf-8 -o tls=yes -xu "$mailLogin" -xp "$mailPass"
           fi
        fi
    fi
fi

exit 0