New Backup Method with Backblaze Using RCLONE

Post Reply
User avatar
webxtek
Posts: 67
Joined: Wed Nov 18, 2020 7:43 pm
Been thanked: 2 times

Hi everyone,

Back in 2020, I created a guide viewtopic.php?t=275 for setting up backups on MyVestaCP. However, with a fresh installation of Debian 12.9, I decided to take a different approach to optimize the process. Instead of using the old method, I now use RCLONE to sync my backup folder with Backblaze, which offers more flexibility and security.

Here's how you can set it up:
  • Log into the Admin Panel: Start by logging into your MyVestaCP admin panel.
  • Access the Backup Section: In the panel, go to the "Server" section and click on "Configure".
Screenshot 2025-01-20 120055.png
Screenshot 2025-01-20 120055.png (36.97 KiB)
  • Enable Local Backup: In the backup section, enable "Local Backup" by setting it to "Yes". This will allow MyVestaCP to make local backups before syncing them to the cloud.
Screenshot 2025-01-20 123241.png
Screenshot 2025-01-20 123241.png (37.5 KiB)
  • choose how often you want the server to do the backups on cron tab edit the cron for v-backup-users
  • Go on your Backblaze panel, create a 1) bucket and 2) create some Application Keys, make sure those keys have access to your bucket...
To sync with Backblaze, you need to install RCLONE on your server. Here's how you do it:
  • SSH into your server as root and install RCLONE:

    Code: Select all

    curl https://rclone.org/install.sh | sudo bash
  • Once installed, you can configure RCLONE by running:

    Code: Select all

    rclone config
    I named it "backup-weekly" because I will be using it as a weekly backup, but you can name it whatever you want.
  • Next, modify the v-backup-users script located in /usr/local/vesta/bin/

    Code: Select all

    sudo nano /usr/local/vesta/bin/v-backup-users
    You can replace the existing script with the following content:

Code: Select all

#!/bin/bash

scriptname="v-backup-users"
for pid in $(pidof -x "$scriptname"); do
    if [ $pid != $$ ]; then
        echo "[$(date)] : $scriptname : Process is already running with PID $pid" | tee -a $log
        exit 1
    fi
done

source /etc/profile
source $VESTA/func/main.sh
source $VESTA/conf/vesta.conf

ALLOW_MYSQL_REPAIR=1
if [ $# -ge 1 ]; then
    ALLOW_MYSQL_REPAIR=$1
fi

log=$VESTA/log/backup.log
summary=$VESTA/log/backup_summary.log
month_year=$(date +"%B-%Y" | tr ' ' '-')
remote_base_path="backup-weekly:x078-WEEKLY/Backups/$month_year"

NOTIFY_ADMIN_FULL_BACKUP=$(grep CONTACT $VESTA/data/users/admin/user.conf | cut -f 2 -d \')

if [ -z "$NOTIFY_ADMIN_FULL_BACKUP" ]; then
    echo "[$(date)] : Admin email not configured. Exiting." | tee -a $log
    exit 1
fi

touch $log
mv $log $log-$(date +"%Y-%m-%d--%H:%M:%S")
touch $summary
echo "Backup Summary - $(date)" > $summary
echo "=====================" >> $summary

if [ -d "/backup" ]; then
    find /backup -mindepth 1 -delete
    echo "[$(date)] : Directory /backup cleaned before starting the backup process." | tee -a $log
else
    mkdir -p /backup
    echo "[$(date)] : Directory /backup created." | tee -a $log
fi

if [ $ALLOW_MYSQL_REPAIR -eq 1 ]; then
    echo "[$(date)] : Database repair started." | tee -a $log
    nice -n 19 ionice -c 3 mariadb-check --all-databases --check --auto-repair | tee -a $log 2>&1
fi

FINAL_STATUS='SUCCESS'
SUCCESS_USERS=()
FAILED_USERS=()

for user in $(grep '@' /etc/passwd | cut -f1 -d:); do
    if [ ! -f "$VESTA/data/users/$user/user.conf" ]; then
        continue
    fi
    check_suspend=$(grep "SUSPENDED='no'" $VESTA/data/users/$user/user.conf)
    if [ ! -z "$check_suspend" ]; then
        echo "[$(date)] : Starting backup for user $user." | tee -a $log
        nice -n 19 ionice -c 3 $BIN/v-backup-user $user | tee -a $log 2>&1
        STATUS=$?
        
        # Check if backup command failed
        if [ $STATUS -ne 0 ]; then
            echo "[$(date)] : Error creating backup for user $user (exit code: $STATUS)." | tee -a $log
            FINAL_STATUS='CONTAINS ERRORS !!!'
            FAILED_USERS+=("$user")
            continue
        fi
        
        # Get all files created for this user (including .tar and .log files)
        user_backup_files=$(ls /backup 2>/dev/null | grep "^$user\.")
        
        if [ -z "$user_backup_files" ]; then
            echo "[$(date)] : No backup files found for user $user." | tee -a $log
            FINAL_STATUS='CONTAINS ERRORS !!!'
            FAILED_USERS+=("$user")
            continue
        fi
        
        # Check if we have valid .tar backup files
        user_tar_files=$(echo "$user_backup_files" | grep "\.tar$")
        user_log_files=$(echo "$user_backup_files" | grep "\.log$")
        
        if [ -z "$user_tar_files" ] && [ ! -z "$user_log_files" ]; then
            echo "[$(date)] : Backup failed for user $user - only log files created, no valid backup archive." | tee -a $log
            FINAL_STATUS='CONTAINS ERRORS !!!'
            FAILED_USERS+=("$user")
            # Clean up log files that shouldn't be uploaded as backups
            for log_file in $user_log_files; do
                rm -f "/backup/$log_file"
                echo "[$(date)] : Removed error log file $log_file for user $user." | tee -a $log
            done
            continue
        fi
        
        # Validate .tar files for integrity and minimum size
        valid_tar_files=""
        for tar_file in $user_tar_files; do
            if [ ! -f "/backup/$tar_file" ]; then
                echo "[$(date)] : Backup file /backup/$tar_file not found for user $user." | tee -a $log
                FINAL_STATUS='CONTAINS ERRORS !!!'
                FAILED_USERS+=("$user")
                continue 2
            fi
            
            # Check if file size is 0 or very small (less than 1MB might indicate issues)
            file_size=$(stat -c%s "/backup/$tar_file" 2>/dev/null || echo "0")
            if [ "$file_size" -eq 0 ]; then
                echo "[$(date)] : Backup file $tar_file for user $user is empty (0 bytes)." | tee -a $log
                FINAL_STATUS='CONTAINS ERRORS !!!'
                FAILED_USERS+=("$user")
                rm -f "/backup/$tar_file"
                continue 2
            elif [ "$file_size" -lt 1048576 ]; then  # Less than 1MB
                echo "[$(date)] : Warning: Backup file $tar_file for user $user is very small ($file_size bytes). This might indicate an incomplete backup." | tee -a $log
                # Continue but log the warning - small backups might be legitimate for minimal users
            fi
            
            # Test tar file integrity
            if ! tar -tf "/backup/$tar_file" >/dev/null 2>&1; then
                echo "[$(date)] : Backup file $tar_file for user $user is corrupted or not a valid tar archive." | tee -a $log
                FINAL_STATUS='CONTAINS ERRORS !!!'
                FAILED_USERS+=("$user")
                rm -f "/backup/$tar_file"
                continue 2
            fi
            
            valid_tar_files="$valid_tar_files $tar_file"
        done
        
        # If no valid tar files remain after validation
        if [ -z "$valid_tar_files" ]; then
            echo "[$(date)] : No valid backup files remain for user $user after integrity checks." | tee -a $log
            FINAL_STATUS='CONTAINS ERRORS !!!'
            FAILED_USERS+=("$user")
            continue
        fi
        
        # Upload valid backup files only
        user_remote_path="$remote_base_path/$user"
        backup_success=true
        
        for backup_file in $valid_tar_files; do
            echo "[$(date)] : Uploading $backup_file to $user_remote_path on Backblaze." | tee -a $log
            
            # Check available disk space before upload (basic check)
            available_space=$(df /backup | awk 'NR==2 {print $4}')
            if [ "$available_space" -lt 1048576 ]; then  # Less than 1GB free
                echo "[$(date)] : Warning: Low disk space detected ($available_space KB free) during backup of $user." | tee -a $log
            fi
            
            rclone copy "/backup/$backup_file" "$user_remote_path/" | tee -a $log 2>&1
            upload_status=$?
            
            if [ $upload_status -ne 0 ]; then
                echo "[$(date)] : Error uploading $backup_file to Backblaze (exit code: $upload_status)." | tee -a $log
                FINAL_STATUS='CONTAINS ERRORS !!!'
                backup_success=false
            else
                # Verify file was actually uploaded by checking if rclone can see it
                echo "[$(date)] : Verifying upload of $backup_file..." | tee -a $log
                if rclone lsf "$user_remote_path/$backup_file" >/dev/null 2>&1; then
                    rm -f "/backup/$backup_file"
                    echo "[$(date)] : $backup_file uploaded successfully and removed locally." | tee -a $log
                else
                    echo "[$(date)] : Upload verification failed for $backup_file - file not found on remote." | tee -a $log
                    FINAL_STATUS='CONTAINS ERRORS !!!'
                    backup_success=false
                fi
            fi
        done
        
        # Clean up any remaining log files after successful backup upload
        for log_file in $user_log_files; do
            rm -f "/backup/$log_file"
            echo "[$(date)] : Cleaned up log file $log_file for user $user." | tee -a $log
        done
        
        # Determine final status for this user
        if [ "$backup_success" = true ]; then
            SUCCESS_USERS+=("$user")
        else
            FAILED_USERS+=("$user")
        fi
    fi
done

if [ "$FINAL_STATUS" = "SUCCESS" ]; then
    find /backup -mindepth 1 -delete
    echo "[$(date)] : Process completed successfully. Directory /backup cleaned." | tee -a $log
else
    echo "[$(date)] : Errors occurred during the process. Directory /backup was not cleaned." | tee -a $log
fi

echo "Backup Summary" >> $summary
echo "=====================" >> $summary
echo "Overall Status: $FINAL_STATUS" >> $summary
echo "" >> $summary
echo "Successful Backups:" >> $summary
if [ ${#SUCCESS_USERS[@]} -eq 0 ]; then
    echo "  - None" >> $summary
else
    for user in "${SUCCESS_USERS[@]}"; do
        echo "  - $user" >> $summary
    done
fi
echo "" >> $summary
echo "Failed Backups:" >> $summary
if [ ${#FAILED_USERS[@]} -eq 0 ]; then
    echo "  - None" >> $summary
else
    for user in "${FAILED_USERS[@]}"; do
        echo "  - $user" >> $summary
    done
fi

email_subject="Backup Summary Report for $HOSTNAME"
if [ "$FINAL_STATUS" != "SUCCESS" ]; then
    email_subject="[ERROR] Backup Summary Report for $HOSTNAME - CONTAINS ERRORS"
else
    email_subject="[SUCCESS] Backup Summary Report for $HOSTNAME"
fi
log_attachment="/tmp/backup_log_$(date +"%Y-%m-%d_%H-%M-%S").log"
cp $log $log_attachment

(
    echo "Subject: $email_subject"
    echo "MIME-Version: 1.0"
    echo "Content-Type: multipart/mixed; boundary=\"boundary\""
    echo
    echo "--boundary"
    echo "Content-Type: text/plain; charset=UTF-8"
    echo
    cat $summary
    echo "--boundary"
    echo "Content-Type: text/plain; name=$(basename $log_attachment)"
    echo "Content-Disposition: attachment; filename=$(basename $log_attachment)"
    echo
    cat $log_attachment
    echo "--boundary--"
) | /usr/sbin/sendmail "$NOTIFY_ADMIN_FULL_BACKUP"

rm -f $log_attachment

exit
  • Don't forget to change line 23 with your BUCKET NAME and the name you gave the RCLONE:

    Code: Select all

    remote_base_path="backup-weekly:BUCKETNAME/Backups/$month_year"
Last edited by webxtek on Tue Jun 03, 2025 12:40 am, edited 3 times in total.
User avatar
webxtek
Posts: 67
Joined: Wed Nov 18, 2020 7:43 pm
Been thanked: 2 times

  • You can manually test the script through SSH before scheduling it with cron. To do so, run the following command to execute the backup for all users:

    Code: Select all

    sudo /usr/local/vesta/bin/v-backup-users
    This command will start the backup process for all users, and you can check the generated logs to ensure the script is working properly.
User avatar
webxtek
Posts: 67
Joined: Wed Nov 18, 2020 7:43 pm
Been thanked: 2 times

To monitor the log as new entries are added, run the following command:

Code: Select all

sudo tail -f /usr/local/vesta/log/backup.log
This will display the log updates live, which is particularly useful to track the backup script's progress in real-time.
If you need to review the entire log file:

Code: Select all

sudo cat /usr/local/vesta/log/backup.log
or

Code: Select all

sudo less /usr/local/vesta/log/backup.log
less allows you to scroll through the log file, while cat will print the entire content at once.
User avatar
webxtek
Posts: 67
Joined: Wed Nov 18, 2020 7:43 pm
Been thanked: 2 times

Ensure Your Custom v-backup-users Still Exists if myVesta Updates Before Backup with a Cron Job

I’ve set up a way to ensure my custom v-backup-users script remains active before backups, even if myVesta updates overwrite it. This uses a cron job to restore the custom version 1 hour before the backup. Here’s a quick guide:

Step 1: Back Up Your Custom Script
Save your modified v-backup-users in a safe location:

Code: Select all

mkdir -p /root/myvesta-custom-scripts
cp /usr/local/vesta/bin/v-backup-users /root/myvesta-custom-scripts/v-backup-users
Step 2: Create a Script to Restore Your Custom Version
Create /root/vesta-patch.sh to restore your custom script:

Code: Select all

nano /root/vesta-patch.sh
Add this content:

Code: Select all

#!/bin/bash
# Check and replace if the script differs
if ! cmp -s "/root/myvesta-custom-scripts/v-backup-users" "/usr/local/vesta/bin/v-backup-users"; then
    cp /root/myvesta-custom-scripts/v-backup-users /usr/local/vesta/bin/v-backup-users
    chmod 755 /usr/local/vesta/bin/v-backup-users
fi
Save (Ctrl+O, Enter, Ctrl+X) and make it executable:

Code: Select all

chmod 755 /root/vesta-patch.sh

Step 3: Test the Script

Code: Select all

/root/vesta-patch.sh
Check if it updated correctly:

Code: Select all

diff /usr/local/vesta/bin/v-backup-users /root/myvesta-custom-scripts/v-backup-users
(No output means success.)

Step 4: Find Your Backup Schedule on your Vesta panel and set a Cron Job to Run 1 Hour Before Backup

Code: Select all

crontab -e
Add a line to run /root/vesta-patch.sh 1 hour before the backup. For my case:

Code: Select all

0 1 * * 7 /root/vesta-patch.sh
Save and verify:

Code: Select all

crontab -l

--------------------------------------
if you ever decide to modify v-backup-users, update the copy:

Code: Select all

cp /usr/local/vesta/bin/v-backup-users /root/myvesta-custom-scripts/v-backup-users
Post Reply