Script for a backup ?

[Bristow 0]

At present, I clone my proxmox virtual machine for my backups and from time to time, I manually run a dump of the database and images.

I’d like to automate this backup via a cron. Do you have a script that allows me to :

• database dump
• rsync images to remote media

Perhaps you have something better to suggest? Thanks

[falko]

@Bristow-0 you may already do this. But you mentioned you clone the vm and not back it up. Would it not be best to run a proxmox backup of the fog server VM to another location like a Nas/share located elsewhere.

[Bristow 0]

@falko Yes, I already clone my VM on a remote space. But if my clone isn’t correct, I lose everything.
So I’d like a backup script that allows me to automate a database backup and an rsync to a remote share. You can never be too careful.

[rodluz]

@Bristow-0 A while back, I was trying to find an answer to your question, but I couldn’t find anything, so I just wrote it myself. It’s not perfect, but this could be a good starting point.

I have it set up on a Proxmox LXC container running Ubuntu 22.04 and it’s running on a cron job weekly. If you want to go this route, you just have to set up a few things on the container first.

• Make sure you can send emails from the container. I use msmtp, but you can use something else that you prefer, just make sure to change the command in the script.
• Make sure you can SSH into your FOG server from the container with an SSH key so that you don’t have to deal with username/password.
• Not extremely necessary but recommended. Add a mount point, to the container, of a separate storage location to save all the data to.
  • In the script, I have it set to /mnt/FOGBackups

Make sure that you read through the script and change the FOG information.

For some reason the forum doesn’t show the bash script correctly, so I am adding a link to a public gist with the code: https://gist.github.com/rluzuriaga/a2cd00cbff9a5cc70d0fb93afdd15566

Hope this helps!

[Bristow 0]

Thanks a lot @rodluz !

I took your script and adapted it. I use it directly on my FOG server, which simplifies things a bit. I also changed the mail to a notification sent to my smartphone with ntfy, which I find easy to use.

I also mounted my NAS share in NFS directly on /mnt/FOGBackups.

Here’s my script:

#!/bin/bash

fogServerAddress="172.X.Y.Z"  # Change to actual FOG server address

backupDate=$(date +"%Y%m%d")

backupDir="/mnt/FOGBackups/$backupDate"
backupDirImages="/mnt/FOGBackups/images"

snapinLocation="/opt/fog/snapins"
reportLocation="/var/www/fog/lib/reports"
imageLocation="/images"

failedBackupDB=0
failedBackupSnapins=0
failedBackupReports=0
failedBackupImages=0

[[ ! -d $backupDir ]] && mkdir -p $backupDir/{mysql,snapins,reports,logs} >/dev/null 2>&1
[[ ! -d $backupDir/mysql || $backupDir/snapins || $backupDir/reports || $backupDir/logs ]] && mkdir -p $backupDir/{mysql,snapins,reports,logs} >/dev/null 2>&1

backupDB() {
    wget --no-check-certificate --post-data="nojson=1" -O $backupDir/mysql/fog.sql "http://$fogServerAddress/fog/management/export.php?type=sql" 2>>$backupDir/logs/error.log 1>>$backupDir/logs/progress.log 2>&1
    stat=$?
    if [[ ! $stat -eq 0 ]]; then
        echo "Failed to backup database!"
        failedBackupDB=1
    else
        echo "Database backed up."
    fi
}

backupSnapins() {
    cp -r $snapinLocation $backupDir/snapins/ 2>>$backupDir/logs/error.log 1>>$backupDir/logs/progress.log 2>&1
    stat=$?
    if [[ ! $stat -eq 0 ]]; then
        echo "Failed to backup snapins!"
        failedBackupSnapins=1
    else
        echo "Snapins backed up."
    fi
}

backupReports() {
    cp -r $reportLocation $backupDir/reports/ 2>>$backupDir/logs/error.log 1>>$backupDir/logs/progress.log 2>&1
    stat=$?
    if [[ ! $stat -eq 0 ]]; then
        echo "Failed to backup reports!"
        failedBackupReports=1
    else
        echo "Reports backed up."
    fi
}

backupImages() {
    rsync -auv $imageLocation $backupDirImages 2>>$backupDir/logs/error.log 1>>$backupDir/logs/progress.log 2>&1
    stat=$?
    if [[ ! $stat -eq 0 ]]; then
        echo "Failed to backup images!"
        failedBackupImages=1
    else
        echo "Images backed up."
    fi
}

checkForFailures() {
    echo "Running checkForFailures()"
    if [[ $failedBackupDB -eq 1 ]]; then
        message="$message \nFailed to backup database."
    fi

    if [[ $failedBackupSnapins -eq 1 ]]; then
        message="$message \nFailed to backup snapins."
    fi

    if [[ $failedBackupReports -eq 1 ]]; then
        message="$message \nFailed to backup reports."
    fi
    if [[ $failedBackupImages -eq 1 ]]; then
        message="$message \nFailed to backup images."
    fi
}

sendNotif() {
	echo "Running sendNotif()"
    if [[ ! -z $message ]]; then
        {
            curl -d "FOG Backup Failed\n $message" ntfy.sh/XYZATEHFJICH
		}
	else
		curl -d "FOG Backup Successful!" ntfy.sh/XYZATEHFJICH
	fi
}

backupDB
backupSnapins
backupReports
backupImages

checkForFailures
sendNotif

[Bristow 0]

I’m revisiting this script, which no longer works for backing up the database because authentication is now required.

Is it possible to modify this line to include authentication? Or, quite simply, is there a way to back up the database via the command line?

wget --no-check-certificate --post-data="nojson=1" -O $backupDir/mysql/fog.sql "http://$fogServerAddress/fog/management/export.php?type=sql" 2>>$backupDir/logs/error.log 1>>$backupDir/logs/progress.log 2>&1

Thanks in advance

[Tom Elliott]

@Bristow-0 I’m working on a fix for this, however it will likley mean you need ot setup and get your api token and user api token.

Basically, my thought is to create an API endpoint that allows you to get the database backup, but securely.

The reason:

All your AD and username secrets are stored in the database, so anyone who can download the database now has all the access credentials to all the data in your environment.

I think you can imagine the repercussions this could cause and I apologize we even had this open wide gap for any period of time.

I want this to continue to work for you, but we need this to be secure too, working on it now:

The issue is documented as well the api endpoint documentation:
https://github.com/FOGProject/fogproject/issues/827

Hope this helps.

Testing this I did:

curl -ik -X GET "https://10.255.20.1/fog/system/export" -H "fog-api-token: $FOG_API" -H "fog-user-token: $FOG_USR" -o 'test_db.sql'

Of course changing the IP to yours and setting your variables where required would make sense.

Your script would be adjusted to:

#!/bin/bash

fogServerAddress="172.X.Y.Z"  # Change to actual FOG server address

fogApiToken="YOURFOGAPITOKEN" # Change to FOG Configuration -> Settings -> FOG API Token (ensure api is enabled for global access)
fogUsrToken="YOURFOGUSERAPITOKEN" # Change to User -> API Token (ensure api is enabled for user.)

backupDate=$(date +"%Y%m%d")

backupDir="/mnt/FOGBackups/$backupDate"
backupDirImages="/mnt/FOGBackups/images"

snapinLocation="/opt/fog/snapins"
reportLocation="/var/www/fog/lib/reports"
imageLocation="/images"

failedBackupDB=0
failedBackupSnapins=0
failedBackupReports=0
failedBackupImages=0

[[ ! -d $backupDir ]] && mkdir -p $backupDir/{mysql,snapins,reports,logs} >/dev/null 2>&1
[[ ! -d $backupDir/mysql || $backupDir/snapins || $backupDir/reports || $backupDir/logs ]] && mkdir -p $backupDir/{mysql,snapins,reports,logs} >/dev/null 2>&1

backupDB() {
    curl -ik -X GET "http://$fogServer/fog/system/export" -H "fog-api-token: $fogApiToken" -H "fog-user-token: $fogUsrToken" -o $backupDir/mysql/fog.sql 2>>$backupDir/logs/error.log 1>>$backupDir/logs/progress.log 2>&1
    stat=$?
    if [[ ! $stat -eq 0 ]]; then
        echo "Failed to backup database!"
        failedBackupDB=1
    else
        echo "Database backed up."
    fi
}

backupSnapins() {
    cp -r $snapinLocation $backupDir/snapins/ 2>>$backupDir/logs/error.log 1>>$backupDir/logs/progress.log 2>&1
    stat=$?
    if [[ ! $stat -eq 0 ]]; then
        echo "Failed to backup snapins!"
        failedBackupSnapins=1
    else
        echo "Snapins backed up."
    fi
}

backupReports() {
    cp -r $reportLocation $backupDir/reports/ 2>>$backupDir/logs/error.log 1>>$backupDir/logs/progress.log 2>&1
    stat=$?
    if [[ ! $stat -eq 0 ]]; then
        echo "Failed to backup reports!"
        failedBackupReports=1
    else
        echo "Reports backed up."
    fi
}

backupImages() {
    rsync -auv $imageLocation $backupDirImages 2>>$backupDir/logs/error.log 1>>$backupDir/logs/progress.log 2>&1
    stat=$?
    if [[ ! $stat -eq 0 ]]; then
        echo "Failed to backup images!"
        failedBackupImages=1
    else
        echo "Images backed up."
    fi
}

checkForFailures() {
    echo "Running checkForFailures()"
    if [[ $failedBackupDB -eq 1 ]]; then
        message="$message \nFailed to backup database."
    fi

    if [[ $failedBackupSnapins -eq 1 ]]; then
        message="$message \nFailed to backup snapins."
    fi

    if [[ $failedBackupReports -eq 1 ]]; then
        message="$message \nFailed to backup reports."
    fi
    if [[ $failedBackupImages -eq 1 ]]; then
        message="$message \nFailed to backup images."
    fi
}

sendNotif() {
	echo "Running sendNotif()"
    if [[ ! -z $message ]]; then
        {
            curl -d "FOG Backup Failed\n $message" ntfy.sh/XYZATEHFJICH
		}
	else
		curl -d "FOG Backup Successful!" ntfy.sh/XYZATEHFJICH
	fi
}

backupDB
backupSnapins
backupReports
backupImages

checkForFailures
sendNotif

[Bristow 0]

@Tom-Elliott Thanks for your quick reply!

Yes, I fully understand the need for security; in fact, it did surprise me (even though I don’t have any AD in my setup).

I’ll be testing the command very soon and will get back to you (I’m not at my school at the moment). Do I need to upgrade the FOG server?

[Tom Elliott]

@Bristow-0 You will need to upgrade to either dev-branch or wait until later today/tomorrow and update to stable. I backported this into the dev-branch as well the working-1.6 branch for consistency and ease.