shell script to sync folder to storage node.

[Greg Plamondon]

I have modified a shell script that was in the fogproject git hub to update all storage nodes.

I can get the shell script to run from an ssh session but not from a cronjob.
does anyone have any suggestions?

#!/bin/bash
export TERM=xterm
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
array=( 10fog-helpdesk 11storage 13storage 14storage 15storage 17storage 19storage 21storage 23storage 25storage 27storage 28storage 29storage 30storage 32storage 34storage 36storage 38storage 43storage 44storage )
LOCKFILE=/tmp/synclock.txt
if [ -e ${LOCKFILE} ] && kill -0 `cat ${LOCKFILE}`; then
    echo "already running"
    exit
fi
# make sure the lockfile is removed when we exit and then claim it
trap "rm -f ${LOCKFILE}; exit" INT TERM EXIT
echo $$ > ${LOCKFILE}

clear
echo
echo
echo "Syncing defined systems."
echo
for i in "${array[@]}"
do
    printf "FOG Sync Started at: $i..."
    successCheck=$(rsync --progress --stats --prune-empty-dirs --delete --bwlimit=500 -avz /images/Sync root@$i:/images/ > /var/log/fog/shopsync.$i.log; rsync --progress --stats --prune-empty-dirs --delete --bwlimit=500 -avz /images/drivers root@$i:/images/ > /var/log/fog/driversync.$i.log;)
    if [[ "$successCheck" -eq 0 ]]; then
        printf "Success!\n"
    else
        printf "Failed!\n"
    fi
    printf "\n"
done
#sleep 1000
rm -f ${LOCKFILE}

I have ssh-copy-id root@10fog-helpdesk to imort my key
I am able too shh root@10fog-helpdesk without a password with no issues.
This is what the cronjob log looks like.

FOG Sync Started at: 10fog-helpdesk...Permission denied, please try again.
Permission denied, please try again.
Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: unexplained error (code 255) at io.c(226) [sender=3.1.2]
Permission denied, please try again.
Permission denied, please try again.
Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: unexplained error (code 255) at io.c(226) [sender=3.1.2]
Success!

Any pointers?

Thanks!

[Sebastian Roth]

@Greg-Plamondon How did you setup the cronjob? Or to be more specific - which user runs the cronjob? See here: https://serverfault.com/questions/352835/crontab-running-as-a-specific-user

[Wayne Workman]

Have you tried ssh aliases?

For example, if you are root and set the file /root/.ssh/config as such:

Host fog-server
    User root
    HostName 10.0.0.28
    Port 22

You can then just type ssh fog-server and bam, you’re in.
Makes scripting a bit simpler… Particularly makes getting around in your systems faster while in the shell.

Agreeing with @Sebastian-Roth though. If ssh works manually, but doesn’t in cron, it’s maybe a user issue. You can figure out who it’s executing as like such:
whoami > /tmp/whoami.txt 2>&1
Put that towards the top of your script, see what it says.

[Greg Plamondon]

@Sebastian-Roth the cronjob runs hourly as root.

[Greg Plamondon]

@Wayne-Workman contents of /tmp/whoami.txt is “root”

[Wayne Workman]

@Greg-Plamondon My advice: start eliminating variables, start reducing complexity. Create a new script that has a single command to sync one of your nodes - don’t use any variables, hard code everything. See if that works. If it doesn’t, start eliminating arguments one by one until you find the one that is problematic. Also try to sync test directories. Also look at the permissions of the directories you are syncing both locally and remotely.

[Sebastian Roth]

@Greg-Plamondon A quick search engine foo revealed that in cron the environment is not set appropriately to use the SSH keys: https://stackoverflow.com/a/50999470

I think the most easy solution in your case is to use rsync ... -avze "ssh -i /full/path/to/ssh_private_key" /images/Sync root@$i:/images/ ...

[george1421]

While I haven’t really looked hard into this I have often wondered if FOG should replace its replicator with rsync. I know its a bit off topic here. But from what I remember that rsync can be run in a couple of different modes rsync->ssh as well as rsync(sender)->rsync(receiver). I haven’t really looked into it (at all) but I wonder if the rsync->rsync solution would give a better experience?

[Sebastian Roth]

@george1421 I have started to think about getting rid of FTP and NFS altogether because both are very old, rubbish protocols that cause head aces as soon as you leave the local subnet. But it’s a long way to get there and I don’t see this happen soon.

[Greg Plamondon]

@Sebastian-Roth Thanks for the input,
I was able to figure out what was going wrong. when I was ssh-copy-d root@storagenode it was copying the sshkey i use to login to the server. I needed to use ssh-copy-id -i /root/.ssh/id_rsa and then your suggestio of rsync ... -avze "ssh -i /full/path/to/ssh_private_key" /images/Sync root@$i:/images/ ... worked great.

Thanks!