shell script to sync folder to storage node.


  • Testers

    I have modified a shell script that was in the fogproject git hub to update all storage nodes.

    I can get the shell script to run from an ssh session but not from a cronjob.
    does anyone have any suggestions?

    #!/bin/bash
    export TERM=xterm
    PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
    array=( 10fog-helpdesk 11storage 13storage 14storage 15storage 17storage 19storage 21storage 23storage 25storage 27storage 28storage 29storage 30storage 32storage 34storage 36storage 38storage 43storage 44storage )
    LOCKFILE=/tmp/synclock.txt
    if [ -e ${LOCKFILE} ] && kill -0 `cat ${LOCKFILE}`; then
        echo "already running"
        exit
    fi
    # make sure the lockfile is removed when we exit and then claim it
    trap "rm -f ${LOCKFILE}; exit" INT TERM EXIT
    echo $$ > ${LOCKFILE}
    
    clear
    echo
    echo
    echo "Syncing defined systems."
    echo
    for i in "${array[@]}"
    do
        printf "FOG Sync Started at: $i..."
        successCheck=$(rsync --progress --stats --prune-empty-dirs --delete --bwlimit=500 -avz /images/Sync root@$i:/images/ > /var/log/fog/shopsync.$i.log; rsync --progress --stats --prune-empty-dirs --delete --bwlimit=500 -avz /images/drivers root@$i:/images/ > /var/log/fog/driversync.$i.log;)
        if [[ "$successCheck" -eq 0 ]]; then
            printf "Success!\n"
        else
            printf "Failed!\n"
        fi
        printf "\n"
    done
    #sleep 1000
    rm -f ${LOCKFILE}
    

    I have ssh-copy-id root@10fog-helpdesk to imort my key
    I am able too shh root@10fog-helpdesk without a password with no issues.
    This is what the cronjob log looks like.

    FOG Sync Started at: 10fog-helpdesk...Permission denied, please try again.
    Permission denied, please try again.
    Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).
    rsync: connection unexpectedly closed (0 bytes received so far) [sender]
    rsync error: unexplained error (code 255) at io.c(226) [sender=3.1.2]
    Permission denied, please try again.
    Permission denied, please try again.
    Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).
    rsync: connection unexpectedly closed (0 bytes received so far) [sender]
    rsync error: unexplained error (code 255) at io.c(226) [sender=3.1.2]
    Success!
    

    Any pointers?

    Thanks!


  • Testers

    @Sebastian-Roth Thanks for the input,
    I was able to figure out what was going wrong. when I was ssh-copy-d root@storagenode it was copying the sshkey i use to login to the server. I needed to use ssh-copy-id -i /root/.ssh/id_rsa and then your suggestio of rsync ... -avze "ssh -i /full/path/to/ssh_private_key" /images/Sync root@$i:/images/ ... worked great.

    Thanks!


  • Developer

    @george1421 I have started to think about getting rid of FTP and NFS altogether because both are very old, rubbish protocols that cause head aces as soon as you leave the local subnet. But it’s a long way to get there and I don’t see this happen soon.


  • Moderator

    While I haven’t really looked hard into this I have often wondered if FOG should replace its replicator with rsync. I know its a bit off topic here. But from what I remember that rsync can be run in a couple of different modes rsync->ssh as well as rsync(sender)->rsync(receiver). I haven’t really looked into it (at all) but I wonder if the rsync->rsync solution would give a better experience?


  • Developer

    @Greg-Plamondon A quick search engine foo revealed that in cron the environment is not set appropriately to use the SSH keys: https://stackoverflow.com/a/50999470

    I think the most easy solution in your case is to use rsync ... -avze "ssh -i /full/path/to/ssh_private_key" /images/Sync root@$i:/images/ ...



  • @Greg-Plamondon My advice: start eliminating variables, start reducing complexity. Create a new script that has a single command to sync one of your nodes - don’t use any variables, hard code everything. See if that works. If it doesn’t, start eliminating arguments one by one until you find the one that is problematic. Also try to sync test directories. Also look at the permissions of the directories you are syncing both locally and remotely.


  • Testers

    @Wayne-Workman contents of /tmp/whoami.txt is “root”


  • Testers

    @Sebastian-Roth the cronjob runs hourly as root.



  • Have you tried ssh aliases?

    For example, if you are root and set the file /root/.ssh/config as such:

    Host fog-server
        User root
        HostName 10.0.0.28
        Port 22
    

    You can then just type ssh fog-server and bam, you’re in.
    Makes scripting a bit simpler… Particularly makes getting around in your systems faster while in the shell.

    Agreeing with @Sebastian-Roth though. If ssh works manually, but doesn’t in cron, it’s maybe a user issue. You can figure out who it’s executing as like such:
    whoami > /tmp/whoami.txt 2>&1
    Put that towards the top of your script, see what it says.


  • Developer

    @Greg-Plamondon How did you setup the cronjob? Or to be more specific - which user runs the cronjob? See here: https://serverfault.com/questions/352835/crontab-running-as-a-specific-user


Log in to reply
 

504
Online

6.3k
Users

13.7k
Topics

128.9k
Posts