• Recent
  • Unsolved
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Register
  • Login
  • Recent
  • Unsolved
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Register
  • Login

shell script to sync folder to storage node.

Scheduled Pinned Locked Moved Solved
Linux Problems
4
10
1.0k
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • G
    Greg Plamondon Testers
    last edited by Greg Plamondon Oct 21, 2019, 2:49 PM Oct 21, 2019, 8:47 PM

    I have modified a shell script that was in the fogproject git hub to update all storage nodes.

    I can get the shell script to run from an ssh session but not from a cronjob.
    does anyone have any suggestions?

    #!/bin/bash
    export TERM=xterm
    PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
    array=( 10fog-helpdesk 11storage 13storage 14storage 15storage 17storage 19storage 21storage 23storage 25storage 27storage 28storage 29storage 30storage 32storage 34storage 36storage 38storage 43storage 44storage )
    LOCKFILE=/tmp/synclock.txt
    if [ -e ${LOCKFILE} ] && kill -0 `cat ${LOCKFILE}`; then
        echo "already running"
        exit
    fi
    # make sure the lockfile is removed when we exit and then claim it
    trap "rm -f ${LOCKFILE}; exit" INT TERM EXIT
    echo $$ > ${LOCKFILE}
    
    clear
    echo
    echo
    echo "Syncing defined systems."
    echo
    for i in "${array[@]}"
    do
        printf "FOG Sync Started at: $i..."
        successCheck=$(rsync --progress --stats --prune-empty-dirs --delete --bwlimit=500 -avz /images/Sync root@$i:/images/ > /var/log/fog/shopsync.$i.log; rsync --progress --stats --prune-empty-dirs --delete --bwlimit=500 -avz /images/drivers root@$i:/images/ > /var/log/fog/driversync.$i.log;)
        if [[ "$successCheck" -eq 0 ]]; then
            printf "Success!\n"
        else
            printf "Failed!\n"
        fi
        printf "\n"
    done
    #sleep 1000
    rm -f ${LOCKFILE}
    

    I have ssh-copy-id root@10fog-helpdesk to imort my key
    I am able too shh root@10fog-helpdesk without a password with no issues.
    This is what the cronjob log looks like.

    FOG Sync Started at: 10fog-helpdesk...Permission denied, please try again.
    Permission denied, please try again.
    Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).
    rsync: connection unexpectedly closed (0 bytes received so far) [sender]
    rsync error: unexplained error (code 255) at io.c(226) [sender=3.1.2]
    Permission denied, please try again.
    Permission denied, please try again.
    Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).
    rsync: connection unexpectedly closed (0 bytes received so far) [sender]
    rsync error: unexplained error (code 255) at io.c(226) [sender=3.1.2]
    Success!
    

    Any pointers?

    Thanks!

    W 1 Reply Last reply Oct 22, 2019, 2:01 AM Reply Quote 0
    • S
      Sebastian Roth Moderator
      last edited by Oct 21, 2019, 8:51 PM

      @Greg-Plamondon How did you setup the cronjob? Or to be more specific - which user runs the cronjob? See here: https://serverfault.com/questions/352835/crontab-running-as-a-specific-user

      Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

      Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

      G 1 Reply Last reply Oct 24, 2019, 1:54 PM Reply Quote 1
      • W
        Wayne Workman @Greg Plamondon
        last edited by Wayne Workman Oct 21, 2019, 8:13 PM Oct 22, 2019, 2:01 AM

        Have you tried ssh aliases?

        For example, if you are root and set the file /root/.ssh/config as such:

        Host fog-server
            User root
            HostName 10.0.0.28
            Port 22
        

        You can then just type ssh fog-server and bam, you’re in.
        Makes scripting a bit simpler… Particularly makes getting around in your systems faster while in the shell.

        Agreeing with @Sebastian-Roth though. If ssh works manually, but doesn’t in cron, it’s maybe a user issue. You can figure out who it’s executing as like such:
        whoami > /tmp/whoami.txt 2>&1
        Put that towards the top of your script, see what it says.

        Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
        Daily Clean Installation Results:
        https://fogtesting.fogproject.us/
        FOG Reporting:
        https://fog-external-reporting-results.fogproject.us/

        G 1 Reply Last reply Oct 24, 2019, 2:01 PM Reply Quote 0
        • G
          Greg Plamondon Testers @Sebastian Roth
          last edited by Oct 24, 2019, 1:54 PM

          @Sebastian-Roth the cronjob runs hourly as root.

          1 Reply Last reply Reply Quote 0
          • G
            Greg Plamondon Testers @Wayne Workman
            last edited by Oct 24, 2019, 2:01 PM

            @Wayne-Workman contents of /tmp/whoami.txt is “root”

            W 1 Reply Last reply Oct 25, 2019, 3:42 AM Reply Quote 1
            • W
              Wayne Workman @Greg Plamondon
              last edited by Wayne Workman Oct 24, 2019, 9:42 PM Oct 25, 2019, 3:42 AM

              @Greg-Plamondon My advice: start eliminating variables, start reducing complexity. Create a new script that has a single command to sync one of your nodes - don’t use any variables, hard code everything. See if that works. If it doesn’t, start eliminating arguments one by one until you find the one that is problematic. Also try to sync test directories. Also look at the permissions of the directories you are syncing both locally and remotely.

              Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
              Daily Clean Installation Results:
              https://fogtesting.fogproject.us/
              FOG Reporting:
              https://fog-external-reporting-results.fogproject.us/

              1 Reply Last reply Reply Quote 0
              • S
                Sebastian Roth Moderator
                last edited by Sebastian Roth Oct 25, 2019, 3:45 AM Oct 25, 2019, 9:44 AM

                @Greg-Plamondon A quick search engine foo revealed that in cron the environment is not set appropriately to use the SSH keys: https://stackoverflow.com/a/50999470

                I think the most easy solution in your case is to use rsync ... -avze "ssh -i /full/path/to/ssh_private_key" /images/Sync root@$i:/images/ ...

                Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                G 1 Reply Last reply Oct 25, 2019, 6:26 PM Reply Quote 0
                • G
                  george1421 Moderator
                  last edited by Oct 25, 2019, 11:50 AM

                  While I haven’t really looked hard into this I have often wondered if FOG should replace its replicator with rsync. I know its a bit off topic here. But from what I remember that rsync can be run in a couple of different modes rsync->ssh as well as rsync(sender)->rsync(receiver). I haven’t really looked into it (at all) but I wonder if the rsync->rsync solution would give a better experience?

                  Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!

                  1 Reply Last reply Reply Quote 0
                  • S
                    Sebastian Roth Moderator
                    last edited by Oct 25, 2019, 12:36 PM

                    @george1421 I have started to think about getting rid of FTP and NFS altogether because both are very old, rubbish protocols that cause head aces as soon as you leave the local subnet. But it’s a long way to get there and I don’t see this happen soon.

                    Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                    Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                    1 Reply Last reply Reply Quote 1
                    • G
                      Greg Plamondon Testers @Sebastian Roth
                      last edited by Oct 25, 2019, 6:26 PM

                      @Sebastian-Roth Thanks for the input,
                      I was able to figure out what was going wrong. when I was ssh-copy-d root@storagenode it was copying the sshkey i use to login to the server. I needed to use ssh-copy-id -i /root/.ssh/id_rsa and then your suggestio of rsync ... -avze "ssh -i /full/path/to/ssh_private_key" /images/Sync root@$i:/images/ ... worked great.

                      Thanks!

                      1 Reply Last reply Reply Quote 0
                      • 1 / 1
                      1 / 1
                      • First post
                        2/10
                        Last post

                      149

                      Online

                      12.0k

                      Users

                      17.3k

                      Topics

                      155.2k

                      Posts
                      Copyright © 2012-2024 FOG Project