Post Download Scripts - Not Executing



  • Server
    • FOG Version: 1.3.0-rc11
    • OS: Ubuntu 14.04
    Description

    I am near the end of what I need from the FOG project, and everything seems to be functioning pretty solid so far. However, I am having issues kicking off post download scripts.

    I have followed the following two articles to no success…

    My /images/postdownloadscripts/fog.postdownload looks like…

    #!/bin/sh
    
    . ${postdownpath}update_swap_uuid.sh
    

    And update_swap_uuid.sh looks like…

    #!/bin/bash
    
    ORIGINAL=$(grep -v '#' /etc/fstab | grep swap | cut -d ' ' -f1 | cut -d '=' -f2)
    CURRENT=$(blkid | grep 'swap' | cut -d ' ' -f2 | cut -d '"' -f2)
    TIME=$(date +%Y%m%d%H%M%S)
    
    if [ "$ORIGINAL" = "$CURRENT" ];
    then
      exit
    fi
    
    # NOTE: Backup 'fstab'
    cp /etc/fstab "/etc/fstab.$TIME.backup"
    
    # NOTE: Overwrite ORIGINAL with CURRENT swap UUID
    sed -i -e "s/$ORIGINAL/$CURRENT/g" /etc/fstab
    

    I have had no issues using the update script itself, so I am not sure why FOG would have an issue with it.

    I have also given ‘755’ rights to the whole postdownloadscripts/ directory.

    Am I forgetting something or doing something incorrectly? I have tried little permutations of editing the fog.postdownload and update scripts, as well as hunting for settings and permissions to allow access to these contents.

    Also, where would I look for any indication that these are being run? I have looked very closely through the entire process and have seen no output text about running this particular region of scripts.

    Many thanks for your time!

    -Dustin



  • I had to table these efforts, as I have been unable to successfully clone and deploy an extended linux image without having the swap drive hiccup during the cloning/imaging process. I will return to this as soon as time permits, but I have a few tentative workarounds for this milestone.

    -Dustin


  • Senior Developer

    @dholtz-docbox You’re absolutely right that postdownload scripts is a very powerful tool. It can also be very destructive, but “with great power comes great responsibility…” or something like that.

    Essentially, postdownload scripts can be ANYTHING you want them to be. They have the power to iterate over the freshly imaged system to change whatever you may need, however you may need to do it.

    Yes, links can change, but chances are you’ll know of those changes the first time you go to work with them. With post download scripts, you can modify EVERYTHING, even HOW the scripts run. For example, on the fog.postdownload you can add if statements to do different things based on whatever you deem necessary. You don’t, necessarily, even need to just source your own scripts, you could just script it directly in the fog.postdownload script. I prefer separating my scripts for modularity reasons.

    I don’t mind giving input here and there as it helps EVERYBODY.



  • @Tom-Elliott : What is the best way for me to learn what I can do in the FOS? I couldn’t locate a primary resource on the FOG wiki regarding it and its available functionality. I wouldn’t mind knowing more about how people use this side of FOG, it is starting to feel like it’s the strongest tool in the whole arsenal.

    Also, I was able to capture + deploy the image fine if I used the physical partitions name in lieu of the UUID.

    /dev/sda5, none, swap, sw, 0, 0
    

    My primary concern in this solution is the what-if scenario. Should I be concerned about whether this link will change on its own? That would be my primary concern, correct? Someone changing the sym-link between /dev/sda5 and its underlying UUID?

    Thanks for going back-and-forth with me on this. I don’t have a lot of people to talk with about topics like this.

    -Dustin


  • Senior Developer

    @dholtz-docbox I’m only looping the found partitions to make things a bit more dynamic. But if you know the FOS system recognizes the disk as /dev/sda and you also know the root etc/fstab will be on partition 1, you can forgo the loop and simply mount the /dev/sda1 and make your edits directly.



  • @Tom-Elliott :

    Oh, that’s awesome! I am still learning how to do more with the system through shell scripts, so this is very cool to read through. I will update my current script to reflect this and give it a whirl. It might not be the best solution, but I can figure out another solution if this one works for this particular milestone.

    -Dustin



  • Setting the image to “Multiple Partition Image - Single Disk (Not Resizable) - (2)” did not work either.

    I guess what stumps me is why I am having this issue, or what I need to do to avoid it. There isn’t even anything special about the drive I am imaging, it’s just a fresh installation of Ubuntu 14.04 Desktop. How do you guys generally configure your swap partition? Linux needs this swap partition, so it is awkward to me that I am not seeing more resources on this issue while having received it so easily.

    I am trying to avoid the post download scripts solution at the moment, and am looking into other ways of configuring the system before imaging. At the moment, I have tried just using the physical name in lieu of the UUID for the swap drive and am testing this as I write this.

    I will continue posting my progress.

    -Dustin


  • Senior Developer

    @dholtz-docbox said in Post Download Scripts - Not Executing:

    Server
    • FOG Version: 1.3.0-rc11
    • OS: Ubuntu 14.04
    Description

    I am near the end of what I need from the FOG project, and everything seems to be functioning pretty solid so far. However, I am having issues kicking off post download scripts.

    I have followed the following two articles to no success…

    My /images/postdownloadscripts/fog.postdownload looks like…

    #!/bin/sh
    
    . ${postdownpath}update_swap_uuid.sh
    

    And update_swap_uuid.sh looks like…

    #!/bin/bash
    
    ORIGINAL=$(grep -v '#' /etc/fstab | grep swap | cut -d ' ' -f1 | cut -d '=' -f2)
    CURRENT=$(blkid | grep 'swap' | cut -d ' ' -f2 | cut -d '"' -f2)
    TIME=$(date +%Y%m%d%H%M%S)
    
    if [ "$ORIGINAL" = "$CURRENT" ];
    then
      exit
    fi
    
    # NOTE: Backup 'fstab'
    cp /etc/fstab "/etc/fstab.$TIME.backup"
    
    # NOTE: Overwrite ORIGINAL with CURRENT swap UUID
    sed -i -e "s/$ORIGINAL/$CURRENT/g" /etc/fstab
    

    I have had no issues using the update script itself, so I am not sure why FOG would have an issue with it.

    I have also given ‘755’ rights to the whole postdownloadscripts/ directory.

    Am I forgetting something or doing something incorrectly? I have tried little permutations of editing the fog.postdownload and update scripts, as well as hunting for settings and permissions to allow access to these contents.

    Also, where would I look for any indication that these are being run? I have looked very closely through the entire process and have seen no output text about running this particular region of scripts.

    Many thanks for your time!

    -Dustin

    You can change your update_swap_uuid.sh to something like:

    #!/bin/bash
    # First we need to get the partitions of this disk (typically $hd in resize or single disk nonresize)
    getPartitions "$hd"
    # Now to iterate the parts
    for part in $parts; do
         # Print a nice message
         dots "Mounting partition $part"
         # Attempt the mount
         mount $part /mnt >/dev/null 2>&1
         # It didn't mount, inform and start at top
         if [[ ! $? -eq 0 ]]; then
             echo "Failed to mount the partition"
             continue
         fi
         # Test to see if fstab exists on this part
         # If not un-mount the directory and print the message for the user and start from top of loop
         if [[ ! -f /mnt/etc/fstab ]]; then
             umount /mnt
             echo "Done, fstab not on this partition"
             continue
         fi
         # File was found
         echo "Done"
         # Let the user know what is happening
         dots "Checking and updating swap UUID"
         # Get the fstab uuid currently setup
         ORIGINAL=$(grep -v '#' /mnt/etc/fstab | grep swap | cut -d ' ' -f1 | cut -d '=' -f2)
         # Get the current real partition swap
         CURRENT=$(blkid | grep 'swap' | cut -d ' ' -f2 | cut -d '"' -f2)
         # Get the current date/time
         TIME=$(date +%Y%m%d%H%M%S)
    
         # Check if the original is the same as the current if so
         # if so, unmount and inform start at top
         if [[ $ORIGINAL == $CURRENT ]]; then
             umount /mnt
             echo "Done, UUID unchanged"
             continue
         fi
    
         # NOTE: Backup 'fstab'
         cp /mnt/etc/fstab "/mnt/etc/fstab.$TIME.backup"
    
         # NOTE: Overwrite ORIGINAL with CURRENT swap UUID
         sed -i -e "s/$ORIGINAL/$CURRENT/g" /mnt/etc/fstab >/dev/null 2>&1
         # If the sed fails inform, un-mount, start at top
         if [[ ! $? -eq 0 ]]; then
             echo "Failed to update file"
             umount /mnt
             continue
        fi
        # All succeeded inform and unmount
        echo "Done, UUID updated"
        umount /mnt
    done
    


  • Testing this today - had to leave work last night. I am hoping this will solve it. It feels promising, heh.

    -Dustin


  • Moderator

    @mrayzies You came to the same conclusion I did (as I was finishing mowing the grass tonight, and still thinking about this issue). The fstab being empty is on the FOS Engine linux OS, that I can understand. Your mount command is needed to connect to the target’s local hard drive (which is not mounted automatically by FOS). We have to do this if we want to tweak the unattend.xml in the windows realm of the post install scripts.



  • @mrayzies :

    Oh. My. Gosh. I have been using Single Disk - Resizable this whole time! I thought this was the correct configuration, one DRIVE, which is resizable for each partition. When did this change, out of curiosity? When I first downloaded the FOG Project, I swear the multiple partition selection did not exist. This has me wicked excited! I was just about to go home too. Time for one more test!

    -Dustin



  • @dholtz-docbox

    For this image, what do you have defined for “Image Type” and “Partition”? Perhaps you have something misconfigured here which is why the UUID of the SWAP space is not set properly?

    I’d guess that you should have “Multiple Partition Image - Single Disk (Not Resizable) - (2)” and “Everything - (1)” respectively.


  • Senior Developer

    @dholtz-docbox Yes, you should be good to go. Now I don’t know which partition is which.

    One of the things FOG does automatically though, is reset the UUID to match of what the OS was expecting. So I don’t know why you need to change the UUID to begin with.



  • @Tom-Elliott :

    Each of our machines look similar to…

    # / was on /dev/nvme0n1p2 during installation
    UUID=61d640da-5df9-4a71-b6bc-cc28d8a8c9c8 /               ext4    errors=remount-ro 0       1
    # /boot/efi was on /dev/nvme0n1p1 during installation
    UUID=030B-0954  /boot/efi       vfat    defaults        0       1
    # swap was on /dev/nvme0n1p3 during installation
    UUID=f0207d3c-a9b2-492e-93ca-fe37a59473d6 none            swap    sw              0       0
    

    Are you saying that I can just change these to…

    # / was on /dev/nvme0n1p2 during installation
    /dev/nvme0n1p2 /               ext4    errors=remount-ro 0       1
    # /boot/efi was on /dev/nvme0n1p1 during installation
    /dev/nvme0n1p1  /boot/efi       vfat    defaults        0       1
    # swap was on /dev/nvme0n1p3 during installation
    /dev/nvme0n1p3 none            swap    sw              0       0
    

    … and be good-to-go? Then image with the drives labeled as such, instead? I have been looking around for more details on doing something like this, but all I can find are people who lost their drives UUID and need to re-assign it. It’s overwhelming the amount of issues people have in simply manually assigning it, it really drowns out any other questions about fstab and what you can do with it and when.

    -Dustin


  • Senior Developer

    @dholtz-docbox I need to see the fstab in question, but typically the UUID is used as the “reference” to the device/partition what have you.

    So instead of setting fstab for swap as:

    UUID=SOME-UUID-HERE swap swap defaults 0 0

    You could set as:
    /dev/sda2 swap swap defaults 0 0



  • @Tom-Elliott : That makes sense, regarding the mount. This was also true for the values returned by my CURRENT assignment, via. blkid, where its UUID also did not reflect the host machine’s swap UUID. What I am interested in now is what you mentioned about removing the UUID for the swap and defining it for the partition itself. Would you mind elaborating on this a little for me? I am digging around trying to understand more about how to configure this, and I feel you are hinting on something similar to where I should be headed.

    The image I am handling by-the-way, currently, is just an Ubuntu 14.04 desktop installation. So I have done nothing special to it other than install it, shut the machine down, and capture its image. I figured this would be the best starting place for a lot of things, but it is starting to sound like that is a loaded gun. That I need to configure a few things at a minimum to ensure subsequent images are deployed more smoothly?

    -Dustin


  • Senior Developer

    Sorry to spam, but I do need to say it.

    I think your postdownload script was operating properly, though I’ve not tested it myself.

    You are trying to edit /etc/fstab and as @mrayzies stated, this is the in memory filesystem, not the filesystem of which you’re trying to fix.


  • Senior Developer

    To add on to what @mrayzies stated, you do need to mount the individual partitions and use the mount point to access the etc/fstab.

    As noted in the ORIGINAL=$(grep -v '#' /mnt/etc/fstab | grep swap | cut -d ' ' -f1 | cut -d '=' -f2)


  • Senior Developer

    I guess I can jump in then?

    The swap UUID is already set to what was originally taken. I’m fairly sure this is the expected case. But, for your /etc/fstab file I would recommend removing the UUID for the swap and define it for the partition itself so as to avoid confusion.



  • @george1421 : It could very well be a simple answer to an otherwise superfluously complex problem - of which I may have fabricated :P. Let me look into other ways of handling the definition of the partitions in the fstab.

    I absolutely feel the problem is significantly simpler than I am looking at it from, so this would be a very nice solution if it works. I have no attachment to the UUID’s, I just need all the partitions properly identified, so that Partclone can capture the image and so the drive is registered when the machine boots. In the former, it appears to completely thwart Partclone if any of the drives appear to be “not mounted”. The later causes the machine to hang during boot, prompting the user to either skip drive mounting or to wait for the machine to figure it out.

    Nonetheless, I agree. I believe the post download scripts knowledge will come in great use here soon :) We have a lot of plans for remote imaging, and I am looking to become pretty intimate with the everything FOG provides.

    -Dustin


Log in to reply
 

Looks like your connection to FOG Project was lost, please wait while we try to reconnect.