Post Download Scripts - Not Executing

george1421

@dholtz-docbox When you break out of the fog deploy script. You can check out the entire environment. Just use set | more or set | grep postdownload

dholtz-docbox

@george1421 : That makes sense. I am going through the debug deploy task as we speak - very cool! As soon as I can, I will query this (currently in the middle of Partclone).

I really want to learn more of what you guys did here, it’s really awesome. As someone overcoming the whole understanding-networking-in-depth, going through the process of learning what you guys have done has been great. That said, boy am I ready to develop something where the network doesn’t kick my ass left-and-right!

-Dustin

dholtz-docbox

Okay! My script is being called, and I believe I see the issue…

Every time I deploy a new image, my swap drive has the incorrect UUID in its fstab. So I thought I would just update it after imaging, which led me to post download scripts. Now that I am looking at it, it appears that the ‘fstab’ has no contents at the time of its execution, so it is not able to update the swap drive’s UUID in the fstab after imaging the host machine.

Everything said, there is no way what I am doing is valid. I have seen very little resources on people having this issue, yet it is the very first thing I have hit with every image I captured. What can I do to ensure the swap drive gets assigned the correct UUID? When going through the debug process, I noted sections where the following occurred…

Resetting UUIDs…
Resetting swap systems…
Restoring swap partition

For some reason, I feel like there is an issue in how I am capturing images, or in how the server resets the swap drives. I have to imagine it’s on my end though, given the nature of the issue and lack of articles with other users having the same issue. Is there a better way to handle how the swap drive gets assigned its UUID in it the host machine?

-Dustin

george1421

@dholtz-docbox This is really a question for the @Developers.

But what I wonder is why (if the UUID’s being wrong is the issue) do you not use the physical partitions reference and do away with the uuid reference all together. ?? But this may be just a simplistic thought to a complex question. The Developers would have more insight on what FOG does when it deploys a unix defined target.

But I can say the skills you just learned will help you when you want to do more advanced things like rename a host or change service boot states via the post install script. So all is not lost.

mrayzies

The following might be lies, so take it with a grain of salt til someone more qualified comments:

When your scripts are running, they are running from the perspective of the client; however, the file system you see is not the actual file system on the disk of the client; it’s a temporary file system that comes from the PXE file/kernel you boot. So when you operate on “/etc/fstab”, you are making a change to the fstab of something that is running memory, and thus no changes persist. If you really want to operate on files on the host, you need to mount the disk device. This is not an entirely fool-proof way, but for my scripts, I mount /dev/sda to /mnt and then can operate on files within that and those changes end up in the host. So for your example, you’d need to do something like this in your script:

#!/bin/bash
mount -t ext4 /dev/sda1 /mnt
ORIGINAL=$(grep -v '#' /mnt/etc/fstab | grep swap | cut -d ' ' -f1 | cut -d '=' -f2)
....

/dev/sda has worked well enough for me, though I am sure there are situations in which it would fail.

Secondly, to the heart of your issue, I believe your analysis is correct in that the UUID issue isn’t something you should be fixing in postdownload scripts. UUID’s, if I understand correctly, are unique, based on the hardware ID of the device. To my understanding, FOG should be handing that during the imaging process (similarly to how it handles growing the file system).

dholtz-docbox

@george1421 : It could very well be a simple answer to an otherwise superfluously complex problem - of which I may have fabricated :P. Let me look into other ways of handling the definition of the partitions in the fstab.

I absolutely feel the problem is significantly simpler than I am looking at it from, so this would be a very nice solution if it works. I have no attachment to the UUID’s, I just need all the partitions properly identified, so that Partclone can capture the image and so the drive is registered when the machine boots. In the former, it appears to completely thwart Partclone if any of the drives appear to be “not mounted”. The later causes the machine to hang during boot, prompting the user to either skip drive mounting or to wait for the machine to figure it out.

Nonetheless, I agree. I believe the post download scripts knowledge will come in great use here soon We have a lot of plans for remote imaging, and I am looking to become pretty intimate with the everything FOG provides.

-Dustin

Tom Elliott

I guess I can jump in then?

The swap UUID is already set to what was originally taken. I’m fairly sure this is the expected case. But, for your /etc/fstab file I would recommend removing the UUID for the swap and define it for the partition itself so as to avoid confusion.

Tom Elliott

To add on to what @mrayzies stated, you do need to mount the individual partitions and use the mount point to access the etc/fstab.

As noted in the ORIGINAL=$(grep -v '#' /mnt/etc/fstab | grep swap | cut -d ' ' -f1 | cut -d '=' -f2)

Tom Elliott

Sorry to spam, but I do need to say it.

I think your postdownload script was operating properly, though I’ve not tested it myself.

You are trying to edit /etc/fstab and as @mrayzies stated, this is the in memory filesystem, not the filesystem of which you’re trying to fix.

dholtz-docbox

@Tom-Elliott : That makes sense, regarding the mount. This was also true for the values returned by my CURRENT assignment, via. blkid, where its UUID also did not reflect the host machine’s swap UUID. What I am interested in now is what you mentioned about removing the UUID for the swap and defining it for the partition itself. Would you mind elaborating on this a little for me? I am digging around trying to understand more about how to configure this, and I feel you are hinting on something similar to where I should be headed.

The image I am handling by-the-way, currently, is just an Ubuntu 14.04 desktop installation. So I have done nothing special to it other than install it, shut the machine down, and capture its image. I figured this would be the best starting place for a lot of things, but it is starting to sound like that is a loaded gun. That I need to configure a few things at a minimum to ensure subsequent images are deployed more smoothly?

-Dustin

Tom Elliott

@dholtz-docbox I need to see the fstab in question, but typically the UUID is used as the “reference” to the device/partition what have you.

So instead of setting fstab for swap as:

UUID=SOME-UUID-HERE swap swap defaults 0 0

You could set as:
/dev/sda2 swap swap defaults 0 0

dholtz-docbox

@Tom-Elliott :

Each of our machines look similar to…

# / was on /dev/nvme0n1p2 during installation
UUID=61d640da-5df9-4a71-b6bc-cc28d8a8c9c8 /               ext4    errors=remount-ro 0       1
# /boot/efi was on /dev/nvme0n1p1 during installation
UUID=030B-0954  /boot/efi       vfat    defaults        0       1
# swap was on /dev/nvme0n1p3 during installation
UUID=f0207d3c-a9b2-492e-93ca-fe37a59473d6 none            swap    sw              0       0

Are you saying that I can just change these to…

# / was on /dev/nvme0n1p2 during installation
/dev/nvme0n1p2 /               ext4    errors=remount-ro 0       1
# /boot/efi was on /dev/nvme0n1p1 during installation
/dev/nvme0n1p1  /boot/efi       vfat    defaults        0       1
# swap was on /dev/nvme0n1p3 during installation
/dev/nvme0n1p3 none            swap    sw              0       0

… and be good-to-go? Then image with the drives labeled as such, instead? I have been looking around for more details on doing something like this, but all I can find are people who lost their drives UUID and need to re-assign it. It’s overwhelming the amount of issues people have in simply manually assigning it, it really drowns out any other questions about fstab and what you can do with it and when.

-Dustin

Tom Elliott

@dholtz-docbox Yes, you should be good to go. Now I don’t know which partition is which.

One of the things FOG does automatically though, is reset the UUID to match of what the OS was expecting. So I don’t know why you need to change the UUID to begin with.

mrayzies

@dholtz-docbox

For this image, what do you have defined for “Image Type” and “Partition”? Perhaps you have something misconfigured here which is why the UUID of the SWAP space is not set properly?

I’d guess that you should have “Multiple Partition Image - Single Disk (Not Resizable) - (2)” and “Everything - (1)” respectively.

dholtz-docbox

@mrayzies :

Oh. My. Gosh. I have been using Single Disk - Resizable this whole time! I thought this was the correct configuration, one DRIVE, which is resizable for each partition. When did this change, out of curiosity? When I first downloaded the FOG Project, I swear the multiple partition selection did not exist. This has me wicked excited! I was just about to go home too. Time for one more test!

-Dustin

george1421

@mrayzies You came to the same conclusion I did (as I was finishing mowing the grass tonight, and still thinking about this issue). The fstab being empty is on the FOS Engine linux OS, that I can understand. Your mount command is needed to connect to the target’s local hard drive (which is not mounted automatically by FOS). We have to do this if we want to tweak the unattend.xml in the windows realm of the post install scripts.

dholtz-docbox

Testing this today - had to leave work last night. I am hoping this will solve it. It feels promising, heh.

-Dustin

Tom Elliott

@dholtz-docbox said in Post Download Scripts - Not Executing:

Server

FOG Version: 1.3.0-rc11

OS: Ubuntu 14.04

Description

I am near the end of what I need from the FOG project, and everything seems to be functioning pretty solid so far. However, I am having issues kicking off post download scripts.

I have followed the following two articles to no success…

https://forums.fogproject.org/topic/7740/the-magical-mystical-fog-post-download-script-under-construction/3

https://wiki.fogproject.org/wiki/index.php/Auto_driver_Install

My /images/postdownloadscripts/fog.postdownload looks like…
#!/bin/sh

. ${postdownpath}update_swap_uuid.sh
And update_swap_uuid.sh looks like…
#!/bin/bash

ORIGINAL=$(grep -v '#' /etc/fstab | grep swap | cut -d ' ' -f1 | cut -d '=' -f2)
CURRENT=$(blkid | grep 'swap' | cut -d ' ' -f2 | cut -d '"' -f2)
TIME=$(date +%Y%m%d%H%M%S)

if [ "$ORIGINAL" = "$CURRENT" ];
then
  exit
fi

# NOTE: Backup 'fstab'
cp /etc/fstab "/etc/fstab.$TIME.backup"

# NOTE: Overwrite ORIGINAL with CURRENT swap UUID
sed -i -e "s/$ORIGINAL/$CURRENT/g" /etc/fstab
I have had no issues using the update script itself, so I am not sure why FOG would have an issue with it.

I have also given ‘755’ rights to the whole postdownloadscripts/ directory.

Am I forgetting something or doing something incorrectly? I have tried little permutations of editing the fog.postdownload and update scripts, as well as hunting for settings and permissions to allow access to these contents.

Also, where would I look for any indication that these are being run? I have looked very closely through the entire process and have seen no output text about running this particular region of scripts.

Many thanks for your time!

-Dustin

You can change your update_swap_uuid.sh to something like:

#!/bin/bash
# First we need to get the partitions of this disk (typically $hd in resize or single disk nonresize)
getPartitions "$hd"
# Now to iterate the parts
for part in $parts; do
     # Print a nice message
     dots "Mounting partition $part"
     # Attempt the mount
     mount $part /mnt >/dev/null 2>&1
     # It didn't mount, inform and start at top
     if [[ ! $? -eq 0 ]]; then
         echo "Failed to mount the partition"
         continue
     fi
     # Test to see if fstab exists on this part
     # If not un-mount the directory and print the message for the user and start from top of loop
     if [[ ! -f /mnt/etc/fstab ]]; then
         umount /mnt
         echo "Done, fstab not on this partition"
         continue
     fi
     # File was found
     echo "Done"
     # Let the user know what is happening
     dots "Checking and updating swap UUID"
     # Get the fstab uuid currently setup
     ORIGINAL=$(grep -v '#' /mnt/etc/fstab | grep swap | cut -d ' ' -f1 | cut -d '=' -f2)
     # Get the current real partition swap
     CURRENT=$(blkid | grep 'swap' | cut -d ' ' -f2 | cut -d '"' -f2)
     # Get the current date/time
     TIME=$(date +%Y%m%d%H%M%S)

     # Check if the original is the same as the current if so
     # if so, unmount and inform start at top
     if [[ $ORIGINAL == $CURRENT ]]; then
         umount /mnt
         echo "Done, UUID unchanged"
         continue
     fi

     # NOTE: Backup 'fstab'
     cp /mnt/etc/fstab "/mnt/etc/fstab.$TIME.backup"

     # NOTE: Overwrite ORIGINAL with CURRENT swap UUID
     sed -i -e "s/$ORIGINAL/$CURRENT/g" /mnt/etc/fstab >/dev/null 2>&1
     # If the sed fails inform, un-mount, start at top
     if [[ ! $? -eq 0 ]]; then
         echo "Failed to update file"
         umount /mnt
         continue
    fi
    # All succeeded inform and unmount
    echo "Done, UUID updated"
    umount /mnt
done

dholtz-docbox

Setting the image to “Multiple Partition Image - Single Disk (Not Resizable) - (2)” did not work either.

I guess what stumps me is why I am having this issue, or what I need to do to avoid it. There isn’t even anything special about the drive I am imaging, it’s just a fresh installation of Ubuntu 14.04 Desktop. How do you guys generally configure your swap partition? Linux needs this swap partition, so it is awkward to me that I am not seeing more resources on this issue while having received it so easily.

I am trying to avoid the post download scripts solution at the moment, and am looking into other ways of configuring the system before imaging. At the moment, I have tried just using the physical name in lieu of the UUID for the swap drive and am testing this as I write this.

I will continue posting my progress.

-Dustin

dholtz-docbox

@Tom-Elliott :

Oh, that’s awesome! I am still learning how to do more with the system through shell scripts, so this is very cool to read through. I will update my current script to reflect this and give it a whirl. It might not be the best solution, but I can figure out another solution if this one works for this particular milestone.

-Dustin

Post Download Scripts - Not Executing

Server

Description

182

11.6k

17.1k

154.6k