The magical, mystical FOG post download script


  • Moderator

    In this series if posts I plan on outlining what the FOG post download script is, what its about, and how to use it to your advantage with image deployment.

    I’ve debated with myself if another post was needed since Lee https://forums.fogproject.org/topic/4278/utilizing-postscripts-rename-joindomain-drivers-snapins and I https://forums.fogproject.org/topic/7391/deploying-a-single-golden-image-to-different-hardware-with-fog have written tutorial about them in the past.

    Where I want to explore with this series of posts is more about the background behind its function and to cover some of the script elements from Lee’s and my previously linked tutorials.

    First of all lets get a little background here and some terminology defined. The FOG image deployment system consists of three main components:

    1. The FOG management environment. The FOG management environment is the FOG server itself, which is responsible for planning, scheduling, instructing and documenting the image deployment.
    2. The FOS (FOG OS) target engine {need better name here}. The FOS target engine, or FOS (as it will be known in this document) is responsible for capturing and deploying images from the defined storage node, as well as other actions on the target computer. FOS is actually an FOG Project created high performance linux operating system that was specifically constructed to image target computers. We have to remember for the rest of the tutorial, FOS is a linux based operating system that is used to deploy any OS image to your target computer.
    3. FOG Client. This is an add on utility service for windows and linux, which is installed on the final or target OS. The FOG Client queries the FOG server for instructions and actions. I’m not planning on discussing the FOG client during this tutorial since it is out of scope in regards to post installation scripting.

    The FOG Post install scripts give FOG system admins the ability to inject actions in the image creation process. To allow this custom scripts to run the developers added an external call into the image deployment sequence. Just after the image is placed on the target computer’s storage device, the FOG server calls a script in the /images/postdownloadscripts directory (on the FOG server). That script is called fog.postdownload. The fog.postdownload script is created when the FOG Server is installed. In its default state the script doesn’t do anything much. It is just a place holder to call your custom post install scripts. As I mentioned the fog deployment process calls this fog.postdownload bash script just after the image is placed on the target computer’s storage device. Once all of the external post install scripts have completed the FOG server completes the imaging and data recording steps.

    While the post install scripts are stored on the FOG server in the /images/postinstall directory, they are executed by FOS running on the target host. So you have to remember when writing your post install scripts, they execute from the perspective of the target host. The post install scripts have limited access to resources on the FOG server (outside of the /images directory), but have full access to the target host. As noted above, these scripts run on a linux OS, so any resource (programs) available to linux operating systems can be run against the target host. I want to make this distinction to make it clear that FOS is linux and not MS Windows based. So you can not run MS Windows based applications, like DISM, in your post install script. FOS and linux is very powerful, but also limiting in some ways. As long as you are aware that MS Windows based applications can not run in a post install script then you should have little trouble. There are some cross over applications that are compiled for both MS Windows and linux, you just need to ensure you have the correct application for the OS and architecture (IA32 or X64).


  • Moderator

    @george1421 said in The magical, mystical FOG post download script:

    hostbuilding # ??

    This is an old field in the hosts table left over from much older versions of fog. As far as I can tell, it is not used for anything anymore.


  • Moderator

    #wiki worthy


  • Moderator

    Part 2 FOG Post install script.

    As I noted above, the post install script is called by the FOS target engine just after the image has been copied to the target host computer and before the house keeping and recording steps are performed. (Hint, don’t force a reboot in your post install script or the house keeping and recording steps will not be performed). There is a call to execute the /images/postdownloadscripts/fog.postdownload script. You should hook your custom post install scripts into this bash script file.

    The bash shell environment (created in 1988) is a mix of operating system interface (command processor) and programming environment. In windows terms, think of it as a great, great, grandfather to what MS PowerShell will be one day. If you are a PowerShell maven, a lot of the PS look and feel came from this era of command processors, so programming in bash is just a syntax change, the program flow is very similar. The bash shell script can call linux applications, create run time variables, and produce output that can be written directly to disk storage or filtered through another linux application before being written to disk (we are going to take advantage of this functionality in our fog post install scripts. I’m not going into the details of bash programming. The environment has been around for almost 30 years, there is plenty of examples on the internet to follow. I will go into the specifics of what we need to do in bash to connect to the target computer’s hard drive and query info about the target computer. In our case we can either query the computer directly to find out key elements of its design or since FOG r8050 we can make a call back to FOG to retrieve things that FOG knows about the target computer.

    For our example post install script we are going to create it next to the existing fog.postinstall script. I do recommend that you compartmentalize your fog post install scripts based on their task and not just have a single monolithic (i.e. big a$$) script. Its much easier to maintain smaller and specific code elements. I also recommend that you write your own scripts in your own files and not just use the fog.postdownload script for your scripts. The nature of that FOG owned script may change over time. I would not want changes to the fog.postdownload script to erase your content.

    For our example fog post install script, I’m calling it fog.postexamp and will place it in the /images/postdownloadscripts directory. For clarity you can call this script anything you want, I’ve just kept with the fog naming convention of “fog.<action>”.

    For this example I'm running as root on centos, for other linux distros you will probably need sudo. For this thread assume all commands have a hidden sudo on front

    1. Lets begin to create our script
      touch /images/postdownloadscripts/fog.postexamp
    2. We’ll need to make it runable
      chmod 755 /images/postdownloadscripts/fog.postexamp
    3. With your favorite text editor, edit the fog.postexamp script.
    4. For every bash script you must insert this mandatory line on the very first line of your script. There can be no blank lines above or spaces to the left of this text. So insert this on the very first line of your bash script. #!/bin/bash and press enter. This line tells the command processor to use bash to process this script.
    5. Now I recommend that you enter several lines of comments to describe what this script does, who created it, and any other relevant information. Comment lines for bash begin with a pound symbol (a hash tag for the millennials) and then the remainder of the line is text to the end of the line.
    6. We are going to take advantage of one of the fog developers scripts for several interesting functions later so lets include that script in our script. After your comments enter the following line . /usr/share/fog/lib/funcs.sh (note: again don’t forget that lonely dot plus a space before the path to the script).

    Comment: As I mentioned before the fog post install scripts execute on the FOS client, so the /usr/share/fog/lib/funcs.sh is not on the FOG server, but on the FOS virtual hard drive (a.k.a the inits). So if you are nosy you won’t find this file if you search the path on the fog server.

    1. At this point your fog.postexamp script should look something like this one
    #!/bin/bash
    
    #
    # This is an example script that will show you the recommend way to create a 
    # FOG post install script. There are many ways you can go about this, some will be better than 
    # others. 
    #
    # Author: George1421
    # Version: 1.0
    # Date: 20160611
    #
    
    . /usr/share/fog/lib/funcs.sh
    
    
    1. If you stop right here you have a complete fog post install script. Admittedly it doesn’t do anything special, but you DO have a complete script.
    2. The rest of the script will be up to you to decide what you want your post install script to do. I’ll include more code fragments that you can assemble into your own script in Part 3

  • Moderator

    Part 3 Code snippets

    This first code snippet will connect our post install script to the target computer’s hard drive. This connection may or may not have been made by a previous FOG script. If the connection has already been setup we’ll just error gracefully and continue on with the execution. This is a good programming style in that you need to do a task, perform that task as if no other script has run on this target computer, go through the steps to set it up and then tare it down before you script is over.

    1. Connecting to the local hard drive.
      To connect our post install script to the target computers local hard drive requires a little understand about the deployed operating system and the deployed os hard drive layout (partitions). For windows vista and later there will be at least 2 partitions on the target hard drive. The first partition (1) is the boot partition that contains the windows startup code. The second partition on the target’s hard drive is the C: or windows partition. If there is a D: drive in the target image then there will be a 3rd partition, and so on. For our script this second partition will be the target of our attention. To allow our script to interact with the windows partition on our target computer we will first need to create a mount (or connection) point in FOS and then attach the second partition of the first disk to that mount point. We’ll use the following code snippet.
      To make our code flexible for other operating systems I’m going to assign the partition layout to a variable. For other disk layouts we would just update the variable, but maintain a consistent code after that by referencing the variable. In this case osdiskpart will represent the disk and partition we are interested in.
    # windows 7
    osdiskpart="/dev/sda2";
    
    # create a directory to hang the Windows C: drive partition on in FOS
    # the 2>/dev/null below just redirects any errors from the mkdir command to null. i.e.
    # if the directory already exists, I don't want to know about it, just hide the error. Understand
    # that I could have tested if the directory already existed, but that takes more programming steps
    # I'm just going to try to create it and ignore the error if it already exists. 
    
    mkdir /ntfs 2>/dev/null
    
    # This next command connects the hard drive partition to the directory we just created. You will see the
    # 2>/tmp/mntfail at the end of the mount command. In this case if the connection fails we want to write
    # the output to a text file we can review and test to see if it exists. If the file exists then something went
    # wrong with the connection to the hard disk partition.
    
    mount.ntfs-3g "${osdiskpart}" /ntfs 2>/tmp/mntfail
    
    # this last bit of magic checks to see if the mntfail file exists and if it does then it means the mount
    # failed so there is no need to continue on with the script. 
    mntRet="$?";
    if [ ! "$mntRet" = "0" ]; then
        echo "Failed to mount C:";
        # display what happened
        cat /tmp/mntfail;
        # give the reader a chance to see what the error was
        sleep 12;
        # terminate the post install script
        exit 1;
    fi
    
    1. After the connection to the hard drive is established, your script can make alterations to the Windows disk. You can update configuration files, interact with the registry, delete files, or copy files to and from the FOG server. In our case we want to update some settings in the MS Windows unattend.xml file. Note: the unattend.xml file is only used if the reference image was sysprep'd before image capture. Since we were following the recommend rules and sysprep'd the image before capture this is the current state our reference image
      We have to remember that MS Windows is not case specific, but our post install script is running on a FOS linux environment which IS case important. Remember this as you write your post install scripts /ntfs/windows is not the same as /ntfs/Windows.
    2. For the next part lets assign the path of our unattent.xml file to a variable so its easier to use in with our script.
    # Unattend.xml path (note the case specifics in the file name and path)
    unattendfile="/ntfs/Windows/Panther/unattend.xml";
    
    1. With the $unatendfile variable set we can use it to update… lets say the computer name stored in the unattend.xml file with the computer name defined in fog. The fog developers have already copied the target computer name into the $hostname variable for us so to update the computer name in the unattend file we just need to do some sed (linux application) “regular expression” magic. I’m not going to explain the magic that is going on in this script. If you are interested look up sed and “regular expression” to decode what this command is really doing.
    sed -i -e "s#<ComputerName>\([^<][^<]*\)</ComputerName>#<ComputerName>$hostname</ComputerName>#gi" $unatendfile 
    
    1. Using the same concepts we can update other unattend.xml elements like:
      timezone
    sed -i -e "s#<TimeZone>\([^<][^<]*\)</TimeZone>#<TimeZone>$timezone</TimeZone>#gi" $unattendfile
    

    computer ou

    sed -i -e "s#<MachineObjectOU>\([^<][^<]*\)</MachineObjectOU>#<MachineObjectOU>${oupath}</MachineObjectOU>#gi" $unattendfile
    

    < insert keyboard>
    < insert system language>

    1. Lets say we need to determine what type of computer is the target computer (i.e. desktop, laptop. ?). We can query the SMBIOS using dmidecode to extract we are installing the chassis type using this code snippet.
    chassis=`dmidecode -s chassis-type`;
    chassis="${chassis%"${chassis##*[![:space:]]}"}";  #Remove training space
    chassis="${chassis,,}"; # Convert string to lower
    
    if [ "$chassis" = "laptop" ]; then
        chtype="Portable";
    elif [ "$chassis" = "tablet" ]; then
        chtype="Tablet";
    else
        # We'll default every other chassis type to desktop
        chtype="Desktop";
    fi
    

  • Moderator

    Part 3 Code snippets (cont)

    1. This next one took me quite a while to work out (now that I’m a bit smarter and know more about the inner workings of FOG there is an easier way to go about this. I’ll show you the harder way because its a good learning point). I needed to make some windows configuration changes based on the location where the target computer was being built. An easy way for my script to tell what location the script was running in was to inspect the FOS IP address and compare that address against my known locations. In this case I’m picking up the FOS ip address by this hack. Once I know the IP address I trim off the last two octets since the first two are only relevant. Once I have the network address I can create a case statement to set specific variables.
    myip=`ip route get 8.8.8.8 | awk 'NR==1 {print $NF}' | cut -d "." -f1-2`;
    
    case "${myip}" in
         10.1)
             sitecode="NYC";
             timezone="Eastern Standard Time";
             oupath="ou=computers,ou=nyc,dc=domain,dc=com";
             ;;
         10.2)
             sitecode="LA";
             timezone="Western Standard Time";
             oupath="ou=computers,ou=la,dc=domain,dc=com";
             ;;
         *)
             # Default code for the unknowns
             sitecode="CORP";
             timezone="Eastern Standard Time";
             oupath="ou=computers,ou=corp,dc=domain,dc=com";
             ;;
    esac
    
    1. This next one is an easy hack to tell what the intended architecture (x86 or x84) is of the target system. But first I think I need to give a little background first. If the target system (CPU) has 64 bit support, FOG will send the 64 bit FOS engine to the target computer. That 64 bit FOS engine can deploy any operating system and any architecture since all it does is move bits around. So we can’t rely on what architecture FOS has and assume the target image is the same. A quick check to see if the MS Windows target image is 64 bit is to check if the c:\Windows\SysWOW64 directory exists. If it exists then we can assume it is a 64 bit version of MS Windows. If it doesn’t exist then we can probably assume its a x86 install of MS Windows. The code for that logic would look something like this.
    if [ -d "/ntfs/Windows/SysWOW64" ]
    then
        setarch="x64";
    else
        setarch="x86";
    fi
    
    1. Lets say you need to make a post install script decision based on input from an IT tech during imaging. You can post a question to the IT tech by using this code
      read -p "Enter your room number where the computer will be installed" rn
    
      echo "You answered ${rn}. ";
    
    1. You can also update the windows registry from your post install script. Personally I have not done it this way (update the windows registry from FOS), but that is just personal preference. I know that it works and can be done pretty easy. This method uses some linux trickery to simulate keyboard input to automate a linux interactive application. This method is a bit like using the sendkeys function to send key strokes to a windows application for automation. The magic happens between the <<EOFREG and EOFREG tags. This sequence tells bash to send these characters to the reged program as if someone was typing them into the keyboard. Again with linux to Windows interaction watch your case with file paths. As before the “&>/dev/null” directive sends any error messages from the reged command to null (we don’t care about them).
    regfile="/ntfs/Windows/System32/config/SOFTWARE"
    key="\Microsoft\Windows\CurrentVersion\DevicePath"
    devpath="%SystemRoot%\inf;%SystemRoot%\DRV";
    reged -e "$regfile" &>/dev/null <<EOFREG
    ed $key
    $devpath
    q
    y
    EOFREG
    
    1. You can also patch / replace files on the target system too. In this case we are going to check to see if the file SetupComplete.cmd exists in the source folder then copy it to the destination on the C: drive.
    if [ -f "/images/Drivers/Common/SetupComplete.cmd" ]; then
        cp /images/Drivers/Common/SetupComplete.cmd /ntfs/Windows/Setup/Scripts/SetupComplete.cmd;
    fi
    
    1. FOS can also make web/url calls. You can send http queries to remote web sites for data logging, information collection, or any thing you can think of. You can also collect the response from the remote web site. Understand this is an advanced function that is pretty powerful, but is also somewhat complex since you need to know how to construct a http request, and have to understand the response from the remote web site. There are two ways to submit a http call. One is a POST and the other is a GET.
      Here is an example of a GET request that sends the results into a text file. As you can see from the url you can embed variables into the get request. In this case we are sending the content of the variable $mac to that php page.
    wget -q -O /tmp/hinfo.txt "http://${web}service/hostinfo.php?mac=$mac"
    

    Here is an example of a post request. In this case the mac address is being send as a post. The results of the wget is being sent to the snpchk variable with any errors being sent to the dust bin. In this example the -O switch doesn’t send the output to a file, rather it sends the output to the standard out which is directed into the snpchk variable.

    snpchk=`wget -O - --post-data="mac=${mac}" "http://${web}service/snapcheck.php" 2>/dev/null`
    

    This is two different methods to make http calls from inside FOS. The FOG programmers make use of the wget calls for communication between the FOG management server and the FOS engine.


  • Moderator

    Part 4 Accessing FOG defined variables

    While I haven’t described why we’ve included the funcs.sh script from Part 2, we’ll explore that script a bit more. That script includes a lot of common FOG functions as well as it gives us access to some key kernel parameters. Just by including the script in your post install script you have immediate access to all kernel parameters. These kernel parameters as passed from FOG to FOS so FOS knows what to do with/to the target computer. The kernel parameters are outlines below.

    ftp		# IP Address from Storage Management-General
    hostname	# Host Name from Host Managment-General
    img		# Image Name from Image Management-General
    mac		# Primary MAC from Image Management-General
    osid		# Host Image Index number from Image Management-General
    storage		# IP Address + Image Path from Storage Management-General
    storageip	# IP Address from Storage Management-General
    web		#IP Address + Web root from Storage Management-General
    

  • Moderator

    Part 5 Access FOG host data

    A new feature has been added to the FOG 1.2.0 trunk build starting at r8950. With r8050 FOS can now request host specific data from the FOG server. For example, lets say you entered some data into the other and other1 fields in the host record. You can now access them in your post install script. The following code snippet will request the host info for the computer FOS is imaging.
    The code says (in english) if the $mac variable is defined then call the FOG server with the mac address and output the results into a file. Then we’re going to stat that file file. The results is FOS will create a variable list of all of the host info for you to use in your script.

    if [[ ! -z $mac ]]; then
        wget -q -O /tmp/hinfo.txt "http://${web}service/hostinfo.php?mac=$mac"
        if [[ -f /tmp/hinfo.txt ]]; then
            . /tmp/hinfo.txt
        fi
    fi
    

    After running the above script these additional variables are available for use in your post install script.

    shutdown	# Shut down at the end of imaging
    hostdesc	#Host Description from Host Managment-General
    hostip		# IP address of the FOS client
    hostimageid	# ID of image being deployed
    hostbuilding	# ??
    hostusead	# Join Domain after image task from Host Management-Active Directory
    hostaddomain	# Domain name from Host Management-Active Directory
    hostaduser	# Domain Username from Host Management-Active Directory
    hostadou	# Organizational Unit from Host Management-Active Directory
    hostproductkey=	# Host Product Key from Host Management-Active Directory
    imagename	# Image Name from Image Management-General
    imagedesc	# Image Description from Image Management-General
    imageosid	# Operating System from Image Management-General
    imagepath	# Image Path from Image Management-General (/images/ assumed)
    primaryuser	# Primary User from Host Management-Inventory
    othertag	# Other Tag #1 User from Host Management-Inventory
    othertag1	# Other Tag #2 from Host Management-Inventory
    sysman		# System Manufacturer from Host Management-Inventory (from SMBIOS)
    sysproduct	# System Product from Host Management-Inventory (from SMBIOS)
    sysserial	# System Serial Number from Host Management-Inventory (from SMBIOS)
    mbman		# Motherboard Manufacturer from Host Management-Inventory (from SMBIOS)
    mbserial	# Motherboard Serial Number from Host Management-Inventory (from SMBIOS)
    mbasset		# Motherboard Asset Tag from Host Management-Inventory (from SMBIOS)
    mbproductname	# Motherboard Product Name from Host Management-Inventory (from SMBIOS)
    caseman		# Chassis Manufacturer from Host Management-Inventory (from SMBIOS)
    caseserial	# Chassis Serial from Host Management-Inventory (from SMBIOS)
    caseasset	# Chassis Asset from Host Management-Inventory (from SMBIOS)
    location	# Host Location (name) from Host Management-General
    

    (pause)


  • Moderator

    (place holder)


  • Moderator

    Part 7 Tying it all together

    At this point we need to insert the call to our custom bash script into the FOG task sequence. Please do the following:

    1. After your script has been fully debugged we need to tell the fog post install script to call our script. With your favorite text editor edit /images/postdownloadscripts/fog.postdownload and at the end of the script insert . ${postdownpath}fog.postexamp
      It is important you include the dot prefix a space and then the full path to your post install script. While that lonely dot seems to be out of place, it tells the bash shell something important that we need. It tell the bash shell process to include the environment (bash variables) from the calling script to the called script. I’ll say this again, its important for our bash script to know about the variables the fog developers created for their scripts to run. We will use some of these variables in our example scripts.
    2. After you have added the call to our example bash script save and exit out of the fog.postdownload script.

    Thats it, the FOG install sequence will call the fog.postdownload after the image is on the workstation. The fog.postdownload will call our custom application and then return back to the FOG install sequence to finish up the house keeping functions.


Log in to reply
 

Looks like your connection to FOG Project was lost, please wait while we try to reconnect.