Part 5 Access FOG host data

A new feature has been added to the FOG 1.2.0 trunk build starting at r8950. With r8050 FOS can now request host specific data from the FOG server. For example, lets say you entered some data into the other and other1 fields in the host record. You can now access them in your post install script. The following code snippet will request the host info for the computer FOS is imaging.
The code says (in english) if the $mac variable is defined then call the FOG server with the mac address and output the results into a file. Then we’re going to stat that file file. The results is FOS will create a variable list of all of the host info for you to use in your script.

Using FOG 1.4.1 and earlier you should use this code:

if [[ ! -z $mac ]]; then
    wget -q -O /tmp/hinfo.txt "http://${web}service/hostinfo.php?mac=$mac"
    if [[ -f /tmp/hinfo.txt ]]; then
        . /tmp/hinfo.txt
    fi
fi

As of FOG 1.4.2 curl is preferred over wget. Also the structure of the ${web} variable has changed. Using curl and the updated contents of ${web} we would update the above script to this:

if [[ ! -z $mac ]]; then
    curl -A "" -Lkso /tmp/hinfo.sh ${web}/fog/service/hinfo.php -d "mac=$mac"
    if [[ -f /tmp/hinfo.sh ]]; then
        . /tmp/hinfo.sh
    fi
fi

After running the above script these additional variables are available for use in your post install script.

shutdown	# Shut down at the end of imaging
hostdesc	#Host Description from Host Managment-General
hostip		# IP address of the FOS client
hostimageid	# ID of image being deployed
hostbuilding	# ??
hostusead	# Join Domain after image task from Host Management-Active Directory
hostaddomain	# Domain name from Host Management-Active Directory
hostaduser	# Domain Username from Host Management-Active Directory
hostadou	# Organizational Unit from Host Management-Active Directory
hostproductkey=	# Host Product Key from Host Management-Active Directory
imagename	# Image Name from Image Management-General
imagedesc	# Image Description from Image Management-General
imageosid	# Operating System from Image Management-General
imagepath	# Image Path from Image Management-General (/images/ assumed)
primaryuser	# Primary User from Host Management-Inventory
othertag	# Other Tag #1 User from Host Management-Inventory
othertag1	# Other Tag #2 from Host Management-Inventory
sysman		# System Manufacturer from Host Management-Inventory (from SMBIOS)
sysproduct	# System Product from Host Management-Inventory (from SMBIOS)
sysserial	# System Serial Number from Host Management-Inventory (from SMBIOS)
mbman		# Motherboard Manufacturer from Host Management-Inventory (from SMBIOS)
mbserial	# Motherboard Serial Number from Host Management-Inventory (from SMBIOS)
mbasset		# Motherboard Asset Tag from Host Management-Inventory (from SMBIOS)
mbproductname	# Motherboard Product Name from Host Management-Inventory (from SMBIOS)
caseman		# Chassis Manufacturer from Host Management-Inventory (from SMBIOS)
caseserial	# Chassis Serial from Host Management-Inventory (from SMBIOS)
caseasset	# Chassis Asset from Host Management-Inventory (from SMBIOS)
location	# Host Location (name) from Host Management-General

(pause)

@C-M Yes use a standalone MDT server to build your reference image in a VM and then sysprep. Then capture the vm by pxe booting into FOG. Works very smooth.

@FlowLive First let me apologize for hijacking this thread a bit. But in the end I think I have a workable solution for your environment to allow you to run FOG along side MDT. I understand what you are trying to do by chaining FOG and MDT/WDS. Wayne has provided a way for you to mesh WDS and FOG into a pxelinux boot menu here.
http://www.vcritical.com/2011/06/peaceful-coexistence-wds-and-linux-pxe-servers/

And in my case I have worked out a way to launch a MDT install from the FOG pxe boot menu. This would allow you to launch and install MDT as you do today, but though FOG. It would provide a smooth transition path from MDT deployment to eventually a FOG deployment.

I’m going to say this with the most sincerity. The path you are on is kind of a waste of time to building your own kernel. You can do it that way if you want, but there are now documented ways to get it done without having to mess with syslinux. One of the issues with PXELinux (at least the last time I looked at it) was it only supports tftp booting and all the restrictions round that. Where iPXE can use other boot methods like http. Now if you use ipxe for your boot loader then chaining would work better. Then you could chain to fog using something like a call to

kernel ipxe.krn dhcp && chain http://<fog_ip_address>/fog/service/ipxe/boot.php?mac=${net0/mac}

(stolen from the fog pxe default configuration)

Borrowed from Sebastian’s signature: Please take a look at your apache error log file (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log)

Take a look at this file on the FOG Server, see if there are any exceptions thrown at the time you deleted the host. This will guide the devs to resolve your issue.

Part 4 Accessing FOG defined variables

While I haven’t described why we’ve included the funcs.sh script from Part 2, we’ll explore that script a bit more. That script includes a lot of common FOG functions as well as it gives us access to some key kernel parameters. Just by including the script in your post install script you have immediate access to all kernel parameters. These kernel parameters as passed from FOG to FOS so FOS knows what to do with/to the target computer. The kernel parameters are outlines below.

ftp		# IP Address from Storage Management-General
hostname	# Host Name from Host Managment-General
img		# Image Name from Image Management-General
mac		# Primary MAC from Image Management-General
osid		# Host Image Index number from Image Management-General
storage		# IP Address + Image Path from Storage Management-General
storageip	# IP Address from Storage Management-General
web		#IP Address + Web root from Storage Management-General

@chimchild OK then (understand this configuration is not supported by the FOG Project, my musings are simply my own).

In this future setup I would do the following.

1. Install linux on your selected hardware with a single nic installed.
2. Copy the fog 1.2.0 stable installer to the linux box, but don’t install it just yet
3. Assign a ip address for the main nic so that it is static and on the lab subnet.
4. With the main network adapter connected to the lab lan, install fog 1.2.0 stable
5. Once fog is setup and functional install the second network adapter on the campus network
6. Assign a static ip address to this nic and make sure the gateway is set on this interface only to point to your internet router on your campus network.
7. Once that is done make sure you can ping devices on your campus network from your FOG server as well as ping (or connect to) devices on the internet.
8. Now to turn your linux box into a router all you need to do is turn a switch on in the linux kernel. You can do it a few different ways. The simplest way to turn in on right away is echo 1 > /proc/sys/net/ipv4/ip_forward that will work until your FOG server is rebooted. To make it a forever change you need to edit the /etc/sysctl.conf file and add in net.ipv4.ip_forward = 1 then finally resync the settings with this command sysctl -p /etc/sysctl.conf
9. OK so now your fog server is a router. There is 2 things you need to do. In your ISP router (or next upstream router) create a static route that defines the LAB subnet, and make it accessible via the FOG server’s network interface on the campus network. That will tell your ISP router how to send data to your LAB network. Then on the LAB network you need to tell those devices that the default route off the LAB network is via the FOG server’s interface on the LAB network.

Once all of those steps are completed your fog server will act as a router between the lab network and the campus network as well as act as a pxe boot server for your lab network. The key is to set your FOG server primary nic on the subnet where you want the pxe booting and image deployment to happen first.

@FlowLive

Looking and the chain load link that you sent. I did see this reference

  set netX/next-server ip.of.wds.server
  imgexec tftp://${netX/next-server}/boot/x86/wdsnbp.com

What I think I might have missed is in my guess is setting the network adapter 0:

set wdsserver:ipv4 192.168.1.88
set net0/next-server ${wdsserver}

Its interesting that the chain does work (kind of) that it does gram the wdsnnp.com file, but there is something in the referral that its missing to continue.

@Wayne-Workman TBH I did not read the entire thread so if this has already been asked my apologies,

What is the results of showmount -e 127.0.0.1?

======
OP: If I understand this correctly you only have one FOG server but you have 2 hard drives in this physical server with each not having enough space for all of your images, do I understand this right? If so I think I would take a little different and less complex approach. There is two routes I can think of

1. Setup LVM and just add those disks to an LVM volume group and then create a LVM logical volume. Let LVM decide where to span the files on those two physical disks. Temp mount that LVM volume over /mnt and then move the content of /images to that LVM volume. Unmount /mnt and remount that new LVM volume over /images and be done with it. (make sure you update fstab). That way you are running a standard FOG configuration without having to deviate from the norm.
2. On your existing FOG server create 2 directories under /images like olifant and koei. Mount those two drives over those directories. Then when you create your image definitions make sure you pick one of the two drives. So if you create an image name called WIN7X64 make sure the destination path isn’t /images/WIN7X64 but /images/koei/WIN7X64 instead. Its not as clean as doing it at the OS level as with LVM but it should work.

Part 3 Code snippets (cont)

7. This next one took me quite a while to work out (now that I’m a bit smarter and know more about the inner workings of FOG there is an easier way to go about this. I’ll show you the harder way because its a good learning point). I needed to make some windows configuration changes based on the location where the target computer was being built. An easy way for my script to tell what location the script was running in was to inspect the FOS IP address and compare that address against my known locations. In this case I’m picking up the FOS ip address by this hack. Once I know the IP address I trim off the last two octets since the first two are only relevant. Once I have the network address I can create a case statement to set specific variables.

myip=`ip route get 8.8.8.8 | awk 'NR==1 {print $NF}' | cut -d "." -f1-2`;

case "${myip}" in
     10.1)
         sitecode="NYC";
         timezone="Eastern Standard Time";
         oupath="ou=computers,ou=nyc,dc=domain,dc=com";
         ;;
     10.2)
         sitecode="LA";
         timezone="Western Standard Time";
         oupath="ou=computers,ou=la,dc=domain,dc=com";
         ;;
     *)
         # Default code for the unknowns
         sitecode="CORP";
         timezone="Eastern Standard Time";
         oupath="ou=computers,ou=corp,dc=domain,dc=com";
         ;;
esac

8. This next one is an easy hack to tell what the intended architecture (x86 or x84) is of the target system. But first I think I need to give a little background first. If the target system (CPU) has 64 bit support, FOG will send the 64 bit FOS engine to the target computer. That 64 bit FOS engine can deploy any operating system and any architecture since all it does is move bits around. So we can’t rely on what architecture FOS has and assume the target image is the same. A quick check to see if the MS Windows target image is 64 bit is to check if the c:\Windows\SysWOW64 directory exists. If it exists then we can assume it is a 64 bit version of MS Windows. If it doesn’t exist then we can probably assume its a x86 install of MS Windows. The code for that logic would look something like this.

if [ -d "/ntfs/Windows/SysWOW64" ]
then
    setarch="x64";
else
    setarch="x86";
fi

9. Lets say you need to make a post install script decision based on input from an IT tech during imaging. You can post a question to the IT tech by using this code

  read -p "Enter your room number where the computer will be installed" rn

  echo "You answered ${rn}. ";

10. You can also update the windows registry from your post install script. Personally I have not done it this way (update the windows registry from FOS), but that is just personal preference. I know that it works and can be done pretty easy. This method uses some linux trickery to simulate keyboard input to automate a linux interactive application. This method is a bit like using the sendkeys function to send key strokes to a windows application for automation. The magic happens between the <<EOFREG and EOFREG tags. This sequence tells bash to send these characters to the reged program as if someone was typing them into the keyboard. Again with linux to Windows interaction watch your case with file paths. As before the “&>/dev/null” directive sends any error messages from the reged command to null (we don’t care about them).

regfile="/ntfs/Windows/System32/config/SOFTWARE"
key="\Microsoft\Windows\CurrentVersion\DevicePath"
devpath="%SystemRoot%\inf;%SystemRoot%\DRV";
reged -e "$regfile" &>/dev/null <<EOFREG
ed $key
$devpath
q
y
EOFREG

11. You can also patch / replace files on the target system too. In this case we are going to check to see if the file SetupComplete.cmd exists in the source folder then copy it to the destination on the drive.

if [ -f "/images/Drivers/Common/SetupComplete.cmd" ]; then
    cp /images/Drivers/Common/SetupComplete.cmd /ntfs/Windows/Setup/Scripts/SetupComplete.cmd;
fi

12. FOS can also make web/url calls. You can send http queries to remote web sites for data logging, information collection, or any thing you can think of. You can also collect the response from the remote web site. Understand this is an advanced function that is pretty powerful, but is also somewhat complex since you need to know how to construct a http request, and have to understand the response from the remote web site. There are two ways to submit a http call. One is a POST and the other is a GET.
    Here is an example of a GET request that sends the results into a text file. As you can see from the url you can embed variables into the get request. In this case we are sending the content of the variable $mac to that php page.

wget -q -O /tmp/hinfo.txt "http://${web}service/hostinfo.php?mac=$mac"

Here is an example of a post request. In this case the mac address is being send as a post. The results of the wget is being sent to the snpchk variable with any errors being sent to the dust bin. In this example the -O switch doesn’t send the output to a file, rather it sends the output to the standard out which is directed into the snpchk variable.

snpchk=`wget -O - --post-data="mac=${mac}" "http://${web}service/snapcheck.php" 2>/dev/null`

This is two different methods to make http calls from inside FOS. The FOG programmers make use of the wget calls for communication between the FOG management server and the FOS engine.

@Wayne-Workman said in Planning out a FOG install.:

@george1421 NAT? And why fog stable?

NAT was not requested. So if the address space is unique to the LAB why not just make it routable and not have to mess with nat. The OP can do do NAT if he wants, but that setup is way beyond the scope of what I posted. (yes I know you can do it with iptables).

FOG stable, I guess I didn’t explain that one very well, and I missed a step.
FOG Stable because that doesn’t have to connect back to the fogproject site for kernel downloads. That installer should be self contained. It will have to be, because the FOG server is not acting like a router yet. But we need the fog installer to be sure to select the NIC connected to the lab network. That way the multicasting and other FOG settings will be configured correctly.

Now the part that should be step 10
10. Download and install the FOG 1.2.0 trunk version and update the stable version of fog 1.2.0 to the latest trunk. Now that routing is working correctly the FOG installer can reach the fogproject.org web site to pickup the checksums and files.

@FlowLive said:

Now I read that it’s not possible to change the menu order of things, WDS being all the way down, can’t I just bump it up a little?

Where it should be, way at the bottom. Where do you think the competition would put it, front and center??

I’m glad/happy you got that wds thing fixed out. From what I’ve read those forward / backslashes are a PITA to get worked out. It looks like you got the right combination to get it done.

Can you confirm if the fog (linux user) account password is still consistent with what is defined in the FOG management GUI? (I’m guessing here) The error may be misleading in that its not really a database error, but the step before where the FOS engine logs into the FOG server via ftp and moves the image from the /images/dev to /images directory.

ref: https://wiki.fogproject.org/wiki/index.php?title=Troubleshoot_FTP

Part 4 Configuring FOG

Before we get to setting up fog I want to give you one warning. Every time you mount and unmount that boot.wim file, we found that the file grows by a certain percentage. Even if you change nothing. The key/goal is to keep that boot.wim file as small as possible. A small boot.wim file means that the file will transfer faster to the target computer, boot faster, and require less RAM to hold the virtual drive. The point is if you need to open the boot.wim to add additional drivers, you are better off just recreating it from the original winpe.wim file, run through the script and before you unmount it add your custom drivers (or automate the custom driver install with DISM as we are doing with the Dell CAB).

Now setting up FOG

1. On your fog server create the following directory with this command
   md /var/www/html/fog/iso
2. Copy one or both of the ISOs you created in Part 2 and/or Part 3 to that directory (use putty, tftp, or ftp to get the iso files onto the FOG server, then move them to the iso directory)
3. Make sure the ISOs mode are World:Read
4. Log into the FOG Management GUI
5. Go Settings->
6. Fill in the following details for the WinPE x86:
   Menu Item: Win7Setup.x86
   Description: Windows 7 DVD x86 Boot
   Parameters:
   initrd http://<ip_addr_fog_server>/fog/iso/winpe_x86.iso
   chain memdisk iso raw
   Menu show with: All Hosts
7. Fill in the following details for the WinPE x64:
   Menu Item: Win7Setup.x64
   Description: Windows 7 DVD x64 Boot
   Parameters:
   initrd http://<ip_addr_fog_server>/fog/iso/winpe_amd64.iso
   chain memdisk iso raw
   Menu show with: All Hosts

From a practical standpoint if you wanted to host 4 different version of windows for installation you could do it with one winPE environment with some way to select which share you connect to. Or create 4 different boot ISOs (one for each OS). While I haven’t tried this, the WinPE 3.0 boot image we created here should launch the setup installers for any OS greater or equal to Windows 7.

@Wayne-Workman said:

it’s not that big of a deal to support all the different distros…

I agree. Really the biggest part of support isn’t FOG (once we move off a trunk build environment).

The bigger support issues are beyond the reach of the fog project.

1. PXE booting (dhcp and initial iPXE image download via tftp)
2. Networking related (mainly spanning tree)
3. Target system hardware (i.e. new network driver of the month club)
4. UEFI bios inconsistencies (emerging problem child)

This issues are mainly target system environmental and not something that FOG can manage directly. I can say that with FOG 2.0 that will enter a new realm since FOG 2.0 will run inside node.js (think of how java was publicized as a cross platform operating environment) and not touch or be dependent on the OS directly. As long as the OS supports node.js then FOG will run on it. Heck node.js runs on a Raspberry Pi (I’m not suggesting that is an ideal platform for FOG, maybe some day though).

@need2 said:

I’m late to the party, but no I never got WDS and FOG to play well together.

@FlowLive was just able to do this. Its a bit down in the thread, but he has FOG acting as the PXE boot server then you can select the WDS environment from the FOG Menu.

I went the other way around and setup FOG to be able to boot the MDT litetouch image so that you can deploy through MDT from the FOG boot menu. Two different ways to have FOG and MDT/WDS coexist.

@pmonstad I know I’m being a type “A” person this AM, but can you confirm that you can log in from a windows pc to the fog server using FTP and the fog (linux user) account?

Part 3 Building a WinPE x64 boot image

The steps are pretty quick (same as part 2).

You can either just copy and paste them, or if you build a custom WinPE environment often create a batch file so you can just launch the batch file and come back a few minutes later and its done.

Just a reminder again, you need to launch the WAIK command line tool with “Run as Administrator” to get the necessary rights to mount the wim.

# Setup your WinPE amd64 Environment
copype x86 C:\winPE_amd64
copy "C:\Program Files\Windows AIK\Tools\PETools\amd64\winpe.wim" C:\winpe_amd64\ISO\Sources\Boot.wim
copy "C:\Program Files\Windows AIK\Tools\amd64\Imagex.exe" C:\winpe_amd64\ISO\

# Create a directory to hang your boot.wim on and then hang it
mkdir c:\img
imagex /mountrw C:\winPE_amd64\ISO\sources\boot.wim  1 c:\img

# Inject the x64 drivers from the extracted Dell cab file
DISM /Image:c:\img /Add-Driver /Driver:c:\winpe\x64 /Recurse /ForceUnsigned

####################################
Using notepad edit C:\img\windows\system32\startnet.cmd

  @echo off
  echo Setting up WinPE
  wpeinit

  echo Connecting to the remote share
  net use z: \\<server_name>\<share_name> /user:<domain\uid> <pass>
  z:
  setup.exe
####################################

# This tip was submitted by KnightRaven, if you remove this file booting the iso image won't prompt
# you to press enter to boot from the CD
del  C:\winPE_amd64\ISO\boot\bootfix.bin


# This command unmounts the wim. If you want the wim alone its this directory C:\winPE_amd64\ISO\sources\boot.wim  
DISM /Unmount-WIM /MountDir:c:\img /commit

# This will covert the wim into a bootable iso image. 
oscdimg -n -bC:\winpe_amd64\etfsboot.com C:\winpe_amd64\ISO C:\winpe_amd64\winpe_amd64.iso

For the net use command net use z: \\<server_name>\<share_name> /user:<domain\uid> <pass> replace the bracketed fields with the server name and share you created in Part 1. For the uid and pass enter the proper domain (or workstation) account required to access that share.

That is it for the WinPE x64 build out. Proceed to Part 4

@Wayne-Workman Great video. Yeah, your image size isn’t large enough to find the standard transfer rates. Just as the transfer settles down its done. (not a bad thing). In our virtual environment we can pull a sustained 6-7GB transfer rate. That is with a VM with 2 vCPU and 2GB of RAM running on a 24 core processor. But again as long as the server can move the image from disk to net pretty quick, all of the heavy lifting is done by the client. On a e6410 with a HDD we can get no more than 4.5GB/min exchanging that HDD with SSD increases the transfer to 6.1GB/min on the same hardware.

Just for comparison, when I was working with that intel NUC as the FOG server. That dual core celeron with an 256GB SSD was able to transfer at 5GB/min to a e6410 with a SSD. So really the FOG server isn’t the key to deployment speed. It more compression used and target computer performance.

But either way FOG is darn impressive that it can wake the computer up and deploy a new image in under 3 minutes. With that you could almost reimage a whole computer lab {between classes} at any school system. That is impressive.

A18 will work as long as you turn off the legacy ROMS. I did see that post too, btw. I just had to try it and it worked with the same results as you. So no better or worse then.

@pmonstad said in not possible to capture images, database connection error:

Deploy works OK

That still makes me think its an ftp issue. Really if its a database issue I would think the FOG console would have issues or … deploy may work since it only uses kernel parameters.

I could see registration fail if there were database user account issues.