Introduction

First I have to say this article contains the results of many brilliant people and is not my content. I’m only assembling this information into a consistent document instead of spread around buried in posts and responses. My intent is to not dig into the details behind the scripts or how to tweak them for your needs. You can read the links below to figure out why things are being done the way they are. I wanted to create a tutorial that was as close to a cut and paste to get driver injection going in your environment. Now I will primarily focus on Dell hardware for the main reason that Dell does supply driver archive files (known as .CABs) that can be downloaded and extracted quickly to create the driver structure. I’m sure that HP, Lenovo, and others have similar driver packs.

You can download the Dell driver cabs for your hardware from here: http://en.community.dell.com/techcenter/enterprise-client/w/wiki/2065.dell-command-deploy-driver-packs-for-enterprise-client-os-deployment

Reference links:
https://forums.fogproject.org/topic/4278/utilizing-postscripts-rename-joindomain-drivers-snapins
https://forums.fogproject.org/topic/7740/the-magical-mystical-fog-post-download-script-under-construction
https://forums.fogproject.org/topic/7740/the-magical-mystical-fog-post-download-script
https://forums.fogproject.org/topic/8878/fog-drivers-script-will-not-run-correctly-in-postdownloadscripts/46

In this series if posts I plan on outlining what the FOG post download script is, what its about, and how to use it to your advantage with image deployment.

I’ve debated with myself if another post was needed since Lee https://forums.fogproject.org/topic/4278/utilizing-postscripts-rename-joindomain-drivers-snapins and I https://forums.fogproject.org/topic/7391/deploying-a-single-golden-image-to-different-hardware-with-fog have written tutorial about them in the past.

Where I want to explore with this series of posts is more about the background behind its function and to cover some of the script elements from Lee’s and my previously linked tutorials.

First of all lets get a little background here and some terminology defined. The FOG image deployment system consists of three main components:

1. The FOG management environment. The FOG management environment is the FOG server itself, which is responsible for planning, scheduling, instructing and documenting the image deployment.
2. The FOS (FOG OS) target engine {need better name here}. The FOS target engine, or FOS (as it will be known in this document) is responsible for capturing and deploying images from the defined storage node, as well as other actions on the target computer. FOS is actually an FOG Project created high performance linux operating system that was specifically constructed to image target computers. We have to remember for the rest of the tutorial, FOS is a linux based operating system that is used to deploy any OS image to your target computer.
3. FOG Client. This is an add on utility service for windows and linux, which is installed on the final or target OS. The FOG Client queries the FOG server for instructions and actions. I’m not planning on discussing the FOG client during this tutorial since it is out of scope in regards to post installation scripting.

The FOG Post install scripts give FOG system admins the ability to inject actions in the image creation process. To allow this custom scripts to run the developers added an external call into the image deployment sequence. Just after the image is placed on the target computer’s storage device, the FOG server calls a script in the /images/postdownloadscripts directory (on the FOG server). That script is called fog.postdownload. The fog.postdownload script is created when the FOG Server is installed. In its default state the script doesn’t do anything much. It is just a place holder to call your custom post install scripts. As I mentioned the fog deployment process calls this fog.postdownload bash script just after the image is placed on the target computer’s storage device. Once all of the external post install scripts have completed the FOG server completes the imaging and data recording steps.

While the post install scripts are stored on the FOG server in the /images/postinstall directory, they are executed by FOS running on the target host. So you have to remember when writing your post install scripts, they execute from the perspective of the target host. The post install scripts have limited access to resources on the FOG server (outside of the /images directory), but have full access to the target host. As noted above, these scripts run on a linux OS, so any resource (programs) available to linux operating systems can be run against the target host. I want to make this distinction to make it clear that FOS is linux and not MS Windows based. So you can not run MS Windows based applications, like DISM, in your post install script. FOS and linux is very powerful, but also limiting in some ways. As long as you are aware that MS Windows based applications can not run in a post install script then you should have little trouble. There are some cross over applications that are compiled for both MS Windows and linux, you just need to ensure you have the correct application for the OS and architecture (IA32 or X64).

The developers found this issue to why the log file grew so big (after almost a year of log file collection). The log file will now be capped at the log file maximum set by the fog configuration settings. This was just a fluke case that caused this file to grow to an abnormal size.

The fix / log file limiting code will be in 1.3.5RC11 when its released.

What you need to understand that the FOS Engine (the customized linux operating system that captures and deployed images on the target computer) is a complete linux OS. It is built with 2 halves. The first part is the the kernel (bzImage) which contains the core linux functions as well as the compiled in device drivers. And the second part is the virtual hard drive (inits) which contains the linux utilities, programs, and fog command scripts. When you PXE boot a target computer into the iPXE menu, certain iPXE menu options will send the bzImage and inits files to the target computer. The iPXE menu will also send specific kernel parameters to tell the FOS Engine (linux) how to react when it boots.

You can take the FOS Engine (bzImage and inits) and just as easily boot directly from a usb flash if you use grub as your boot loader. The FOS Engine will boot from the USB stick without requiring the FOG server to be online. The FOS Engine won’t do much without the proper kernel parameters being passed from the FOG server. But in the end FOS is a specialized, high performance, standalone linux OS.

Now for specifics, I don’t have the details from inside FOS, but I suspect there is something in /etc/init.d that calls the main fog script called /bin/fog that script. The fog master script reads in the passed kernel parameters and then selects the proper task to execute.

Preface

The bits I’m going to cover here are the general outline of what needs to be done to create and deploy a hardware independent image across your fleet of computers. In this tutorial I’m going to discuss how to do this with Dell computers. I know this process works with Lenovo, Intel NUC, and a few others with some caveats. I’m going to touch on some steps that you need to do in MDT to build your universal reference image, but I’m not going to discuss how to setup MDT to create your reference images. There are plenty of examples on the internet on how to do this.

This process makes use of a custom script that gets executed post image deployment but prior to the reboot of the FOS client. This step is vial since we need to make some tweaks to the windows environment pre first windows boot. This is the key. I can’t/won’t share that script I created because of my contract with my employer, because it is derived work because of my employment. So that script is intellectual property of my company. I’ve held the job title of unemployed, its not a great title. The pay stinks, the stress level is high, but the hours are great. I don’t plan on going there any time soon, so no script for you. BUT, sitting here on my zorin (ubuntu) dell laptop I should be able to reverse engineer the important parts to give the crafty scripters here the tools they need to create their own post install script.

Our master image is created following the standard Microsoft SOE guidelines, meaning MDT to build the reference image, sysprep to reseal the image, use a custom unattend.xml file (required to make this process work for Win10) and some disk imaging tool to capture and deploy the image to the target hardware. In this process sysprep is mandatory since we want to create a single image that can be deployed to any hardware. The generalize process of sysprep removes all hardware references (for the reference image) so that when windows first boots it goes through the hardware discovery process. Without sysprepping the image the process I’m going to discuss will not work. So use sysprep.

When I started out creating this process for FOG, I began with the process we created for deploying Windows XP from a usb stick using Ghost. In that process we would automate the image deployment with ghost to lay a hardware independent image onto the target hardware and then detect the current hardware using a DOS program that would query smbios to get the target hardware. Then once the target hardware was known we would move the correct driver pack into a location where windows would find it on the first boot. This worked extremely well with Windows XP. So I took the knowledge that I had from that process and tried to do something similar with FOG.

I do have to say I did not think up this entire process all by my self. I did start the design base on the information found on this wiki page. https://wiki.fogproject.org/wiki/index.php?title=Auto_driver_Install This page and associated scripts gave me just what I needed to take our xp/ghost process and covert it to windows 7/FOG. So without that wiki page the remainder could not be possible.

@julianh A SSD for only the OS will not add much value making FOG go fast. The critical data path is from /images -> nfs -> network -> target computer.

NOTE: Changes in FOG's code since this article was written have made it harder to use NAS' as storage node with FOG. If you use a NAS with FOG 1.5.x and beyond the FOG replicator will continue to cycle and recopy files over and over again.

Part 1 NAS Setup

So far I’ve setup what should work from the synology NAS side of the fog storage node. This has NOT been proven to work just yet. So far the synology nas has been configured to what “should work”. On my test NAS I’m using DSM 6.0.

The following is just my short hand notes that will be used to create the actual tutorial. I’m out of time tonight to complete the docs for this.

The first thing we need to do is setup our NAS with the required network shares. To do this you need access to the Synology NAS’s web console. Log into the web console as admin and do the following.

1. Control Panel->Shared Folder
   Create new share
   Name: images
   Location: volume 1
   Checked Hide this shared folder in My Network Places
   Checked Hide sub-folders and files from users without permissions
   Press OK
   NFS Permission (tab)
   Create new Permission
   Hostname or IP: *
   Privilege: Read/Write
   Squash: No mapping
   Security: sys
   Checked Enable asynchronous
   Checked Allow users to access mounted subfolders

2. Control Panel->Shared Folder
   Create new share
   Name: snapins
   Location: volume 1
   Checked Hide this shared folder in My Network Places
   Checked Hide sub-folders and files from users without permissions
   Press OK
   NFS Permision (tab)
   Create new Permission
   Hostname or IP: *
   Privilege: Read/Write
   Squash: No mapping
   Security: sys
   Checked Enable asynchronous

3. Control Panel->Shared Folder
   Create new Share
   Name: tftpboot
   Location: volume 1
   Checked Hide this shared folder in My Network Places
   Checked Hide sub-folders and files from users without permissions
   Press OK
   NFS Permission (tab)
   Create new Permission (we only need this nfs shared for setting up the storage node)
   Hostname or IP: *
   Privilege: Read/Write
   Squash: No mapping
   Security: sys
   Checked Enable asynchronous

4. Control Panel->File Services
   Select Win/Mac/NFS (tab)
   Checked Enable NFS

5. Control Panel->File Services
   Select FTP (tab)
   Checked Enable FTP service (no encryption)
   Checked Use the default port range

6. Control Panel->File Services
   Select TFTP (tab)
   Checked Enable TFTP service
   TFTP root folder: tftpboot (this is the share name we created above)

7. Control Panel->User
   Select User (tab)
   Create user foguser
   Name: foguser
   Description: FOG User
   Password: fogremote1 (pick your own secure password)
   Conform Password: fogremote1
   Checked Disallow the user to change account password
   Press Next
   Join to group: users
   Press Next
   images: RW
   snapins: RW
   tftpboot: RO
   Press Next
   (Assign quota as needed)
   Press Next
   Assign application permissions: None
   Press Next
   Speed limiting: None
   Press Next
   Press Apply

8. Control Panel->User
   Select the Advanced (tab)
   (scroll way at bottom)
   Under User Home
   Checked Enable user home service
   Press Apply

That completes the setup of the Synology NAS.

In the next part we’ll test the network shares we setup above and create the remaining flag files and directory structure needed to transform the Synology NAS into a FOG storage node.

@oraniko What hardware are you using?

On my campus I have 15 standard hardware where I deploy 1 image to then and then copy the appropriate driver files during FOG imaging.

Ref: https://forums.fogproject.org/topic/7391/deploying-a-single-golden-image-to-different-hardware-with-fog
Ref: https://forums.fogproject.org/topic/8889/fog-post-install-script-for-win-driver-injection
Ref: https://forums.fogproject.org/topic/7740/the-magical-mystical-fog-post-download-script
Ref: https://forums.fogproject.org/topic/4278/utilizing-postscripts-rename-joindomain-drivers-snapins

To answer several requests in the forum about changing the keyboard mapping in the FOG iPXE menu. The only way to do change the keyboard mappings is to recompile the iPXE kernels from their source code since the keyboard layout is defined/set at compile time and not run time.

By following the process below you should be able to compile your own copies of the ipxe boot kernels (undionly.kpxe, ipxe.efi or any other FOG Project supplied boot kernel).
Understand: This method of creating your own boot kernels is not supported by the FOG Project Developers. If your make your own boot kernels and your computers fail to pxe boot, YOU are responsible for fixing any issue (or rerun the fog installer to restore the files to what the developers have created and tested).

Be aware that FOG uses another open source application called iPXE to provide the dynamic menuing structure. iPXE is created and manged by the iPXE Project team and not the FOG Developers.

This tutorial is only an example of how one might compile your own iPXE boot kernels on your fog server.

If your fog server is Ubuntu based you will need to install the compiler and development tools

sudo apt-get install build-essential liblzma-dev

If your fog server is Centos based you will need to install the compiler tools and required library

yum groupinstall "Development Tools" 
yum install xz-devel

Change to your home directory and clone the ipxe source files

git clone http://git.ipxe.org/ipxe.git ipxe

Make back up copies of the files we ar e about to change.

mv ~/ipxe/src/config/console.h ~/ipxe/src/config/console.h.sav
mv ~/ipxe/src/config/general.h ~/ipxe/src/config/general.h.sav
mv ~/ipxe/src/config/settings.h ~/ipxe/src/config/settings.h.sav

If you have the FOG Project installer files already downloaded on your fog server, you can just copy the FOG Project customized files to the iPXE source directories

cp <path_to_installer_files>/fogproject/src/ipxe/src/config/* ~/ipxe/src/config
cp <path_to_installer_files>/fogproject/src/ipxe/src/ipxescript ~/ipxe/src

If you DO NOT have the FOG Project installer files you can download only the specific files needed from the FOG Project GitHub site to create your custom iPXE boot kernels:

wget https://raw.githubusercontent.com/FOGProject/fogproject/blob/master/src/ipxe/src/config/console.h -O ~/ipxe/src/config/console.h
wget https://raw.githubusercontent.com/FOGProject/fogproject/blob/master/src/ipxe/src/config/general.h -O ~/ipxe/src/config/general.h
wget https://raw.githubusercontent.com/FOGProject/fogproject/blob/master/src/ipxe/src/config/settings.h -O ~/ipxe/src/config/settings.h 
wget https://raw.githubusercontent.com/FOGProject/fogproject/blob/master/src/ipxe/src/ipxescript -O ~/ipxe/src/ipxescript

Use the linux command sed to change the keyboard mapping. To use the following command replace XX with the proper keyboard code for your location.
sed -i 's/KEYBOARD_MAP\tus/KEYBOARD_MAP\tXX/g' ~/ipxe/src/config/console.h

For example if you want to change the US keyboard to the French keyboard in the FOG iPXE menus you would replace XX with fr as in this example:

sed -i 's/KEYBOARD_MAP\tus/KEYBOARD_MAP\tfr/g' ~/ipxe/src/config/console.h

To be sure the settings are correct, you can run the following command to confirm the keyboard mappings have been changed.

grep -e 'KEYBOARD_MAP' ~/ipxe/src/config/console.h

[Editor note, insert expected output here]

All supported iPXE keymaps are listed on this page:
https://github.com/ipxe/ipxe/tree/master/src/hci/keymap

Now that you have the keyboard mapping set, lets compile iPXE…
Change to the iPXE source directory and run make clean to remove any unneeded bits

cd ~/ipxe/src
make clean

Build the BIOS boot kernel undionly.kpxe, rename the original undionly.kpxe file, and finally copy over the custom iPXE boot kernel.

make bin/undionly.kpxe EMBED=~/fogproject/src/ipxe/src/ipxescript
sudo mv /tftpboot/undionly.kpxe /tftpboot/undionly.kpxe.old
sudo cp bin/undionly.kpxe /tftpboot

Purge any prevoiusly built object files (aways good practice when you update configuration files.

make clean

If you have the FOG Project installer files already downloaded on your fog server, you can just copy the FOG Project customized files to the iPXE source directories

cp <path_to_installer_files>/fogproject/src/ipxe/src-efi/config/* ~/ipxe/src/config

If you DO NOT have the FOG Project installer files you can download only the specific files needed from the FOG Project GitHub site to create your custom iPXE boot kernels:

wget https://raw.githubusercontent.com/FOGProject/fogproject/blob/master/src/ipxe/src-efi/config/console.h -O ~/ipxe/src/config/console.h
wget https://raw.githubusercontent.com/FOGProject/fogproject/blob/master/src/ipxe/src-efi/config/general.h -O ~/ipxe/src/config/general.h
wget https://raw.githubusercontent.com/FOGProject/fogproject/blob/master/src/ipxe/src-efi/config/settings.h -O ~/ipxe/src/config/settings.h

Use the same sed command above to update the UEFI boot files. Hint: Don’t forget to replace XX with your desired keyboard map.

sed -i 's/KEYBOARD_MAP\tus/KEYBOARD_MAP\tXX/g' ~/ipxe/src/config/console.h

Build the UEFI boot kernel ipxe.efi, rename the original ipxe.efi file, and finally copy over the custom iPXE boot kernel.

make bin-x86_64-efi/ipxe.efi EMBED=~/fogproject/src/ipxe/src/ipxescript
sudo mv /tftpboot/ipxe.efi /tftpboot/ipxe.efi.old
sudo cp ~/ipxe/src/bin-x86_64-efi/ipxe.efi /tftpboot

If you need snponly.efi (roughly equivelent to BIOS’ undionly.kpxe) run these commands (Hint: This compile command will run really quick since all of the hard work was done in the previous step).

make bin-x86_64-efi/snponly.efi EMBED=~/fogproject/src/ipxe/src/ipxescript
sudo mv /tftpboot/snponly.efi /tftpboot/snponly.efi.old
sudo cp ~/ipxe/src/bin-x86_64-efi/snponly.efi /tftpboot

Clean up after yourself in case you need to build these kernels over again.

make clean

@ITSolutions said

Disadvantages in my personal opinion are few:

• Need to have Linux knowledge(although having Linux knowledge is a advantage in the industry)
  The other disadvantages I used to see, when I first found FOG in the .3x days have been corrected/added. I would like to see what others feel are disadvantages.

I can see the linux knowledge thing being a big issue for those that grew up in the MS Windows era. Once the system is setup you don’t need to step into linux at all. But every once an a while you need access to the linux command shell, so you will need someone in your company with some familiarity with linux.

Other things I can see that are limitations…
Access control. Right now there isn’t a real good way to limit what IT techs can do in the system. Like certain IT techs can only deploy images 1,2,3 to location C only.

Having a centralized login authentication system with either AD/LDAP.

Linux hardware drivers lag a bit behind windows drivers. The fact is that most hardware vendors develop hardware for the windows folks and then it seems like linux is either an after thought or some really smart linux people hack into the hardware and come up with their own linux drivers. (thinking about the surface pro 4 as I write this).

@Jamaal The tutorial was more of a proof of concept than something that should be used in production. Did it work, yeah. Would I probably use it… maybe not but some of the dedup capabilities in 2016 and 2019 make it sound intreaging.

For your case if you have additional space on your VM host server you might consider adding an additional virtual disk to your fog server then adding it to the LVM group for your fog server. There are a number of ways to go about this, but that discussion should be moved to a new thread.

d1.partitions:

label: gpt
label-id: B4C26CFC-6D5A-4CE1-94D1-0F4E18D8D657
device: /dev/nvme0n1
unit: sectors
first-lba: 34
last-lba: 500118158

/dev/nvme0n1p1 : start= 2048, size= 532480, type=C12A7328-F81F-11D2-BA4B-00A0C93EC93B, uuid=79E1F6C3-BEF0-$
/dev/nvme0n1p2 : start= 534528, size= 32768, type=E3C9E316-0B5C-4DB8-817D-F92DF00215AE, uuid=55A3D53F-A47D-$
/dev/nvme0n1p3 : start= 567296, size= 497502208, type=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7, uuid=622E9706-DECA-$
/dev/nvme0n1p4 : start= 498069504, size= 2048000, type=DE94BBA4-06D1-4D40-A16A-BFD50179D6AC, uuid=524E378B-F6E3-$

its failing to restore the mbr partition.

Captured image is from a 500GB sata, attempting to restore to a 250GB nvme drive.

Question would block size be an issue where the sata would be 512B and the nvme would be 4K (guess), would that cause the mbr fail to deploy?

@Jim-Holcomb If you look on the fog server in /images/aiov910 directory does d1.mbr exist?

@Jim-Holcomb Ah OK it looks like you beat me to it.

So lets get a screen shot of the host management page for this target computer. We need to understand where that /dev/md0 is coming from.

Lets start out with rescheduling a capture or deploy but tick the check box that says debug before you submit the task.

Then pxe boot the target computer after a few screens of text you need to clear with the enter key you should be dropped to a linux command prompt, at that linux command prompt key in the following lsblk and grab a screen shot of the output.

The issue I’m having is with the disk driver called /dev/md0 that would indicate a raid controller.

@Zourous You will need to assign an IP address range to this imaging network that is different than your home LAN network. So if your home LAN network is 192.168.1.x/24 make this imaging network 192.168.12.0/24 and the fog server 192.168.12.1 on the imaging network. For the HOME LAN network leave that interface to dhcp and let your home network assign the IP address or (maybe a better idea) assign the FOG server a static IP address on the HOME LAN network so that you always know the management interface on is 192.168.1.x address.

When you setup FOG you will define the imaging network to the ethernet adapter and the dhcp server will bind to that interface only.

Lastly you will probably need a small (cheap) switch or a cross over cable to connect the FOG server to the target computer.

This configuration will work.

@Zourous I can read your request a few different ways.

1. You want to setup a fog server with an imaging network and a business (home) network. Your business network will be connected to the internet over home wifi. Is this possible? Yes. The fog server will need to supply dhcp and dns services to your imaging network. This is a normal function in fog if you pick these options while installing FOG. You will need to know the linux name of the imaging network interface as well as configure the imaging network interface with a static IP address before you install the FOG software.

2. You want to directly connect a PC to the FOG server using a single cable for imaging. You don’t plan on using a network switch on your imaging network. Can I plug the ethernet cable directly between the fog server and the target computer? Yes. You will need to use an ethernet cross over cable for this. Some network ports will auto detect which mode to use (mdi or mdi-x) but using a cross over cable will work in all instances.

X. If you want your FOG server to act as a router between your imaging network and the internet, is that possible? Yes. You will need to make a few changes in linux to convert your FOG server into a router so the clients on your imaging network can reach the internet (for what ever reason).

@ZrytyADHD said in Booting accros networks:

You have a number of problems here.

1. FOG doesn’t like it when you change the fog server’s IP address after FOG is installed. There are several steps that you must do if you decide to change the FOG server IP address after fog is installed.

2. Why are you mixing WDS and FOG in your booting process? You can do manage it but I would go about it a bit differently.

3. Your section of the isc dhcp server is just wrong in my opinion.

if exists user-class and option user-class = "iPXE" {
        filename "http://10.0.0.68/boot.php";
    } else {

        if option arch = 00:00 {

            filename "undionly-mj2.kpxe";
        } else {
#UEFI Bootujemy z WDS UEFI
        next-server 10.249.4.20;
        filename "boot\\x64\\wdsmgfw.efi"
        }
    }

a. I’m not aware of a user class in the dhcp discover packet. There is a vendor class
Vendor class identifier: PXEClient:Arch:00007:UNDI:003016

b. The following line is wrong for several reasons

filename "http://10.0.0.68/boot.php";

b.1 you are telling the PXE rom to load a http protocol. Almost all PXE roms use tftp not http.
b.2 The path to the boot.php is wrong if you are trying to call the boot.php on the fog server the path is <fog_server_ip>/fog/service/ipxe/boot.php

For this I would just use filename "default.ipxe"; but that still will not work because you don’t mention the boot loader (undionly.kpxe or ipxe.efu)

c. You are saying if the option is bios then load undionly-mj2.kpxe (not a fog supplied boot loader) but if everything else call WDS. But only do this if user-class == “iPXE”

d. uefi pxe booting into FOG is not supported

There are ways to fix this, dhcp booting, but please explain why you are mixing FOG and WDS. What is your goal of doing this? Will it be a for ever solution?

@foggystarz TBH I don’t know. If it was me I’d copy them to the fog server in a different directory than the one you have working and try just the cdrom initrd file (keep the network kernel one from the working [linux] ) and try to boot. See where you end up.

See the linux has the NFS services turned on where the vmlinuz one probably doesn’t. What you need (maybe) is the logic in the virtual hard drive [initrd] to start up the live runtime and not the installer.

@brakcounty There should be a default menu timeout function in grub. So that if no entry is selected grub will pick the default menu. This is akin to how iPXE works.